Spark Data Engineer

BT-454 – Spark Data Engineer

Location: Tysons, VA

Required skills: 

• Demonstrates experience building and managing data pipelines
• Demonstrates experience with Python
• Demonstrates experience with cloud computing, using AWS services
• Demonstrates experience processing data using Apache Spark
• Demonstrates experience with an RDBMS (PostgreSQL, Oracle, MySQL) and writing SQL queries
• Demonstrates experience with Linux and shell scripting
• Demonstrates experience analyzing data in different file formats like CSV, XML, JSON, Avro, Parquet, etc.
• Demonstrates experience writing and validating unit tests

Highly desired: 

• Experience with NiFi, Apache AirFlow, or an equivalent solution or tool for orchestrating data pipelines
• Experience with Java or Scala
• Experience administering an EMR/Spark cluster
• Experience conducting performance tuning of a Spark job
• Experience developing cloud-based security solutions
• Experience following a configuration management process to review and deploy code as part of releases