quality data pipelines driving analytic solutions from diverse, disparate sources of data.
This role requires a working understanding of data architecture, data engineering, and data analysis. The ideal candidate is a skilled data engineer with experience creating data products supporting analytic solutions. They are able to identify and implement solutions in a highly technical environment and work as part of a technical, cross-functional team. Strong problem-solving and troubleshooting skills are a must.
This role focuses on designing, developing, and maintaining datasets within AWS with oversight and support from Lead Developers. Day-to-day responsibilities include:
- Coordinating, building, and managing new data ingests
- Implementing updates, fixes, and optimizations across suite of production jobs
- Providing feedback on and enacting changes for improvements across the team including both technical and process updates
- Learning about and becoming familiar with data assets, processing jobs and pipelines, and tools
- Partnering with data analysts and data scientists to design, build, and deploy aggregation processes
Top Skills:
- Strong SQL, Python scripting (PySpark, control scripts), Write and review code
- Building and managing data pipeline, ETLs, Query and data pipeline optimization
- AWS experience (EMR, Lambda, Glue, Step Functions, Athena) and Airflow (MWAA, AWS-native orchestration)
- Hadoop/Hive experience, Spark experience (including PySpark)
- BI experience: Tableau, SQL-based reporting, Excel
- Experience working with large data sets (billions of records per day)
- Strong communication and collaboration skills, PowerPoint /presentation skills
- Ability to learn new technologies and skills to incorporate into existing work
Nice to have :
- GitLab /Bit Bucket Experience
- Terraform / AWS Cloud Formation Experience
- Additional BI development exposure
- Experience with hybrid Analyst/Developer workflows