Senior Data Engineer (AWS)

u'll be responsible for designing, building, and maintaining scalable, production-grade data pipelines that can handle large volumes of batch and streaming data.

This role requires deep, hands-on experience with AWS services and a track record of building reliable, secure, and high-performance pipelines at scale. You'll collaborate closely with data scientists, analysts, and software engineers to ensure data is accessible, reliable, and optimized for analysis and decision-making.

If you're passionate about solving complex data challenges, driving reliability and efficiency, and mentoring others, this is a high-impact opportunity to make a real difference in how data is used across the company.

Where you'll work

You must be located in the US to be considered for this position. You can choose to work remotely or from our offices in Austin or Dallas.

What you'll be doing

• Designing, developing, and optimizing scalable, high-performance data pipelines for both batch and streaming ingestion in AWS.

• Building data workflows with AWS Glue, Lambda, Step Functions, Kinesis, Firehose, and CloudWatch.

• Partnering with data architects and analysts to design and implement data models that support efficient storage, retrieval, and analytics.

• Establishing monitoring, alerting, and automated recovery mechanisms to ensure pipeline reliability and minimize downtime.

• Enforcing security, privacy, and compliance standards using tools like Secrets Manager, IAM, and encryption best practices.

• Proactively identifying opportunities to improve pipeline performance, reduce latency, and optimize costs.

• Staying current with AWS advancements and advocating for modern data engineering practices across the team.

• Mentoring junior engineers and contributing to the evolution of our data engineering standards.

What you'll need

• 7+ years of experience in Data Engineering or a similar role, with at least 5 years of deep, hands-on AWS experience.

• Expertise with AWS ingestion and processing services (Glue, Kinesis, Firehose, Lambda, Step Functions).

• Strong experience in monitoring and troubleshooting pipelines using CloudWatch, logs, alarms, and dashboards.

• Proficiency with Python (preferred) or other programming languages for data workflows.

• Strong SQL skills and experience with data modeling and data warehousing concepts.

• Hands-on experience with orchestration tools (Airflow, Step Functions) and workflow automation.

• Knowledge of data governance, security, and integration principles.

• Proven track record of optimizing pipelines for performance and cost efficiency.

• Excellent problem-solving and communication skills, with the ability to lead discussions and drive technical decisions.

Perks of working for ShiftKey

• Inclusive and collaborative work environment where all voices are valued.

• Hybrid-friendly office spaces designed to be fun and engaging.

• Comprehensive health, vision, and dental coverage.

• Benefits begin on your first day.

• Generous PTO and company-paid holidays, including flexible floating holidays.

• 100% 401(k) employer match up to 6%.

• Paid parental leave.

• Wellness support, including access to mental health resources.

#LI-Remote #LI-AT1

Our Commitment to Diversity

ShiftKey is an equal opportunity employer and is committed to embracing diverse and individual perspectives shaping our culture and contributing to the solutions that transform the world. We actively work to create an inclusive environment for employees to thrive.

EEO Statement

ShiftKey does not discriminate based on race, religion, color, sex, gender, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by applicable law. All employment is decided based on qualifications, merit, and business needs. ShiftKey is committed to providing reasonable accommodation to applicants with a mental or physical disability, please contact: [email protected]