Associate Manager - Data Engineer

parent, global operating rhythm that ensures always-on access to high-quality data for stakeholders across the company

• Responsible for day-to-day data collection, transportation, maintenance/curation and access to the PepsiCo corporate data asset

• Work cross-functionally across the enterprise to centralize data and standardize it for use by business, data science or other stakeholders

• Increase awareness about available data and democratize access to it across the company

Job Description

As a member of the data engineering team, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into various source systems, rest data on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. As a member of the data engineering team, you will help lead the development of very large and complex data applications into public cloud environments directly impacting the design, architecture, and implementation of PepsiCo's flagship data products around topics like revenue management, supply chain, manufacturing, and logistics. You will work closely with process owners, product owners and business users. You'll be working in a hybrid environment with in-house, on-premise data sources as well as cloud and remote systems.

Responsibilities

• Design, develop & tune data products, applications, and integrations on large-scale data platforms (SQL Server, Databricks, SSIS, SAP Hana) with an emphasis on performance, reliability, and scalability, and most of all quality.

• Analyze the business needs, profile large data sets and build custom data models and applications to drive the business decision making and customers experience.

• Participating in all aspects of data development activities, including design, coding, code review, testing, bug fixing, and code/API documentation.

• Collaborate with software quality engineers and other cross-functional teams in all phases of testing, documentation, and training

• Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide insights to make better decisions

• Implement Azure products and services, including Azure Data Lake Storage, Azure Data Factory, Azure Functions, Event Hub, Azure Stream Analytics, Azure Databricks, etc., as well as conventional data warehouse tools, to create data migration and data engineering solutions.

• Perform various phases of the development lifecycle, including tasks such as design, cloud engineering (covering infrastructure, network, security, and administration), data ingestion, data preparation, data modeling, testing, setting up continuous integration and continuous deployment (CICD) pipelines, optimizing performance, managing deployments, enabling data consumption, facilitating business intelligence, configuring alerting, and providing production support.

• Proficiency in one or more programming languages such as Python, SQL, PySpark or Scala

• Expertise in writing stored procedures and SQL queries

• Uses Azure services such as Azure SQL DW (Synapse), ADLS, Azure Event Hub, Azure Data Factory to improve and speed up delivery of our data products and services.

• Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.

• Implement batch and streaming data pipelines using cloud technologies

• Work effectively in an Agile/Scrum environment - writing / decomposing user stories, participate in grooming and estimation sessions etc.

• Skilled in analyzing problem statements, breaking down problems

• Develops and operationalizes data pipelines to make data available for consumption (BI, Advanced analytics, Services).

• Strong data analytics, quantitative skills, problem-solving skills, comfortable navigating data-driven business environments

• Develops and operationalizes data pipelines to make data available for consumption (BI, Advanced analytics, Services).

• Works in tandem with data architects and data/BI engineers to design data pipelines and recommends ongoing optimization of data storage, data ingestion, data quality, and orchestration.

• Grow with the support of your team and help others on the team grow by providing thoughtful feedback and uplifting those around you.

• Work both independently and collaboratively within a fast-paced development team, with clear, positive, and constructive communication.

Qualifications

• 11+ years of overall technology experience that includes at least 4+ years of hands-on software development, data engineering, and systems architecture.

• 4+ years of experience with Data Lake Infrastructure, Data Warehousing, and Data Analytics tools.

• 4+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc.).

• 2+ years in cloud data engineering experience in Azure.

• Fluent with Azure cloud services. Azure Certification is a plus.

• Experience with integration of multi cloud services with on-premises technologies.

• Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.

• Experience with data profiling and data quality tools like Apache Griffin, Deequ, and Great Expectations.

• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.

• Experience with at least one MPP database technology such as Redshift, Synapse or SnowFlake.

• Experience with running and scaling applications on the cloud infrastructure and containerized services like Kubernetes.

• Experience with version control systems like Github and deployment & CI tools.

• Experience with Azure Data Factory, Azure Databricks and Azure Machine learning tools.

• Experience with building solutions in the retail or in the supply chain space is a plus

• Working knowledge of agile development, including DevOps and DataOps concepts.

• Familiarity with business intelligence tools (such as PowerBI).

Education

• Bachelor's degree in Computer Science, Computer Engineering, Technology, Information Systems (CIS/MIS), Engineering or related technical discipline, or equivalent experience/training

Skills, Abilities, Knowledge

• Excellent communication skills, both verbal and written, along with the ability to influence and demonstrate confidence in communications with senior level management.

• Proven track record of leading, mentoring data teams.

• Strong change manager. Comfortable with change, especially that which arises through company growth.

• Ability to understand and translate business requirements into data and technical requirements.

• High degree of organization and ability to manage multiple, competing projects and priorities simultaneously.

• Positive and flexible attitude to enable adjusting to different needs in an ever-changing environment.

• Strong leadership, organizational and interpersonal skills; comfortable managing trade-offs.

• Foster a team culture of accountability, communication, and self-management.

• Proactively drives impact and engagement while bringing others along.

• Consistently attain/exceed individual and team goals

• Ability to lead others without direct authority in a matrixed environment.

Competencies

• Highly influential and having the ability to educate challenging stakeholders on the role of data and its purpose in the business.
• Understands both the engineering and business side of the Data Products released.
• Places the user in the center of decision making.
• Teams up and collaborates for speed, agility, and innovation.
• Experience with and embraces agile methodologies.
• Strong negotiation and decision-making skill.
• Experience managing and working with globally distributed teams.