Lead Data Engineer

Grid Automation (GA) business to identify areas where the business can leverage common framework for data storage and data analytics to drive efficiency, and quickly develop POCs to solve critical problems for our customers and build state-of-the-art models and deploy them on edge or cloud based systems.

Job Description

The Lead Data Engineer will be responsible for:

• Design and maintain database architectures, schemas, and data models tailored to grid innovation and energy system applications.

• Apply efficient storage technologies (Relational, Data lakes, NoSQL etc.) to manage data storage, access and data security.

• Build, optimize, and maintain reliable data pipelines for data ingestion, cleaning, transformation, and feature extraction from structured and unstructured sources.

• Build and maintain integrations with internal and external data sources and APIs.

• Identify and integrate new datasets that can be leveraged through our product capabilities.

• Automate Integration of data from various sources in a unified format and create transformation structure to based on business specific utilization needs.

• Monitor and optimize data pipeline for scalable system.

• Establish appropriate data quality checks and follow data governance policies.

• Work closely with other adjacency functions such as Data Scientists and ML engineers to cater relevant data expectations.

• Apply data governance policies and implement data quality checks to ensure data integrity across systems.

• Collaborate with cross-functional teams of product management, R&D, and other functions, to understand their needs and develop innovative solutions.

QUALIFICATIONS/REQUIREMENTS:

• PhD/Masters/Bachelors in Data Science, computer science, electrical and computer engineering, specifically in the computer and electric power engineering field with hands-on experience in data engineering.

• Proven experience in the energy, smart infrastructure, or industrial automation sectors, with hands-on project experience in building and managing data pipelines, typically acquired through a minimum of 5 years of service.

• 5+ years' experience working with professionals of statistical techniques, artificial intelligence (AI) and machine learning (ML), to understand the requirement of both structured and unstructured databases.

• Experience in data pipeline creation.

• Database management experience with relational (e.g., PostgreSQL) and NoSQL (e.g., Cassandra, MongoDB) databases and data warehousing technologies such as, Snowflake or Redshift.

• Familiarity with cloud platforms like AWS, Azure, or GCP for deploying and managing data systems.

• Extensive experience with ETL processes (Extract, Transform, Load) and automating data pipeline workflows.

• Experience with data visualization tools such as Tableau, Power BI, or similar platforms for building reports and dashboards.

• Familiarity with big data tools and technologies, such as Hadoop, Kafka, and Spark.

• Able to share ideas and work well in a team environment, proactive approach to tasks displaying initiative.

• Flexible and adaptable; open to change and modification of tasks, working in multi-tasking environment.

DESIRED CHARACTERISTICS:

• 5+ years of industry experience

• Ability of using data scientific programming tools or languages, such as Python, Scala, Java, SQL etc.

• Hands-on on ETL (Extract, Transform and Load) and CRUD process.

• Experience with different cloud platforms such as AWS, GCP, Azure etc.

• Experience with version control systems e.g., Git

• Familiarity with big data technologies such as Hadoop, Kafka etc.

• Understanding of data warehousing principles and technologies.

• Experience with data visualization tools (e.g., Tableau, Power BI).

• Experience in GraphDB, MongoDB, SQL/NoSQL, MS Access, databases.

• Strong communication skills and a proactive and open approach to conflict resolution.

• Strong organizational skills, self-motivated, and self-directed.

Additional Information

Relocation Assistance Provided: No