AI Data Engineer

of data extraction.

• Write production-grade SQL and PySpark code to create data architecture.

• Consolidate SQL databases from multiple sources, data cleaning, and manipulation in preparation for analytics and machine learning.

• Use data visualization tools such as Power BI to create professional quality dashboards and reports.

• Write good quality documentation for data processing for different projects to ensure reproducibility.

• Responsible to ensure compliance with applicable external and internal regulations, procedures, and guidelines.

• Living Hitachi Energy's core values safety and integrity, which means taking responsibility for your own actions while caring for your colleagues and the business.

Your Background:

• BE / B.Tech in Computer Science, Data Science, or related discipline and at least 5 years of related working experience.

• 5 years of data engineering experience, with understanding lake house architecture, data integration framework, ETL/ELT pipeline, orchestration/monitoring, star schema data modeling.

• 5 years of experience with Python/PySpark and SQL.( Proficient in PySpark, Python, and Spark SQL).

• 2-3 years of hands-on data engineering experience using Databricks as the main tool (meaning >60% of their time is using Databricks instead of just occasionally).

• 2-3 years of hands-on experience with different Databricks components (DLT, workflow, Unity catalog, SQL warehouse, CI/CD) in addition to using notebooks.

• Experience in Microsoft Power BI.

• Proficiency in both spoken & written English language is required.

Apply now

Current employees apply here