Data Engineer

for designing, developing, and maintaining our data infrastructure and systems. You will collaborate with cross-functional teams to understand data requirements, implement data pipelines, and ensure the availability, reliability, and scalability of our data solutions.
You can program in several languages and understands the end-to-end software development cycle including CI/CD and software release. you will also be responsible for developing and maintaining Power BI reports and dashboards, ensuring they meet business requirements and provide actionable insights. You will work closely with stakeholders to gather requirements and deliver high-quality data visualizations that drive decision-making
"
"* Design, develop, and maintain scalable data pipelines and data integration processes to extract, transform, and load (ETL) data from various sources into our data warehouse or data lake.

• Collaborate with stakeholders to understand data requirements and translate them into efficient and scalable data engineering solutions.
• Optimize data models, database schemas, and data processing algorithms to ensure efficient and high-performance data storage and retrieval.
• Implement and maintain data quality and data governance processes, including data cleansing, validation, and metadata management.
• Work closely with data scientists, analysts, and business intelligence teams to support their data needs and enable data-driven decision-making.
• Develop and implement data security and privacy measures to ensure compliance with regulations and industry best practices.
• Monitor and troubleshoot data pipelines, identifying and resolving performance or data quality issues in a timely manner.
• Stay up to date with emerging technologies and trends in the data engineering field, evaluating and recommending new tools and frameworks to enhance data processing and analytics capabilities.
• Build insights using various BI tools - Power BI
• Collaborate with infrastructure and operations teams to ensure the availability, reliability, and scalability of data systems and infrastructure.
• Mentor and provide technical guidance to junior data engineers, promoting best practices and knowledge sharing."" "

Qualifications:

"Required: Bachelor's degree from an accredited institution
"
• 3+ years of experience in Data Engineering and Power BI

• 3+ years of experience in data analytics

Skills:

• Apache Spark, Python
• Azure experience (Data Bricks, Docker, Function App)
• Git
• Working knowledge of Airflow
• Knowledge of Kubernetes and Docker
• Power BI:
o Data Visualization: Proficient in creating interactive and visually appealing dashboards and reports using Power BI.
o Data Modeling: Experience in designing and implementing data models, including relationships, hierarchies, and calculated columns/measures.
o DAX (Data Analysis Expressions): Strong knowledge of DAX for creating complex calculations and aggregations.
o Power Query: Skilled in using Power Query for data transformation and preparation.
o Integration: Ability to integrate Power BI with various data sources such as SQL databases, Excel, and cloud services.
o Performance Optimization: Experience in optimizing Power BI reports for performance and scalability.
o Security: Knowledge of implementing row-level security and managing user access within Power BI.
o Collaboration: Experience in sharing and collaborating on Power BI reports and dashboards within an organization.
o Best Practices: Familiarity with Power BI best practices and staying updated with the latest features and updates.
• Proficient with using Power BI
• Background in SQL and experience working with relational databases.
• Bachelor Degree in Computer Science or Software Engineering or Information Technoogy
• Experience on Cloud Development Platforms - Azure & AWS and their associated data storage options
• Experience on CI/CD (Continuous Integration/Delivery) i.e. Jenkins, GIT, Travis-CI
• Virtual build environments (Container, VMs and Microservices) and Container orchestration - Docker Swarm, Kubernetes/Red Hat Openshift.
• Relational & non-relational database systems - SQL, Postgres SQL, NoSQL, MongoDB, CosmosDB
"