Data Engineer - Scala

Description

We are looking for an enthusiastic Data Engineer - Scala to work to work at the forefront of our cloud modernisation within our Credit & Verification Services. This is a hybrid role requiring travelling to Hyderabad office 40% per month.

Background:

This is an incredibly exciting time for the Experian UKI Region, as we look to build our presence in the UK and Hyderabad and work on a technology transformation to meet our aspiration to significantly scale our business over the next five years. This an opportunity to join Credit & Verification Services on this journey and be part of a collaborative team that uses Agile DevSecOps principles to deliver business value.

Credit and Verification Services currently comprises nearly 100 engineering teams who deliver over 200 products achieving significant revenue per annum for our UK Business. Our unique culture and agile ways of working offer a great opportunity to those looking to join an experienced diverse set of problem solvers to design, build and maintain our products. We pride ourselves in excellence, adopting best practices and holding ourselves to the highest standards.

Main responsibilities:

• You will be #LI-hybrid based in Hyderabad and reporting to Director Engineering.

• Develop and maintain batch data processing pipelines using Scala and Apache Spark.

• Optimize Spark jobs for performance and scalability in a cloud-native environment (AWS).

• You will collaborate with data engineers and analysts to understand data requirements and deliver high-quality solutions.

• Ensure data integrity, reliability, and consistency across systems.

• Participate in code reviews, design discussions, and agile development practices.

• Monitor and troubleshoot production jobs, ensuring resolution of issues.

Qualifications

• 3-6 years of software development experience, with at least 2 years in Scala.

• Write clean, scalable and re-usable code that implements SOLID principles, common design patterns where applicable and adheres to published coding standards

• Strong understanding of functional programming and distributed computing.

• Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins

• Hands-on experience with Apache Spark for batch processing.

• Familiarity with AWS services such as S3, AWS Glue, Lambda, Step Functions, AWS Batch and CloudWatch.

• Proficiency in working with structured and semi-structured data formats (e.g., Parquet, JSON).

• Experience of the full development lifecycle/application Lifecycle Management Tools - BitBucket, Jira

• Experience with data lake architectures and ETL frameworks.

• Exposure to containerization tools (Docker, Kubernetes)

• Knowledge of data governance and security best practices

• Relevant certifications in AWS, Spark, or Scala.

Additional Information

Our uniqueness is that we celebrate yours. Experian's culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI, work/life balance, development, authenticity, collaboration, wellness, reward & recognition, volunteering... the list goes on. Experian's people first approach is award-winning; World's Best Workplaces™ 2024 (Fortune Global Top 25), Great Place To Work™ in 24 countries, and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site and Glassdoor to understand why.

Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is a critical part of Experian's DNA and practices, and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work, irrespective of their gender, ethnicity, religion, color, sexuality, physical ability or age. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.

Benefits

Experian care for employee's work life balance, health, safety and wellbeing. In support of this endeavor, we offer best-in-class family well-being benefits, enhanced medical benefits and paid time off.

Experian Careers - Creating a better tomorrow together

Find out what its like to work for Experian by clicking here