Big Data Spark Lead Engineer

in lieu of every year of education.

• At least 4 years of experience in Information Technology.
• Candidate must be located within commuting distance of Austin, TX / Sunnyvale, CA or be willing to relocate to the area. This position may require travel within the US.
• Experience with Hadoop distributed frameworks while handling large amount of data using Spark or PySpark and Hadoop Ecosystems.
• Proven experience in data engineering, data architecture, or a related field
• Experience with Spark or PySpark is required.
• Strong understanding of data modeling, data warehousing, and ETL concepts
• Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop or Spark
• Experience with Scala or Python is required.

Preferred Q****ualifications:

• Experience in understanding of Design Patterns, ability to discuss tradeoffs between RDBMS vs Distributed Storage
• At least 3 years of Experience with Spark or PySpark, Python, Scala and Data Engineering
• Experience in design and implementing a tiered data architecture that integrates analytics data from multiple sources in an efficient and effective manner.
• Experience with data orchestration tools like Airflow is a nice to have.
• Excellent problem-solving and analytical skills, and the ability to work well under tight deadlines.
• Excellent interpersonal skills and the ability to collaborate effectively with cross-functional teams.
• Experience in developing data models and mapping rules to transform raw data into actionable insights and reports.
• Experience in collaborating with the analytics and business teams to understand their requirements, with cross-functional teams to define and implement data governance policies and standards.
• Experience in developing data validation and reconciliation processes to ensure data quality and accuracy is met.
• Experience with development and maintenance of user documentation, including data models, mapping rules, and data dictionaries.

The job entails extensive amount of travel. The job also entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face.

Estimated annual compensation range for candidate based in the below locations will be:
Sunnyvale, CA- 90751 - 139208
Along with competitive pay, as a full-time Infosys employee you are also eligible for the following benefits :-

• Medical/Dental/Vision/Life Insurance
• Long-term/Short-term Disability
• Health and Dependent Care Reimbursement Accounts
• Insurance (Accident, Critical Illness , Hospital Indemnity, Legal)
• 401(k) plan and contributions dependent on salary level
• Paid holidays plus Paid Time Off