Software Engineer III - Data Engineer

opriate systems

• Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development

• Recommend improvements and modifications on new and existing data and ETL pipelines.

• Create data analytics for data scientists to innovate, build and optimize our ecosystem

• Assemble large, complex data sets that meet functional and non-functional business requirements

• Be responsible for ingesting data into our data lake and providing frameworks and services for operating on that data including the use of Spark

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 3+ years applied experience

• Hands-on practical experience in system design, application development, testing, and operational stability

• Proficient in coding in one or more languages

• Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages and database querying languages

• Overall knowledge of the Software Development Life Cycle

• Analyze, debug and correct issues with data pipelines

• Operate on or build solution required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, AWS and Spark technologies.

• Collaborate with product teams to resolve data quality issues promptly.

Preferred qualifications, capabilities, and skills

• Experience implementing Data pipelines preferably in connection with Hadoop or Spark with Java or Python
• Exceptional coding and design skills in Java or Python and PYSPARK
• Experience with API calls for data extraction
• Good knowledge of using SQL and analytical queries
• Experience in JAVA,PYSARK and AWS Implementations