Lead Software Engineer - AWS, Data Engineering, Python/Java, Spark

and debugs code written by others

• Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems

• Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture

• Leads communities of practice across Software Engineering to drive awareness and use of new and leading-edge technologies

• Adds to team culture of diversity, opportunity, inclusion, and respect

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years of applied experience.

• Design and implementation on Operational Database or Data Warehouse and experience in integrating various data sources.

• Experience in Data Encryption, aware of various Authentication methods, Resiliency planning etc.

• Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages and database querying language.

• Hands-on practical experience in developing spark-based Frameworks for end-to-end ETL, ELT & reporting solutions using key components like Spark SQL & Spark Streaming.

• Proficient in coding in one or more Coding languages - Java, Scala, Python,

• Experience with Relational and No SQL databases,

• Cloud implementation experience with AWS including AWS Data Services: Proficiency in Lake formation, Glue ETL (or) EMR, S3, Glue Catalog, Athena, Kinesis (or) MSK, Airflow (or) Lambda + Step Functions + Event Bridge

• Experience in Data De/Serialization: Expertise in at least 2 of the formats: Parquet, Iceberg, AVRO, JSON-LD

• Experience in AWS Data Security: Good Understanding of security concepts such as: Lake formation, IAM, Service roles, Encryption, KMS, Secrets Manager

• Experience working of the financial services industry and their IT systems. Practical cloud native experience AWS.

Preferred qualifications, capabilities, and skills

• Experience in Snowflake nice to have.
• Experience in Data Steaming nice to have