Lead Software Engineer

We are seeking a highly skilled Lead Data Engineer to join our R&D Data Engineering Team. In this role, you will be a key player in shaping the architecture and technical direction of our data platform, ensuring that it meets the evolving needs of the business while adhering to best practices and industry standards.

Requirements

• Bachelor’s Degree or equivalent
• At least 7 years of Relevant Experience
• At least 5+ Years in Software Development: Demonstrated experience in software development, with a focus on Big Data technologies.
• At least 3+ Years in Distributed Data Processing: Proven experience in building scalable distributed data processing solutions.
• At least 3+ Years in Database Design: Expertise in database design and development, with a strong focus on data model design.
• Strong Proficiency with Apache Spark and Airflow: Extensive hands-on experience with these technologies, leveraging them for data processing and orchestration.
• Python Proficiency: Advanced proficiency in Python for data processing and building services.
• Experience with Databricks and Snowflake: Practical experience with these platforms, including their use in cloud-based data pipelines.
• Familiarity with Delta Lake or Apache Iceberg: Experience working with these data storage to decouple storage from processing engines.
• Cloud-Based Solutions Expertise: Proven experience in designing and implementing cloud-based data pipelines, with specific expertise in AWS services such as S3, RDS, EMR, and AWS Glue.
• CI/CD Best Practices: Strong understanding and application of CI/CD principles