Lead Data Engineer

Role Overview

The Lead Data Engineer will be a key operational and hands-on engineer delivering Business tenancies for SwapClear and Listed Rates as part of the data platform strategy execution across all areas of the business. The role will involve designing and implementing modern, scalable data solutions using a range of new and emerging technologies contributing to innovation in the data space.

What You Will Do

Develop and support the AWS & Snowflake based cloud data platform, help support and migrate data from legacy platforms, and play a key part in the roll-out of a new cloud-based analytics platform.

Why It Might Be a Fit

The role would suit an ambitious individual keen to develop their skills and knowledge, with a strong background in data engineering and experience working with Agile Delivery teams.

Requirements

• Design and development of automated data pipelines and data structures
• 10+ years of working with Python Programming
• 3+ years hands on data engineering experience for data warehouse solutions using Snowflake, SQL, dbt and Airflow
• 5+ years hand on experience building Event stream processing systems on Kafka or similar technology and experience in building event based services
• 2+ years hands on knowledge of building CI/CD pipelines using GitLab
• 5+ years working with Agile Delivery teams (Scrum, Kanban) and JIRA
• Ability to build automated testing to test data pipelines
• Support transition from on-premises solutions to cloud first strategy (AWS &Snowflake)
• Up-to-date knowledge of technology standards, industry trends, emerging technologies and software development best practices
• Demonstrable experience of ITIL processes within a 2nd/3rd line support environment

Benefits

• healthcare
• retirement planning
• paid volunteering days
• wellbeing initiatives