Data Engineer position at BT Group, responsible for designing, developing, and maintaining end-to-end data pipelines using Ab Initio, Apache Spark, and Apache NiFi. Opportunity to work on exciting data products and explore new technologies.
Requirements
- Proficient in building Big Data solutions using Hadoop or GCP or AWS.
- Skilled in technologies such as Scala, Java, Python, Spark (PySpark/Scala), Kafka, Kinesis, BigQuery, Dataflow, BigTable, Terraform, and SQL.
- Experience with Terraform, Kafka, Kinesis, SQL are nice to have.
- Strong expertise in Ab Initio (GDE, PDL, Conduct>IT, Express>IT, EME) with a focus on parallel processing and component optimisation.
- Hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and performance tuning.
- Practical experience with Apache NiFi for flow design, processor tuning, error handling, and security configurations.
- Good understanding of Infrastructure as Code toolsets.
- Experienced in implementing Continuous Integration and Deployment strategies.
- Proficient in using Atlassian tools such as Confluence and JIRA.
- Experienced in version control using Git or SVN.
- Able to clearly present technical issues, progress, and outcomes to team members and Product Owners.
- Demonstrates enthusiasm for learning and applying new technologies.
- Experience or interest in managing and developing team members.
- Capable of building new solutions as well as supporting existing data products and platforms.
Benefits
- 10% on target annual bonus
- Access to an online private GP 24/7 for you and your immediate family
- Market-leading paid carers leave with up to 2 weeks off
- Equalized maternity, paternity, and adoption leave – 18 weeks’ full pay and 8 weeks’ half pay
- Discounted EE and BT products, including mobile and broadband
- Market leading Pension scheme – 5% from you and 10% from us
- Holiday purchase scheme