We are seeking a highly skilled and detail-oriented Data Engineering Test Automation Engineer to ensure the quality, reliability, and performance of our data pipelines and platforms.
Requirements
- Design, develop, and maintain automated test scripts for data pipelines, ETL jobs, and data integrations.
- Validate data accuracy, completeness, transformations, and integrity across multiple systems.
- Collaborate with data engineers to define test cases and establish data quality metrics.
- Develop reusable test automation frameworks and CI/CD integrations.
- Perform performance and load testing for data systems.
- Maintain test data management and data mocking strategies.
- Identify and track data quality issues, ensuring timely resolution.
- Perform root cause analysis and drive corrective actions.
- Contribute to QA ceremonies and drive continuous improvement in QA processes and culture.
- Build and automate end-to-end data pipeline validations across ingestion, transformation, and consumption layers using Databricks, Apache Spark, and AWS services.