Senior Data Engineer Pyspark with Data modelling
VirtusaDubai, United Arab Emirates
Posted March 19, 2026
Full Time
About the Company
Virtusa is a global product and platform engineering firm headquartered in Massachusetts, delivering end‑to‑end digital transformation services that accelerate growth, profitability, and sustainability for enterprises worldwide. Leveraging a domain‑driven approach, Virtusa blends strategy, design, and engineering expertise to create industry‑specific solutions in areas such as AI, cloud, data analytics, and customer experience. The company serves a broad portfolio of sectors—including financial services, healthcare, media, travel, manufacturing, and technology—through a network of global delivery centers. Virtusa’s distinctive value lies in its ability to combine deep industry knowledge with cutting‑edge technology to deliver measurable business impact and continuous innovation.
Job Description
We are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team.
Requirements
- Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy.
- Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
- Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
- Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
- Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
- Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
- Monitor pipeline performance, troubleshoot issue.
