We are seeking a highly skilled Senior Data Engineer with strong expertise in PySpark and Python to join our team for a leading banking client in Dubai. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and data architectures to support advanced analytics and business intelligence initiatives.
Requirements
- Design, develop, and maintain scalable data pipelines using PySpark and Python
- Work with large-scale distributed systems (Hadoop/Spark ecosystem)
- Build and optimize ETL/ELT workflows for structured and unstructured data
- Collaborate with data scientists, analysts, and stakeholders to deliver data solutions
- Ensure data quality, integrity, and governance across platforms
- Implement performance tuning and optimization for Spark jobs
- Work with cloud-based data platforms (AWS / Azure / GCP) if applicable
- Develop and maintain data models, schemas, and metadata management
- Troubleshoot production issues and provide timely resolutions