We are seeking a Data Engineer to contribute to our next level of growth and expansion, with a focus on building enterprise-grade data pipelines, unified Analytics ID, and scalable feature store.
Requirements
- Design and build production-grade data pipelines in Databricks using Spark/PySpark and SQL
- Develop and maintain an Analytics ID stitching pipeline using deterministic and probabilistic matching techniques
- Build and manage modular data marts (Identity, Behavior, Demographics) with independent refresh cadences
- Implement and maintain a scalable feature store supporting downstream analytics and data science use cases
- Own the end-to-end data lifecycle: ingestion, transformation, validation, deployment, monitoring, and optimization
- Develop data quality frameworks including schema drift detection, anomaly monitoring, match-rate validation, and automated deduplication audits
- Implement CI/CD processes for multi-environment promotion (dev/staging/prod) in Databricks environments
- Coordinate orchestration workflows and manage dependencies using Databricks Workflows or similar tools
- Collaborate closely with Data Architects and Client stakeholders to translate business rules into scalable technical solutions
- Produce comprehensive technical documentation including data contracts, lineage maps, architecture diagrams, and operational runbooks
Benefits
- Learning Opportunities
- Mentoring and Development
- Travel opportunities to attend industry conferences and meet clients
- Flexible working options
- Company-provided equipment
- Special day rewards to celebrate birthdays, work anniversaries, and other personal milestones