Senior Data Engineer - Databricks

MediaRadar, an industry leader in marketing intelligence, is seeking a Senior Data Engineer to build and maintain scalable data solutions using Azure Databricks, Apache Spark, and modern data lakehouse architectures.

Requirements

• Design, development, and maintenance of scalable ETL/ELT pipelines on Azure Databricks using Apache Spark (PySpark/Spark SQL)
• Design and implement both batch and real-time data ingestion and transformation processes
• Build and manage Delta Lake tables, schemas, and data models to support efficient querying and analytics
• Write optimized SQL queries for large datasets using Spark SQL and PostgreSQL
• Develop, schedule, and monitor workflows using Databricks Workflows, Airflow or similar orchestration tools
• Design, build, and deploy cloud-native, containerized applications on Azure Kubernetes Service (AKS) and integrate with Azure services
• Ensure data quality, governance, and compliance through validation, documentation, and secure practices
• Collaborate with data analysts, data architects, and business stakeholders to translate requirements into technical solutions
• Contribute to and enforce best practices in data engineering, including version control (Git), CI/CD pipelines, and coding standards
• Continuously enhance data systems for improved performance, reliability, and scalability
• Mentor junior engineers and help evolve team practices and documentation
• Stay up to date on emerging trends, technologies, and best practices in the data engineering space
• Work effectively within an agile, cross-functional project team

Benefits

• Generous Paid Time Off
• 401k Matching
• Retirement Plan
• Visa Sponsorship
• Four Day Work Week
• Generous Parental Leave
• Tuition Reimbursement
• Relocation Assistance