We are seeking a hands-on Data Engineer with strong Databricks and PySpark experience to build scalable data pipelines and analytics applications within a modern data platform.
Requirements
- Design, develop, and maintain scalable data pipelines using PySpark on Databricks
- Build analytics data models and transformation workflows for enterprise reporting and analytics
- Migrate legacy ETL workloads from platforms such as Informatica and Teradata to Databricks
- Develop Databricks-native dashboards and analytics applications to replace traditional BI tools
- Build lightweight Python-based data applications (e.g., FastAPI) to expose and interact with data
- Integrate Databricks pipelines with APIs and application services
- Implement Slowly Changing Dimensions (SCD) and dimensional data modeling techniques
- Develop reusable data engineering frameworks and standardized pipelines
- Optimize Spark workloads for performance, scalability, and cost efficiency
- Collaborate with analytics and business teams to deliver user-facing data solutions
- Leverage AI-assisted coding tools (e.g., Copilot, ChatGPT) to improve development productivity
- Contribute to best practices for modern data engineering and analytics application development
Benefits
- 401k Matching
- Retirement Plan
- Generous Paid Time Off
- Relocation Assistance