Software Mind develops solutions that make an impact for companies around the globe. We're seeking a Senior Data Engineer with proven expertise in Databricks PySpark development and comprehensive data modeling experience to join our Data Engineering Team and drive the evolution of our Azure-based Data Analytics Platform.
Requirements
- Databricks expertise: proficient in Databricks Lakehouse architecture and PySpark development
- Data modeling mastery: extensive experience in data model definition including fact tables, dimension tables, measures, grain analysis, SCD Type 2, surrogate keys, late-arriving records handling, overlap joins, and incremental loads with idempotency
- Programming: advanced Python and PySpark skills for ETL/ELT development
- Databricks optimization: deep knowledge of optimize, zOrder, Liquid clustering, ACID transactions and performance tuning
- Event-based architecture: proven experience in designing and implementing event-driven data solutions
- Azure data platform: experience working with Azure-based datasets and data pipelines
- SQL proficiency: strong SQL skills for complex data transformations
- Large-scale data processing: experienced in handling large and complex datasets efficiently
- CI/CD: experience in developing automated deployment pipelines
- Networking fundamentals: understanding of basic networking concepts
- Agile methodology: familiar with Scrum and agile development practices
- Understanding of stream processing challenges and Spark Structured Streaming
- Experience with Infrastructure as Code (Terraform, Bicep)
- Experience with containerized applications (Azure Container Apps, Kubernetes)
- Knowledge of Azure cloud native solutions (Azure Data Factory, Azure Function App, Azure Container Instances)
Benefits
- Flexible employment and remote work
- International projects with leading global clients
- International business trips
- Non-corporate atmosphere
- Language classes
- Internal & external training
- Private healthcare and insurance
- Multisport card
- Well-being initiatives