I am a Lead Talent Acquisition Specialist at STI (Software Technology Inc) and currently looking for a Data Engineer. The job is a long-term contract, and the ideal candidate will have experience working in Azure Databricks, Apache Spark, and developing and deploying data pipelines for streaming and batch data sources.
Requirements
- Experience working in Azure Databricks, Apache Spark
- Proficient programming in Scala/Python/Java
- Experience developing and deploying data pipelines for streaming and batch data sources
- Experience creating data models and implementing business logic using tools and languages listed
- Working knowledge in Kafka, Structured Streaming, DataFrame API, SQL, NoSQL Database
- Comfortable with API, Azure Datalake, Git, Notebooks, Spark Cluster, Spark Jobs, Performance tuning
- Excellent communication skills
- Familiarity with Power BI, Delta Lake, Lambda Architecture, Azure Data Factory, Azure Synapse a plus
- Telecom domain experience not necessary but really helpful