Unlock the power of data to drive innovation and transform business outcomes as a Senior Fullstack Developer on our Data Engineering team. Collaborate with cross-functional teams to build and manage robust data ecosystems using Python, SQL, AWS, PySpark, and Databricks.
Requirements
- Design, build, and optimize scalable data pipelines using PySpark, Databricks, and SQL on AWS cloud platforms.
- Collaborate with data analysts, data scientists, and business users to understand data requirements and ensure reliable, high-quality data delivery.
- Implement batch and streaming data ingestion frameworks from a variety of sources (structured, semi-structured, and unstructured data).
- Develop reusable, parameterized ETL/ELT components and data ingestion frameworks.
- Perform data transformation, cleansing, validation, and enrichment using Python and PySpark.
- Build and maintain data models, data marts, and logical/physical data structures that support BI, analytics, and AI initiatives.
- Apply best practices in software engineering, version control (Git), code reviews, and agile development processes.
- Ensure data pipelines are well-tested, monitored, and robust with proper logging and alerting mechanisms.
- Optimize performance of distributed data processing workflows and large datasets.
- Leverage AWS services (such as S3, Glue, Lambda, EMR, Redshift, Athena) for data orchestration and lakehouse architecture design.
- Participate in data governance practices and ensure compliance with data privacy, security, and quality standards.
- Contribute to documentation of processes, workflows, metadata, and lineage using tools such as Data Catalogs or Collibra (if applicable).
- Drive continuous improvement in engineering practices, tools, and automation to increase productivity and delivery quality.
Benefits
- Comprehensive benefits package
- Competitive salary
- Opportunities for career growth and professional development
- Collaborative and dynamic work environment
- Hybrid work schedule with flexibility