This role has a specialized focus on building and maintaining robust, scalable, and automated data pipelines and plays a key role in optimizing our data infrastructure and enabling efficient data delivery across the organization.
Requirements
- Design, build, and maintain scalable and resilient CI/CD pipelines for data applications and infrastructure, with a focus on Snowflake, dbt, and related data tools.
- Implement and manage Snowflake dbt projects for data transformation, including developing dbt models, tests, and documentation, and integrating dbt into CI/CD workflows.
- Develop and manage infrastructure as code (IaC) using Terraform to provision and configure cloud resources for data storage, processing, and analytics on GCP.
- Automate the deployment, monitoring, and management of Snowflake data warehouse environments, ensuring optimal performance, security, and cost-effectiveness.
- Collaborate with data engineers and data scientists to understand their requirements and provide robust, automated solutions for data ingestion, processing, and delivery.
- Implement and manage monitoring, logging, and alerting systems for data pipelines and infrastructure to ensure high availability and proactive issue resolution.
- Develop and maintain robust automation scripts and tools, primarily using Python, to streamline operational tasks, manage data pipelines, and improve efficiency; Bash scripting for system-level tasks is also required.
- Ensure security best practices are implemented and maintained across the data infrastructure and pipelines.
- Troubleshoot and resolve issues related to data infrastructure, pipelines, and deployments in a timely manner.
- Participate in code reviews for infrastructure code, dbt models, and automation scripts.
- Document system architectures, configurations, and operational procedures.
- Stay current with emerging DevOps technologies, data engineering tools, and cloud best practices, particularly related to Snowflake, dbt, and Terraform.
- Optimize data pipelines for performance, scalability, and cost.
- Support and contribute to data governance and data quality initiatives from an operational perspective.
- Help implement AI features
Benefits
- Fully remote
- Flexible timings
- Market competitive compensation
- Insane learning and growth
