Senior Data Engineer – Real-Time & Distributed Systems (GCP)
Innodata Inc.Remote job
Posted March 14, 2025
Full Time
About the Company
Innodata is a leading data engineering company providing AI technology solutions to major technology companies and leading companies across financial services, insurance, technology, law, and medicine.
Job Description
We are a leading data engineering company seeking a Senior Data Engineer to design, build, and optimize scalable data pipelines for batch and real-time processing. The ideal candidate will have advanced proficiency in Python, experience with big data pipelines, and hands-on expertise with workflow orchestration tools.
Requirements
- Advanced proficiency in Python for backend and large-scale data processing
- Strong experience building and managing big data pipelines in production environments
- Hands-on expertise with workflow orchestration tools such as Airflow or Google Cloud Composer
- Proven experience in batch and streaming data processing using: Apache Spark, Apache Beam (Dataflow)
- Experience designing and operating event-driven systems using Pub/Sub
- Strong understanding of distributed systems architecture and scalability patterns
- Experience managing globally distributed, low-latency datasets
- Hands-on experience with NoSQL databases and/or Google Cloud Spanner
- Strong knowledge of system reliability, fault tolerance, and performance optimization
