Genpact is seeking a Lead Consultant, Databricks, Pyspark, AWS to build and deploy cutting-edge AI solutions in a fast-paced, innovation-driven environment. The successful candidate will have experience in Data Engineering, strong analytical and technical problem-solving skills, and expertise in Databricks, Python, and Pyspark.
Requirements
- Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.
- Work with architect and lead engineers for solutions to meet functional and non-functional requirements.
- Demonstrated knowledge of relevant industry trends and standards.
- Demonstrate strong analytical and technical problem-solving skills.
- Must have experience in Data Engineering domain.
- Must have implemented at least 4 project end-to-end in Databricks.
- Must have at least experience on databricks which consists of various components as below
- Must have skills: Azure data factory, Azure data bricks, Python and Pyspark
- Expert with database technologies and ETL tools.
- Hands-on experience on designing and developing scripts for custom ETL processes and automation in Azure data factory, Azure databricks, Python, Pyspark etc.
- Good knowledge of AZURE, AWS, GCP Cloud platform services stack
- Hands-on experience on designing and developing scripts for custom ETL processes and automation in Azure data factory, Azure databricks,Delta lake, Databricks workflows orchestration, Python, Pyspark etc.
- Good Knowledge on Unity Catalog implementation.
- Good Knowledge on integration with other tools like – DBT, other transformation tools.
- Good knowledge on Unity Catalog integration with Snowlflake
- Must be well versed with Databricks Lakehouse concept and its implementation in enterprise environments.
- Must have good understanding to create complex data pipeline
- Must have good knowledge of Data structure & algorithms.
- Must be strong in SQL and sprak-sql.
- Must have strong performance optimization skills to improve efficiency and reduce cost.
- Must have worked on both Batch and streaming data pipeline.
- Must have extensive knowledge of Spark and Hive data processing framework.
- Must have worked on any cloud (Azure, AWS, GCP) and most common services like ADLS/S3, ADF/Lambda, CosmosDB/DynamoDB, ASB/SQS, Cloud databases.
- Must be strong in writing unit test case and integration test
- Must have strong communication skills and have worked on the team of size 5 plus
- Must have great attitude towards learning new skills and upskilling the existing skills.
Benefits
- Lead AI-first transformation – Build and scale AI solutions that redefine industries
- Make an impact – Drive change for global enterprises and solve business challenges that matter
- Accelerate your career—Gain hands-on experience, world-class training, mentorship, and AI certifications to advance your skills
- Grow with the best – Learn from top engineers, data scientists, and AI experts in a dynamic, fast-moving workplace
- Committed to ethical AI – Work in an environment where governance, transparency, and security are at the core of everything we build
- Thrive in a values-driven culture – Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress
