Sr Developer - Data Engineering - Mississauga - Canada
PhotonMississauga, Ontario, Canada
Posted February 6, 2025
Full Time
About the Company
Photon is a global leader in AI and digital solutions, accelerating AI adoption and enabling Fortune 100 clients to stay agile and future-ready. With expertise in IoT, API management, big data analytics, and more, Photon combines AI management, digital innovation, and engineering excellence to drive lasting transformation for F500 clients across billions of touchpoints daily. Employing thousands worldwide, Photon's comprehensive suite of services positions them as a distinctive player in the converging digital and AI landscape.
Job Description
This role involves developing, testing, and deploying data processing applications using Spark and Scala. The position requires close collaboration with data scientists and business analysts to build and optimize data solutions. They will also be responsible for ensuring data integrity and performance across all processing tasks.
Requirements
- Minimum 8 years of experience in application development
- Strong hands-on experience in Apache Spark, Scala, and Spark SQL
- Hands-on experience with Cloudera Hadoop components such as HDFS, Hive, Impala, HBase, Kafka, and Sqoop
- Familiarity with other Big Data technologies including Apache Kafka, Flume, Oozie, and Nifi
- Experience building and optimizing ETL pipelines using Spark
- Experience building and optimizing data warehousing concepts