Scala/Spark Developer - Jersey City
PhotonJersey City, New Jersey, United States
Posted February 11, 2025
48000 - 169000 USD year
Full Time
About the Company
Photon is a global leader in AI and digital solutions, accelerating AI adoption and enabling Fortune 100 clients to stay agile and future-ready. With expertise in IoT, API management, big data analytics, and more, Photon combines AI management, digital innovation, and engineering excellence to drive lasting transformation for F500 clients across billions of touchpoints daily. Employing thousands worldwide, Photon's comprehensive suite of services positions them as a distinctive player in the converging digital and AI landscape.
Job Description
This Scala/Spark Developer role focuses on building and optimizing data processing applications using Apache Spark and Scala, particularly within the Cloudera Hadoop ecosystem. The position involves collaborating with stakeholders to understand data requirements, designing and implementing data pipelines, and ensuring data integrity. Experience with Hadoop and related technologies is required.
Requirements
- Minimum 8 years of experience in application development
- Strong hands-on experience in Apache Spark, Scala, and Spark SQL for distributed data processing
- Hands-on experience with Cloudera Hadoop (CDH) components such as HDFS, Hive, Impala, HBase, Kafka, and Sqoop
- Familiarity with other Big Data technologies, including Apache Kafka, Flume, Oozie, and Nifi
- Experience building and optimizing ETL pipelines using Spark
- Experience with SQL and NoSQL databases such as HBase, Hive, and PostgreSQL
- Knowledge of data warehousing concepts, dimensional modeling, and data lakes
- Ability to troubleshoot and optimize Spark and Cloudera platform performance
- Familiarity with version control tools like Git and CI/CD tools (e.g., Jenkins, GitLab)
Benefits
- Medical, vision, and dental benefits
- 401k retirement plan
- Variable pay/incentives
- Paid time off
- Paid holidays