Data Engineer

Data Engineer responsible for designing and building data transformation and conversion pipelines, processing large volumes of financial information on AWS, and collaborating with BA to understand accounting business rules.

Requirements

• Designing and building data transformation and conversion pipelines
• Processing, transforming, and loading large volumes of financial information
• Developing and maintaining ETL/ELT pipelines using Apache Spark and AWS Glue
• Implementing data transformation and conversion logic
• Managing data catalogs in AWS Glue Data Catalog
• Optimizing queries on S3/Redshift/RDS
• Ensuring data quality, traceability, and lineage
• Developing Python scripts for automation, data cleaning, and validation
• Collaborating with BA to understand accounting business rules
• Monitoring and optimizing Spark job performance
• Documenting data flows, models, and data dictionaries