Data Engineer

The Data Engineer - Advanced is responsible for designing, implementing, and maintaining complex data pipelines and architectures. The role involves working with large-scale data platforms, ensuring data quality, and optimizing data flow and collection for cross-functional teams.

Requirements

• Develops, maintains and documents physical data models, data dictionaries and data flow diagrams
• Leads and participates in gathering complex and often unique business requirements by defining the business problem and data requirements
• Reinforces the value and use of data to business subject matter experts
• Stay up-to-date with the latest technologies and trends in data engineering and apply them to improve existing data pipelines
• Optimize data delivery and automate manual processes to improve efficiency and reliability
• Assists data scientists in integrating diverse datasets for machine learning, and deep learning models
• Integrates standards for platforms and frameworks for source code management, work item and project tracking, DevOps and enterprise architecture
• Build large-scale batch and / or real-time data pipelines with data processing frameworks in cloud platforms and / or on-premises
• Manages and secure the flow of structured and unstructured data from multiple sources
• Lead and mentor Data Engineers - Associate, providing guidance and support in their professional development
• Design and implement solutions for real-time data processing and streaming