Automation Engineer

rockITdata is seeking an Automation Engineer to design and develop automated metadata harvesting pipelines using Python and SQL. The role involves integrating AI/ML capabilities to enable intelligent metadata extraction and implements drift detection systems for ongoing metadata maintenance.

Requirements

• Design and develop automated metadata harvesting architecture
• Build Python/SQL-based data pipelines for metadata extraction from diverse systems
• Integrate AI tools for intelligent metadata classification and tagging
• Implement Human-in-the-Loop (HITL) validation workflows for AI-assisted harvesting
• Deploy harvesting automation across enterprise systems for scalable metadata capture
• Develop drift detection and alerting mechanisms for metadata change management
• Create and maintain technical documentation for all automation components
• Support empirical testing in sandbox environments for technology evaluations
• Optimize harvesting processes to achieve efficiency gains
• Coordinate with QA teams on automation quality standards
• Troubleshoot and resolve pipeline issues across production environments