Data Engineer, Platform

Basis is a nonprofit applied AI research organization seeking a Data Engineer to build trustworthy data pipelines with comprehensive provenance and quality gates. The ideal candidate has experience with ML data pipelines, understands the full lifecycle from raw data through model training and evaluation, and brings rigor to data provenance, lineage tracking, and quality assurance.

Requirements

• Have demonstrated significant achievements in data engineering for ML/AI systems.
• Possess strong proficiency in data technologies including SQL, Python for data processing, distributed computing frameworks, and workflow orchestration tools.
• Have experience with cloud data platforms, including data warehouses, data lakes, object storage, and streaming systems.
• Understand ML data requirements, including feature engineering, training/validation/test splits, data versioning, and experiment reproducibility.
• Be skilled at data quality and governance, including implementing validation frameworks, anomaly detection, data lineage tracking, and metadata management.

Benefits

• Competitive salary
• Exceptional candidates may be eligible for additional benefits