Applied AI Researcher, Benchmarking
Distyl AISan Francisco, California, United States • New York, New York, United States
Posted October 16, 2025
Full Time
About the Company
Distyl AI is dedicated to revolutionizing enterprises' success in the AI-assisted economy by providing customer-centric solutions tailored to unique data, workflows, and employee needs. Leveraging proprietary tools and strategic alliances like OpenAI, Distyl develops high-integrity, reliable generative AI products that enhance institutional AI readiness globally.
Job Description
Distyl AI develops AI-native technologies for businesses, partnering with Fortune 1000 companies. They are seeking Applied AI Researchers to define evaluation frameworks for evaluating complex systems and shaping industry standards. The company values creative research and collaboration within an AI-native mindset.
Requirements
- Experience designing and running evaluations for models/systems
- Statistical and analytical rigor in experiment design
- Expertise in compound AI systems, agentic collaboration, and techniques like ensemble methods
- Proven track record of research results (publications, Twitter posts, etc.)
- Proficiency in AI tools like ChatGPT, Cursor, and Perplexity
- Strong programming and data analysis skills
- Understanding of biases towards showing vs telling
- Biases towards showing vs telling
Benefits
- Competitive salary and benefits package
- Equity options
- Medical/dental/vision coverage
- 401K plan
- Commuter benefits
- Lunch provided in office
