Boomi is looking for a Senior Software Quality Engineer - AI to bridge traditional software quality engineering with cutting-edge AI quality practices. As a Senior Software Quality Engineer, you will design and implement comprehensive testing strategies that ensure the reliability of backend infrastructure and the quality of Generative AI features.
Requirements
- Design and execute comprehensive test plans for backend services, APIs, and microservices architectures.
- Partner with Backend Engineers, AI Engineers, and Product teams to understand requirements and identify quality risks early in the development cycle.
- Develop automated testing frameworks using tools like Pytest, Playwright, unittest, and integration testing libraries.
- Build end-to-end test suites that validate the integration between backend systems, AI agents, databases & services.
- Advocate for quality-first practices, influencing architectural decisions and embedding testing into the development lifecycle.
- Design and implement automated evaluation frameworks for Generative AI features, including LLM / SLM model testing, prompt/ output validation, and behavioral assessment of agentic workflows.
- Develop quality metrics and evaluation methodologies for LLM-based applications, assessing accuracy, consistency, reliability, quality of AI/ML models.
- Create and maintain curated evaluation datasets and synthetic test data that cover edge cases, adversarial scenarios, and real-world variability.
- Leverage AI evaluation tools and frameworks & establish observability and monitoring requirements to quality assessment.
- Establish observability and monitoring requirements including structured logging, metrics collection, and tracing for both backend services and AI agent behavior.
- Implement quality gates and acceptance criteria and create test documentation including test plans, automation architecture, quality reports, and runbooks.
- Build strategies for testing in cloud environments & containerized execution.
- Implement performance testing, load testing, and reliability testing for production backend services and AI inference pipelines.
- Define and champion quality standards, best practices, and testing methodologies for both traditional backend systems and AI applications.
- Conduct code reviews with a focus on testability, quality patterns, and maintainability.
- Mentor junior quality engineers, sharing expertise in automation frameworks, AI testing approaches, and quality engineering principles.
- Communicate quality insights, risk assessments, and test results effectively to technical and non-technical stakeholders using tools like Jira and Confluence.
Benefits
- Generous Paid Time Off
- 401k Matching
- Retirement Plan
- Visa Sponsorship
- Four Day Work Week
- Generous Parental Leave
- Tuition Reimbursement
- Relocation Assistance
