Listed on Himalayas. You apply through their page. We never charge applicants.
Pay
$200K–$300K/yr
Published in the job listing
Prepare for this job
Free on every jobLikely interview questions
At a glance
- Experience asked
- 3+ years
- Education
- Master's
Skills mentioned
About the role
About the Role
Join a small, fast-moving team building autonomous AI capabilities for executive productivity. You will develop agent capabilities and evaluation systems that help ensure important tasks are completed reliably, and that improvements are proven before reaching users.
What You'll Do
Build agents that complete delegated work end to end, with minimal need for executive oversight.
Define and measure agent quality so the team can confidently assess how often tasks are completed correctly.
Develop new agent capabilities for the tools executives already use, and evaluate readiness before launch.
Investigate production failures, identify underlying causes, and ship fixes that prevent repeat issues.
Build detection systems to surface known failure patterns and unexpected issues at scale.
Improve agent recovery when tools or websites fail.
Create repeatable evaluation workflows that let engineers measure the impact of changes.
What We're Looking For
At least 3 years in data science or machine learning roles building or delivering evaluation systems, metrics, or quality measurement for products or systems.
Production-quality Python and SQL skills for analysis, large datasets, and automated pipelines.
Experience designing evaluation frameworks, including measurable success criteria, representative datasets, appropriate metrics, and reliable benchmarks.
A track record of relevant impact, sustained experience in a role with demonstrated progression, and a degree from a strong engineering program.
Experience with applied AI or machine learning systems and a collaborative, iterative approach to solving ambiguous problems.
Relevant tools and technologies include Python, SQL, Django, React, TypeScript, harness engineering, Braintrust, and Raindrop.
Compensation & Benefits
Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.
Location
On-site in Palo Alto, California. The team works in the office four days per week.
Originally posted on Himalayas