E
Embrace Software IncListed on Himalayas
Senior QA Engineer
Remote · United States
Source: Himalayas. This job was published on Himalayas; you apply there.
RemoteFull-timeSeniorDeveloper
Apply on Himalayas ↗
Listed on Himalayas. You apply through their page. We never charge applicants.
Estimated pay
Estimate$140K–$185K/yr
Estimate — the employer hasn't published pay.
Based on 32 similar senior QA Engineer jobs in United States that show pay. The range is the middle half of their posted pay (25th to 75th percentile), not an offer — ask the employer for the real range.
QA & Testing salaries →Prepare for this job
Free on every jobLikely interview questions
At a glance
- Experience asked
- 3+ years
- Education
- No degree required
- Benefits mentioned
- Health insurance
Skills mentioned
llmci/cdsecurityagilesaastypescriptjavascriptpython
About the role
This is a remote position.
Embrace Healthcare Group powers some of the most innovative healthcare technology solutions in North America, with a focus on enhancing clinical efficiency and compliance across emergency care settings.
EPOWERdoc, a leading SaaS platform within the Embrace Healthcare Group portfolio, specializes in Emergency Department Information Systems (EDIS) tailored for hospitals of all sizes. Designed by clinicians for clinicians, EPOWERdoc streamlines emergency room documentation with intuitive workflows, customizable templates, peer-reviewed clinical content, and real-time coding assistance—helping providers reduce documentation time and focus more on patient care.
We’re looking for a Senior QA Engineer to build EPOWERdoc’s next-generation quality engineering practice, AI-native from day one. This is a hands-on builder role: you’ll choose and architect our test automation stack, write much of the code yourself, and use AI coding agents every day to design, generate, maintain, and triage tests.
You’ll also own how we test AI itself. You’ll design the evaluation harnesses and guardrails that show EPOWERdoc’s AI-powered capabilities are accurate, safe, and dependable enough for an emergency department. You’ll partner closely with developers, product managers, designers, and clinical content experts.
Key Responsibilities:
- Lead QA strategy: Own the end-to-end quality strategy for EPOWERdoc, defining test plans, risk-based coverage goals, and quality gates for every release.
- Architect the QA tech stack: Evaluate, select, and document the tooling for UI, API, performance, accessibility, and security testing, recording each decision and its trade-offs in architecture decision records.
- Build automation and expand coverage: Design and maintain scalable test frameworks for web applications, APIs, and integrations. Set standards clear enough that engineers and AI agents can extend them reliably. Use these frameworks to replace manual regression with automated coverage of critical clinical workflows such as triage, documentation, orders, and discharge.
- Performance and reliability: Build load, stress, and soak tests that model real emergency department peak volumes, establish performance baselines, and catch regressions before they reach production.
- Own CI/CD quality: Integrate automated tests into CI/CD pipelines so every pull request and build is validated quickly. Use parallel execution and AI-assisted test impact analysis to keep feedback loops short as the suite grows.
- Defect management: Work with engineering teams to identify, document, prioritise, and resolve defects early in the development lifecycle, and drive root-cause analysis.
- Exploratory testing: Lead exploratory testing of new features and complex clinical user flows, combining human judgement with AI-driven exploration where automation alone adds less value.
- Agile partnership: Bring a quality perspective to sprint planning, refinement, and retrospectives, including estimates, acceptance criteria, and release readiness.
- Quality advocacy: Champion a quality-first culture from code reviews to release management. Report on quality metrics and trends such as escaped defects, flakiness, and test cycle time.
- Security and compliance: Ensure testing practices, test data, and the use of AI tools meet HIPAA and healthcare data security standards, with no real PHI in test environments or AI prompts.
AI-NATIVE QUALITY ENGINEERING
AI is part of how this team works every day. In this role you will:
- AI-assisted test design: Use LLMs to turn user stories, acceptance criteria, and clinical specifications into test scenarios, edge cases, and boundary conditions, then review and sharpen them with sound test-design judgement.
- Agentic test authoring: Use AI coding agents and Playwright’s MCP server and test agents (planner, generator, and healer) to create and maintain automated tests. Hold generated code to the same review and quality bar as production code.
- Self-healing and failure triage: Apply AI to locator healing, flaky-test detection, and automated failure analysis: clustering failures, summarising likely root causes, and routing defects to the right owner.
- Synthetic test data: Make realistic, PHI-free synthetic patient records and clinical scenarios the default for functional, performance, and AI evaluation testing, using tools such as Synthea or LLM-driven generators.
- Testing AI features: Design evaluation frameworks for AI-powered EPOWERdoc capabilities. These include golden datasets, scoring rubrics, LLM-as-judge grading calibrated against human review, and regression gates that run whenever a model, prompt, or knowledge source changes.
- AI safety and guardrails: Test AI features for hallucination, clinical and coding accuracy, bias, PHI leakage, prompt injection, and graceful failure. Work with clinical experts to define what “good enough for the ED” means in measurable terms.
- Measure and share impact: Track how AI changes test authoring time and maintenance effort, and share what works with other Embrace engineering teams.
THE STACK YOU’LL SHAPE
This is our starting point, not a mandate. You’ll own the final decisions and the reasoning behind them.
- UI and end-to-end: Playwright with TypeScript, including visual regression and automated accessibility checks (axe-core).
- API and integration: Playwright API testing, contract testing (for example Pact), and validation of HL7 and FHIR interfaces.
- Performance: Locust or k6 for load, stress, and soak testing.
- Security: OWASP ZAP and dependency scanning in CI, plus the OWASP Top 10 for LLM Applications as the baseline for AI features.
- AI tooling: AI coding agents (such as Claude Code, GitHub Copilot, or Cursor), Playwright MCP and test agents, and LLM evaluation frameworks such as Promptfoo, DeepEval, or Ragas, with tracing through Langfuse or LangSmith.
- CI/CD and reporting: Tests running on every pull request, with Allure or a similar dashboard for trends, flakiness, and release readiness.
WHAT SUCCESS LOOKS LIKE
- First 30 days: You understand EPOWERdoc’s clinical workflows, existing test assets, and release process. You’ve proposed the target QA stack and AI-native approach as written architecture decision records.
- By 90 days: Playwright smoke and regression suites cover the most critical clinical workflows and run on every pull request. A performance baseline exists for peak load. AI-assisted test authoring is in daily use with measured results.
- By 6 months: Manual regression effort is a small fraction of today’s. Flaky tests are tracked and trending down. An evaluation harness gates every release of AI-powered features.
Requirements
- Experience: 5+ years of professional QA or software testing experience, including at least 3 years building test automation, ideally in a SaaS or web application environment.
- QA foundations: Solid understanding of QA methodologies, risk-based testing, test design techniques, and software development life cycles.
- Programming: Strong coding skills in TypeScript or JavaScript and working proficiency in Python. You write test code with the same care as production code.
- Test automation: Hands-on experience with modern automation tools, with Playwright strongly preferred (Cypress or Selenium also considered). This includes building scalable frameworks from the ground up, managing test data, keeping suites stable and fast, and a record of well-reasoned tooling decisions.
- AI-native practice: Daily use of AI coding agents in test engineering, with examples you can walk us through, and the judgement to spot when AI output is wrong or shallow.
- Testing AI systems: Understanding of how LLM features fail and how to test non-deterministic outputs with eval datasets, rubric or model-based grading, and pass thresholds.
- Performance testing: Hands-on experience with Locust, k6, JMeter, or similar tools, including designing realistic workload models and interpreting the results.
- API and CI/CD: Solid experience with API testing, CI/CD tools, and Git, and with running automated suites within build pipelines.
- Communication and ownership: Ability to explain quality risks clearly to engineers, product managers, and leadership, with strong analytical skills and a proactive, self-starter attitude.
- Ways of working: Experience working in a highly remote, fast-paced, and agile environment across time zones.
- Education: Bachelor’s degree in Computer Science, Software Engineering, or a related field (or equivalent experience).
Nice to have
- Healthcare experience: Work in healthcare or health tech, especially EHR, EDIS, or other regulated clinical software (for example, ONC Health IT certification testing). This includes familiarity with HL7 v2, FHIR, and HIPAA requirements for test data and environments.
- AI tooling experience: Hands-on experience with LLM evaluation and observability tools, or building MCP servers and custom AI agents for engineering teams.
- Security and accessibility testing: Experience testing web and AI applications, including prompt-injection testing, red-teaming, and WCAG.
Benefits
- Competitive salary aligned with UK working hours, ensuring fair and transparent compensation.
- Global Team Collaboration—work alongside extremely talented professionals from around the world.
- Professional Growth Support with access to training programs designed to enhance your skills and expand your knowledge.
- Clear Career Progression and excellent opportunities for long-term advancement within the organization.
- Comprehensive Health Benefits to support your wellbeing and peace of mind.
- Flexible Remote Work Options that empower you to balance work and life effectively.
Originally posted on Himalayas