Generative AI Engineer

Accountabilities

• Design, develop, test, deploy, and maintain generative AI solutions that ground large language model responses in authoritative healthcare program documentation.

• Build and maintain document ingestion pipelines covering parsing, chunking, metadata tagging, embedding generation, and vector or semantic search indexing.

• Translate user needs and stakeholder requirements into effective prompts, context strategies, system requirements, and evaluation criteria in collaboration with cross-functional teams and subject matter experts.

• Implement AI responses that include source citations and direct links to underlying documentation, ensuring that users can trace answers back to authoritative materials.

• Design and implement responsible AI guardrails, including content filtering, scope restrictions, PII and PHI protections, and appropriate refusal handling for out-of-scope questions.

• Develop evaluation frameworks and test datasets to measure retrieval quality, answer accuracy, groundedness, consistency, and overall system performance.

• Continuously monitor, evaluate, and tune AI solutions to improve reliability while balancing accuracy, latency, cost, security, and specific use-case requirements.

• Design AI applications capable of supporting multiple foundation models and providers and evaluate model performance against business and technical requirements.

• Deploy and operate solutions using approved AWS services and within applicable federal security controls, contributing to documentation required for security authorization and AI governance reviews.

• Implement logging, monitoring, and audit trails that support transparency, operational oversight, cost management, and responsible AI governance.

• Write clean, reusable, well-documented code and contribute to code reviews, automated testing, deployment processes, and ongoing system maintenance.

• Collaborate with technical and non-technical stakeholders to communicate AI capabilities, limitations, risks, and performance clearly.

• Stay current with emerging generative AI technologies, foundation models, development frameworks, security practices, and federal AI guidance.

Requirements:

• You have a master’s degree in computer science, data science, engineering, or another relevant technical field.

• You have at least 5 years of experience developing software and cloud-based solutions, including recent hands-on experience designing and deploying LLM or generative AI applications.

• You have strong proficiency in Python and experience working with modern LLM APIs, SDKs, orchestration frameworks, and AI/ML libraries such as Hugging Face, LangChain, LlamaIndex, or equivalent technologies.

• You have hands-on experience designing and deploying generative AI systems involving embeddings, vector databases, and hybrid or semantic search.

• You have experience with AWS AI and data services such as Amazon Bedrock, Amazon OpenSearch, Amazon Kendra, SageMaker, Lambda, or S3.

• You have practical experience with prompt engineering, context engineering, LLM evaluation, and techniques for reducing hallucinations and ensuring responses remain grounded in source material.

• You understand how to evaluate and select foundation models based on factors including accuracy, latency, cost, security, and specific application requirements.

• You have experience working within Agile software development environments and understand modern software engineering and delivery practices.

• You have experience building and maintaining CI/CD pipelines and deploying infrastructure using tools such as GitHub Actions, CloudFormation, Terraform, and/or Jenkins.

• You have strong problem-solving, communication, and collaboration skills, including the ability to explain complex AI capabilities and limitations to non-technical stakeholders.

• You can work effectively both independently and as part of a multidisciplinary team, with strong attention to detail and a professional approach to documentation and presentation.

• Experience supporting federal health programs, including program policy, regulatory guidance, or customer and stakeholder support operations, is highly desirable.

• Experience deploying AI solutions in federal environments, including FedRAMP-authorized services, ATO or security authorization processes, and NIST 800-53 controls, is preferred.

• Familiarity with responsible AI and federal AI governance frameworks, including the NIST AI Risk Management Framework and OMB AI guidance, is advantageous.

• Knowledge of healthcare security and compliance requirements such as HIPAA and HITECH is preferred.

• Experience building conversational or search interfaces, including familiarity with Section 508 accessibility requirements, is a plus.

• Relevant AWS certification, such as AWS Certified Machine Learning Engineer – Associate, AWS Certified AI Practitioner, Generative AI Developer – Professional, Solutions Architect, or an equivalent credential, is desirable.

Benefits:

• Opportunity to work on mission-critical generative AI solutions supporting healthcare organizations and access to authoritative program information.

• Hands-on exposure to advanced LLM technologies, retrieval-augmented generation, semantic search, AI evaluation, and responsible AI practices.

• Opportunity to work with AWS cloud services and modern AI engineering tools and frameworks.

• Collaborative environment involving engineers, analysts, project managers, subject matter experts, and client stakeholders.

• Opportunity to contribute to secure and responsible AI solutions operating within federal security and governance requirements.

• Professional growth through exposure to emerging AI technologies, federal AI guidance, and complex healthcare use cases.