Senior Full Stack AI Engineer (Rapid Prototyping & Analytics )

Responsibilities

• Drive Prompt’s mission to improve healthcare through modern technology including AI
• Lead AI projects from ideation → architecture → production → iteration until tools are widely adopted and loved!
• Design, build, and deploy end-to-end AI systems across both traditional ML and LLM-based workflows
• Partner with stakeholders across the company to understand workflows, define success criteria, and deliver practical solutions
• Supporting internal as well as product-facing AI initiatives as needed
• Design and implement evaluation, monitoring, and observability metrics (accuracy, cost, latency, and business impact)
• Build guardrails around LLM pipelines including validation and human-in-the-loop workflows
• Develop rapid proofs-of-concept and prototypes, then harden them into reliable systems
• Help grow and lead the AI team, including training and development of junior team members

Qualifications

• 4+ years of experience building AI/ML systems in industry or academia, with meaningful production ownership
• Strong experience building and operating AI systems end-to-end, not just models in isolation
• Strong experience with LLM-based systems (programmatic prompt optimization, evaluation, prompting, structured outputs, RAG, multi-step workflows)
• Solid understanding of evaluation and measurement for AI systems (offline metrics, online monitoring, regressions, cost tracking)
• Ability to work closely with non-technical stakeholders to translate workflows into effective AI solutions
• Strong software engineering fundamentals and ability to ship maintainable, production-grade systems
• Track record of benchmarking and iteratively improving accuracy, cost, and latency
• Excellent communication skills, with the ability to explain complex AI concepts to non-technical stakeholders
• Python (>3.9)
• Docker (Basics)
• AWS familiarity (S3, RDS, EC2) preferred
• Strong problem-solving mindset and ability to operate effectively in ambiguous, fast-paced settings
• Willingness to learn and deeply understand the healthcare sector, specifically physical therapy, occupational therapy, and speech therapy
• Please only apply if you are interested in a heavy code contribution role; this is not an engineering management position, though growth opportunities exist over time

Nice-to-Haves

• Experience building internal AI tools or platforms used across multiple business functions
• Experience designing evaluation harnesses for LLM workflows (golden datasets, regression tests, human review loops)
• Experience setting up analytics and dashboards to track AI usage, outcomes, and ROI
• Familiarity with workflow automation and business systems (CRMs, support tools, analytics platforms)
• Experience with FastAPI, Streamlit, Pydantic, SQLAlchemy, PostgreSQL
• Experience with retrieval systems (vector search, hybrid search, grounding and citation strategies)
• Experience building healthcare software and/or data processing pipelines