Product Manager, Inference Platform

The role

Once a model is deployed, keeping it fast, reliable, and economical at scale is where production inference is won or lost. You'll own the surface that makes that happen: how deployments autoscale, how traffic is routed, how the system fails over, and how workloads scale across clusters and regions. You'll own these as products end to end - both how they work under the hood and how customers configure and observe them - and you'll help set and define the roadmap that infrastructure and product teams alike can build towards. This space is largely still evolving - think Cloud Infrastructure in mid-2000s. Your job is to make it 10x easier to reliably scale and serve AI models in production and set the market standard.

Impact and outcomes you'll drive

• You will own how workloads scale and where they land — autoscaling to demand (up under load, down to zero when idle) and a single placement policy expressing region, compliance regime, and capacity preference, with compliance-bound workloads given right-of-way on sensitive capacity.
• You will make production inference reliable by default — every request reaches a healthy replica, rolling deploys never drop traffic, region-aware routing with multi-region / active-active and fallback as first-class policy, and health-aware recovery from stuck or bad replicas.
• You will build the release engine beneath safe rollouts — the traffic-shifting that powers canary/shadow/A/B, warm-ups, drain, and probes.
• You will push the cost/performance frontier for serving AI at scale — latency, throughput, uptime, and cost-efficiency, plus a measurable decline in MTTR through self-serve incident management.

What we're looking for

• 8+ years in product management, including deep experience with infrastructure, distributed systems, or ML serving.
• You reason fluently about scaling, routing, failover, and the cost/performance frontier — and you earn the respect of staff engineers doing it.
• You’ve owned capabilities end to end, backend through UX, rather than a single slice.
• You drive cross-team roadmaps and the dependencies beneath them, and you're at your best defining a category that doesn't fully exist yet.

Bonus: hands-on with GPU infrastructure, Kubernetes, or serving frameworks like vLLM, TensorRT-LLM, or SGLang

Not for You if

• You don't like getting technical.
• You prefer only strategy, UX, or writing great docs over doing whatever it takes to ship great products for customers.
• You lean toward applied AI over building platform, systems, GPUs, models, and scaling platforms and infrastructure.
• You're not interested in the foundational, sometimes unglamorous work of making AI systems reliable and scalable at the infrastructure level.