Note: The job is a remote job and is open to candidates in USA. Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. They are seeking a Senior Technical Program Manager to drive execution of large-scale ML infrastructure and AI tooling initiatives, owning the end-to-end delivery of programs that span model serving infrastructure, ML pipelines, and internal AI tooling.
Responsibilities
- Own end-to-end delivery of AI infrastructure programs—from model training pipelines and experiment tracking to inference serving and production monitoring
- Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling (e.g., vector databases, model servers, evaluation frameworks, prompt engineering platforms)
- Serve as connective tissue between ML research, ML engineering, product, and data teams to align on ML system requirements, capability roadmaps, and deployment timelines
- Drive cost and latency optimization for real-time inference workloads at scale
- Build lightweight internal tools and processes to accelerate ML iteration cycles (experiment tracking, model versioning, A/B testing infrastructure)
- Identify and resolve technical bottlenecks in training pipelines, serving infrastructure, and model evaluation workflows
- Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems
Skills
- 5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent)
- Strong technical acumen in ML systems—ideally hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer
- Experience coordinating cross-functional ML programs (e.g., model training → evaluation → serving → monitoring)
- Proven ability to translate ML/research requirements into robust, scalable infrastructure
- Comfortable working in ambiguity and helping teams navigate complex technical tradeoffs (e.g., accuracy vs. latency vs. cost)
- Excellent communication with both technical and non-technical stakeholders
- Familiarity with high-growth or startup environments
- Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar)
- Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching strategies)
- Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC, or similar)
- Background in feature stores, vector databases, or real-time ML systems
- Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure
- Experience with multi-region model serving or edge deployment
- Hands-on with relevant frameworks (PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML)
Benefits
- Offers Equity
- Offers Bonus
- 10% Annual Bonus
Company Overview
Company H1B Sponsorship