Lead Data Platform Engineer - 11623

What You'll Do

• Manage end-to-end Data pipeline (ETL jobs) within agreed SLAs.
• Manage AWS core and big data services (S3, IAM, EMR, Redshift, etc..)
• Running applications in containers (ECS, Docker)
• Lead Day 2 operational lifecycle for ML and GenAI infrastructure. This includes designing, deploying, and maintaining high-availability production LLM serving platforms, implementing automated scaling, self-healing, and infrastructure-as-code patterns. Focus on proactive reliability, model performance observability, and continuous cost optimization for high-compute AI workloads.
• Collaborate closely with our product development and engineering teams to create AI-driven features
• Drive cloud operations consistency by automating platform maintenance, standardizing infrastructure configurations (IaC), and implementing robust release management processes to minimize drift across multi-cloud environments.
• Manage AWS infrastructure using code (Terraform, Chef, etc..)
• Administering applications running in Linux operating system.
• Enable application and system monitoring for better observability.
• Application and infrastructure support for ETL jobs and data pipelines including participating in an on-call rotation for after-hours emergencies.
• Collaborate with platform and Dev teams to plan and deploy product releases and patch Linux/ECS clusters.
• Ability to participate in design reviews, code reviews, and troubleshooting incidents.
• Ability to operate in a high-pressure environment and troubleshoot complex issues quickly while successfully handling multiple priorities.
• Ability to record, write, and review RCAs.

What You Will Bring to Coupa

• Bachelor's Degree and at least 8+ years of experience managing Big Data technologies and Data Pipelines.
• Sound knowledge and experience in Linux administration and troubleshooting.
• 5+ years of experience in managing cloud infrastructure and platforms, such as AWS and Azure
• Familiar with the current engineering landscape in the generative AI space and have a strong interest in AI and related technologies.
• Strong expertise in MLOps and production-grade LLM operations. Proven track record in managing high-availability model inference clusters, automating model lifecycle management, and implementing advanced observability (latency, throughput, and error rate monitoring) specifically for AI workloads.
• Have Bash or Python scripting experience
• Experience with containerization, Amazon ECS, EKS/ Azure AKS
• Experience with tools like Chef, Ansible, Jenkins, Rundeck, or equivalent
• Experience with source control systems such as Git and operating in complex branching strategies
• Experience with Infrastructure as Code products like Terraform, helm charts
• Good understanding of DNS and Load balancers setup and troubleshooting
• Experience in Big Data platforms/Data lakes and managing Business Intelligence tools (like looker..)
• Knowledge in ApacheSpark architecture and troubleshooting Java applications.
• Basic understanding of MySQL Server and general database knowledge
• Excellent written and verbal communication with a passion for solving the problem
• Confidence in your ability to own and deliver projects and issues to resolution on your own & can think and act globally
• Deep experience in Day 2 cloud operations, including automated incident remediation, capacity planning, and managing large-scale production cloud environments with a focus on performance and reliability.