Senior ML Operations Engineer II - Monetization
Overview
Fetchfetch.comPosted 13 days ago
United Statesعن بُعد
Quick apply- مستوى الخبرة
- Senior
- نوع الوظيفة
- دوام كامل
- التخصص
- Process engineering
Responsibilities
- Own the complete model lifecycle from experimentation through validation, deployment, versioning, retraining, rollback, and retirement.
- Establish clear model promotion criteria, release processes, and rollback paths that enable teams to ship improvements confidently.
- Build monitoring for data quality, drift, prediction quality, service reliability, latency, and infrastructure cost.
- Partner with data scientists to define meaningful offline and production evaluation signals, including for use cases with delayed outcome labels.
- Use production evidence to improve model performance, platform reliability, deployment speed, and infrastructure efficiency.
- Build reusable training, evaluation, and deployment pipelines that support multiple models and modeling approaches.
- Develop dependable orchestration workflows covering dependencies, scheduling, retries, backfills, and failure recovery.
- Translate scientific requirements into maintainable production systems with clear interfaces, ownership, and operational expectations.
- Build and maintain an accessible model hub connecting model documentation, evaluation results, release history, versioning, and monitoring.
- Create tested CI/CD workflows, reproducible environments, actionable alerts, incident-response procedures, and operational runbooks.
- Define MLOps priorities in partnership with Data Science and Engineering, identifying risks before they become production issues.
- Establish reusable engineering standards, mentor teammates, and reduce the operational burden placed on data scientists.
Requirements
- 5+ years managing production ML systems beyond initial deployment, including monitoring, retraining, incident response, and continuous improvement.
- Strong production software development skills in Python and proficiency with SQL.
- Hands-on experience using Spark for distributed data processing.
- Experience with a workflow orchestration engine for scheduling, dependency management, retries, backfills, and recovery.
- Hands-on experience building infrastructure with Terraform.
- Experience deploying and operating containerized production workloads on Kubernetes.
- Experience building reusable training, evaluation, or deployment infrastructure used by multiple models, scientists, or teams.
- Experience implementing model and dataset versioning, experiment tracking, lineage, and reproducible releases.
- Ability to distinguish service or infrastructure failures from changes in model effectiveness.
- Experience building CI/CD pipelines, monitoring, alerting, rollback mechanisms, and recovery procedures for production ML systems.
- Demonstrated ability to balance model quality, delivery speed, reliability, and infrastructure cost.
- Strong cross-functional communication and experience partnering with data scientists to preserve scientific validity as models move into production.
- Ability to explain a production incident or model degradation issue, how it was detected, and what improvements followed.
- Experience developing platform services, infrastructure, or internal tooling in Go.
- Experience building or owning a model registry, model hub, or similar system used by technical and nontechnical stakeholders.
- Experience supporting ML systems where outcome labels are delayed or model quality cannot be measured immediately.
- Track record of improving deployment speed, model quality, system reliability, infrastructure cost, or manual operational effort.
- Experience designing configurable ML infrastructure that supports several modeling frameworks or approaches.
- Evidence that reusable infrastructure, standards, or documentation were adopted across multiple teams.
- Experience mentoring engineers or data scientists and influencing technical priorities across an ML portfolio.
- Experience working in environments with sensitive information and designing documentation or monitoring around appropriate access boundaries.
Key skills
- CI/CD
- Data Science
- Machine Learning
- Python
- SQL
- Apache Spark
- Terraform
- Kubernetes
Benefits & perks
- Remote friendly