AutomateEdge · MLOps Engineer · Skills

MLOps Engineer Skills Matrix: 2026 Proficiency Ladder

The MLOps Engineer skill map below is calibrated against 2026 hiring signals — specifically, skills that appear in ≥35% of MLOps / ML Platform Engineer / ML Infrastructure postings across the AutomateEdge jobs pipeline over the last 90 days. Proficiency levels follow a four-step ladder: Beginner (can follow tutorials), Intermediate (can build a solution to a new problem), Advanced (can debug production incidents), Expert (can design systems from first principles). An MLOps Engineer clears Advanced in container orchestration, CI/CD, and observability by week 24. One note on math: you do not need to be able to derive backpropagation, but you must understand what is expensive about it — batch size, activation memory, parameter count, gradient accumulation — so you can make sensible GPU scheduling decisions.

Container orchestration and infrastructure

Kubernetes is the operating system of MLOps. Everything else — serving, scheduling, autoscaling, feature computation — runs on top of it. You need to understand Kubernetes beyond 'I can apply a YAML' — operators, custom resources, HPA, VPA, pod disruption budgets, node affinity for GPU workloads, and the debugging workflow when a training pod OOMs at 3 a.m. Kubernetes 1.30+ in 2026 ships Gateway API as stable; know it alongside Ingress.

SkillTarget levelNote
Kubernetes — cluster ops, workloads, networking Advanced Deployments, StatefulSets, DaemonSets, Services, Ingress/Gateway API, RBAC, namespaces. Comfortable reading events and logs to triage a failed pod without the docs open.
Kubernetes — custom resources and operators Intermediate CRDs, controller reconciliation loops, Kubeflow PipelineRun / TrainingJob resources. Most ML platform work is operators writing operators.
Kubernetes — GPU scheduling and resource quotas Intermediate nvidia.com/gpu resource requests, NUMA affinity, priority classes, ResourceQuota, LimitRange. Misaligned GPU scheduling is the #1 cause of silent under-utilisation.
Helm chart authoring and management Intermediate Templates, values overrides, chart dependencies, Helmfile for multi-chart stacks. Every ML platform component you ship should be a Helm chart.
Docker — multi-stage builds and layer optimisation Advanced COPY --from, .dockerignore, BuildKit secrets, distroless base images, image size discipline. A 12 GB CUDA image is a footgun; a 2 GB one takes 6x less time to pull on every scale-up event.
Infrastructure-as-Code — Terraform or Pulumi Intermediate State management, modules, provider locking, plan/apply workflow, drift detection. Pulumi (Python-native) is gaining ground in ML orgs; Terraform still dominates ops teams.
Linux internals and networking Intermediate cgroups v2, namespaces, iptables/eBPF basics, NFS/object-store mounts, /proc debugging. You will debug network policies and host-level issues that kubectl logs alone cannot surface.

CI/CD for ML systems

A CI/CD pipeline for ML is not a software CI/CD pipeline with a training step bolted on. It must version data alongside code, gate on model quality metrics not just test pass rates, handle artefact sizes measured in gigabytes, and coordinate handoffs between training, evaluation, registry, and serving environments. Getting this right is the highest-leverage infrastructure investment an ML team can make.

SkillTarget levelNote
GitHub Actions or GitLab CI for ML pipelines Advanced Multi-job workflows, matrix builds, artifact caching, self-hosted runners on GPU nodes, secrets management. Build pipelines that lint, test, train on a small slice, evaluate against a gate, and promote to the model registry — all in one YAML.
Argo Workflows — DAG-based ML pipeline orchestration Intermediate Workflow CRD, steps vs DAG templates, artifact passing, retry policies, resource templates for GPU tasks. Argo Workflows is the de-facto Kubernetes-native orchestrator for training + eval pipelines.
Model quality gating — eval-driven promotion Intermediate Define promotion criteria before you write the training code. Gating on accuracy delta, latency p99, data-drift score, and fairness metrics. A CI pipeline that promotes a model on 'accuracy >= 0.8' with no baseline comparison is broken by design.
Data + artefact versioning with DVC or LakeFS Intermediate DVC remote storage, pipeline stages, dvc repro, LakeFS branches for datasets. Reproducibility requires locking data commits alongside code commits — every ML engineer learns this the hard way once.
Container image scanning and supply-chain hygiene Beginner Trivy, Grype, or Snyk in CI. SBOM generation. ML images pull dozens of large transitive dependencies from PyPI — scanning is non-optional in security-conscious orgs.

Model serving and inference infrastructure

Model serving is where ML meets real SRE work — SLOs, capacity planning, traffic management, rollback. The stack has fragmented: vLLM and SGLang have become the standard for generative-model serving; Triton Inference Server remains the standard for discriminative models and computer vision; TorchServe is a reasonable middle ground. Ray Serve covers the heterogeneous-model case. Know at least two serving runtimes well enough to choose between them.

SkillTarget levelNote
vLLM and SGLang — generative model serving Intermediate vLLM's paged attention, continuous batching, async engine, LoRA adapter hot-swap. SGLang's structured generation, RadixAttention, multi-modal routing. These are the 2026 production stack for LLM serving — not TGI, not a plain transformers loop.
NVIDIA Triton Inference Server Intermediate Model repository layout, ensemble pipelines, dynamic batching, ONNX / TensorRT backends, gRPC and HTTP protocols, perf_analyzer. Triton is the right answer for CV models, embedding services, and multi-model latency-sensitive pipelines.
Ray Serve — multi-model and heterogeneous serving Intermediate Deployments, replica scaling, request batching, model composition graphs, Ray 2.x actor model. Covers the 'I have twelve models and they share compute' case that neither Triton nor vLLM handles natively.
Kubernetes-native serving patterns Advanced HPA on custom metrics (requests/sec, GPU utilisation), Knative for scale-to-zero, node pools with GPU taints, preemptible GPU nodes, rollout strategies (canary, blue-green via Argo Rollouts). Autoscaling a serving fleet is the most common MLOps ops fire.
Inference optimisation — quantisation, compilation, batching Intermediate INT8/FP8/INT4 quantisation trade-offs, TensorRT compilation, torch.compile, CUDA graphs, flash-attention. Know enough to cut GPU cost by 2-5x on a real serving workload; know when to stop optimising and buy more hardware.

Monitoring, observability, and drift detection

ML systems fail silently. A software service that stops working pages instantly; a model that produces subtly wrong answers may go undetected for weeks. The MLOps Engineer builds the layer that converts 'model went bad' from a discovery event into a monitored, alerted, automatically-gated condition. Statistical drift detection, latency SLOs, cost dashboards, and model-quality proxies all belong here.

SkillTarget levelNote
Prometheus + Grafana — ML-specific dashboards Advanced Custom metrics from serving containers (inference latency, batch size, GPU memory, request rate), PromQL queries, Grafana alert rules, recording rules for efficiency. Every serving component you ship should export /metrics — non-negotiable.
OpenTelemetry — distributed tracing for ML pipelines Intermediate Instrumenting training scripts, pipeline steps, and serving handlers with spans and attributes. Trace from feature lookup → model inference → postprocessing in one waterfall. Particularly valuable in multi-model serving and RAG pipelines.
Data and model drift detection Advanced PSI, KS test, MMD, Evidently AI report generation, Arize / WhyLabs / Fiddler for production monitoring. Understand the difference between covariate drift (input distribution shifts) and concept drift (the relationship between inputs and outputs changes). False-positive drift alerts are as expensive as missed drift.
Model quality proxies — surrogate metrics Intermediate When ground-truth labels arrive late (or never), you need surrogate metrics: confidence calibration, output distribution entropy, proxy tasks, implicit feedback signals. Define these before go-live, not after the first incident.
Cost observability — GPU and API spend dashboards Intermediate DCGM metrics for GPU utilisation and memory bandwidth, cost-per-inference attribution, idle-GPU alerts, spot-instance reclamation handling. In 2026 GPU cost is the dominant operational expense for ML-heavy companies; the engineer who instruments it becomes indispensable.

Data pipelines and feature stores

Features are the interface between the data world and the model world. Training-serving skew — where the features a model was trained on differ subtly from the features it receives in production — is one of the top causes of production ML failures. Feature stores exist to close that gap. Airflow, Prefect, and Dagster are the orchestration layer; Feast is the most widely adopted open-source feature store; Redis and DynamoDB back the low-latency online store.

SkillTarget levelNote
Airflow, Prefect, or Dagster — pipeline orchestration Intermediate DAG authoring, task dependencies, sensors, retry policies, schedule intervals, backfill. Know at least one in depth. In 2026 Prefect and Dagster have both pulled significantly ahead of raw Airflow for usability in Python-native teams.
Feast — feature store architecture Intermediate Feature views, entity keys, online vs offline store split, feature retrieval for training vs serving, push-based and pull-based materialisation. The central tool for eliminating training-serving skew.
Streaming data with Kafka or Kinesis Beginner Topics, consumer groups, at-least-once delivery semantics, KSQL for lightweight transformations. Online feature computation for real-time models requires a streaming backbone — understand the operational cost before committing to it.
Data quality and validation — Great Expectations or Deequ Intermediate Expectation suites, validation checkpoints, data docs. Data validation upstream of training is 10x cheaper than debugging a model that was trained on corrupted data.
Columnar storage and query — Parquet, Delta Lake, DuckDB Intermediate Parquet schema evolution, Delta Lake transaction log for offline feature stores, DuckDB for ad-hoc analytics on feature snapshots. The offline store is always larger and always cheaper per query when you use the right format.

MLflow, model registry, and governance

A model registry is the source of truth for what is running in production. Every model that serves traffic must be traceable to a training run, a dataset version, a set of hyperparameters, and an author. MLflow is the most widely-adopted open-source tool for this; Vertex AI Model Registry and SageMaker Model Registry are the cloud-native equivalents. Model governance — who can promote a model to production, under what conditions — becomes critical at regulated companies and at scale.

SkillTarget levelNote
MLflow — tracking, registry, and serving Advanced Experiment tracking (params, metrics, artefacts), the Model Registry lifecycle (Staging → Production → Archived), MLflow Models serving, custom Python models, model signatures. MLflow is the backbone of most MLOps pipelines outside of full cloud lock-in.
Model lineage and reproducibility Intermediate Linking a deployed model version to the exact code commit, dataset hash, container image digest, and hyperparameter set that produced it. This is the chain of custody that enables incident post-mortems and regulatory audits.
A/B testing and shadow deployment infrastructure Intermediate Traffic splitting at the proxy layer (Istio, Argo Rollouts, nginx weighted upstreams), logging both model variants' outputs, statistical significance testing for quality deltas, feature flags for model routing. The mechanism that separates 'we think the new model is better' from 'we know the new model is better.'
Canary and blue-green deployment for ML models Intermediate Staged rollout percentages, automatic rollback triggers on latency or quality SLO breach, the difference between a software canary (binary right/wrong) and an ML canary (stochastic quality degradation). Argo Rollouts is the Kubernetes-native implementation.
Model cards and documentation Beginner Intended use, evaluation metrics, training data sources, known limitations, fairness analysis. Model cards are increasingly required at regulated companies and are a professional-grade habit regardless.

Suggested learning sequence

How long to 'Intermediate' across the board?

Full-time learner: 6-8 months to production-ready MLOps skills. Part-time (10 h/week): 12-15 months. The longest poles are Kubernetes depth and production incident exposure — both require hands-on repetition, not just reading. The engineers who progress fastest deliberately break their own clusters and debug them.

Frequently asked questions

Do I need to know how to train models to be an MLOps Engineer?

You need to understand what is expensive and why — batch size effects on GPU memory, gradient accumulation, checkpoint frequency, why full fine-tuning costs 10x more than LoRA. You do not need to design architectures from scratch or understand backpropagation's calculus. The boundary is: you can read a training config and know what to ask, not that you can author one blindly.

Is this role basically SRE with ML bolted on?

Partially — the on-call, capacity planning, and reliability posture are very SRE-adjacent. But the domain knowledge required (training-serving skew, drift detection, model governance, feature store design) is distinct from traditional SRE. The highest-value MLOps engineers understand both worlds; the weakest ones are SREs who learned to deploy models, or ML engineers who learned to write Kubernetes YAMLs, without filling in the other half.

Kubernetes feels enormous. Do I need to know all of it?

No. Focus on: workloads (Deployments, Jobs, CronJobs), services + networking (ClusterIP, Ingress/Gateway), storage (PVCs, StorageClasses), RBAC, and the GPU resource model. That is ~70% of what you touch day-to-day. Operators, custom resources, and admission webhooks come later, organically, when a platform feature requires them.

Should I learn Terraform or Pulumi?

Terraform if you join an existing infra team with Terraform already in use. Pulumi if you are starting fresh or in a Python-native ML org — writing infrastructure in Python with type hints and testing is a significant ergonomic win for ML engineers. Both are worth knowing at Intermediate depth.

vLLM, SGLang, Triton, Ray Serve — which serving runtime should I learn first?

vLLM, then Triton. vLLM is where the industry has landed for generative models in 2026 and has the most community momentum. Triton is the right tool for discriminative models, embedding services, and latency-critical CV pipelines. Ray Serve is worth learning after those two, for the heterogeneous-model orchestration case.

How much Python do I need vs Go or Rust?

Python is primary — it is the language of ML tooling, orchestrators, and glue code. Advanced Python (async, type hints, profiling, C-extension calling) is mandatory. Go and Rust at Intermediate are increasingly useful for writing high-performance serving sidecars, admission webhook controllers, and custom Kubernetes operators — the kind of work that Kubernetes-native ML platforms generate. You will not be blocked without Go/Rust, but senior MLOps roles at platform-centric companies often expect it.

What does training-serving skew actually mean in practice?

A concrete example: you train a recommendation model with a feature called user_age_bucket computed by a Python function during batch training. In production, the same feature is computed by a different service in Java with a slightly different bucketing boundary. The model's predictions are subtly degraded for users near the bucket boundaries — and there is no error, no crash, just worse recommendations that degrade over weeks. Feature stores solve this by making the feature computation a single artifact used by both training and serving. This is the category of problem MLOps Engineers exist to eliminate.

Is the MLOps role being automated away by managed ML platforms?

Managed platforms (Vertex AI, SageMaker, Databricks) abstract a great deal of the undifferentiated heavy lifting. What they do not abstract is: organisation-specific compliance constraints, multi-cloud or hybrid infrastructure, the 'last 20%' of performance optimisation that determines whether GPU costs are sustainable, and cross-team coordination. The platforms eliminate the lowest-value MLOps work; the highest-value work grows as the company's ML footprint does.

Continue on the MLOps Engineer track