MLOps Engineer Portfolio Projects: 15 Shippable Ideas

These projects are ordered by platform-building dependency — each earlier one builds skills the later ones require. A strong MLOps portfolio is built of systems, not notebooks: live URLs, working CI pipelines, instrumented dashboards, and post-mortems from the things you deliberately broke. Every project here can run on a $10/month VPS with a few hours of GPU rental; most can run entirely on free-tier cloud resources.

1. Model registry from scratch using MLflow

Stand up a self-hosted MLflow tracking server with a PostgreSQL backend and an S3-compatible artefact store (MinIO). Train three variants of a binary classifier (logistic regression, random forest, XGBoost) on the same dataset. Log all parameters, metrics, and model artefacts to your registry. Transition the winner through Staging → Production using the MLflow registry lifecycle. Build a minimal Python client that fetches the Production model version and runs inference. This project teaches the artefact management discipline that separates professional MLOps work from ad-hoc model files dropped in S3.

Evaluation rubric (what recruiters check on your GitHub)

• MLflow server, PostgreSQL, and MinIO all run via Docker Compose with health checks
• Three models logged with full parameter and metric provenance
• Model transition history visible in the MLflow UI — Staging → Production documented
• Client code uses `mlflow.pyfunc.load_model` with the registry URI, not a hardcoded file path
• README explains the artefact store separation and why file-based artefacts are anti-patterns in shared teams

2. CI/CD pipeline for ML: lint, test, train, evaluate, register

Build a GitHub Actions pipeline that runs on every pull request and every merge to main. The pipeline must: lint with ruff, run pytest on feature engineering code, train a model on a small held-out slice of a public dataset, evaluate against a quality gate (must beat the current Production model's accuracy by ≥0.5%), and push a new Staging candidate to MLflow only if all gates pass. The training step should run on a self-hosted runner or GitHub-hosted runner with DVC pulling the data from a versioned remote. This project is the CI/CD equivalent of a 'hello world' for MLOps — nearly every interview will ask you to whiteboard a version of it.

Evaluation rubric (what recruiters check on your GitHub)

• Pipeline YAML is fully readable — no inline shell scripts longer than 5 lines
• Quality gate implemented as a Python script, not a bash one-liner — takes current Production model baseline from registry
• DVC data pull step uses a locked .dvc file — any uncommitted data change fails the pipeline
• Failed quality gate produces a structured summary in the Actions run with the delta metrics
• README includes a screenshot of a failing pipeline and a passing pipeline, both with the gate output visible

3. Model serving on Kubernetes with autoscaling and observability

Deploy a text classification model behind a FastAPI serving container to a Kubernetes cluster (k3s or cloud). Expose it via Ingress. Add Prometheus metrics: requests per second, inference latency p50/p95/p99, and model version label. Wire Grafana dashboards. Configure HPA to scale replicas when p95 latency exceeds 200ms. Run a k6 load test that triggers the HPA. Document the incident you had during the load test. This project proves you understand the operational reality of running ML in production, not just deploying it once.

Evaluation rubric (what recruiters check on your GitHub)

• Deployment YAML includes readiness and liveness probes — failing probes are tested, not just written
• Prometheus metrics exported from the FastAPI app using prometheus-client — not pushed via a sidecar
• HPA YAML targets a custom metric (latency p95 from Prometheus adapter) — not CPU, which is a proxy, not the signal
• Grafana dashboard exported as JSON and committed to the repo
• Load test report shows the HPA scale-up event and the latency recovery curve — both captured from Grafana

4. Drift detection system on a production-style data stream

Take a public tabular dataset and simulate production drift: create a 'day 0' training distribution and a 'day 30' inference distribution where several feature distributions have shifted. Build a batch drift-detection job using Evidently AI that computes PSI for numeric features, chi-squared for categorical features, and a data-quality report. Run the job on a schedule (Airflow or Prefect cron). Alert when PSI > 0.2 (via a Slack webhook or email). Write up a one-page post-mortem as if this were a real production drift incident, including impact, root cause, and the remediation steps you would take.

Evaluation rubric (what recruiters check on your GitHub)

• Training vs inference distribution shift clearly documented with before/after histograms
• PSI and chi-squared thresholds justified in the README — not just 'I used 0.2 because it's common'
• Scheduled job produces an HTML report committed to an artefacts store (S3, MinIO)
• Alert fires on a seeded drift injection — tested, not assumed
• Post-mortem section in README covers impact, root cause, remediation, and prevention — the four-part structure

5. Feature store with online-offline parity

Build a working feature store using Feast. Define feature views for a recommendation or fraud-detection use case. Materialise features from a Parquet offline store to a Redis online store. Build a training pipeline that pulls features from the offline store and a serving pipeline that retrieves features from the online store in under 5ms. Deliberately introduce a training-serving skew bug — different bucketing on one feature — and build a validation test that catches it. This project is the single most effective way to demonstrate that you understand why feature stores exist.

Evaluation rubric (what recruiters check on your GitHub)

• Feature views defined with clear entity keys and timestamps — not a notebook ad-hoc schema
• Point-in-time correct feature retrieval demonstrated for training — no data leakage
• Online store latency measured and documented: p50/p99 for feature lookup
• Skew validation test written in pytest — fails with the bug, passes without it
• README explains the training-serving skew concept using your specific project as the example

6. A/B testing infrastructure for two model variants

Deploy two model versions (call them Model A and Model B) behind a traffic-splitting layer. Route 80% of requests to A and 20% to B. Log both models' outputs with a consistent request ID. Build a statistical significance test that runs daily on the accumulated logs and determines whether B's quality metrics are significantly better than A's. Build an automatic promotion script: if B is significantly better for 3 consecutive days, promote B to 100% traffic. This project teaches the measurement discipline that separates rigorous model deployment from opinion-based deployment.

Evaluation rubric (what recruiters check on your GitHub)

• Traffic split implemented at proxy layer — not by the model server choosing which model to use
• Request logs include: model version, input hash, output, timestamp, and any available implicit feedback signal
• Statistical test is pre-registered — the test type (t-test vs Mann-Whitney), metric, threshold, and horizon are documented before the experiment starts
• Automatic promotion script is idempotent — safe to run multiple times without double-promoting
• README documents what happens if neither model is significantly better after the test horizon — 'keep A' logic is explicit

7. Canary deployment for an ML model with automatic rollback

Implement a canary deployment for a model update using Argo Rollouts on Kubernetes. Configure a rollout strategy: 5% → 20% → 50% → 100% canary progression, with each step gated on a Prometheus metric (p99 latency < 300ms, error rate < 0.5%). Build an automatic rollback trigger: if p99 exceeds 400ms for 2 consecutive minutes, the rollout is paused and rolled back. Seed a regression in the canary model to verify the rollback fires. Write up the incident as a post-mortem.

Evaluation rubric (what recruiters check on your GitHub)

• AnalysisTemplate YAML defines the rollback criteria — not manual kubectl commands
• Rollback is triggered by a PromQL query, not a synthetic test flag
• Regression seed is documented: what you changed in the canary model and what metric it broke
• Rollout history is preserved in the Argo Rollouts dashboard — screenshot in README
• Estimated blast radius quantified: max % of traffic that saw the regression before rollback, calculated from the rollout step timing

8. Multi-model serving with vLLM and routing by capability

Deploy two LLMs of different sizes (e.g. Qwen2.5-1.5B and Qwen2.5-7B) on vLLM. Build a routing layer that dispatches requests to the smaller model when request complexity is low (short prompt, simple output schema) and the larger model when complexity is high. Route 100% of structured-output requests through a quality gate that validates the JSON schema. Measure and report cost-per-request for each route. This project teaches the model routing and cost-optimisation skills that senior MLOps roles prize most.

Evaluation rubric (what recruiters check on your GitHub)

• Routing heuristic is documented and tested — not a random split
• Both model versions running simultaneously on separate Kubernetes Deployments with GPU resource limits
• Cost-per-request metric computed from GPU-hours and reported via Prometheus
• Quality gate rejects malformed outputs and routes to the large model for retry — retry logic is bounded (max 1 retry)
• Load test shows the routing percentages holding under 10x normal traffic

9. GPU cost-optimisation dashboard

Build a Grafana dashboard that aggregates GPU utilisation, memory bandwidth, idle time, and cost attribution per team or model. Pull DCGM metrics from all GPU nodes. Implement a cost model: cost_per_hour × utilisation_fraction = attributed_cost. Build an alert for GPUs that are allocated but idle (utilisation < 5%) for more than 30 minutes. Document the three biggest waste sources you found in your cluster and the configuration changes that fixed them. This project is immediately valuable at any company with a GPU fleet and demonstrates measurable business impact.

Evaluation rubric (what recruiters check on your GitHub)

• DCGM metrics flowing from at least one GPU node (can be a $0.50/hr rental GPU) — real hardware, not mocked
• Cost model documented with assumptions: on-demand hourly rate used, overhead factor included
• Idle-GPU alert fires on a manually-induced idle condition — tested before submission
• Dashboard exported as Grafana JSON and committed to the repo
• Three waste findings documented with before/after utilisation metrics — this is the most valuable part of the writeup

10. Prompt version management system

Build a system for managing prompt templates as versioned artefacts. Store templates in a DB with version history, author, and a description of what changed and why. Build a UI (minimal — FastAPI + Jinja2 is fine) for creating and promoting templates through a Draft → Staging → Production lifecycle. Wire prompt version as a logged field in every LLM call, so you can slice model quality metrics by prompt version. This project is directly relevant as LLM applications proliferate and 'we changed the prompt' becomes the most common cause of quality regressions.

Evaluation rubric (what recruiters check on your GitHub)

• Template schema includes: id, version, content, author, created_at, status, change_reason
• Status lifecycle transitions enforced at the API level — cannot go from Draft to Production directly
• Prompt version logged in every LLM response record — query-able in the analysis notebook
• A rollback scenario demonstrated: promote v2 to Production, log 10 calls, roll back to v1, log 10 more — show the version distribution in a query
• README explains why prompt versioning belongs in an MLOps platform alongside model versioning, not in the application code

11. Model lineage tracking: code → data → model → deployment

Build a lineage tracking layer that connects: (1) the git commit of the training code, (2) the DVC data version, (3) the MLflow run ID, (4) the model registry version, and (5) the Kubernetes Deployment that is serving the model. Implement this as a chain of metadata tags passed through the pipeline, persisted in a lineage DB, and queryable via a simple API. Given a production incident, you should be able to answer 'what training data was in scope for the model currently serving request X?' in under 30 seconds. This project demonstrates the audit-trail discipline that compliance-sensitive organisations require.

Evaluation rubric (what recruiters check on your GitHub)

• Lineage chain covers all five links — no gaps
• Query API: GET /lineage/{deployment_name} returns the full chain for the currently serving model version
• Test case: redeploy with a different model version and verify the lineage chain updates correctly
• Lineage ID is propagated as a Kubernetes Deployment annotation — visible via kubectl describe
• README includes a diagram of the lineage chain

12. Inference latency profiler and optimisation report

Profile a transformer model's inference latency end-to-end: tokenisation, model forward pass, de-tokenisation, and network overhead. Use PyTorch Profiler and NVIDIA Nsight Systems to identify the hottest kernels. Apply at least two optimisations from: dynamic batching, INT8 quantisation via bitsandbytes or TensorRT, torch.compile, attention implementation swap (flash-attention-2). Document the before/after latency and throughput for each optimisation. Publish a table of trade-offs. This project is the clearest way to demonstrate that you understand the inference side of the stack, not just the deployment side.

Evaluation rubric (what recruiters check on your GitHub)

• Baseline measured at batch sizes 1, 8, 32 before any optimisation — latency p50/p99 and tokens/second
• PyTorch Profiler trace captured and committed (as HTML) — hottest kernels identified by name
• At least two independent optimisations applied and measured independently — not stacked before measurement
• Trade-off table: latency gain, throughput gain, accuracy impact (if any), memory delta, implementation complexity for each optimisation
• Conclusion section explains which combination you would ship to production and why

13. End-to-end Argo Workflows ML training pipeline

Build an Argo Workflows DAG that orchestrates: (1) data validation with Great Expectations, (2) feature engineering, (3) model training on a GPU job (PyTorchJob via Kubeflow Training Operator or plain Kubernetes Job), (4) evaluation and quality gating, (5) registration in MLflow, and (6) notification (Slack or email) with the evaluation metrics. Each step runs in an isolated container. The pipeline should be re-triggerable on any code or data change. This is the project that best simulates a production-grade training platform and the most common interview-loop system design question in MLOps.

Evaluation rubric (what recruiters check on your GitHub)

• Each step in its own container image — no single monolithic image for the whole pipeline
• Data validation step uses a Great Expectations checkpoint with at least 5 expectations — pipeline halts on validation failure
• Training step uses a Kubernetes Job with GPU resource requests and backoff limits — not a Pod running directly
• Quality gate step compares against the current Production model baseline from MLflow — not a hardcoded threshold
• Notification step fires for both success and failure, with different message content — observed by the reviewer

14. Open-source contribution to an MLOps tool

Pick an active open-source MLOps project: MLflow, Feast, Evidently AI, Ray, Argo Workflows, Kubeflow, DVC, Prefect. Find a good-first-issue, a documentation gap you hit while learning, or a bug you encountered in your projects. Submit a PR. Survive the review process. Ship it. One merged contribution to a production-grade MLOps tool demonstrates engineering maturity that no solo project can replicate.

Evaluation rubric (what recruiters check on your GitHub)

• PR is merged — un-merged PRs are noted, not featured
• Your change includes a test that fails on main and passes on your branch
• You respond to all review comments within 48 hours, professionally
• Commit message follows the project's convention exactly
• README links the merged PR with a one-line description of what you fixed or added

15. Capstone: end-to-end ML platform slice, documented as a system design

Build and document a coherent end-to-end slice of an ML platform: data ingestion → feature store → training pipeline → model registry → serving infrastructure → observability → drift alerting → automated rollback. Every component uses tools from this track. The deliverable is not just the working system — it is a system design document with architecture diagrams, trade-off discussions, and an honest section on what you would do differently at 10x scale. This is the project you present in Staff-level MLOps interviews.

Evaluation rubric (what recruiters check on your GitHub)

• Architecture diagram in the README — one image, clear boxes, arrows with labels
• Each technology choice justified in the design doc — not just 'I used MLflow because it's popular'
• Training-serving skew proof: test demonstrating that your feature store eliminates it
• Chaos experiment: what happens when the MLflow server is unavailable during serving? Documented and tested.
• Live URL or a fully-scripted `make deploy` that stands up the entire stack in under 10 minutes from a clean cluster

Frequently asked questions