ML Engineer Portfolio Projects: 15 Shippable Ideas

These projects are ordered by depth and dependency — each one teaches something the later ones require. A strong ML Engineer portfolio contains 8-10 of these. The projects that get interviews are not the ones that use the most tools — they are the ones that show you understand what you built, why you made each decision, and what the numbers mean. Every project below should have: training/evaluation curves in a W&B report, a README that answers 'what problem, what approach, what result,' and a live demo or reproducible run command.

1. Implement a transformer from scratch — decoder-only language model

Write a decoder-only transformer (GPT-style) in under 500 lines of raw PyTorch — no Hugging Face, no Lightning. Include multi-head self-attention with causal masking, rotary positional embeddings, layer norm, and a standard feedforward block. Train on TinyShakespeare or a comparable small text corpus. Track loss curves in W&B. Compare your implementation against nanoGPT as a reference checkpoint. Write a blog post walking through each architectural decision with the mathematical motivation.

Evaluation rubric (what recruiters check on your GitHub)

• Training loss converges to < 1.5 nats on TinyShakespeare (matches nanoGPT reference within ±0.1)
• Every attention head weights verified against a reference implementation on the same input (numerical match)
• Gradient norms logged throughout training — spike detection and discussion in README
• Ablation: compare RoPE vs learned positional embeddings, documented with loss curves
• Blog post cites the Attention Is All You Need paper section-by-section and maps each equation to code

2. Fine-tune an LLM with LoRA — reproduce benchmark metrics

Fine-tune Llama 3.1 8B or Qwen 2.5 7B on an instruction dataset (Alpaca, ShareGPT, or a domain-specific set you curate) using LoRA via Hugging Face PEFT + Accelerate. Run ablations on LoRA rank (r=4, 8, 16, 32) and alpha, tracking loss and downstream eval metrics. Report ROUGE, perplexity, or a task-specific accuracy depending on your dataset. Compare to a full-fine-tune baseline on the same number of trainable parameters. Quantise the final checkpoint to INT4 and measure inference throughput before and after.

Evaluation rubric (what recruiters check on your GitHub)

• Ablation table: rank 4/8/16/32 × alpha 8/16/32, eval metric for each, training cost per run
• Final model card documenting training config, dataset, and eval results in Hugging Face format
• INT4 quantisation: latency before/after with throughput numbers (tokens/second at batch size 1 and 8)
• Comparison against base model on 50 held-out test examples with LLM-judge scoring
• Training-serving skew check: validate that Tokenizer config is identical in training and serving

3. Distributed training pipeline — train a model across 2+ GPUs

Take your transformer from project 1 (or a medium-sized image model) and train it across 2 or 4 GPUs using PyTorch FSDP. Profile single-GPU vs multi-GPU throughput (samples/second), memory usage per device, and communication overhead using PyTorch Profiler and NVIDIA Nsight. Then add DeepSpeed ZeRO-2 and compare. Document the setup, the results, and the trade-offs. The goal is to demonstrate you understand distributed training beyond 'I added `.to('cuda')`'.

Evaluation rubric (what recruiters check on your GitHub)

• Throughput scaling table: 1 GPU, 2 GPU, 4 GPU — samples/second and GPU utilisation per run
• Memory profile: per-device peak memory with FSDP FULL_SHARD vs SHARD_GRAD_OP vs NO_SHARD
• Communication overhead measured: what fraction of wall-clock time is all-reduce vs compute
• FSDP vs DeepSpeed ZeRO-2 comparison on the same model — documented trade-offs in README
• Checkpoint-resume tested: kill the run mid-epoch, resume from checkpoint, verify loss continuity

4. Reproduce a research paper end-to-end

Pick a paper with a clear, measurable benchmark — LoRA, Mamba, DPO, CLIP, or Diffusion Policy are all reasonable choices depending on your interest area. Re-implement the core algorithm from scratch. Train on the paper's exact dataset (or a documented substitute). Report the primary metric from the paper and compare your result. Write a post-mortem: where did your first implementation differ from the paper, what did you have to fix, and what is still unexplained. Reproducing a paper is the single best signal of research-adjacent ML competence.

Evaluation rubric (what recruiters check on your GitHub)

• Primary paper metric reproduced within 5% of the reported number (document any gap and why)
• Commit-by-commit log that shows iteration: first broken implementation → debugging → working
• Ablation of at least one hyperparameter not ablated in the original paper
• Post-mortem write-up: 3 things the paper under-specified, how you resolved each
• Code is clean, documented, and runnable in under 30 min from a fresh clone

5. Custom optimiser implementation + benchmarking

Implement two optimisers from scratch in PyTorch — Adam and one modern variant (SOAP, Muon, AdaFactor, or Adan). Verify mathematical correctness against PyTorch's reference on the same model and data. Benchmark convergence speed, final validation loss, and wall-clock time on a small language model task. Write up the derivation, the update rule, and the intuition behind each optimizer's design choice. This project demonstrates the math depth that separates ML Engineers from ML practitioners.

Evaluation rubric (what recruiters check on your GitHub)

• Parameter-level numerical match between your Adam and torch.optim.Adam on the first 100 steps
• Convergence curves on a standard benchmark (PTB perplexity, CIFAR-10 accuracy) for both optimisers
• Math write-up: derive the Adam update rule from the EWMA formulation with bias correction
• Sensitivity analysis: how each optimiser responds to 10x learning rate increase
• README explains when each optimiser is the better choice — not just 'Adam is default'

6. A/B testing framework for model improvements

Build an end-to-end A/B testing framework for ML models. Implement statistical power analysis to determine required sample size before launching an experiment. Run a real experiment pitting two model variants (e.g., base model vs LoRA fine-tune, or model with INT8 quantisation vs full precision) against each other on live or simulated traffic. Implement sequential testing (alpha spending) to allow early stopping. Log all predictions and compute the primary metric plus guardrail metrics. Report the result with a proper confidence interval.

Evaluation rubric (what recruiters check on your GitHub)

• Power analysis tool: input effect size + alpha + beta, output required sample size — tested with known analytical results
• Sequential testing implemented: experiment can be stopped early if the effect is conclusive at interim looks
• Guardrail metrics tracked alongside the primary metric — catch regressions the primary metric misses
• Interaction effect check: are any user segments responding differently?
• Final report with confidence intervals, not just point estimates — 'Model B is 2.3% better (95% CI: 0.8%-3.8%)'

7. Model serving system with batching and autoscaling

Build a production-grade model serving endpoint that handles variable request loads. Implement dynamic batching (accumulate requests for up to 20ms before forwarding to the model). Deploy with Ray Serve or vLLM. Add horizontal autoscaling triggered by GPU utilisation. Load test with Locust or k6 at 1x, 5x, and 20x steady-state traffic. Report p50, p95, and p99 latency and throughput at each load level. Compare serving cost per 1000 requests before and after batching optimisation.

Evaluation rubric (what recruiters check on your GitHub)

• Dynamic batching implementation with configurable max_batch_size and max_wait_ms — measured effect on throughput vs latency
• p50/p95/p99 latency table at 1x / 5x / 20x traffic — tested with Locust and plotted
• Autoscaling trigger: replica count increases when GPU utilisation > 80% for 30s, measured in the load test
• Serving cost calculation: tokens/second per dollar at steady-state traffic
• Rollback tested: deploy model v2, confirm v1 is accessible, switch back — all under 60 seconds

8. Feature store prototype with point-in-time correctness

Build a feature store that correctly serves features with point-in-time join semantics — the key property that prevents training-serving skew in time-dependent ML systems. Use Feast for the core store implementation. Define an offline store (Parquet-backed) and an online store (Redis-backed). Show that naive joins would produce a different (biased) training set compared to point-in-time joins on the same data. Deploy a simple model that uses online feature lookup at inference time. Document the skew problem and how point-in-time joins solve it.

Evaluation rubric (what recruiters check on your GitHub)

• Point-in-time correctness demonstration: side-by-side comparison of naive join vs PIT join training labels on the same event set
• Offline → online consistency: verify that the online store returns the same feature values as the offline store for the same entity + timestamp
• Feature freshness SLA documented: how stale can online features be before model accuracy degrades?
• Backfill pipeline: populate historical features for a 6-month window without loading all data into memory
• Training pipeline reads from offline store; serving endpoint reads from online store — both using Feast SDK

9. Train a diffusion model from scratch on a curated image dataset

Train a DDPM (Denoising Diffusion Probabilistic Model) or DDIM from scratch on a narrow, well-defined image domain you curate — pixel art, satellite imagery, medical images, or product photography. Implement the forward (noising) and reverse (denoising) processes, the noise prediction network (a UNet), and the sampling loop. Evaluate with FID score on a held-out test set. Track training in W&B. Ablate the number of diffusion timesteps (100 vs 500 vs 1000) and the noise schedule (linear vs cosine). This project demonstrates the mathematical depth of generative model training.

Evaluation rubric (what recruiters check on your GitHub)

• Forward process: mathematically correct implementation of q(x_t | x_0) with arbitrary timestep — verified against the closed-form formula
• Generated samples visually coherent at 256x256+ resolution, FID score reported and compared to reference DDPM numbers
• Ablation table: timesteps 100 / 500 / 1000 × linear / cosine schedule, FID and training time per configuration
• Inference speed: samples/second for DDPM vs DDIM (the deterministic accelerated sampler)
• Sample diversity vs quality trade-off: vary classifier-free guidance scale and document the effect

10. Experiment tracking dashboard — replay and compare runs

Build a custom experiment tracking dashboard that aggregates runs from W&B, MLflow, or both, and surfaces actionable comparisons. Features: hyperparameter importance analysis (Optuna's fANOVA or a simpler correlation sweep), run comparison with delta-metric highlighting, learning curve smoothing, best-checkpoint selection by eval metric. The goal is a tool you actually want to use for your other projects. This demonstrates software engineering taste alongside ML knowledge.

Evaluation rubric (what recruiters check on your GitHub)

• Hyperparameter importance chart: which hyperparameters account for >80% of val-loss variance across 50+ runs
• Run comparison: select any two runs, see a side-by-side diff of every hyperparameter and metric
• Best-checkpoint-per-run tracked: not just final checkpoint but the epoch with best val metric
• Dashboard updates automatically as new runs complete — no manual refresh
• Export function: download a CSV of all runs with hyperparameters + final metrics for offline analysis

11. Build a retrieval-augmented evaluation harness for LLMs

Build an evaluation harness for an LLM that integrates retrieval. The eval harness should: (1) run a suite of benchmark tasks against a model, (2) compare retrieval-augmented vs non-augmented performance, (3) run LLM-as-judge scoring on open-ended outputs, and (4) track results across model versions in W&B. Make the harness model-agnostic so you can plug in different checkpoints. This project demonstrates evaluation engineering — one of the highest-signal skills in 2026 ML hiring.

Evaluation rubric (what recruiters check on your GitHub)

• Eval harness runs ≥3 task types: multiple-choice, open-ended generation, and retrieval-grounded QA
• LLM-as-judge implementation with defined rubric, prompt reproducibility, and inter-rater reliability check on 20 examples
• Model-agnostic design: swap from HF model to vLLM endpoint with a config change, not a code change
• Regression detection: alert if a new checkpoint underperforms the baseline by >2% on any task
• Retrieval ablation: RAG vs no-RAG comparison with statistical significance test

12. MLOps pipeline — automated retraining with drift detection and rollback

Build a complete MLOps pipeline that monitors a deployed model, detects data drift, triggers automated retraining when drift exceeds a threshold, evaluates the retrained model against the champion, and promotes the challenger only if it passes regression gates. Implement rollback that restores the previous model version in under 2 minutes. Use ZenML or Metaflow for pipeline orchestration. Deploy to Docker Compose on a VPS. This is the graduation project for the MLOps section and demonstrates production-grade ML engineering.

Evaluation rubric (what recruiters check on your GitHub)

• Drift detection: PSI-based input distribution monitoring with configurable alert thresholds, confirmed by injecting known distribution shift
• Automated retraining trigger: drift alert → retraining pipeline launches → challenger model evaluated → promoted or rejected
• Regression gate: challenger model must exceed champion on 3 held-out eval metrics — all three must pass, not just one
• Rollback tested: introduce a deliberately degraded model, confirm rollback restores the previous version in < 2 minutes
• Pipeline reproducibility: every retraining run captures data hash, code commit, and hyperparameters — any run can be replicated

13. GPU kernel optimisation — implement flash attention in Triton

Implement the forward pass of flash attention in Triton (the Python-embedded GPU kernel language). Start with a naive attention baseline in CUDA. Profile with Nsight Compute to identify the memory-bandwidth bottleneck. Implement the tiled, IO-aware forward pass from the FlashAttention-2 paper. Benchmark your kernel against torch.nn.functional.scaled_dot_product_attention and the reference flash-attn package on sequence lengths 512, 1024, 4096, and 8192. This project puts you in the top 5% of ML Engineer candidates. Treat it as an optional stretch goal; attempt only after the core portfolio is complete.

Evaluation rubric (what recruiters check on your GitHub)

• Naive O(N^2) attention in CUDA verified numerically correct against PyTorch reference at FP32
• Triton flash attention implementation numerically correct within FP16 tolerance at all tested sequence lengths
• Benchmark table: naive CUDA / torch SDPA / flash-attn reference / your Triton impl — memory (GB) and TFLOPS at each sequence length
• Nsight profile showing your kernel reaches ≥80% of theoretical memory bandwidth on H100
• Writeup explains the IO complexity argument from the FlashAttention paper in your own words with a diagram

14. End-to-end ML system — train, serve, monitor, and retrain

Capstone. Build a complete ML system on a problem you define: a recommender, a ranker, a classifier, or a generative model. The system must train on a real dataset, serve predictions behind an API endpoint at sub-200ms p99 latency, log prediction distributions for drift monitoring, and have a retraining pipeline that can be triggered by a data volume threshold or a schedule. Deploy to a VPS. Add a public-facing demo. Write a 2000-word technical post-mortem. The capstone is the project you demo in interviews — it should be something you can talk about for 45 minutes.

Evaluation rubric (what recruiters check on your GitHub)

• Architecture diagram in the README — training, serving, monitoring, and retraining components clearly separated
• Sub-200ms p99 latency verified under realistic load (Locust load test report in README)
• Drift monitoring active: describe what metric you are monitoring and what threshold triggers retraining
• One documented retraining cycle: describe what data change triggered it, what improved, what the regression gate was
• Live URL. A recruiter can interact with the system in under 60 seconds without setup.

Frequently asked questions