AI Engineer Portfolio Projects: 15 Shippable Ideas

These projects are ordered by dependency: each earlier one teaches a skill the later ones assume. Every project is AI-Engineer-specific — no model-training projects appear here (those belong on the ML Engineer track). The focus throughout is on shipped, evaluated, production-ready AI features: retrieval systems, agentic workflows, structured-output pipelines, and observable LLM applications. Ship them to the public internet. A recruiter who can click your live URL and see streaming LLM output will form an impression in under 30 seconds — make sure it is the right one.

1. Multi-provider LLM comparison harness

Build a CLI tool that sends the same prompt to OpenAI, Anthropic, and Gemini simultaneously, displays all three responses side by side, and logs token count, latency, and cost-per-call to a SQLite database. Add a `--best-of` flag that uses an LLM judge to score the responses and pick a winner. This is your first eval fixture and your first cost-tracking surface — skills you will use every day as an AI Engineer.

Evaluation rubric (what recruiters check on your GitHub)

• All three providers called in parallel (not sequentially) — measure the latency difference
• Token count + estimated cost logged per call, queryable from the DB
• LLM-judge scoring uses a rubric, not free-form; rubric is stored as a versioned file
• README documents every provider's API key setup and the cost per 1000 test runs
• Error handling covers rate limits, network timeouts, and malformed responses

2. Streaming LLM chat backend with observability

Build a FastAPI backend that streams an LLM response token-by-token to the browser via Server-Sent Events. Persist every conversation turn (user message, assistant response, model, token counts, latency) to SQLite. Add a PII-redaction filter that strips email addresses and phone numbers from logs before writing them. Deploy to a VPS with nginx. This is the foundation of every chat-based AI product.

Evaluation rubric (what recruiters check on your GitHub)

• First token arrives in the browser within 500ms of the user pressing send (measure with browser devtools)
• Every conversation persisted; conversation history reconstructable from DB for any session_id
• PII redaction tested with phone and email inputs — redacted form visible in logs, never the original
• nginx config includes rate-limit headers — no user can send more than 10 requests/minute
• Live URL that a reviewer can open and chat with

3. RAG system over a personal document corpus

Build a retrieval-augmented Q&A system over 50+ documents you actually care about — your company's internal docs, a technical book, a public-domain text corpus. Use pgvector for storage, sentence-transformers for embedding, and a hybrid search (BM25 + dense) strategy. Add a 30-question hand-labelled eval set and score the system using Ragas faithfulness and context_recall metrics. This is the portfolio centrepiece for most AI Engineer job applications.

Evaluation rubric (what recruiters check on your GitHub)

• Chunk strategy documented with before/after quality comparison across at least two strategies
• 30-question eval set with automated Ragas scores; baseline score documented before any prompt tuning
• Hybrid search (BM25 + dense) implemented; ablation showing which strategy wins on your corpus
• Streaming response — user sees tokens arriving, not a loading spinner for 10 seconds
• Prompt injection defense: retrieved content cannot override system-level instructions

4. Prompt regression testing pipeline with CI

Set up a Promptfoo test suite for an existing LLM feature. Define 40+ test cases covering happy paths, edge cases, and adversarial inputs. Wire it into GitHub Actions so every pull request that touches a prompt file runs the full eval suite and blocks merge if any case regresses more than 10%. Add a score-trend dashboard (a simple HTML file generated from the Promptfoo JSON output). This project is rare in portfolios and universally impressive to AI Engineering interviewers.

Evaluation rubric (what recruiters check on your GitHub)

• 40+ test cases including at least 5 adversarial inputs (prompt injection attempts, boundary cases)
• GitHub Actions workflow that fails the PR if quality regresses
• Score-trend chart committed alongside the test run in CI artifacts
• README explains how to add a new test case and what the CI gate protects
• At least one example in the README where the CI gate caught a real regression

5. Structured-output extraction pipeline

Build a pipeline that extracts structured data from unstructured text at scale — job descriptions into normalised fields, research papers into citation graphs, customer support tickets into categorised summaries. Use Instructor + Pydantic for validation; add retry logic for parse failures. Process 1,000 real documents; log accuracy by field. This project demonstrates the skill most enterprise AI work actually requires: turning messy text into reliable structured data.

Evaluation rubric (what recruiters check on your GitHub)

• Pydantic schema covers all target fields with appropriate types, validators, and field descriptions
• Retry logic handles model refusals and malformed JSON — never crashes on bad output
• Field-level accuracy table for 1,000 documents with honest failure-mode analysis
• Async batch processing with concurrency limit (respect API rate limits)
• Cost per 1,000 documents documented; optimisation applied if over $5

6. Function-calling agent for a narrow domain task

Build an agent that does one task reliably: GitHub PR summariser, calendar booking assistant, database query agent for a specific schema, or shell-command assistant with safety gates. Expose 3-5 tools via the provider's function-calling API. Log every tool call. Add a dry-run mode for any destructive action. Resist feature creep — a narrow agent that works 95% of the time is a better portfolio piece than a broad one that works 60% of the time.

Evaluation rubric (what recruiters check on your GitHub)

• Task scope narrowly defined in the README with an explicit 'out of scope' section
• Every destructive tool has a dry_run=True default; explicit confirmation required for real execution
• Tool-call log persisted and queryable — include request, tool name, arguments, response, latency
• At least 20 test cases covering normal flows, bad inputs, and tool failures
• Failure-mode section in README — where the agent loops, hallucinates arguments, or misses the task

7. Semantic search engine for a niche corpus

Build a full semantic search engine over a corpus you choose: papers from a specific arXiv category, Stack Overflow posts for a niche technology, lyrics from a genre. Support both semantic (embedding) and keyword (BM25) search with a ranking blend. Add a web UI with faceted filtering (date, category, score range). Measure and compare retrieval quality via NDCG@10 against a keyword-search baseline. This demonstrates retrieval engineering depth beyond basic RAG.

Evaluation rubric (what recruiters check on your GitHub)

• Corpus ≥ 5,000 documents; ingestion pipeline re-runnable from a single command
• Hybrid ranking (BM25 + dense) with configurable alpha parameter; ablation table in README
• NDCG@10 measurement against 50 hand-judged queries
• Web UI with working faceted filter — not just a text box
• Index rebuild time and search latency (P50/P95) documented

8. LLM-as-judge evaluation framework

Build a reusable LLM-as-judge framework that scores generated text against rubrics you define. Support at minimum: binary pass/fail, 1-5 scale rating, and pairwise preference comparison. Implement bias mitigations (position randomisation for pairwise, temperature=0 for consistency). Validate your judge against 100 human-scored examples and report judge-human agreement rates (Cohen's kappa). This framework will underpin every future eval work — it is worth building well.

Evaluation rubric (what recruiters check on your GitHub)

• Three evaluation modes implemented: binary, scale, pairwise
• Position bias mitigation for pairwise (randomise A/B order, average both orderings)
• Judge-human agreement reported as Cohen's kappa on 100 human-annotated examples — minimum kappa 0.6
• Rubric files versioned in files, not hardcoded strings
• Parallelised scoring — 100 examples should complete in under 60 seconds

9. Production RAG system with cost and quality dashboards

Extend the basic RAG project with a full production observability layer: per-query cost and latency tracking, retrieval quality trending over time, daily Ragas metric snapshots logged to a time-series table, and an admin dashboard that surfaces cost burn rate, p95 latency, and quality trend charts. Add semantic caching and measure its impact on cost. This project demonstrates what it takes to operate an AI feature in production — not just build one.

Evaluation rubric (what recruiters check on your GitHub)

• Per-query cost logged; daily cost summary queryable from the DB
• Ragas scores computed on a sample of 10 queries per day; time-series chart shows quality trend
• Semantic cache hit rate measured and displayed in the dashboard
• P50/P95/P99 latency tracked and alertable (a log message at P99 > 5s is acceptable; a silent slow query is not)
• Admin dashboard live at a password-protected route

10. Multi-modal application (vision + text)

Build an application that combines vision and text modalities using GPT-4o vision or Gemini 2.0 Pro: a receipt data extractor, a diagram Q&A tool, a photo-to-structured-data pipeline, or a visual code-review assistant. Pay close attention to image pre-processing (resolution, format, token cost) and eval design (vision outputs are harder to auto-evaluate than text).

Evaluation rubric (what recruiters check on your GitHub)

• Image size pre-processed to minimise token cost without sacrificing quality — document the trade-off
• Structured output extraction validated with Pydantic — no raw string output fed to downstream logic
• Eval set of 50 images with ground truth labels; accuracy by category reported
• Error handling covers unsupported image formats, oversized images, and refusals
• Cost per processed image documented at three quality settings

11. Agentic research assistant with tool composition

Build a research assistant that can answer complex questions by composing multiple tools: web search (via a search API), code execution (via a sandboxed subprocess), document retrieval (via your RAG system), and calculator tools. The key challenge is reliability: the agent must complete tasks across 5-10 tool calls without derailing. Implement a maximum-step budget and a quality-check step that reviews the answer before returning it.

Evaluation rubric (what recruiters check on your GitHub)

• Four or more distinct tool types implemented and tested independently before integration
• Maximum step budget enforced — agent cannot loop more than 15 turns
• Quality-check step (separate LLM call) validates the answer before return — catches obvious hallucinations
• Complete execution trace (every tool call and response) persisted for debugging
• 20 multi-step benchmark questions with documented accuracy rate

12. Prompt version management system

Build a lightweight prompt management system: store prompts in a database with version history, support A/B deployment (route X% of traffic to version A, Y% to version B), and track eval score per prompt version over time. This does not need to be a full SaaS — a FastAPI backend + simple admin UI + SQLite is the right scope. The goal is to demonstrate you understand prompts as production artifacts, not throwaway strings.

Evaluation rubric (what recruiters check on your GitHub)

• Prompt versions stored with immutable hash + semantic version number
• A/B routing implemented at the API level — `POST /generate` samples between active versions by percentage
• Eval score recorded per version per run; chart shows quality over time
• Rollback to any previous version in one admin action with an audit log entry
• README explains the deployment workflow: how to promote a new prompt version to 100% traffic

13. Real-time collaborative AI writing tool

Build a browser-based writing editor where an AI copilot provides inline suggestions, paragraph completions, and style critiques in real time. Use WebSockets for bidirectional communication, streaming completions for low-latency suggestions, and a debounce strategy to avoid hammering the API on every keystroke. This project stresses all the real-time engineering skills that distinguish AI product engineers from API wrappers.

Evaluation rubric (what recruiters check on your GitHub)

• WebSocket connection maintained across the editing session — reconnect on drop
• Debounce strategy documented: minimum idle time before triggering completion
• Streaming tokens rendered progressively in the editor UI without flickering
• Suggestion acceptance / rejection tracked; acceptance rate logged per session
• Cost per 1,000-word editing session documented; optimisation applied if over $0.10

14. Safety and guardrails layer for an LLM application

Build a reusable input/output safety middleware for an LLM application. Input layer: classify and block prompt injection attempts, detect and redact PII, enforce topic constraints for domain-specific applications. Output layer: detect refusals and off-topic responses, validate structured output, flag potentially harmful content with a confidence score. Measure false-positive rate against 200 benign test inputs — overly aggressive guardrails are a real production problem.

Evaluation rubric (what recruiters check on your GitHub)

• Input classifier tested against 100 adversarial inputs (known prompt injection patterns)
• Output validator catches refusals and off-topic responses reliably
• False-positive rate measured on 200 benign inputs — document the rate and the acceptable threshold
• PII redaction covers at minimum: email, phone, SSN patterns, and credit card patterns
• Middleware is extractable — it should drop into any FastAPI application as a dependency

15. Capstone: end-to-end AI-powered product feature

Build and deploy a complete AI-powered feature that a real user would pay for: a domain-specific Q&A assistant with RAG, a document analysis tool, an AI job-matching engine, or a personalised content recommender. The feature must combine at minimum: retrieval, structured output, streaming, evals, observability, and production deployment. Every component should be wired, measured, and documented. This is your graduation project — the piece you present in interviews and link from your resume.

Evaluation rubric (what recruiters check on your GitHub)

• Architecture diagram in the README — one clear image showing every component and its responsibility
• Ragas or equivalent eval suite with documented baseline and current scores
• LangSmith or Langfuse tracing active in production with a dashboard link in the README
• Cost and latency budget documented per user request
• Live URL that any recruiter can open and interact with in under 60 seconds

Frequently asked questions