Production LLM systems · MCP servers, agents, evals

Shipped to production. And actually used.

I take a business problem to a production AI system, and then get people to actually use it. Head of AI at Crème de la Crème, where I run discovery, design the architecture, write the code and ship it.

Measured results

What these systems changed

Not a list of deliverables, but a list of usage that stuck. Every figure below comes from a shipped, measured system, used by teams I do not manage.

90–95%

of inbound applications decided by AI, with a human kept on the ambiguous cases

50 / 50

hand-decided applications the pipeline agreed with, before go-live

1 call / candidate

the matching engine judge is capped and parallelised, so cost and latency per brief stay flat as the pool grows

30 min → seconds

per data extract, self-serve, without going through an engineer

~7 h / week

returned to the Product Manager by the human-in-the-loop decision pipeline

27

more salespeople given access to an expert tool that licences had kept to a few

Method

From a business problem to a system people use

Discovery with the people who will use the system, architecture with the trade-offs written down, the code itself, a production path that holds up when the system gets it wrong, and adoption tracked after go-live. Evals, guardrails and cost control are first-class concerns, not afterthoughts.

  1. 01

    Discovery

    Discovery with the leadership and with the people who will live with the result, turned into a short use-case portfolio: an owner, a success measure, a blocker list. Including the cases worth killing.

    Tested against six business teams with different risk tolerances.

  2. 02

    Architecture

    Models, data, integration, identity, privacy, governance, evaluation, deployment. Every significant call is written down with what it costs.

    32 trade-offs recorded before a line of the MCP server was written.

  3. 03

    Proof of value

    A prototype on real data early, so the decision to go further rests on evidence rather than on a demo that only works on the happy path.

    An eval harness that had to beat the production baseline, brief by brief.

  4. 04

    Build

    I write the code, front and back, working through coding agents, and what ships goes through the client's engineering review. When a piece needs expertise I do not have, I bring in someone who does and write down who owns it once they leave.

    Six systems built this way, front and back, on the same engagement.

  5. 05

    Production and governance

    Evals against whatever is already running, guardrails wired in rather than suggested, authorization and audit designed in. The question of what happens when the system gets it wrong is answered before it ships.

    Isolation predictions written down before each role switch, then checked.

  6. 06

    Adoption

    Usage tracked after go-live, the build documented so another engineer can take it over, and systems retired when a vendor closes the gap.

    Rolled out to Sales ahead of general availability.

Capabilities

What I actually build

The parts that decide whether a system can be left running on production data.

01

Retrieval & ranking

Multi-signal recall (BM25, skill overlap, vector kNN over OpenAI embeddings stored in pgvector, experience indexes) fused by Reciprocal Rank Fusion, then deterministic weighted scoring, then a bounded LLM-as-a-judge layer with constant cost and latency per request. A brief costs the same to answer whether the pool holds a thousand profiles or fifty thousand.

02

Agents & MCP

MCP servers over real production systems, with OAuth 2.1, tool design and allowlists, sub-agents, packaged skills, and coding agents (Codex, Claude Code) as a normal way of working. Agent Builder and ChatKit where a hosted surface beats a custom one. An agent acts with the rights of the person asking, never those of a shared service account.

03

Evals & reliability

Golden datasets, eval harnesses that must beat the production baseline before a ship decision, schema-validated structured outputs, and multi-state verdicts scored axis by axis so every decision stays explainable.

04

Governance & authorization

Three-layer authorization, personas with data-computed scopes, field-by-field payload projection, deterministic GDPR guards outside the model path, and queryable audit trails with bounded retention. What a departing employee can still see is answered with a date rather than a guess.

05

Stack

TypeScript and Python, React and Node, PostgreSQL with pgvector, Vercel, PostHog. Coding agents are how I build: the architecture is mine, what they produce is corrected rather than accepted, and what ships is reviewed by the client's engineers. The Responses API, structured outputs and embeddings run in production here; several frontier providers run side by side, picked per task rather than by default.

06

Adoption

Discovery with non-technical owners, documentation and handover, usage tracked after go-live, and systems retired when the vendor already in place closes the gap.

Modularcomponents,oratoms,recombinedintolargersystems.ItiswhyInamedmycompanyAtomly AI.

Trusted by

Teams I have worked with

Twelve years of client-facing work: technical discovery with executives, GenAI proofs of concept taken into production pipelines, and 1,000+ professionals trained in applied GenAI.

Chanel
Schneider
Viatris
Stellantis
Institut Pasteur
Rakuten
Robert Half
Primonial
Invivo
Le Parisien
IPTP

What I do

Applied AI, from the business problem to production

I work as the technical owner of an AI portfolio: discovery with the people who will use the system, architecture with the trade-offs written down, the code itself, production, and then adoption. Three ways to work together.

Applied AI architecture & delivery

Embedded with your team as the technical owner of an AI system: discovery, architecture, code, production, adoption. The shape of the work is forward-deployed: I write the code, I work with your engineers, and I stay until the system is in daily use.

AI systems review

An honest read on an existing system or roadmap: what to build, what to buy, what to stop. Evals, guardrails, authorization, cost and latency: the questions that decide whether a demo survives contact with production.

Enablement & training

Hands-on programmes for Product, Tech and business teams. 1,000+ professionals trained in applied GenAI at Chanel, Stellantis, Bouygues Telecom, Rakuten, Generali, Institut Pasteur, Arkéa and Bpifrance.

Testimonials

What clients say

From the enablement side of the work, the part that decides whether a system survives its first month.

Pierre is an exceptional trainer who knows how to make generative AI exciting and accessible to everyone. His depth of expertise, combined with a rare ability to adapt to participants' industries and professions, makes each session a tailored experience. His training sessions are dynamic, inspiring, and always perfectly aligned with clients' concrete challenges. With attentive follow-up and highly precise feedback, Pierre combines professional excellence with the joy of learning!

Christophe Conceicao

Christophe Conceicao

Head of Education & Operational Excellence, Maria Schools

Beyond being one of the best AI experts today, Pierre is a natural teacher, making his training sessions both useful and interesting! He conveys his passion for AI with energy and conviction, and consistently tries to relate examples to cases from the participants' industry.

Guillaume Coulomb

Guillaume Coulomb

Co-founder, On Train

Engagements

Recent engagements

Beyond the production systems: assessments, workshops and training programmes delivered to technical and business teams.

Development
05/2025 - present

Crème de la Crème

Technical owner of the AI portfolio: discovery with the Sales, Product and Engineering teams, architecture, code and production. MCP servers, a hybrid matching engine, human-in-the-loop decisioning, governed data access.

Direct client

MCP
Retrieval
Evals
Training
05/2025

BHG Conseil

Executive coaching on generative AI use cases, focusing on optimising various consulting-related tasks. In-depth exploration of AI Agent creation to automate and enhance consulting processes.

Direct client

Training
Consulting
AI Agents
Training
04/2025

Cube School

Created a comprehensive e-learning program with over 10 hours of video content. Developed a structured curriculum providing a complete introduction to generative AI for students.

Direct client

Training
E-learning
GenAI

Contact

Have a system that needs to reach production?

Tell me the business problem and who is supposed to use the result. I come back with the architecture, what I would build and what I would buy, and a prototype on your own data rather than a deck.

Atomly AI | Pierre Rognion, AI systems in production