Paul Iusztin

🤖 Senior Machine Learning Engineer • MLOps • Contractor ~ Helping startups engineer production-ready ML/AI systems.

I'm the author of the bestseller LLM Engineer's Handbook, lead instructor of the Agentic AI Engineering course, founding AI Engineer of a San Francisco start-up, and obsessed with making knowledge accessible through AI.

With over 10 years of experience in AI and software and 20 apps shipped, I teach AI Engineering as I wanted to at the beginning of my career. End-to-end. From idea to production. From data collection to deploying, monitoring and evaluation. With a focus on AI principles, software patterns and infrastructure systems that will thrive in a future dominated by AI coding tools.

My ultimate goal is to help other engineers escape PoC purgatory and 10x their AI Engineering skills.

.

If you want to become an AI Engineer pro, join Decoding AI for exclusive content on designing, building, and shipping AI software that works: 

🗞️ https://www.decodingai.com

See more:

🌐 Homepage: https://www.pauliusztin.ai
📚 LLM Engineer's Handbook: https://www.amazon.com/LLM-Engineers-Handbook-engineering-production/dp/1836200072/
🤖 Agentic AI Engineering Course: https://academy.towardsai.net/courses/agent-engineering?ref=b3ab31

.

✉️ For collabs, contact me at: p.b.iusztin@gmail.com

Senior Machine Learning Engineer

🟡 Scalable ML batch infrastructure for continuous training & inference

The client, a generative AI leader, struggled with manual training and calling their ML models. As a tedious and unscalable process, they needed to automate their ML processes completely and integrate them with their current web platform.

➞ 𝐃𝐞𝐬𝐢𝐠𝐧𝐞𝐝 and 𝐢𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐞𝐝 a modular 𝐏𝐲𝐭𝐡𝐨𝐧 𝐩𝐚𝐜𝐤𝐚𝐠𝐞 that composes the ML model required steps into 3 main batch pipelines that can easily be automated: feature, training, and inference, 𝐞𝐥𝐢𝐦𝐢𝐧𝐚𝐭𝐢𝐧𝐠 a 𝐭𝐞𝐝𝐢𝐨𝐮𝐬 𝟐-𝐡𝐨𝐮𝐫 𝐦𝐚𝐧𝐮𝐚𝐥 𝐩𝐫𝐨𝐜𝐞𝐬𝐬/𝐯𝐢𝐝𝐞𝐨.
➞ 𝐃𝐞𝐩𝐥𝐨𝐲𝐞𝐝 a scalable & cost-effective 𝐛𝐚𝐭𝐜𝐡 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 and 𝐂𝐈/𝐂𝐃 𝐩𝐢𝐩𝐞𝐥𝐢𝐧𝐞 on top of AWS ECS & SQS that scales in and out based on the number of jobs, 𝐬𝐚𝐯𝐢𝐧𝐠 𝐭𝐡𝐨𝐮𝐬𝐚𝐧𝐝𝐬 𝐨𝐟 𝐝𝐨𝐥𝐥𝐚𝐫𝐬/𝐦𝐨𝐧𝐭𝐡.
➞ 𝐒𝐚𝐯𝐞𝐝 𝐦𝐨𝐧𝐭𝐡𝐬 𝐨𝐟 𝐟𝐮𝐭𝐮𝐫𝐞 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭 by using good software practices to make the Python module & batch cloud infrastructure flexible and composable.
➞ 𝐁𝐮𝐢𝐥𝐭 a 𝐯𝐨𝐢𝐜𝐞 𝐬𝐲𝐧𝐭𝐡𝐞𝐬𝐢𝐬 𝐏𝐨𝐂 in ~1 week required to 𝐜𝐥𝐨𝐬𝐞 the 𝐝𝐞𝐚𝐥 with one of the 𝐜𝐮𝐫𝐫𝐞𝐧𝐭 𝐜𝐥𝐢𝐞𝐧𝐭𝐬.

⚫️ Big data ingestion & archiving pipeline

The client wanted to expand its platform features by securely and safely ingesting, preprocessing, and storing TB of data into their platform.

➞ 𝐂𝐨𝐧𝐭𝐫𝐢𝐛𝐮𝐭𝐞𝐝 to the 𝐝𝐞𝐬𝐢𝐠𝐧 and 𝐢𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 of a 𝐝𝐚𝐭𝐚 𝐢𝐧𝐠𝐞𝐬𝐭𝐢𝐨𝐧 𝐩𝐢𝐩𝐞𝐥𝐢𝐧𝐞 that supports 𝐢𝐧𝐠𝐞𝐬𝐭𝐢𝐧𝐠 >𝟓 𝐓𝐁 of data in 𝐨𝐧𝐞 𝐠𝐨 in a reliable, fast, and secure way.
➞ 𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐞𝐝 a scalable 𝐝𝐚𝐭𝐚 𝐩𝐫𝐞𝐩𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐩𝐢𝐩𝐞𝐥𝐢𝐧𝐞 that supports multiple media types (images, videos, audio, 3D objects) and creates small preview files that are cheap to transfer, store and visualize. The 𝐨𝐫𝐢𝐠𝐢𝐧𝐚𝐥 𝐟𝐢𝐥𝐞𝐬 are 𝐚𝐫𝐜𝐡𝐢𝐯𝐞𝐝 on an affordable storage solution 𝐫𝐞𝐝𝐮𝐜𝐢𝐧𝐠 𝐜𝐨𝐬𝐭𝐬 𝐛𝐲 𝐮𝐩 𝐭𝐨 𝟖𝟏%.

I'm currently designing a second brain AI agent.

Here's a high-level architecture (steal it and build your own):

The goal is to create an agent that can reason over everything I’ve ever written, watched, or saved while automatically picking up facts, events, and preferences about me.

The agent should then utilize that memory to assist me in writing, thinking, and creating.

At the foundation of everything happening is GraphRAG.

Then we have 3 steps:

𝟭/ 𝗜𝗻𝗴𝗲𝘀𝘁𝗶𝗼𝗻 = 𝗺𝗲𝗺𝗼𝗿𝘆

All notes, documents, and links flow through a GraphRAG ingestion pipeline:
 
• Extract entities and relationships
• Summarize and embed content
• Store it as a knowledge graph

This gives me three retrieval modes: semantic search, metadata search, and graph traversal.

That’s how the agent can respond to queries like, “Write a section on building a modular MCP server.”

→ Find MCP-related notes
→ Traverse connected ideas 1–2 hops away.

We then orchestrate the entire pipeline using Prefect, which gives us durable workflows with scheduling, retries, monitoring, and failure recovery.

𝟮/ 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹

On top of that memory sits the agent.

It can:
 
• Query the knowledge graph
• Update long-term memory (episodic, semantic, procedural)
• Call external tools like web search or image generation
• Use a smaller fine-tuned “LLM twin” to match writing style

The agent automatically learns my preferences, facts, and events and writes them back into the knowledge graph.

So it never reasons over everything.
It retrieves only the relevant slice of memory.

This inference loop is also orchestrated with Prefect, this time to make agent calls durable: retries, monitoring, and scalable inference.

𝟯/ 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝘃𝗶𝗮 𝗠𝗖𝗣

The agent is exposed as an MCP server using FastMCP (by Prefect).

This lets me interact with it directly from tools like Claude or Cursor, refining ideas conversationally instead of relying on one-shot prompts.

𝗧𝗵𝗲 𝗰𝗼𝗿𝗲 𝗶𝗱𝗲𝗮 𝗶𝘀 𝘁𝗵𝗶𝘀:

Effective agents are built with structured memory + reliable orchestration.

P.S. Have you tried to build a Second Brain Agent yet?

Senior AI Engineer • Founder @ Decoding AI • Author @ LLM Engineer’s Handbook ~ I ship AI products and teach you about the process.

If you’re building AI agents in 2026,

You need a faster way to make architecture decisions...

95% of agents never hit production.

And it's because teams pick the wrong architecture from day one.

They use workflows when they need an agent.
Multi-agent when a simple pipeline would have worked.
Autonomy where determinism was enough.

So we created something to help with that...

Introducing, Building AI Agents Architecture Cheat Sheet.

It was created by Louis-François Bouchard from Towards AI, Inc. in collaboration with me for our Agentic AI Engineering course that's coming this month.

The idea was to help you make the best architecture decisions.

Specifically, here's what it helps you do:
 
• Decide Workflow vs Single Agent vs Multi-Agent based on real constraints
• Avoid unnecessary autonomy that explodes cost, latency, and debugging time
• Apply the Autonomy Test to know when you actually need an agent
• Design thin agents + heavy tools for reliability
• Add validation loops, observability, and human in the loop (HITL) by design
• Pick the minimum architecture that works before scaling complexity

It also includes:
 
• A practical kickoff decision process
• Architecture questions with concrete build outcomes
• Clear signals for when multi-agent systems actually make sense
• Production rules we use when shipping real agentic systems

This is exactly the kind of reference you want before you write any code.

If you want it, here's how to get it...
 
Link: https://lnkd.in/eb_V3ck7

P.S. Huge credit to Louis-François Bouchard and the Towards AI, Inc. team for putting this cheatsheet together.

I noticed something interesting after publishing LLM Engineers Handbook:

AI engineering is still widely misunderstood.

Many people expected the book to be a step-by-step guide to fine-tuning LLMs.

And this makes sense to an extent...

Fine-tuning is the most visible part of the AI stack.

But this is not what makes you an AI engineer.
It's just part of the stack.

In fact, it's a small commoditized part of the snack.

These days, frameworks like Unsloth AI, Axolotl, and TRL from Hugging Face make fine-tuning relatively straightforward.

The real engineering work is about:
 
• Designing data pipelines that reliably collect, clean, and transform data
• Building infrastructure that scales, recovers, and stays observable
• Orchestrating RAG pipelines, agents, and workflows
• Managing datasets, lineage, evaluations, and monitoring in production

In other words, data is the bottleneck.

If you fine-tune on a random dataset and run a cookbook script, you have learned the “hello world” of fine-tuning.

The challenge is knowing what data you need, how to get it, how to structure it, and how to keep it fresh as systems evolve.

So if you want to grow as an AI or LLM engineer, your focus must shift.

Stop asking, “How do I fine-tune this model?”

Start asking, “How do I build a system that feeds the model the right data, at the right time, reliably?”

This is where production systems succeed or fail

Model architecture tweaks and novel training tricks are important, but that work sits closer to research than engineering.

Both matter.
They are just different jobs.

AI engineering is about shipping systems that work, scale, and survive real users and real data.

And once you see that clearly, your learning priorities change completely.

GraphRAG is becoming mandatory for shipping real-world AI agents.

Here’s why:

Shipping reliable AI agents is mostly a data and memory problem.

But the majority of engineers still treat it like a prompting issue.

An agent fails, and the instinctive reaction is:

“Rewrite the prompt.”
“Add more instructions.”
“Increase the context window.”

Sometimes that helps.
Most of the time, it doesn’t.

Because modern agents rarely fail due to a lack of reasoning power...

They fail because they’re forced to reason over too much, too little, or the wrong context.

As systems grow, context fragments:
 
• Knowledge lives across docs, tickets, dashboards, and chat logs
• Ownership and dependencies exist only in people’s heads
• Old workarounds outlive their original intent
• Stale or irrelevant data quietly pollutes the context window

Longer context windows didn’t fix this...
They just made the problem easier to hide.

Performance degrades.
Contradictions creep in.
The agent repeats itself, hallucinates, or acts on outdated assumptions.

This has a name: context rot.

The real questions shouldn't be towards the prompts.

They should be targeted at the architecture:
 
• How do you store organizational knowledge long-term?
• How do you retrieve only what’s relevant right now?
• How do you prevent bad data from contaminating the context window?
• How do you connect memory, ownership, dependencies, and history?

This is where knowledge graphs and GraphRAG become essential.

Instead of dumping documents into a prompt, you model your domain: services, teams, incidents, decisions (and how they relate).

This is exactly what I'll walk through at AI Superstream: Context Engineering by O’Reilly.

Specifically, we'll cover:
 
• When vanilla RAG is not enough
• How knowledge graphs act as long-term memory for agents
• How GraphRAG keeps context windows small but meaningful
• How to design agent memory that scales in real systems

If you’re building agents meant to survive real users, real data, and real time, this layer is non-negotiable.

Prompting is the surface.
Context engineering is the foundation.

📆: 26th February 2026

🔗: https://lnkd.in/exndWkcS

P.S. Seats are limited.

P.S.S. Huge thank you to Angelina Yang for inviting me to this panel and to O'Reilly for making this event possible.

Most production incidents aren’t hard to fix.

But they are hard to understand.

By the time an alert fires, the problem usually isn’t new…

Someone has seen it before.
There’s a workaround.
There’s history.

But that context is scattered across Slack threads, old runbooks, dashboards, and people’s heads.

As systems grow, this becomes the real failure mode.

Not broken code.
Broken memory.

And this is exactly why GraphRAG matters in production systems.

Without graph-based memory, engineers lose time reconstructing what the organization already knows, instead of fixing the issue at hand.

The way out isn’t more alerts or better dashboards…

It’s designing agents that carry organizational memory.

In the latest episode of Decoding AI Magazine, I walk through how to design a Production Engineer agent that reacts to alerts by:
 
• Identifying affected services and teams
• Understanding how failures propagate through dependencies
• Surfacing the context engineers usually spend hours digging for

The key insight is memory.

We use GraphRAG to model services, teams, incidents, and documentation as a connected system the agent can traverse when it matters most.

If you’re building agentic systems for production environments, this is the mental model that holds up at scale.

The full breakdown will go live tomorrow at 1 PM CET.

Subscribe to Decoding AI Magazine to get it delivered to your inbox.

Link → https://lnkd.in/et8fmmBQ

I was a true believer in clean architecture.

I honestly thought it would help me structure my Python AI projects.

This resulted in countless hours wasted obsessively dividing my projects into four folders:
 
• Domain
• Application
• Infrastructure
• Interface

It felt right.
But it was a complete nightmare to maintain.

It wasn’t until I realized that these layers should be virtual concepts that I finally cracked it.

Because Python doesn’t force structure on you.
It gives you freedom.

And that freedom is exactly why AI projects fall apart.

If you don’t impose some structure, you get spaghetti code.
If you impose too much structure, you get ceremony and friction.

Most advice pushes you to one extreme or the other.

Either rigid Clean Architecture, copied from Java, or tool-specific templates that lock you into a framework.

Neither worked for me.

What finally worked was treating clean architecture as a set of boundaries.

The domain became “pure business logic.”
The application became orchestration.
Infrastructure became swappable details.
Interfaces became replaceable entry points.

But none of that needed to be enforced dogmatically in folders.

Once I made that shift, things got simpler.

AI agents became reusable.
Workflows became easier to change.
Switching models or storage stopped rippling through the codebase.

If your Python AI project feels messy and fragile, or clean but painful to work in...

You’re probably applying the right ideas in the wrong way.

Btw, I wrote down the exact mental model and project structure that finally worked for me while building AI agents and workflows.

Sharing it here in case it helps someone else…

Link: https://lnkd.in/eGsU7ww3

P.S. If clean architecture ever made your Python project worse before it made it better, you’re not alone :)

I created an AI agent to write a Substack post (and it went viral).

Nobody even noticed it was AI. The secret?

How we evaluated the agent.

Most people think creating accurate agents is about prompt magic, but that's not the case.

Your evaluation process can make or break everything.

So here's the exact framework we used:

𝟭/ 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗯𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝘂𝘀𝗲 𝗰𝗮𝘀𝗲

You can’t design agents or evals in the abstract.

For us, the requirements were explicit:
 
• 10–20 minute technical articles
• Markdown output
• Clear structure and narrative flow
• Mermaid diagrams
• Grounded research

The agent wasn’t allowed to “be creative.”
It was either correct or incorrect.

𝟮/ 𝗧𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗲 𝗿𝗲𝗾𝘂𝗶𝗿𝗲𝗺𝗲𝗻𝘁𝘀 𝗶𝗻𝘁𝗼 𝗯𝗶𝗻𝗮𝗿𝘆 𝗰𝗵𝗲𝗰𝗸𝘀

We didn’t use vague 1–5 scores.

We wrote yes/no metrics:
 
• Did the structure match expectations?
• Did it follow the storyline?
• Did it stay grounded in provided research?
• Were diagrams, code blocks, and formatting correct?

Binary metrics force clarity.
They also make finding failures easier and debuggable.

𝟯/ 𝗦𝗲𝗽𝗮𝗿𝗮𝘁𝗲 𝗴𝗿𝗼𝘂𝗻𝗱 𝘁𝗿𝘂𝘁𝗵 𝗳𝗿𝗼𝗺 𝗻𝗼𝗻-𝗴𝗿𝗼𝘂𝗻𝗱 𝘁𝗿𝘂𝘁𝗵

Some things can be checked directly.
Others can’t.

So we designed two layers:
 
• With ground truth: direct comparison to references
• Without ground truth: style, input coherence, topicality

Both matter.

One anchors the system.
The other preserves flexibility where it’s needed.

𝟰/ 𝗕𝘂𝗶𝗹𝗱 𝗮𝗻 𝗟𝗟𝗠-𝗮𝘀-𝗮-𝗷𝘂𝗱𝗴𝗲

Each metric had:
 
• A custom prompt
• 20–30 few-shot examples
• A binary decision
• A written reason

Why?

It tells you why something failed, not just that it did.

𝟱. 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗲 𝘁𝗵𝗲 𝗲𝘃𝗮𝗹𝘂𝗮𝘁𝗼𝗿

We labeled another 20–30 samples ourselves.

Human vs model agreement became the metric.

Once that loop was stable, iteration became engineering:
 
• Swap models
• Push changes
• Score outputs
• Debug with confidence

This framework scales to any agent system.

And it’s exactly the kind of thinking we go deep on in the Agentic AI Engineering course.

We’ll be going live in February 🚀

If you want to learn how to:
 
• Build agents with real evaluation layers
• Design LLM-as-judge systems correctly
• Move from “it works” to “it’s reliable”
 
Join the waitlist here → https://lnkd.in/dtxZuZpb
 
P.S. If you’re building agents without evals, you’re in for a bumpy ride.

In 4 days, I will be live on the DataTalksClub podcast, taking live QAs on AI Engineering, Skill Stack, Agents, LLMOps, and How to Ship AI Products.

Together with Alexey Grigorev we will chat about: 

• The principles behind production-grade systems
• How to build reliable agentic workflows​
• ​The LLMOps practices that keep models stable​
• ​​The patterns I've learned from building more than 20 AI applications

Plus answering all your amazing questions!

The podcast is live, virtual and FREE on 26 Jan.

Join here: https://luma.com/fx3wplwn

See you there ✌️

Thank you Alexey Grigorev for inviting me! So excited about this one.

FastMCP is the best way to build or hook to MCP servers nowadays. At agent-to-tools or even agent-to-agent communication (where an agent is exposed as a tool), MCP is the clear winner. That's why I'm super thrilled to see how quickly FastMCP is adapting to this change with their dynamic features included in their freshly baked 3.0 version.

DAY 1: FastMCP 3.0 Beta is live 🚀

We’re kicking off this week with a rebuild of the FastMCP foundation.

The foundation is rebuilt to move beyond simple tool scripts and into composable AI infrastructure.

𝙼𝚊𝚓𝚘𝚛 𝚄𝚙𝚍𝚊𝚝𝚎𝚜 👇
🔌 Provider Architecture: A unified system that replaces duplicated code across features for flexible, dynamic server building.

👤 Dynamic Scoping: Servers can now show different tools to different users from a single endpoint.

🧩 Composite Apps: Combine tools from data, platform, and product teams into one authorized interface without collisions.

🔍  Context Optimization: New features for search, curation, and progressive disclosure to keep agent context windows clean.

💡 Agent Skills: Direct instructions on how and when an agent should use specific tools.

𝙱𝚊𝚌𝚔𝚠𝚊𝚛𝚍 𝙲𝚘𝚖𝚙𝚊𝚝𝚒𝚋𝚒𝚕𝚒𝚝𝚢: The core Pythonic API remains the same. This rebuild adds the infrastructure required for dynamic environments without breaking your current v2 servers.

🚀 Install the beta: 𝐩𝐢𝐩 𝐢𝐧𝐬𝐭𝐚𝐥𝐥 𝐟𝐚𝐬𝐭𝐦𝐜𝐩==𝟑.𝟎.𝟎𝐛𝟏
💬 Join the Discord: https://lnkd.in/gVH4jZvE
📖 Docs: https://lnkd.in/dCH4cFmA
📝 Announcement Blog: https://lnkd.in/gWx24H_r
✨ What's New Blog: https://lnkd.in/ghQDjqQF

𝒪𝓃ℯ 𝓂ℴ𝓇ℯ 𝓉𝒽𝒾𝓃ℊ.

FastMCP is how you BUILD MCP servers.

Tomorrow we're showing you how you take them to a whole new level. 🌅

taplio