Aurimas Griciūnas

Senior Solutions Architect

Follow me to learn about Data Engineering and ML Systems | Author of SwirlAI Newsletter | Public Speaker

Tired of all the noise surrounding AI space in recent years? Having a hard time keeping up with all of the news in the field and distinguishing what is really important? Follow me and subscribe to my newsletter to get actionable insights on building production ready AI Systems and keep track of only the important signal of what is happening in the AI world.

With over a decade of experience in Data Science, AI/ML Engineering, and MLOps, I’m also dedicated to helping practitioners level up their skills in MLOps, Data Engineering, Machine Learning, and related fields. I share my knowledge in a clear and accessible way to make learning straightforward and actionable.

If you want to partner with SwirlAI and get your product in front of 40K+ AI/ML/Data Engineers and Leaders via our Newsletter or 135K+ AI/ML/Data Engineers and Leaders via LinkedIn, contact me directly.

Head of Product

Leading Product Strategy.

Fusion of 𝗥𝗔𝗚 (Retrieval Augmented Generation) and 𝗖𝗔𝗚 (Cache Augmented Generation). How can you benefit from it as AI Engineer?

Let’s see what it looks like and what additional considerations should be taken into account.

Here are example steps to implement CAG + RAG architecture:

𝘋𝘢𝘵𝘢 𝘗𝘳𝘦𝘱𝘳𝘰𝘤𝘦𝘴𝘴𝘪𝘯𝘨:

𝟭. We use only rarely changing data sources for Cache Augmented Generation. On top of the requirement of data changing rarely we should also think about which of the sources are often hit by relevant queries. Once we have this information, only then we pre-compute all of this selected data into a KV Cache of the LLM. Cache it in memory. This only needs to be done once, the following steps can be run multiple times without recomputing the initial cache.
𝟮. For RAG, if necessary, precompute and store vector embeddings in a compatible database to be searched later in step 4. Sometimes simpler data types are enough for RAG, a regular database might suffice.

𝘘𝘶𝘦𝘳𝘺 𝘗𝘢𝘵𝘩:

We can now utilise the preprocessed data.

𝟯. Compose a prompt including user query and the system prompt with instructions on how cached context and retrieved external context should be used by the LLM.
𝟰. Embed a user query to be used for semantic search via vector DBs and query the context store to retrieve relevant data. If semantic search is not required, query other sources, like real time databases or web.
𝟱. Enrich the final prompt with external context retrieved in step 4.
𝟲. Return the final answer to the user.

𝘚𝘰𝘮𝘦 𝘊𝘰𝘯𝘴𝘪𝘥𝘦𝘳𝘢𝘵𝘪𝘰𝘯𝘴:

➡️ Context window is not infinite and even while some models boast enormous context window sizes, the needle in the haystack problem has not yet been solved so use available context wisely and cache only the data you really need.

✅ For some business cases, specific datasets are extremely valuable to be passed to the model as cache. Think about an assistant that has to always comply with a lengthy set of internal rules stored in multiple documents. 
✅ While CAG has been popularised for Open Source just recently, it is already viable for some time via Prompt Caching features in OpenAI and Anthropic APIs. It is really easy to start prototyping there.
✅ You should always separate hot and cold data sources, only use cold (data that changes rarely) in your cache, otherwise the data will go stale and the application will go out of sync.

❌ Be very careful about what you cache as the data will be available for all users to query.
❌ It is very hard to ensure RBAC for cached data unless you have a separate model with its own cache per role.

Learn how to apply the technique in my End-to-end AI Engineering Bootcamp. Next cohort kicking off on January 12th: https://lnkd.in/dagWE5r3

🎁 Code LASTCHANCE for 10% off.

Founder @ SwirlAI • Ex-CPO @ neptune.ai (Acquired by OpenAI) • UpSkilling the Next Generation of AI Talent • Author of SwirlAI Newsletter • Public Speaker

𝗠𝗖𝗣 plus 𝗔𝟮𝗔, here is how they complement each other 👇

Protocol wars continue to rage, let's understand how Googles A2A (Agent2Agent) protocol is different from MCP and how they complement each other (read till the end).

𝘔𝘰𝘷𝘪𝘯𝘨 𝘱𝘪𝘦𝘤𝘦𝘴 𝘪𝘯 𝘔𝘊𝘗:

𝟭. MCP Host - Programs using LLMs at the core that want to access data through MCP.

❗️ When combined with A2A, an Agent becomes MCP Host.

𝟮. MCP Client - Clients that maintain 1:1 connections with servers.
𝟯. MCP Server - Lightweight programs that each expose specific capabilities through the standardised Model Context Protocol.
𝟰. Local Data Sources - Your computer’s files, databases, and services that MCP servers can securely access.
𝟱. Remote Data Sources - External systems available over the internet (e.g., through APIs) that MCP servers can connect to.

𝘌𝘯𝘵𝘦𝘳 𝘈2𝘈:

Where MCP falls short, A2A tries to help. In multi-Agent applications where state is not necessarily shared

𝟲. Agents (MCP Hosts) would implement and communicate via A2A protocol, that enables:

➡️ Secure Collaboration.
➡️ Task and State Management.
➡️ User Experience Negotiation.
➡️ Capability discovery - similar to MCP tools.

𝗛𝗼𝗻𝗲𝘀𝘁 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀:

❗️ Open protocols for Agent communication are important but they are just a piece of the picture, there will be a need for standards that govern all of the existing protocols and other missing pieces. E.g.

❓ How do we standardise tracing and Observability in multi-agent IOA systems?
❓ How do we retain the identity of the running job of an AI Agent instance if the communication standard is not unified in different parts of the pipeline?
❓ ... 

✅ More on this in my future posts, stay tuned!

Let me know your thoughts in the comments. 👇

A breakdown of 𝗗𝗮𝘁𝗮 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲𝘀 𝗶𝗻 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 👇 And yes, it can also be used for LLM based systems!

It is critical to ensure Data Quality and Integrity upstream of ML Training and Inference Pipelines, trying to do that in the downstream systems will cause unavoidable failure when working at scale.

There is a ton of work to be done on the Data Lake or LakeHouse layer. 𝗦𝗲𝗲 𝘁𝗵𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗯𝗲𝗹𝗼𝘄.

𝘌𝘹𝘢𝘮𝘱𝘭𝘦 𝘢𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵𝘶𝘳𝘦 𝘧𝘰𝘳 𝘢 𝘱𝘳𝘰𝘥𝘶𝘤𝘵𝘪𝘰𝘯 𝘨𝘳𝘢𝘥𝘦 𝘦𝘯𝘥-𝘵𝘰-𝘦𝘯𝘥 𝘥𝘢𝘵𝘢 𝘧𝘭𝘰𝘸:

𝟭: Schema changes are implemented in version control, once approved - they are pushed to the Applications generating the Data, Databases holding the Data and a central Data Contract Registry.

Applications push generated Data to Kafka Topics:

𝟮: Events emitted directly by the Application Services.

👉 This also includes IoT Fleets and Website Activity Tracking.

𝟮.𝟭: Raw Data Topics for CDC streams.

𝟯: A Flink Application(s) consumes Data from Raw Data streams and validates it against schemas in the Contract Registry.
𝟰: Data that does not meet the contract is pushed to Dead Letter Topic.
𝟱: Data that meets the contract is pushed to Validated Data Topic.
𝟲: Data from the Validated Data Topic is pushed to object storage for additional Validation.
𝟳: On a schedule Data in the Object Storage is validated against additional SLAs in Data Contracts and is pushed to the Data Warehouse to be Transformed and Modeled for Analytical purposes.
𝟴: Modeled and Curated data is pushed to the Feature Store System for further Feature Engineering.
𝟴.𝟭: Real Time Features are ingested into the Feature Store directly from Validated Data Topic (5).

👉 Ensuring Data Quality here is complicated since checks against SLAs is hard to perform.

𝟵: High Quality Data is used in Machine Learning Training Pipelines.
𝟭𝟬: The same Data is used for Feature Serving in Inference.

Note: ML Systems are plagued by other Data related issues like Data and Concept Drifts. These are silent failures and while they can be monitored, we can’t include it in the Data Contract.

Let me know your thoughts! 👇

The main 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 𝗧𝘆𝗽𝗲𝘀 👇

It’s no secret that an AI Agent is a piece of software wrapping an LLM or multiple LLMs as a reasoning engine to guide its execution flow. The deployment types for these agents also closely resemble those of software or ML applications, here are the main ones:

➡️ 𝗕𝗮𝘁𝗰𝗵:

- You run your Agents as a part of ETL/ELT Process on a given schedule.
- An agent takes the form of a CLI script most of the time and is scheduled to run periodically via tools like Apache Airflow.
- You consume the accumulated work, connect to external context and tools if needed and save the agent's results to a batch storage.
- Computation results can be loaded into a real-time storage and used for real time applications as well.

➡️ 𝗘𝗺𝗯𝗲𝗱𝗱𝗲𝗱 𝗶𝗻 𝗮 𝗦𝘁𝗿𝗲𝗮𝗺 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻:

- You apply your agentic logic as a part of a Stream Processing Pipeline.
- While work events are continuously piped through your Streaming Data Pipelines, an agent application applies computation on this data and returns it to the system - most likely another Streaming Storage.
- This deployment type can also easily connect to any required external context storage or tools.
- Outputs can be consumed by multiple applications subscribing to the Inference Stream.

✅ This is an ideal deployment type for Ambient Agents, they can listen for events flowing through the system and pick up the ones dedicated to them.

➡️ 𝗥𝗲𝗮𝗹 𝗧𝗶𝗺𝗲:

- You expose your agent as a Backend Service (REST or gRPC).
- This agent retrieves any required external context from external context stores as well as utilising tools available to it.
- Computation can be requested by any application in real time.

➡️ 𝗘𝗱𝗴𝗲:

- You embed your agent's logic directly into the application that runs on a user device.
- This method provides the lowest latency and improves privacy.
- Data in most cases is generated and lives inside of the device significantly improving the security.

Learn how to apply these in practice in my End-to-end AI Engineering Bootcamp: https://lnkd.in/dagWE5r3

Kicking off next Monday, use code LASTCHANCE for 10% discount!

What types of deployments are you mostly working on? Let me know in the comments! 👇

Founder @ SwirlAI • UpSkilling the Next Generation of AI Talent • Author of SwirlAI Newsletter • Public Speaker

𝗔𝗜 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 is a must have in your tool belt as an AI Engineer. 𝗧𝗿𝗮𝗰𝗶𝗻𝗴 sits at the core of it, why is it important?

Tracing and instrumentation of software have been around for decades now. With AI systems resembling regular software even more, we are now moving the practice here as well (with a few key differences).

Let’s look into the process of tracing from a perspective of a naive RAG system.

𝘍𝘦𝘸 𝘥𝘦𝘧𝘪𝘯𝘪𝘵𝘪𝘰𝘯𝘴:

𝘼) An Orchestrator in the GenAI system application is the central piece of software that orchestrates the end-to-end process. Think of apps using LangChain, LlamaIndex or Haystack.
𝘽) Trace is the end-to-end application flow from the entry point till the answer is produced, it is composed of smaller pieces called spans.
𝘾) Span is a smaller piece of the application flow that represents an atomic action like a function call or a database query. They can be sequential, or run in parallel.

ℹ️ As part of span we capture general metadata like start and end time, inputs and outputs of the span. On top of this metadata we track information specific to the GenAI system elements.

What might a trace look like for a naive RAG system?

𝟭. A query that has been submitted to the chat application.
𝟮. The query is embedded into a vector. 

✅ Additional metadata like input token count is persisted with the span so that we can estimate the cost of the procedure.

𝟯. ANN lookup performed against the Vector DB to retrieve the most relevant context. 

✅ Additional metadata about the query is persisted as part of the span together with the retrieved pieces of context and their relevance.

𝟰. A prompt is constructed from the system prompt and retrieved context.
𝟱. The prompt is passed to the LLM to construct the answer. 

✅ Additional metadata about input and output token count is captured together with the span so that we can estimate the cost of the procedure.

𝘞𝘩𝘺 𝘪𝘴 𝘵𝘳𝘢𝘤𝘪𝘯𝘨 𝘰𝘧 𝘎𝘦𝘯𝘈𝘐 𝘴𝘺𝘴𝘵𝘦𝘮𝘴 𝘪𝘮𝘱𝘰𝘳𝘵𝘢𝘯𝘵?

- These applications are usually complex chains, errors can happen in different steps of your application. E.g. Embedding of query is taking longer than expected or you have reached API limits of LLM provider.
- Cost for calling LLM APIs will be variable depending on the length of inputs and produced outputs. You would usually trace this information and analyze it to help forecast expenses.
- GenAI systems are non-deterministic and will deteriorate over time. They need to be evaluated on span level rather than input/output of the entire system so that you can tune each piece separately.
- …

Are you tracing your Agents? Let me know in the comments 👇

Don't get fooled, building a 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗴𝗿𝗮𝗱𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) 𝗯𝗮𝘀𝗲𝗱 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺 is a challenging task. Read until the end to understand why 👇

Here are some of the moving parts in the RAG based systems that you will need to take care of and continuously tune in order to achieve desired results:

𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹:

𝘍 ) Chunking - how do you chunk the data that you will use for external context.

- Small, Large chunks.
- Sliding or tumbling window for chunking.
- Retrieve parent or linked chunks when searching or just use originally retrieved data.

𝘊 ) Choosing the embedding model to embed and query and external context to/from the latent space. Considering Contextual embeddings.

𝘋 ) Vector Database.

- Which Database to choose.
- Where to host.
- What metadata to store together with embeddings.
- Indexing strategy.

𝘌 ) Vector Search

- Choice of similarity measure.
- Choosing the query path - metadata first vs. ANN first.
- Hybrid search.

𝘎 ) Heuristics - business rules applied to your retrieval procedure.

- Time importance.
- Reranking.
- Duplicate context (diversity ranking).
- Source retrieval.
- Conditional document preprocessing.

𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻:

𝘈 ) LLM - Choosing the right Large Language Model to power your application.

✅ It is becoming less of a headache the further we are into the LLM craze. The performance of available LLMs are converging, both open source and proprietary. The main choice nowadays is around using a proprietary model or self-hosting.

𝘉 ) Prompt Engineering - having context available for usage in your prompts does not free you from the hard work of engineering the prompts. You will still need to align the system to produce outputs that you desire and prevent jailbreak scenarios.

And let’s not forget the less popular part:

𝘏) Observing, Evaluating, Monitoring and Securing your application in production!

What other pieces of the system am I missing? Let me know in the comments 👇

If you want to start learning Kubernetes as an AI Engineer, look no further 👇

Some time ago I created a 2 part series on the main concepts you should know as a developer working with K8s. Examples and walkthroughs included - everything runs locally!

In the two articles you will learn:

➡️ Why Kubernetes for AI and Data Engineering.
➡️ General Kubernetes Resources for application configuration.
➡️ How to deploy applications.
➡️ Different ways to deploy applications and when to use which.
➡️ Visual explanation for all of the above.

Since I have resumed the newsletter, there will be many hands-on AI projects utilising Kubernetes for deployment. Be sure to sharpen your skills if you intend to follow! This resource is perfect for that.

Link to the first article: https://lnkd.in/ed2HfHjV

It is completely free 😉

Follow for Your Daily Dose of AI, Software Development & System Design Tips | Exploring AI SaaS - Tinkering, Testing, Learning | Everything I write reflects my personal thoughts and has nothing to do with my employer. 👍

'Exploring AI SaaS - Tinkering, Testing, Learning'

As a technology enthusiast, I find immense joy in delving into new technologies and, more importantly, sharing the knowledge I gain with others every single day.

About Me:

🔸 A dynamic Team & Technical Lead, I thrive in collaborative environments, where I can lead and contribute as an individual to the team's success.

🔸 My expertise spans a wide array of projects, from web-based endeavors as a skilled Full Stack Developer to Java backend projects as a Backend Developer, and even complex Big Data initiatives as a proficient Data Engineer.

🔸 With an in-depth understanding of the end-to-end software development lifecycle, including Agile methodology and CI/CD practices, I consistently deliver efficient and scalable solutions.

🔸 Building distributed systems is my forte, leveraging cutting-edge technologies to create robust and scalable architectures.


Key Technical Strengths:

🔹 My proficiency in Data Structures & Algorithms forms the bedrock of my problem-solving abilities, enabling me to tackle complex challenges head-on.

🔹 I am well-versed in programming languages such as Java and Python, harnessing their power to craft elegant and efficient code.

🔹 My Big Data expertise includes mastery of Spark, Kafka, HDFS, Hive, HBase, and Flume, enabling me to process vast amounts of data with ease.

🔹 I excel in CI/CD practices, utilizing tools like Docker, Kubernetes, Red Hat Openshift, Jenkins, and GIT to ensure seamless and automated workflows.

🔹 I am well-versed in various frameworks, including Spring Boot for Microservices and Spring Batch, alongside my proficiency in Angular for crafting dynamic user interfaces.

🔹 My database prowess extends to MySQL, Oracle, and Cassandra, as well as HBase, ensuring optimal data storage and retrieval.

🔹 As a 5x Certified Microsoft Azure Engineer, I possess comprehensive cloud expertise, enabling me to architect and deploy scalable solutions.

I approach my work with boundless passion, consistently seeking to surpass expectations through continuous learning, exploration, and contributions that make a difference.

I look forward to connecting with like-minded individuals and exploring new opportunities to make a positive impact. 

Let's connect and explore exciting possibilities together! 🤝

Mayank A.

#softwaredevelopment

----

In case you missed my last post on bringing fully open-source MCP-Apps into copilots with CopilotKit and AG-UI, it’s definitely worth a read.
https://lnkd.in/gdwxXx58

Finally, a 100% open-source MCP-Apps infrastructure.

With just a few lines of code, CopilotKit now lets you bring MCP-Apps into your copilot via AG-UI.

Agents can return interactive mini-apps inside your copilot, not just text.

➡️ It works with any agent framework (ADK, LangChain, Mastra, Strands, Pydantic, etc).
➡️ Your app and MCP server communicate bidirectionally.
➡️ AG-UI middleware lets you connect to different types of apps.
➡️ MCP-Apps are treated as frontend tools the agent can invoke.

This brings OpenAI-style MCP-Apps into your own product, without being locked to a single model, platform, or provider.

Demo ↓
https://lnkd.in/drHyUHzG

For context: CopilotKit is an open-source framework for building in-app AI copilots and the team behind the AG-UI protocol.

If you’re building agentic apps, this adds a real UI interaction layer inside your copilot.

Bringing MCP-Apps into copilots is quickly becoming a standard across the ecosystem.
This is part of the agentic future, and where generative UI is headed.

Definitely worth checking out.

All open source ↓
GitHub (28k ⭐): https://lnkd.in/dva6MJED

MCP Apps, A2UI, AG-UI

A big shift is happening for agentic applications.

But what do these three have in common?

They’re all different types of Generative UI.

This a new term that's been coined around and if you look carefully, all of the biggest companies like Anthropic, OpenAI, and Google are shifting towards this type of user interactivity:

From → screens users navigate, To → outcomes users request

Generative UI refers to any user interface that is partially or fully produced by an AI agent, rather than authored exclusively by human designers and developers.

CopilotKit just released a 2026 guide on Generative UI, including the newly released MCP Apps!

I’ve attached it to the post.

The AI ecosystem moves so fast and is often fragmented due to misalignments and lack of day-0 cooperation. But there has been a standout platform that has been lightning fast in keeping up with the ecosystem… CopilotKit

And that’s why I enjoy sharing their resources!

CopilotKit (the developing team behind the AG-UI protocol) is an 𝐨𝐩𝐞𝐧-𝐬𝐨𝐮𝐫𝐜𝐞 framework for building AI copilots directly into apps. Their goal is to be the plug-and-play home for building agentic applications.

Including integrations with:
➡️ All agent frameworks (LangChain, ADK, Mastra, CrewAI, Pydantic, AWS Stands, etc)
➡️ Agentic protocols (MCP, A2A…)
➡️ Generative UI specs (MCP Apps, A2UI, Open-JSON-UI…)
…and so much more

All handled through their Agent-User Interaction protocol!

If you’ve been stuck building your agentic app and Generative UI, start here.
Check out their GitHub: https://lnkd.in/dva6MJED

Generative UI Guide

Our RAG system is 90% accurate.

Sounds great until you realize: that 10% is destroying user trust.

Here's what's happening:
9 out of 10 queries: Perfect answers. Users love it.
1 out of 10 queries: Complete hallucination. Users lose confidence.

The trust problem with LLMs:

> Users don't know which answers to trust.
> One hallucination makes them question everything, even the correct answers.

It's like a doctor who's right 90% of the time. Would you trust them?

Why 90% isn't good enough:

In traditional software:
> 90% uptime is terrible
> 99.9% is standard
> 99.99% is expected

In LLM applications:
> Many teams are at 80-90% accuracy
> Think that's acceptable
> Don't realize it's killing adoption

What's causing the 10%:

After debugging, we found:
40% = Retrieval returned irrelevant documents
30% = LLM ignored good documents and hallucinated
20% = Documents were relevant but contradictory
10% = Prompt was ambiguous

You can only see this with proper observability.

We implemented Opik (open-source LLM observability):
> Traces every retrieval
> Scores document relevance
> Flags hallucinations automatically
> Shows when LLM ignores context
> Catches bad outputs before users see them.

Built by Comet. Works with LangGraph, LangChain, etc. Self-hostable or cloud.

Check the comments for the GitHub Repo.

taplio

The Battle

Aurimas Griciūnas

Mayank A.

Followers

Posts

Engagements

Likes

Performance ScoreAurimas Griciūnas's LinkedIn Performance

Performance ScoreMayank A.'s LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Aurimas Griciūnas

Mayank A.

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Aurimas Griciūnas's LinkedIn Performance

Performance ScoreMayank A.'s LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreAurimas Griciūnas's LinkedIn Performance