I create content about deep learning | The DevRel and Deep Learning Dude

I don’t wanna hop on call to learn about or see a demo of what you’re building. Too busy hacking on VLMs, curating multimodal datasets, and contributing to the open source community. 

Harpreet Sahota 🥑

It was great to be back in Boston last week and host another awesome meetup!

Thank you for everyone who showed up, our awesome lineup of speakers for taking time out of their schedules for delivering such informative talks: Anku Rani Hanhui Wang Sage Simhon Sharut G. 

And for Voxel51 for sponsoring!

If you missed the event, checkout the comments for where you can find the recordings of the talks

🤖 Hacker-in-Residence @ Voxel51| 👨🏽‍💻 AI/ML Engineer | 👷🏽‍♀️ Technical Developer Advocate | Learn. Do. Write. Teach. Repeat.

No alternative text description for this image

i built a ComfyUI-inspired visual, node-based workflow editor for FiftyOne

now you can build dataset curation, analysis, and model evaluation pipelines by connecting nodes on a canvas, without writing code

give it a try and let me know what you think, completely open source and all you have to is `pip install fiftyone` and follow the instructions on the readme

everyone is hype about gtc, but the 3d vision conference is this week and i'm much hype for that

i made a repo and a dataset to explore the papers

checkout the repo here:
 https://lnkd.in/gRFz6Kb7

here's a dataset that you can use to explore the papers:
 https://lnkd.in/g2RA87fD

it's been a long, long while since i've done a live stream on linkedin, but it's happening in t-mius 8 minutes!

if you're interested in learning about how to work with video datasets using open source tools, open data, and open models...this is the session for you!

Join us on Feb 26 for Exploring Video Datasets with FiftyOne and Vision-Language Models. Register for the Zoom: https://lnkd.in/gCk7gGPi

Video is the hardest modality to work with — more data, more complexity, and annotation workflows that don’t scale. This hands-on session by Harpreet Sahota 🥑 shows how to understand what’s inside massive video datasets without manually watching thousands of clips.

You’ll learn how to:
🎬 Navigate and explore video datasets in the FiftyOne App and understand dataset structure
🔎 Compute embeddings with Qwen3-VL to enable semantic search, zero-shot classification, and clustering
📝 Generate video descriptions and localize events using vision-language models like Qwen3-VL and Molmo2
🧭 Visualize patterns and trends through embedding projections in FiftyOne
📏 Evaluate model outputs against Action100M’s hierarchical annotations

Perfect for ML engineers, data scientists, and computer vision teams working with large-scale video data who want practical workflows for exploration, curation, and debugging.

#computervision #machinelearning #ai #datascience #videoanalytics #visionlanguagemodels #mlops #opensource

Derek Van de Streek, Swati Bannore Jain and 11 other attendees

Feb 26 - Exploring Video Datasets with FiftyOne and Vision-Language Models

excited to be back in stuttart! if you're in the area and want to speak at the meetup on april 21st, send me a dm

Chief Community Officer @ Voxel51 | Startup Advisor

Are you based in the Stuttgart area? Consider speaking at the upcoming April 21 - Stuttgart AI, ML and Computer Vision Meetup: https://voxel51.com/speak

#computervision #ai #artificialintelligence #machinevision
#machinelearning #datascience #opensource #stuttgart

i've learned a lot from Eric Ma over the years, from the first time I saw one of his talks on principled data science workflows to talks he's given for my communities over the years, and his latest work on getting the most from coding agents.

check out his next talk, and support the SciPy community as well!

Together with my teammates, we solve biological problems with network science, deep learning and Bayesian methods.

As a data scientist, how much of your week is spent on work a coding agent could do?

I once planned multiple days of work for a machine learning experiment last year. This was an experiment that I had put off for an entire quarter because, as a team lead, I didn't have the time to block out the 3-4 afternoons that I thought I'd need. 

Instead, I described the problem to Claude Opus 4.5 via OpenCode, and it wrote the code, ran the training, found issues, switched strategies, and finished the whole thing in 20 minutes, complete with a written report drafted *with* figures referenced.

The data scientists who figure this out now will have an edge.

I'm teaching a workshop on how to do this flavour of agentic data science to raise funds for SciPy 2026's Financial Aid Program. You'll learn:

- Prescriptive prompting for experiments
- Project structure for agents
- Logging and verification strategies to enable automated control
- Report writing with AI
- Staying in control during EDA
- The minimalist version first

These are the same techniques I use in my personal projects and my teammates to compress data science timelines from days to minutes. I think you'll have a chance to learn something good while doing something good!

Details:

- Date: March 27 & April 3, 2026
- Time: 2:00-4:00 PM EST
- Venue: Google Meet (calendar invite will be provided)
- 40 spots available
- $1,000 minimum donation, paid directly to NumFOCUS (tax-deductible), student pricing: $200!

Video recordings available if you can't attend live.

Sign up here: https://lnkd.in/eFjc36Tb. I'm looking forward to seeing what you'll build with these techniques! 

P.S. If you can't make it but share it with others who have the means and would benefit from the workshop, I would deeply appreciate it!

Open article: Agentic Data Science Workshop | SciPy 2026 Fundraiser by agent-assisted-data-science.vercel.app

Agentic Data Science Workshop | SciPy 2026 Fundraiser

We just released a new AWS workshop that I’m really excited to finally share 👇
https://lnkd.in/eiNprTd7

This came from something I keep seeing across autonomy, robotics, and computer vision teams lately.

Most teams are not blocked by models anymore.
They are blocked by data and iteration speed.

So we wanted to show what a practical workflow actually looks like:
🚗 generate synthetic driving data
 🧠 auto label using foundation models
 🔧 fine tune a small vision language model
 ☁️ deploy and iterate using AWS SageMaker
 👀 inspect and improve everything inside FiftyOne

The goal was simple.

Build something real that teams could follow end to end.

Huge credit to Ryan Patrick Sweeney, Mark Vinciguerra, and Derick Thompson for driving this forward, along with great collaboration between Amazon Web Services (AWS), NVIDIA Omniverse, and Voxel51 🙌

Also excited that Voxel51 is now live on AWS Marketplace, which makes getting started a lot easier for teams working on computer vision and robotics workflows!

Personally, I think this is how a lot of Physical AI systems will get built going forward.

Faster loops between data, models, and deployment instead of long waiting cycles.

If you’re working on robotics, AV, or multimodal systems, I’d genuinely love to hear what you’re building.

More coming soon 🚀

#PhysicalAI #ComputerVision #Robotics #MultimodalAI #SyntheticData #AWS #MachineLearning #AutonomousSystems

Open article: GitHub - aws-samples/sample-finetuning-small-vlms-synthetic-data by github.com

Contribute to aws-samples/sample-finetuning-small-vlms-synthetic-data development by creating an account on GitHub.

GitHub - aws-samples/sample-finetuning-small-vlms-synthetic-data

I’m so happy to have my boi Wiz present at one of our meets!

AI @ NVIDIA - 🏗️ Build | 🚢 Ship | 🚀 Share

View Chris "The Wiz 🪄" Alexiuk’s profile

skills.md - you've seen it, you've probably used it (I assume if you're following this LinkedIn account, you're someone who uses tools like OpenCode/etc.

I'm excited to be yappin' about skills at an upcoming Voxel51 / Hugging Face virtual event with a line up that boggles the mind: 

Ben Burtenshaw, Daniel Ávila Arias, and AdonAI Vera.

My presentation will focus on something near and dear to my heart: Skills as documentation. 

The days of documentation aren't over - humans, and agents, still need to understand how things work - but they're changing quickly. Not everyone wants to read a thorough getting started guide that may be built for a specific system (and if it's not, it'll be ~2K words long) Sometimes, you just want to clone a repository - and get cooking. 

Skills help bridge that gap - context for LLMs, but tailored for a human experience. 

Can't wait for the event, come on out and ask any skill questions you have!

looking forward to hosting this workshop tomorrow, join me!

Join Harpreet Sahota 🥑 on Feb 26 for a virtual workshop to learn how to use Facebook's Action100M dataset and FiftyOne to build an end-to-end workflow - https://lnkd.in/gFvwbQni

What you'll learn:

* Navigate and explore video data in the FiftyOne App, filter samples, and understand dataset structure

* Compute embeddings with Qwen3-VL to enable semantic search, zero-shot classification, and clustering

* Generate descriptions and localize events using vision-language models like Qwen3-VL and Molmo2

* Visualize patterns in your data through embedding projections and the FiftyOne App

* Evaluate model outputs against Action100M's hierarchical annotations to validate what the models actually capture

By the end of the session, you'll have a reusable toolkit for understanding any video dataset at scale, whether you're curating training data, debugging model performance, or exploring a new domain.

***********

Want to build better computer vision models? FiftyOne is an open source toolkit from Voxel51 (our Meetup sponsor) that helps you curate datasets, evaluate model performance, visualize embeddings, catch annotation errors, and eliminate duplicate images—all in one place.

“pip install fiftyone” is all it takes to get started - https://docs.voxel51.com/

#computervision #ai #artificialintelligence #machinevision #machinelearning #datascience #opensource

Richmond Alake is a highly experienced Machine Learning Architect and Engineer with over five years of expertise in the field. He specializes in Computer Vision and Deep Learning and has a proven track record of successfully developing and integrating deep learning models to solve a wide range of problems, such as motion detection, object detection, and pose estimation. Throughout his career, he has worked with a diverse range of clients, including large conglomerates, financial institutions, and small startups. In addition to his professional work, Richmond also serves as an AI advisor to a number of startups in the UK and the US.

With a background in building websites and mobile applications, Richmond is a firm believer in using technology to solve everyday problems. He has extensive knowledge of Machine Learning and has written over 200 articles on the subject, gaining over a million views. He was recognized as one of Medium's top AI writers in 2020/2021 and has collaborated with companies such as O'Reilly, BuiltIn and Nvidia to develop effective educational and informative learning materials on AI.

Currently, Richmond Alake is a Machine Learning Architect at Slalom Build UK. As the first hire of the machine learning practice in the UK division, he is responsible for helping organizations move from machine learning research to productionization and assisting maturing organizations in promoting AI models into existing infrastructure to drive commercial and business value. His main role as an ML Architect is to assist organizations in developing and maintaining machine learning pipelines by implementing MLOps principles, techniques, and tooling. He is well-versed in Feature Stores and has conducted internal training for Data Engineers, Data Scientists, and ML Engineers.

Richmond Alake

Day 87/100 of Agent Memory 🧠

Memory is the battleground of 2026.
But most teams are still treating it as an afterthought. 

Let me break down why that's a problem, depending on where you sit.

1️⃣ If you're an AI Developer:
The infrastructure behind agent memory is not trivial. You need to think about how you ingest, encode, store, retrieve, and forget. That last one doesn't come naturally when you think about context. It does when you think about memory.
File systems are easy to build. But the moment you need security, concurrency, and real-time retrieval at scale, you've just started building a database on top of a file system. Skip that step. Start with the right infrastructure.

2️⃣ If you're a Technical AI leader:
The gap between an agent prototype and a production agent is almost always a memory problem. Stateless calls are elegant for simple tasks. But real workflows unfold across many steps, require context from previous actions, and depend on multiple tool outputs over time. The question you should be asking your team right now is not "does our agent work?" It's "what does our agent remember...and how?"

3️⃣If you're a Product Leader:
Memory is becoming a moat. An agent that remembers your users, their preferences, their workflows, and their history compounds in value over time. The difference between a stateful agent and a stateless one, at scale, is not just a feature gap; it becomes a retention gap, a trust gap, and eventually a revenue gap. The organizations investing in memory infrastructure today are building something competitors cannot easily replicate.

I went deep on all of this in my latest podcast episode on the Tessl podcast with Simon Maple. We talk about the lifecycle of agent memory, the file system vs database debate, and where continuous learning takes us next.

Link in the comments 👇

#100DaysOfAgentMemory #AgentMemory

The AI Stack Engineer | AI Strategist and Educator

Day 95/100 of Agent Memory  🧠

Yesterday, I wrote about Anthropic's 1M context window going generally available at standard pricing and what it does and does not change for memory engineering. 

But one thing I forgot to mention that Agostino Calamia brought to my attention is Latency.

Processing 900K tokens costs the same per token as processing 9K tokens. 
It does not take the same time.

Prefill latency, the time the model spends reading and processing your input before generating a single output token, scales with context length. 

At 1M tokens, you are looking at prefill times that can run into minutes before you see your first response token. Anthropic's announcement addresses pricing. 

It says nothing about processing speed 🤔

This matters more than it sounds for agent systems specifically. The workloads where 1M context is immediately and genuinely practical are the ones where latency is not the primary constraint: batch document analysis, offline research pipelines, contract review, codebase audits. Load everything in, wait, get a high-quality result. That workflow is now significantly cheaper and meaningfully more reliable than it was a week ago.

Real-time agentic loops are a different conversation. 

An agent operating within a user-facing product, making sequential tool calls, waiting for responses, and iterating toward an answer, is sensitive to every added second of prefill time. 

The obvious mitigation here is prompt caching. 

If the bulk of your context is stable across turns, whether that is a system prompt, a large document set, or a populated knowledge base, you can cache the KV state of that prefix and avoid recomputing it on every request. 

Anthropic supports prompt caching, and at 1M context, it becomes less of a nice-to-have and more of an architectural necessity. 

#100DaysOfAgentMemory #AgentMemory #MemoryEngineering

Day 93/100 of Agent Memory 🧠

The more I keep building memory-aware agents, the more I refine the mental model of where memory operations fit within the harness, also solidifying the principles of memory engineering.

Halfway through building the memory engineering notebook(see comments), I had what looked like a complete system. Six memory types backed by Oracle AI Database, vector search across all of them, a tool-calling loop, entity extraction, and workflow persistence. 

The architecture made sense on paper, and the code ran without errors.

Then I tested it with a multi-turn conversation, and the agent started behaving in ways I could not explain. Forgetting context from earlier in the thread, producing inconsistent entity extractions, and returning incomplete conversation history that did not match what I had written to the database. 

The actual problem turned out to be that I had left too many memory operations inside the model's decision boundary...I was trying to go full autonomous here.

During a long tool-call loop, the model was quietly skipping writes it had deemed unnecessary, making state-management decisions I had never asked it to make.

The fix enforced a cleaner agent-engineering principle: the harness owns state integrity, and the model owns strategy within those boundaries.

Context assembly, message persistence, tool output offloading, workflow recording, and entity extraction all became programmatic because the correct answer to "should this happen right now" was always yes. 

If you leave answer persistence to the model, it will occasionally decide the answer is not worth saving. Confidence and accuracy are different things, and the harness cannot afford to conflate them.

Context compaction through summarisation sits on both sides of that line. 
You can make it agent-triggered, letting the model judge when a thread is long enough to warrant compaction. You can make it programmatic, firing automatically once unsummarized message count crosses a threshold. 
Both are valid depending on how predictable your conversation lengths are. 

What the harness should never do is leave compaction unaddressed, because an agent with no compaction strategy is simply accumulating tokens until something breaks.

Web search and loop termination stayed agent-triggered without ambiguity, because no static rule can replicate what the model is reasoning about in a specific turn. The moment the boundary was explicit and enforced, the agent stabilized.

The full technical notebook: https://lnkd.in/ebjVRni8

#100DaysOfAgentMemory #AgentMemory #MemoryEngineering #OracleAIDatabase

Day 94/100 of Agent Memory 🧠

Anthropic announced last week that the full 1M context window is now generally available for Claude Opus 4.6 and Sonnet 4.6 at standard pricing, no long-context premium. 

A 900K-token request costs the same per token as a 9K one. 
The internet's response was predictable: RAG is dead, context and memory engineering are unnecessary, just put everything in context. 

Again, for the 100x time. RAG is not dead. 

Stuffing an entire knowledge base into every request is expensive, even at flat pricing; it cannot incorporate data updated after the prompt was assembled, and it asks the model to reason over everything rather than retrieve what is relevant. 

The 1M window raises the threshold at which you need RAG. 
It does not eliminate the need to understand retrieval techniques and optimization approaches.

The benchmark numbers are worth reading carefully rather than just citing. Opus 4.6 scores 78.3% on MRCR v2 at 1M tokens, the multi-needle retrieval test that hides eight specific facts across a million-token prompt. Gemini 3.1 Pro also reaches 1M, and drops to 25.9% recall accuracy when it gets there. 

A LLM's context window, which degrades to coin-flip retrieval accuracy at full length, is not a 1M context window in any meaningful sense. And in mission-critical AI workloads, losing even a percentage point in accuracy can be significant.

Here is the insight the announcement actually contains, and it is a good one: loading a full codebase, a complete contract set, or the entire trace of a long-running agent without chunking or summarisation is now practical in a way it was not before.

With this increase in accuracy and context window, AI Developers can easily pass in unsummarised context and offloaded tool logs into the context window when needed, without worrying about significant accuracy loss. But emphasis on the "when needed".

Less overhead from chunking, retrieval, and re-stating context across turns is great. And the teams that will benefit from this are those already seeing efficiency gains and thinking carefully about what belongs in context and why. 

The economics of harness design do shift in concrete ways:
1️⃣ Programmatic compaction triggered at a token threshold becomes less urgent when that threshold is five times further away
2️⃣ Agent-triggered summarisation becomes genuinely discretionary rather than a programmatic implementation after every few turns

The full technical notebook on memory engineering with Oracle AI Database is in the comments.

#100DaysOfAgentMemory #AgentMemory #MemoryEngineering #OracleAIDatabase #AIAgents

Day 97/100 of Agent Memory 🧠

I am hoping more people realise this now more than ever.

Harrison Chase, creator of LangChain and one of the most influential voices in the agent engineering space, said it plainly this week: "memory is a moat."

Memory is what turns an AI system into an AI asset. If your organisation has not internalised that yet, the gap is already widening.

The real IP is institutional memory (this can be both semantic and procedural memory, depending on the memory unit characteristics)

The quoted reply in the post says it better than I could: the internal agent that knows your data models, your API quirks, your edge cases from three years of production incidents, that is the moat. 

The OSS framework is a commodity. 
The memory it runs on is the actual competitive advantage.

If you have an AI leader, especially one associated with data, who still does not understand the unique benefits of treating agent memory as a first-class citizen of AI product and agent engineering, both architecturally and organisation-wide, do two things:

1️⃣ Jump ship.
2️⃣ Enrol in my 2-day bootcamp on AI Memory Management.

Lol, joking. 
Kind of.

Every major player in the agent stack has now planted a flag on memory. And many are implementing it as a foundational capability.

Long story short memory is a moat and that you need to own it.

I started writing about agent memory because I believed it was the most under appreciated dimension of agent engineering. I believed memory was not a nice-to-have feature, but an architectural discipline.

The field has moved considerably in that time. 
The argument no longer needs to be made.
The evidence is making it for us.

If you are serious about owning Memory, then look to adding Oracle AI Database to your infra as the unified memory core for your AI Agents

#100DaysOfAgentMemory #AgentMemory #AIAgents

Day 90/100 Of AgentMemory

This one is fitting for a Day 90
Yann LeCun just raised $1.03 billion for AMI Labs.

I read their mission statement carefully:
"Building AI systems that understand the real world, have persistent memory, can reason and plan, and are controllable and safe."

Persistent memory is second on the list, and it's not an afterthought.

LeCun is arguably one of the longest-serving researchers in the history of this field. Turing Award winner. Decades at the frontier. And the problem he has chosen to put a billion dollars behind includes the same memory challenge that has been unsolved since 1989, when catastrophic interference was first identified: neural networks trained on new tasks would completely overwrite what they had previously learned.

That was the same era when LeCun was doing foundational work on convolutional networks (LeNet, the grandfather of AlexNet). The memory problem was known then. It remained unsolved at the architecture level for decades.

The field spent years trying to solve it inside the model. 
I think RNNs and LSTMs were the best attempt, using hidden states and gating mechanisms to carry memory across sequences. They partially worked.

So, yeah, what we are seeing now is an old problem with a trendy name being addressed with a different approach: persistent, retrievable memory that lives entirely outside the model weights, in databases that have been solving storage and retrieval for decades. 

LLMs finally have the comprehension and reasoning to make that external memory useful at inference time in ways earlier architectures could not.
I have said throughout this series that agent memory is the last battleground for every player in the agent stack. 

AMI Labs just put a billion dollars behind that thesis.

But I bet their approach will lie solely within the model, whereas I personally think we will need more of a compound system approach for Agent memory...maybe I'm wrong...or maybe part of that funding is to build a database 🤔

Predictions aside, here is what I think comes next.
The agent memory phase is beginning to shift.
The next frontier is continuous learning. 

Not just retrieving what an agent was told, but also agents that update what they know from experience, without forgetting what they already learned. The gap between external memory stores and true continual learning is where the next area of research is going to play out.

The Oracle AI Database team will be providing practical guidance as this shift happens. Because the distance between research and production is where most organisations get lost, and practical infrastructure built on decades of database engineering has a significant role to play in closing it.

Let's start the shift from Agent Memory to Continual Learning...

#AgentMemory #ContinualLearning #OracleAI #AI #100DaysOfAgentMemory

Day 91/100 of Agent Memory 🧠
I've been talking to a lot of startup founders (see the comments section), enterprise architects, and developers lately. One thing we all agree on: the agent memory landscape is fascinating.

It's one of the few spaces where everyone has a seat at the table. 

Research labs are solving agent memory. 
Developers are solving agent memory. 
Startups are solving agent memory.

Nearly every key player in the agent stack is staking a claim to it.

But not everyone will win.

There's a silent war happening in 2026, and not the headline AI war. The one underneath it. The agent memory war. And two things will determine who comes out on top: infrastructure and talent.

1️⃣ Infrastructure.
Agent memory is not the destination. Think of it as a door. And that door opens into a room called continual learning, the ability for AI systems to persistently update and adapt over time based on new experiences. Most of what we call "agent memory solutions" today relies on in-context learning. Clever retrieval, smart prompting, context stuffing. Useful, but limited.

Real learning happens in the model weights and parameters. That's where you cross the threshold from memory retrieval into genuine continual learning.

Over the coming months, I'll be mapping that transition in detail, showing how agent memory connects to and enables continual learning, and what developers need to understand to get there.

2️⃣ Talent
Agent memory is easy to understand. It's even easier to implement...just look at startups that have raised over the years. But in a large organization It is not as easy to execute. 

Execution means knowing which memory types apply to which agent architectures. It means directing teams on what to build, how to talk about it, and how to match memory patterns to real use cases. Different agents, different application modes, different memory requirements.

What I keep seeing in some organizations: someone hears "agent memory," spins up a deck, ships a half-baked demo, and calls it solved.

That is a direct path into the 95% of AI projects that never reach production.

If someone walks up to you with an agent memory pitch and it's all slides and no infrastructure, that's a red flag 🚩(and I love a good slide)

Oracle AI Database is the agent memory core. It's what makes agent memory work in production. Oracle Cloud is the platform for continual learning. It's the arena where agents don't just retrieve, they adapt and keep learning.

Going to share more learnings on this soon.

#100DaysOfAgentMemory #AgentMemory #ContinualLearning #MemoryEngineering

taplio

The Battle

Harpreet Sahota 🥑

Richmond Alake

Followers

Posts

Engagements

Likes

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance

Performance ScoreRichmond Alake's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Harpreet Sahota 🥑

Richmond Alake

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Harpreet Sahota 🥑's LinkedIn Performance

Performance ScoreRichmond Alake's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreHarpreet Sahota 🥑's LinkedIn Performance