Read my just released 📖 The Hundred-Page Language Models Book: https://thelmbook.com

Read my bestselling 📖 The Hundred-Page Machine Learning Book: https://themlbook.com

Read 📖 Machine Learning Engineering: https://mlebook.com

Subscribe to my weekly ✉️ Artificial Intelligence newsletter: https://www.linkedin.com/newsletters/artificial-intelligence-6598352935271358464/ and to my weekly ✉️ Data Science newsletter: https://www.linkedin.com/newsletters/7102511020270608384/

About me:

Ph.D. in Artificial Intelligence, passionate about data, fluent in English, French, and Russian. Solid scientific programming and team leadership skills, with over 20 years of experience working on various computing projects, including several of my own startups.

More than 15 years of hands-on experience in automated data analysis, machine learning, and natural language processing.

Trained a Transformer from scratch and fine-tuned pretrained transformers for various tasks. Built a robot that crawls the internet, finds websites with business-critical information, and retrieves updated information periodically. Developed an enterprise chatbot that doesn’t hallucinate.

Expert in Python and Java with several years of daily design and development experience in big data contexts.

Specialties: machine learning, natural language processing, conversational interfaces (chatbots), information retrieval.

Andriy Burkov

LLMs process text from left to right — each token can only look back at what came before it, never forward. This means that when you write a long prompt with context at the beginning and a question at the end, the model answers the question having "seen" the context, but the context tokens were generated without any awareness of what question was coming. This asymmetry is a basic structural property of how these models work.

The paper asks what happens if you just send the prompt twice in a row, so that every part of the input gets a second pass where it can attend to every other part. The answer is that accuracy goes up across seven different benchmarks and seven different models (from the Gemini, ChatGPT, Claude, and DeepSeek series of LLMs), with no increase in the length of the model's output and no meaningful increase in response time — because processing the input is done in parallel by the hardware anyway.

There are no new losses to compute, no finetuning, no clever prompt engineering beyond the repetition itself. 

The gap between this technique and doing nothing is sometimes small, sometimes large (one model went from 21% to 97% on a task involving finding a name in a list). If you are thinking about how to get better results from these models without paying for longer outputs or slower responses, that's a fairly concrete and low-effort finding.

Read with AI tutor: https://lnkd.in/ene242cx

Get the PDF: https://lnkd.in/e9tbUTNv

PhD in AI, author of 📖 The Hundred-Page Language Models Book and 📖 The Hundred-Page Machine Learning Book

This paper discusses methods to extend LLMs into interactive agentic systems that incorporate planning, tool use, and search.

The authors from Meta, Amazon, Deepmind, and other top private and public labs consider in-context approaches that adapt during inference without altering weights and post-training methods that refine the model through reinforcement learning.

Read with an AI tutor: https://lnkd.in/efJShRmH

Read alone: https://lnkd.in/ecBVandw

Spent some time with RedisVL recently and it's a surprisingly practical tool. It's a Python library that lets you use Redis as a vector database with such features as semantic search, LLM caching, and chat memory. What I like is that it builds on redis-py, so it's not some new ecosystem to learn. If your team already runs Redis, you can add vector search and retrieval to your AI pipeline without introducing another database. Fewer moving parts is always a win.

Here's the link to the docs: https://fandf.co/4qixIzY

Disclaimer: Redis was last week's sponsor of my weekly AI newsletter.

taplio