I make learning Data Science effortless for you | Data Science @ Bosch

Hi 👋🏻
I'm Banias, AI Engineer at Netze BW.

Applying new technologies and methods in Data Science & AI is my passion.

Through my experience and continual learning, I've gained valuable insights and tips that I love sharing with Data Scientists, Machine Learning Engineers and builders like you.

If you're looking for practical advice, helpful resources, and articles on the latest trends or best practices in the field, you're in the right place.

Let's connect and stay updated on the exciting world of Data Science together. Feel free to reach out, read my articles, and let's grow our knowledge and network. :)


Note: My views are my own and don‘t represent my employer.

Banias Baabe

Running a 70B LLM on a single 4GB GPU is possible with 𝗔𝗶𝗿𝗟𝗟𝗠.

𝗔𝗶𝗿𝗟𝗟𝗠 optimizes inference memory usage so you can run big models without quantization, distillation, or pruning.

It runs inference layer by layer, loading only the layer that is needed for the next compute step, then freeing that memory before moving on.​

To make that practical, it first preprocesses the original Hugging Face weights and re-shards them by transformer layer (instead of giant multi‑GB shards), so each load is focused and smaller.​

This matters because disk I/O becomes the bottleneck when you stream weights.

The main thing that still sits in GPU memory is the KV cache for attention, which is much smaller than full model weights for typical context lengths.​

𝗔𝗶𝗿𝗟𝗟𝗠 also adds prefetching so it can overlap loading the next chunk of weights with the current compute, which helps claw back some speed.

And if you want more throughput, it supports an optional “model compression” mode using block-wise quantization of weights to shrink what needs to be loaded from disk, with the project claiming up to ~3x speedup and minimal accuracy loss.

🔗 Link to repo: github(.)com/lyogavin/airllm


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

I teach AI Engineers the concepts and tools they need to stay ahead | AI Engineer @ Netze BW

Stop destroying your documents with 512-token chunks.

Traditional RAG is breaking your tables, splitting your diagrams, and mixing up your model specifications.

I just read a great article from Alex Vesa about a working data modeling approach for complex technical documents.

The problem? Most RAG systems treat a 500-page service manual like a blog post. They chunk blindly. A table that spans three pages gets shredded into meaningless fragments.

Here's what actually works:

✅ Pages, not chunks
Use full pages as your retrieval unit. That's how humans read technical docs anyway.
✅ Three-page sliding windows
Process each page with its neighbors (N-1, N, N+1). Now your system knows when content continues across boundaries.
✅ Treat tables like they matter
Extract structure. Generate metadata. Flatten for search. Keep all three representations.
✅ Multi-vector indexing
Dense embeddings for semantic search. Sparse vectors for exact matches. ColBERT for precision. Combine them.
✅ Domain-specific metadata
Model applicability, component types, cross-page relationships. Not just title and date.

Link ➙ https://lnkd.in/dJ9aJKAB

Thinking about using 𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿 for your next project?

You might want to think again.

Here's the reality of using it in production:
1️⃣ 𝗜𝗩𝗙𝗙𝗹𝗮𝘁 𝗶𝗻𝗱𝗲𝘅𝗲𝘀 𝗻𝗲𝗲𝗱 𝗰𝗼𝗻𝘀𝘁𝗮𝗻𝘁 𝗿𝗲𝗯𝘂𝗶𝗹𝗱𝘀. New vectors throw off your cluster distribution over time.
2️⃣ 𝗛𝗡𝗦𝗪 𝗶𝗻𝗱𝗲𝘅𝗲𝘀 𝗮𝗿𝗲 𝗺𝗲𝗺𝗼𝗿𝘆 𝗵𝗼𝗴𝘀. We're talking 10GB of memory for hours just to build an index for millions of vectors.
3️⃣ 𝗜𝗻𝘀𝗲𝗿𝘁𝗶𝗻𝗴 𝗻𝗲𝘄 𝘃𝗲𝗰𝘁𝗼𝗿𝘀 𝗶𝘀 𝗮 𝗽𝗮𝗶𝗻. With IVFFlat, you have to reindex preiodically. With HNSW, you're looking at graph traversals and memory allocation for every new vector.
4️⃣ 𝗠𝗲𝘁𝗮𝗱𝗮𝘁𝗮 𝗳𝗶𝗹𝘁𝗲𝗿𝗶𝗻𝗴 𝗶𝘀𝗻'𝘁 𝗰𝗹𝗲𝗮𝗿-𝗰𝘂𝘁. Pre-filtering is great for selective filters, but post-filtering can give you incomplete results, and it only gets worse with more filters.
5️⃣ 𝗡𝗼 𝗯𝘂𝗶𝗹𝘁-𝗶𝗻 𝗵𝘆𝗯𝗿𝗶𝗱 𝘀𝗲𝗮𝗿𝗰𝗵. You'll have to figure out how to balance vector similarity with text relevance and normalize scores on your own.

A new extension, pgvectorscale, does fix some of these problems with more memory-efficient indexing and better filter performance. But it isn't supported out-of-the-box on Azure and AWS, so you'd have to manage your own Postgres instance.

𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿 is a great tool, but it's not a silver bullet. Make sure you're ready for the operational overhead before you commit.

🔗 Link to article: https://lnkd.in/d6jSRy2y


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Tired of blocking I/O messing up your async Python apps?

Try 𝗮𝗶𝗼𝗳𝗶𝗹𝗲𝘀 - the game-changer for handling files in asyncio without the headaches.

It's an Apache-licensed library that wraps standard file ops into async magic.

Why you should use it:
• Mirrors Python's built-in file API, so it's super familiar.
• Supports buffered text, binary files, and even tempfiles asynchronously.
• Delegates to a thread pool to keep your event loop flying.


🔗 Link to repo: github(dot)com/Tinche/aiofiles

---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Vector databases are bloated.

What if you could get the same power with 97% less space?

Try 𝗟𝗘𝗔𝗡𝗡, the world's smallest vector index.

It transforms your laptop into a private and powerful RAG system.

You can finally RAG on everything.
-> Your entire file system
-> Millions of local emails
-> 💻 Your entire codebase
-> 💬 All your chat histories

Why is 𝗟𝗘𝗔𝗡𝗡 interesting?
✅ 𝗜𝘁'𝘀 𝗶𝗻𝗰𝗿𝗲𝗱𝗶𝗯𝗹𝘆 𝗹𝗶𝗴𝗵𝘁𝘄𝗲𝗶𝗴𝗵𝘁: Index 60 million documents in just 6GB instead of 201GB.
✅ 𝗜𝘁'𝘀 𝟭𝟬𝟬% 𝗽𝗿𝗶𝘃𝗮𝘁𝗲: Your data never leaves your laptop. No cloud costs, period.
✅ 𝗧𝗵𝗲𝗿𝗲 𝗶𝘀 𝗻𝗼 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝗹𝗼𝘀𝘀: Get the same search quality as heavyweight solutions.

🔗 Link to repo: github(dot)com/yichuan-w/LEANN


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Do you want to identify which lines use the most amount of memory in your Python script?

Try 𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿.

𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿 is a Python module to make a line-by-line analysis of memory consumption for Python functions.

Below you can see how to use memory_profiler within your Python script:

• Decorate the function you want to profile with @𝗽𝗿𝗼𝗳𝗶𝗹𝗲.
• Run the script by passing the option -𝗺 𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿 to load the 𝗺𝗲𝗺𝗼𝗿𝘆_𝗽𝗿𝗼𝗳𝗶𝗹𝗲𝗿 module.

Link to project: pypi(dot)org/project/memory-profiler


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

What, if an API call in your program fails?

Because of, let’s say, an instable internet connection?

This is not so uncommon.

You usually should have some retry mechanism in your program.

With 𝐭𝐞𝐧𝐚𝐜𝐢𝐭𝐲 in Python, this isn’t a problem anymore.

𝐭𝐞𝐧𝐚𝐜𝐢𝐭𝐲 offers a retrying behavior with a decorator with powerful features like:

- Define Stop Conditions
- Define Wait Conditions
- Customize retrying on Exception
- Retry on Coroutines

One of my favorite libraries!

Check it out: github(dot)com/jd/tenacity


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

OpenAI's Responses API isn't what they're telling you it is.

They are talking about how it's "faster, more flexible, easier" than the old chat completions API.

But here's what's really going on.

OpenAI hides their reasoning traces.

Unlike Claude, DeepSeek, or Qwen, you can't see GPT-5's chain of thought.

This creates a massive problem for developers using the stateless chat completions API.

Without access to reasoning traces, you can't pass them back into the conversation.

Result? GPT-5 performs worse than it should in your applications.

The Responses API is the workaround.

✅ It's stateful (OpenAI stores conversation history on their servers)
✅ They maintain the hidden reasoning traces in the backend
✅ They plug it into conversations automatically
✅ They strip it out before sending responses back to you

You get full model capability without ever seeing the reasoning.

Here's the uncomfortable truth:

The old stateless API is actually simpler.

Prefix caching works in both. Parallel tool calling works in both.

There's nothing inherently better about stateful architecture here.

The Responses API only exists because OpenAI won't show you what their models are thinking.

Meanwhile, Anthropic keeps their reasoning traces completely transparent.

If you're building with AI APIs in 2026, this matters.

The question is whether you're comfortable with your provider hiding critical information from you.

Link to article: https://lnkd.in/drZjD9Xj


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

AI makes your team faster.

But it also puts your data at risk.

Files are moving to new tools constantly.

AI models are training on things they shouldn't see.

And old security tools just can't see what is happening.

They are blind to the context that actually matters.

That is why I'm watching Cyberhaven’s launch on February 3.

They are revealing a new platform that combines data security and protection into one tool.

It's built to handle how messy and fast data moves today.

Join the live event to see:
📍 A live demo of this new AI-native platform.
📍 How to spot the few dangerous actions hidden inside millions of normal ones.
📍 Real stories from leaders on why old rules don't work anymore.

Don't wait for a data leak to find the holes in your security.

Grab your spot here: https://lnkd.in/dC-aGjjM

Struggling with slow data processing for AI training?

Try 𝗹𝗶𝘁𝗱𝗮𝘁𝗮, the Python library that transforms and optimizes datasets at scale!

𝗹𝗶𝘁𝗱𝗮𝘁𝗮 offers:
🟢 Streaming large cloud datasets without local downloads
🟢 Accelerating training by 20x with optimized data
🟢 Parallelizing tasks across multiple machines effortlessly
🟢 Securely handling sensitive data with advanced encryption

Perfect for images, text, video, and more.

Check it out: github(dot)com/Lightning-AI/litdata


---
♻️ Found this useful? Share it with another builder.
➕ For daily practical AI and Python posts, follow Banias Baabe.

Banco BICE

Advanced Analytics and AI Lead

Data Scientist

Universidad de Santiago de Chile

Dicté el curso de "Data Science para el Product Manager" en la USACH. Este módulo forma parte del diplomado de Product Manager de la USACH. Mi labor como docente fue dictar, preparar las clases y evaluar a los 25 alumnos que pasaron el módulo de Data Science.

taplio

The Battle

Banias Baabe

Daniel Vargas

Followers

Posts

Engagements

Likes

Performance ScoreBanias Baabe's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Banias Baabe

Daniel Vargas

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Banias Baabe's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreBanias Baabe's LinkedIn Performance