ByteByteGo

Co-authored 4 books:
     - System Design Interview - Volume 1
     - System Design Interview - Volume 2
     - Coding Interview Patterns
     - Machine Learning System Design Interview

- Write weekly ByteByteGo newsletter on system design topics to 1,000,000+ subscribers.
- Grow the ByteByteGo YouTube channel to 1,000,000+ subscribers.

Author & Co-Founder

Twitter

Software Engineer

Apple

Zynga

Oracle

Software Developer

Modeln N

Associate Member of Technical Staff

Follow me for system design & book-writing tips.

Alex Xu is a software engineer and author. His book System Design Interview - An Insider’s Guide is one of Amazon's best-selling books in the “Distributed Systems & Computing” category.  As of December 2020, his book is being translated into 7 different languages (Traditional Chinese, Russian, Japanese, Spanish, Polish, Simplified Chinese, and Korean).

--------------
Newsletter: https://blog.bytebytego.com
Twitter: https://twitter.com/alexxubyte
YouTube: https://www.youtube.com/c/ByteByteGo

Alex Xu

Database Types You Should Know in 2025

There’s no such thing as a one-size-fits-all database anymore. Modern applications rely on multiple database types, from real-time analytics to vector search for AI. Knowing which type to use can make or break your system’s performance.

Relational: Traditional row-and-column databases, great for structured data and transactions.

Columnar: Optimized for analytics, storing data by columns for fast aggregations.

Key-Value: Stores data as simple key–value pairs, enabling fast lookups. 

In-memory: Stores data in RAM for ultra-low latency lookups, ideal for caching or session management.

Wide-Column: Handles massive amounts of semi-structured data across distributed nodes.

Time-series: Specialized for metrics, logs, and sensor data with time as a primary dimension.

Immutable Ledger: Ensures tamper-proof, cryptographically verifiable transaction logs.

Graph: Models complex relationships, perfect for social networks and fraud detection

Document: Flexible JSON-like storage, great for modern apps with evolving schemas.

Geospatial: Manages location-aware data such as maps, routes, and spatial queries.

Text-search: Full-text indexing and search with ranking, filters, and analytics.

Blob: Stores unstructured objects like images, videos, and files.

Vector: Powers AI/ML apps by enabling similarity search across embeddings.

Over to you: Which database type do you think will grow fastest in the next 5 years?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

bytebytego

linkedin.com/company/the-rundown-ai

JSESSIONID

li_at

Talks about #systemdesign, #techinterview, and #bookpublishing

Author of 4 Bestselling Books | Co-Founder of ByteByteGo

Cloudflare is much more than just a CDN and DDoS protection service.

Let’s do a quick comparison of Cloudflare, AWS, and Azure.

Cloudflare has rapidly expanded beyond its traditional CDN roots, launching a suite of modern developer-first services like Workers, R2, D1, and so on. These offerings position it as a serious edge-native alternative to other cloud providers.

Here are the key cloud capabilities that Cloudflare supports:

1 - Edge Compute and Serverless
2 - Object and Blob Storage
3 - Relational Databases
4 - Containers
5 - Sandboxes
6 - Workflows
7 - AI Agents SDK
8 - Vector and AI search
9 - Data Connectivity
10 - AI Infrastructure
11 - Content Delivery Network
12 - DNS
13 - Load Balancing

Over to you: Have you used Cloudflare’s new offerings? What are your thoughts on them?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

No alternative text description for this image

HTTP vs. HTTPS

When you open a website, the difference between HTTP and HTTPS decides whether your data travels safely or in plain sight. Here’s what actually happens under the hood:

HTTP:
- Sends data in plain text, anyone on the network can intercept it.
- The client and server perform a simple TCP handshake: SYN, SYN-ACK, ACK
- Fast but completely insecure. Passwords, tokens, and forms can all be read in transit.

HTTPS (SSL/TLS):
- Step 1: TCP Handshake: Standard connection setup.
- Step 2: Certificate Check: Client says hello. Server responds with hello and its SSL/TLS certificate. That certificate contains the server's public key and is signed by a trusted Certificate Authority. 

Your browser verifies this certificate is legitimate, not expired, and actually belongs to the domain you're trying to reach. This proves you're talking to the real server, not some attacker pretending to be it.

- Step 3: Key Exchange: Here's where asymmetric encryption happens. The server has a public key and a private key. Client generates a session key, encrypts it with the server's public key, and sends it over. Only the server can decrypt this with its private key.

Both sides now have the same session key that nobody else could have intercepted. This becomes the symmetric encryption key for the rest of the session.

- Step 4: Data Transmission: Now every request and response gets encrypted with that session key using symmetric encryption.

Over to you: What’s your go-to tool for debugging TLS issues, openssl, curl -v, or something else?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Karnav P.

Apache Kafka vs. RabbitMQ

Kafka and RabbitMQ both handle messages, but they solve fundamentally different problems. Understanding the difference matters when designing distributed systems.

Kafka is a distributed log. Producers append messages to partitions. Those messages stick around based on retention policy, not because someone consumed them. Consumers pull messages at their own pace using offsets. You can rewind, replay, reprocess everything. It is designed for high throughput event streaming where multiple consumers need the same data independently.

RabbitMQ is a message broker. Producers publish messages to exchanges. Those exchanges route to queues based on binding keys and patterns (direct, topic, fanout). Messages get pushed to consumers and then deleted once acknowledged. It is built for task distribution and traditional messaging workflows.

The common mistake is using Kafka like a queue or RabbitMQ like an event log. They're different tools built for different use cases.

Over to you:  If you had to explain when NOT to use Kafka, what would you say?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Explaining 9 types of API testing. 
 
🔹 Smoke Testing 
This is done after API development is complete. Simply validate if the APIs are working and nothing breaks. 
 
🔹 Functional Testing 
This creates a test plan based on the functional requirements and compares the results with the expected results. 
 
🔹 Integration Testing 
This test combines several API calls to perform end-to-end tests. The intra-service communications and data transmissions are tested. 
 
🔹 Regression Testing 
This test ensures that bug fixes or new features shouldn’t break the existing behaviors of APIs. 
 
🔹 Load Testing 
This tests applications’ performance by simulating different loads. Then we can calculate the capacity of the application. 
 
🔹 Stress Testing 
We deliberately create high loads to the APIs and test if the APIs are able to function normally. 
 
🔹 Security Testing 
This tests the APIs against all possible external threats. 
 
🔹 UI Testing 
This tests the UI interactions with the APIs to make sure the data can be displayed properly. 
 
🔹 Fuzz Testing 
This injects invalid or unexpected input data into the API and tries to crash the API. In this way, it identifies the API vulnerabilities. 

--
We just launched Become an AI Engineer | Learn by Doing: Cohort 2. If you missed Cohort 1, now’s your chance to join us for Cohort 2.

Check it out here: https://bit.ly/3LdaWuw

#AI #AIEngineer #MachineLearning 
.

The AI Agent Tech Stack

1. Foundation Models: Large-scale pre-trained language models that serve as the “brains” of AI agents, enabling capabilities like reasoning, text generation, coding, and question answering.
2. Data Storage: This layer handles vector databases and memory storage systems used by AI agents to store and retrieve context, embeddings, or documents.
3. Agent Development Frameworks: These frameworks help developers build, orchestrate, and manage multi-step AI agents and their workflows.
4. Observability: This category enables monitoring, debugging, and logging of AI agent behavior and performance in real-time.
5. Tool Execution: These platforms allow AI agents to interface with real-world tools (for example, APIs, browsers, external systems) to complete complex tasks.
6. Memory Management: These systems manage long-term and short-term memory for agents, helping them retain useful context and learn from past interactions.

Over to you: What else will you add to the list?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Big Data Pipeline Cheatsheet for AWS, Azure, and Google Cloud

Each platform offers a comprehensive suite of services that cover the entire lifecycle:

1 - Ingestion: Collecting data from various sources
2 - Data Lake: Storing raw data
3 - Computation: Processing and analyzing data
4 - Data Warehouse: Storing structured data
5 - Presentation: Visualizing and reporting insights

AWS uses services like Kinesis for data streaming, S3 for storage, EMR for processing, RedShift for warehousing, and QuickSight for visualization.

Azure’s pipeline includes Event Hubs for ingestion, Data Lake Store for storage, Databricks for processing, Cosmos DB for warehousing, and Power BI for presentation.

GCP offers PubSub for data streaming, Cloud Storage for data lakes, DataProc and DataFlow for processing, BigQuery for warehousing, and Data Studio for visualization.

Over to you: What else would you add to the pipeline?

--
LAST DAY to enroll in Become an AI Engineer Learn by Doing, Cohort 2. If you missed Cohort 1, now is your chance to join us in Cohort 2.

Check it out here: https://bit.ly/3LdaWuw

#AI #AIEngineer #MachineLearning 
.

Things Every Developer Should Know: Concurrency is 𝐍𝐎𝐓 parallelism.
.
.
In system design, it is important to understand the difference between concurrency and parallelism. 

As Rob Pyke(one of the creators of GoLang) stated:“ Concurrency is about 𝐝𝐞𝐚𝐥𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 lots of things at once. Parallelism is about 𝐝𝐨𝐢𝐧𝐠 lots of things at once." This distinction emphasizes that concurrency is more about the 𝐝𝐞𝐬𝐢𝐠𝐧 of a program, while parallelism is about the 𝐞𝐱𝐞𝐜𝐮𝐭𝐢𝐨𝐧. 

Concurrency is about dealing with multiple things at once. It involves structuring a program to handle multiple tasks simultaneously, where the tasks can start, run, and complete in overlapping time periods, but not necessarily at the same instant. 

Concurrency is about the composition of independently executing processes and describes a program's ability to manage multiple tasks by making progress on them without necessarily completing one before it starts another.

Parallelism, on the other hand, refers to the simultaneous execution of multiple computations. It is the technique of running two or more tasks or computations at the same time, utilizing multiple processors or cores within a computer to perform several operations concurrently. Parallelism requires hardware with multiple processing units, and its primary goal is to increase the throughput and computational speed of a system.

In practical terms, concurrency enables a program to remain responsive to input, perform background tasks, and handle multiple operations in a seemingly simultaneous manner, even on a single-core processor. It's particularly useful in I/O-bound and high-latency operations where programs need to wait for external events, such as file, network, or user interactions.

Parallelism, with its ability to perform multiple operations at the same time, is crucial in CPU-bound tasks where computational speed and throughput are the bottlenecks. Applications that require heavy mathematical computations, data analysis, image processing, and real-time processing can significantly benefit from parallel execution.

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

5 REST API Authentication Methods

1. Basic Authentication: Clients include a Base64-encoded username and password in every request header, which is simple but insecure since credentials are transmitted in plaintext. Useful in quick prototypes or internal services over secure networks.
2. Session Authentication: After login, the server creates a session record and issues a cookie. Subsequent requests send that cookie so the server can validate user state. Used in traditional web-apps.
3. Token Authentication: Clients authenticate once to receive a signed token, then present the token on each request for stateless authentication. Used in single-page applications and modern APIs that require scalable, stateless authentication.
4. OAuth-Based Authentication: Clients obtain an access token via an authorization grant from an OAuth provider, then use that token to call resource servers on the user’s behalf. Used in cases of third-party integrations or apps that need delegated access to user data.
5. API Key Authentication: Clients present a predefined key (often in headers or query strings) with each request. The server verifies the key to authorize access. Used in service-to-service or machine-to-machine APIs where simple credential checks are sufficient.

Over to you: Which other API Authentication method have you seen?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Git Merge vs. Rebase vs. Squash Commit! 
. 
. 
What are the differences? 
 
When we 𝐦𝐞𝐫𝐠𝐞 𝐜𝐡𝐚𝐧𝐠𝐞𝐬 from one Git branch to another, we can use ‘git merge’ or ‘git rebase’. The diagram below shows how the two commands work. 
 
𝐆𝐢𝐭 𝐌𝐞𝐫𝐠𝐞 
This creates a new commit G’ in the main branch. G’ ties the histories of both main and feature branches. 
 
Git merge is 𝐧𝐨𝐧-𝐝𝐞𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐯𝐞. Neither the main nor the feature branch is changed. 
 
𝐆𝐢𝐭 𝐑𝐞𝐛𝐚𝐬𝐞 
Git rebase moves the feature branch histories to the head of the main branch. It creates new commits E’, F’, and G’ for each commit in the feature branch. 
 
The benefit of rebase is that it has 𝐥𝐢𝐧𝐞𝐚𝐫 𝐜𝐨𝐦𝐦𝐢𝐭 𝐡𝐢𝐬𝐭𝐨𝐫𝐲. 
 
Rebase can be dangerous if “the golden rule of git rebase” is not followed. 
 
𝐓𝐡𝐞 𝐆𝐨𝐥𝐝𝐞𝐧 𝐑𝐮𝐥𝐞 𝐨𝐟 𝐆𝐢𝐭 𝐑𝐞𝐛𝐚𝐬𝐞 
Never use it on public branches!

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Top 20 System Design Concepts You Should Know

1. Load Balancing: Distributes traffic across multiple servers for reliability and availability.
2. Caching: Stores frequently accessed data in memory for faster access.
3. Database Sharding: Splits databases to handle large-scale data growth.
4. Replication: Copies data across replicas for availability and fault tolerance.
5. CAP Theorem: Trade-off between consistency, availability, and partition tolerance.
6. Consistent Hashing: Distributes load evenly in dynamic server environments.
7. Message Queues: Decouples services using asynchronous event-driven architecture.
8. Rate Limiting: Controls request frequency to prevent system overload.
9. API Gateway: Centralized entry point for routing API requests.
10. Microservices: Breaks systems into independent, loosely coupled services.
11. Service Discovery: Locates services dynamically in distributed systems.
12. CDN: Delivers content from edge servers for speed.
13. Database Indexing: Speeds up queries by indexing important fields.
14. Data Partitioning: Divides data across nodes for scalability and performance.
15. Eventual Consistency: Guarantees consistency over time in distributed databases
16. WebSockets: Enables bi-directional communication for live updates.
17. Scalability: Increases capacity by upgrading or adding machines.
18. Fault Tolerance: Ensures system availability during hardware/software failures.
19. Monitoring: Tracks metrics and logs to understand system health.
20. Authentication & Authorization: Controls user access and verifies identity securely.

Over to you: Which other System Design concept will you add to the list?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Virtualization vs. Containerization

Before containers simplified deployment, virtualization changed how we used hardware. Both isolate workloads, but they do it differently.

- Virtualization (Hardware-level isolation): Each virtual machine runs a complete operating system, Windows, Fedora, or Ubuntu, with its own kernel, drivers, and libraries. The hypervisor (VMware ESXi, Hyper-V, KVM) sits directly on hardware and emulates physical machines for each guest OS. 

This makes VMs heavy but isolated. Need Windows and Linux on the same box? VMs handle it easily. Startup time for a typical VM is in minutes because you're booting an entire operating system from scratch.

- Containerization (OS-level isolation): Containers share the host operating system's kernel. No separate OS per container. Just isolated processes with their own filesystem and dependencies.

The container engine (Docker, containerd, CRI-O, Podman) manages lifecycle, networking, and isolation, but it all runs on top of a single shared kernel. Lightweight and fast. Containers start in milliseconds because you're not booting an OS, just launching a process.

But here's the catch: all containers on a host must be compatible with that host's kernel. Can't run Windows containers on a Linux host (without nested virtualization tricks).

Over to you: What's your go-to setup: containers in VMs, bare metal containers, or something else?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

What is a REST API?

REST (Representational State Transfer) is an architectural style for building APIs that use HTTP for communication. To be considered RESTful, an API should follow six key constraints:

- Client-Server: Separates the user interface from data storage and processing. This allows each side to evolve independently.

- Stateless: Each request contains all the information needed to process it; the server does not store session state between requests.

- Uniform Interface: Consistent resource naming and formats across the API, e.g., “/products”, “/users”.

- Cacheable: Responses explicitly indicate if they can be cached (e.g., Cache-Control) to improve performance.

- Layered System: Requests may pass through multiple layers (load balancers, auth services, etc.) before reaching the API server, but to the client it appears as a single endpoint. 

- Code on Demand (Optional): The server can send executable code (like JavaScript) to the client to extend its functionality.


Over to you: Which REST constraint do you think is most often overlooked in real-world APIs?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

The HTTP Mindmap

HTTP has evolved from HTTP/1.1 to HTTP/2, and now HTTP/3, which uses the QUIC protocol over UDP for improved performance. Today, it’s the backbone of almost everything on the internet, from browsers and APIs to streaming, cloud, and AI systems.

At the foundation, we have underlying protocols. TCP/IP for IPv4 and IPv6 traffic. Unix domain sockets for local communication. HTTP/3 running over UDP instead of TCP. These handle the actual data transport before HTTP even comes into play.

Security wraps around everything. HTTPS isn't optional anymore. WebSockets power real-time connections. Web servers manage workloads. CDNs distribute content globally. DNS resolves everything to IPs. Proxies (forward, reverse, and API gateways) route, filter, and secure traffic in between.

Web services exchange data in different formats, REST with JSON, SOAP for enterprise systems, RPC for direct calls, and GraphQL for flexible queries. Crawlers and bots index the web, guided by robots.txt files that set the boundaries.

The network world connects everything. LANs, WANs, and protocols like FTP for file transfers, IMAP/POP3 for email, and BitTorrent for peer-to-peer communication. For observability, packet capture tools like Wireshark, tcpdump, and OpenTelemetry let developers peek under the hood to understand performance, latency, and behavior across the stack.

Over to you: HTTP has been evolving for 30+ years, what do you think the next big shift will be?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

How DNS Works

You type a domain name and hit enter, but what actually happens before that webpage loads is more complex than most people realize. DNS is the phonebook of the internet, and every request you make triggers a chain of lookups across multiple servers. 

Step 1: Someone types bytebytego. com into their browser and presses enter. 

Step 2: Before doing anything, the browser looks for a cached IP address. Operating system cache gets checked too. 

Step 3: Cache miss triggers a DNS query. The browser sends a query to the configured DNS resolver, usually provided by your ISP or a service like Google DNS or Cloudflare.

Step 4: Resolver checks its own cache. 

Step 5-6: If the resolver doesn't have the answer cached, it asks the root servers, “Where can I find .com?” For bytebytego. com, the root server responds with the address of the .com TLD name server. 

Step 7-8: Resolver queries the .com TLD server. TLD server returns the authoritative server address. 

Step 9-10: This server has the actual A/AAAA record mapping the domain to an IP address. The resolver finally gets the answer: 172. 67. 21. 11 for bytebytego. com.

Step 11-12: The IP gets cached at the resolver level for future lookups, and returned to the browser.

Step 13-14: The browser stores this for its own future use, and uses the IP to make the actual HTTP request. 

Step 15: The web server returns the requested content.

All this happens in milliseconds, before your first page even starts loading.

Over to you: Which DNS tools or commands do you rely on most, dig, nslookup, or something else?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Linux file permission illustrated. 

𝐎𝐰𝐧𝐞𝐫𝐬𝐡𝐢𝐩
Every file or directory is assigned 3 types of owner:

🔹Owner: the owner is the user who created the file or directory.
🔹Group: a group can have multiple users. All users in the group have the same permissions to access the file or directory.
🔹Other: other means those users who are not owners or members of the group.

𝐏𝐞𝐫𝐦𝐢𝐬𝐬𝐢𝐨𝐧
There are only three types of permissions for a file or directory.

🔹Read (r): the read permission allows the user to read a file.
🔹Write (w): the write permission allows the user to change the content of the file.
🔹Execute (x): the execute permission allows a file to be executed. 

Over to you: chmod 777, good idea?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Types of Virtualization

Virtualization didn’t just make servers efficient, it changed how we build, scale, and deploy everything. Here’s a quick breakdown of the four major types of virtualization you’ll find in modern systems:

1. Traditional (Bare Metal): Applications run directly on the operating system. No virtualization layer, no isolation between processes. All applications share the same OS kernel, libraries, and resources.

2. Virtualized (VM-based): Each VM runs its own complete operating system. The hypervisor sits on physical hardware and emulates entire machines for each guest OS. Each VM thinks it has dedicated hardware even though it's sharing the same physical server.

3. Containerized: Containers share the host operating system's kernel but get isolated runtime environments. Each container has its own filesystem, but they're all using the same underlying OS. The container engine (Docker, containerd, Podman) manages lifecycle, networking, and isolation without needing separate operating systems for each application.

Lightweight and fast. Containers start in milliseconds because you're not booting an OS. Resource usage is dramatically lower than VMs.

4. Containers on VMs: This is what actually runs in production cloud environments. Containers inside VMs, getting benefits from both. Each VM runs its own guest OS with a container engine inside. The hypervisor provides hardware-level isolation between VMs. The container engine provides lightweight application isolation within VMs.

This is the architecture behind Kubernetes clusters on AWS, Azure, and GCP. Your pods are containers, but they're running inside VMs you never directly see or manage.

Over to you: In your experience, which setup strikes the best balance between performance and flexibility?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Vijaya Kalyani K.

Can a web server provide real-time updates? 
. 
. 
An HTTP server cannot automatically initiate a connection to a browser. As a result, the web browser is the initiator. What should we do next to get real-time updates from the HTTP server? 
 
Both the web browser and the HTTP server could be responsible for this task. 
 
🔹Web browsers do the heavy lifting: short polling or long polling. With short polling, the browser will retry until it gets the latest data. With long polling, the HTTP server doesn’t return results until new data has arrived. 
🔹HTTP server and web browser cooperate: WebSocket or SSE (server-sent event). In both cases, the HTTP server could directly send the latest data to the browser after the connection is established. The difference is that SSE is uni-directional, so the browser cannot send a new request to the server, while WebSocket is fully-duplex, so the browser can keep sending new requests. 
 
Over to you: of the four solutions (long polling, short polling, SSE, WebSocket), which ones are commonly used, and for what use cases? 

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

How do AirTags work?

AirTags work by leveraging a combination of Bluetooth technology and the vast network of Apple devices to help you locate your lost items. 
Here's a breakdown of how they function: 

1. Bluetooth Signal: Each AirTag emits a secure Bluetooth signal that can be detected by nearby Apple devices (iPhones, iPads, etc.) within the Find My network. 

2. Find My Network: When an AirTag comes within range of an Apple device in the Find My network, that device anonymously and securely relays the AirTag's location information to iCloud. 

3. Location Tracking: You can then use the Find My app on your own Apple device to see the approximate location of your AirTag on a map. 

Limitations: 
Please note that AirTags rely on Bluetooth technology and the presence of Apple devices within the Find My network. If your AirTag is in an area with few Apple devices, its location may not be updated as frequently or accurately. 

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Wavicle Data Solutions

Lead Data Engineer

LinkedIn

LinkedIn Learning Instructor

Globant

Senior Data Engineer

CRISIL Limited

Senior Analyst - Data Engineer

Quantiphi

Data Engineer

Tata Consultancy Services

System Engineer

Assistant System Engineer

Assistant System Engineer - Trainee

InsideSherpa

KPMG Data Analytics Consulting Virtual Internship

Monisha Enterprises

•	Data mining and Extraction Transformation Loading using R programming.
•	Data cleaning and processing for Data analysis done using Python using NumPy, pandas, SciPy, Scikit. Data modelling and prediction was done using algorithms – Logistic Regression, Time Series Forecasting, Clustering and Classification.
•	Data Visualization done using Tableau for generating reports to enhance the productivity of the products.
•	Built a Data Warehouse on MSSQL Server for ERP System of the enterprise to organize the data for future processing.

Data Analyst Intern

Open to Collaborate | Storyteller | Lead Data Engineer@Wavicle| Linkedin Top Voice 2024 | Globant | Linkedin Learning Instructor | 2xGCP & AWS Certified | LICAP'2022

Data Engineer with 7+ years of experience working on data engineering, analysis, and modelling projects for various domains, such as banking, healthcare, sales, and e-commerce. I leverage my strong statistical and analytical ability to explore business ideas and provide innovative solutions. 

Skilled with technologies and strong intellect with enthusiasm as an avid learner with challenging opportunities. I’m working as a LinkedIn Learning Instructor to bring change as a data evangelist.

Post Graduated in Master of Technology in Data Science and Engineering from Birla Institute of Technology and Science, Pilani in 2023, where I learned advanced concepts and techniques in data science and engineering, machine learning, cloud computing, and visualization. 

I hold multiple certifications in GCP, AWS, and Azure, and I enjoy working with cloud technologies and big data platforms.

I am passionate about data science and women empowerment, and I have been recognized as the 
- Linkedin Top Voice 2024 
- Data Science and Women Empowerment Thought Leader by Global AI Hub. 

I am also a proud participant of the LinkedIn Creator Accelerator Program 2022, where I share my insights and experiences with the data science community. 

I am open to collaborate and share ideas, or explore various opportunities in the data field.

Pooja Jain

73% of data pipeline failures happen because of dependency issues, not bad code.
Cron jobs don't care if yesterday's data never arrived. They'll happily run on empty tables and produce garbage.

Airflow? It checks, retries, alerts, and refuses to cascade your failure downstream.

→ Beyond scheduling Airflow gives an important feature of 𝗗𝗔𝗚𝘀(𝗗𝗶𝗿𝗲𝗰𝘁 𝗔𝗰𝘆𝗰𝗹𝗶𝗰 𝗚𝗿𝗮𝗽𝗵𝘀), that helps to monitor, retry and gives flexibility to organize pipelines with minimum/less code.

🤔 Data Engineers often misinterpret Airflow just as an orchestration tool.

Apache Airflow is an open-source platform designed to programmatically author, schedule, retry and monitor workflows. 

🎼Think of it as the "conductor of an orchestra", where each musician (your data tasks, like data extraction, transformation, loading, or model training) knows when to play, what to play, and what happens if they make a mistake.

Most data engineers automate the wrong things at the wrong time.

Here's the framework I use after 8 years of building production systems:
✅ AUTOMATE WHEN: 
→ Task runs daily/weekly 
→ Human errors cause outages 
→ Work blocks other priorities
→ Team growth = more manual work
Examples: Reports, schema checks, alerts

❌ DON'T AUTOMATE WHEN: 
→ Task happens quarterly 
→ Requirements change weekly 
→ Process isn't understood yet 
→ Manual steps reveal insights

My rule: If it’s done 3+ times, script it; 10+ times, automate it; fails 5+ times, redesign it.
Automate what matters, when it matters—not everything!

Airflow isn’t just about running tasks.

🕑 It’s about building trustworthy, scalable, and observable pipelines that run reliably at 2am — even when you’re not watching.

Airflow gives structure to your chaos.
DAGs give clarity to your pipelines. 
That’s why DEs love them ❤️

Real Example: 
Instead of: 
❌ Cron jobs that fail silently 
❌ Manual dependency management
❌ No visibility into failures
You get: 
✅ Visual workflow monitoring 
✅ Automatic failure notifications
✅ Smart task scheduling 
✅ Easy debugging & restarting

Here's an amazing Airflow cheatsheet with some fundamentals concepts and interview questions to prepare for data engineers. 👇

Carousel Credits -  Futurense.com

Willing to learn more, here's the link to Astronomer free learning path - https://lnkd.in/g8-EWBda

Also, level up with Marc Lamberti's Udemy course on Airflow 3 - https://lnkd.in/gwArwaME

taplio

The Battle

Alex Xu

Pooja Jain

Followers

Posts

Engagements

Likes

Performance ScoreAlex Xu's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Alex Xu

Pooja Jain

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Alex Xu's LinkedIn Performance

Performance ScorePooja Jain's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreAlex Xu's LinkedIn Performance