ByteByteGo

Co-authored 4 books:
     - System Design Interview - Volume 1
     - System Design Interview - Volume 2
     - Coding Interview Patterns
     - Machine Learning System Design Interview

- Write weekly ByteByteGo newsletter on system design topics to 1,000,000+ subscribers.
- Grow the ByteByteGo YouTube channel to 1,000,000+ subscribers.

Author & Co-Founder

Twitter

Software Engineer

Apple

Zynga

Oracle

Software Developer

Modeln N

Associate Member of Technical Staff

Follow me for system design & book-writing tips.

Alex Xu is a software engineer and author. His book System Design Interview - An Insider’s Guide is one of Amazon's best-selling books in the “Distributed Systems & Computing” category.  As of December 2020, his book is being translated into 7 different languages (Traditional Chinese, Russian, Japanese, Spanish, Polish, Simplified Chinese, and Korean).

--------------
Newsletter: https://blog.bytebytego.com
Twitter: https://twitter.com/alexxubyte
YouTube: https://www.youtube.com/c/ByteByteGo

Alex Xu

Database Types You Should Know in 2025

There’s no such thing as a one-size-fits-all database anymore. Modern applications rely on multiple database types, from real-time analytics to vector search for AI. Knowing which type to use can make or break your system’s performance.

Relational: Traditional row-and-column databases, great for structured data and transactions.

Columnar: Optimized for analytics, storing data by columns for fast aggregations.

Key-Value: Stores data as simple key–value pairs, enabling fast lookups. 

In-memory: Stores data in RAM for ultra-low latency lookups, ideal for caching or session management.

Wide-Column: Handles massive amounts of semi-structured data across distributed nodes.

Time-series: Specialized for metrics, logs, and sensor data with time as a primary dimension.

Immutable Ledger: Ensures tamper-proof, cryptographically verifiable transaction logs.

Graph: Models complex relationships, perfect for social networks and fraud detection

Document: Flexible JSON-like storage, great for modern apps with evolving schemas.

Geospatial: Manages location-aware data such as maps, routes, and spatial queries.

Text-search: Full-text indexing and search with ranking, filters, and analytics.

Blob: Stores unstructured objects like images, videos, and files.

Vector: Powers AI/ML apps by enabling similarity search across embeddings.

Over to you: Which database type do you think will grow fastest in the next 5 years?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

bytebytego

linkedin.com/company/the-rundown-ai

JSESSIONID

li_at

Talks about #systemdesign, #techinterview, and #bookpublishing

Author of 4 Bestselling Books | Co-Founder of ByteByteGo

Cloudflare is much more than just a CDN and DDoS protection service.

Let’s do a quick comparison of Cloudflare, AWS, and Azure.

Cloudflare has rapidly expanded beyond its traditional CDN roots, launching a suite of modern developer-first services like Workers, R2, D1, and so on. These offerings position it as a serious edge-native alternative to other cloud providers.

Here are the key cloud capabilities that Cloudflare supports:

1 - Edge Compute and Serverless
2 - Object and Blob Storage
3 - Relational Databases
4 - Containers
5 - Sandboxes
6 - Workflows
7 - AI Agents SDK
8 - Vector and AI search
9 - Data Connectivity
10 - AI Infrastructure
11 - Content Delivery Network
12 - DNS
13 - Load Balancing

Over to you: Have you used Cloudflare’s new offerings? What are your thoughts on them?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

No alternative text description for this image

HTTP vs. HTTPS

When you open a website, the difference between HTTP and HTTPS decides whether your data travels safely or in plain sight. Here’s what actually happens under the hood:

HTTP:
- Sends data in plain text, anyone on the network can intercept it.
- The client and server perform a simple TCP handshake: SYN, SYN-ACK, ACK
- Fast but completely insecure. Passwords, tokens, and forms can all be read in transit.

HTTPS (SSL/TLS):
- Step 1: TCP Handshake: Standard connection setup.
- Step 2: Certificate Check: Client says hello. Server responds with hello and its SSL/TLS certificate. That certificate contains the server's public key and is signed by a trusted Certificate Authority. 

Your browser verifies this certificate is legitimate, not expired, and actually belongs to the domain you're trying to reach. This proves you're talking to the real server, not some attacker pretending to be it.

- Step 3: Key Exchange: Here's where asymmetric encryption happens. The server has a public key and a private key. Client generates a session key, encrypts it with the server's public key, and sends it over. Only the server can decrypt this with its private key.

Both sides now have the same session key that nobody else could have intercepted. This becomes the symmetric encryption key for the rest of the session.

- Step 4: Data Transmission: Now every request and response gets encrypted with that session key using symmetric encryption.

Over to you: What’s your go-to tool for debugging TLS issues, openssl, curl -v, or something else?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Karnav P.

Apache Kafka vs. RabbitMQ

Kafka and RabbitMQ both handle messages, but they solve fundamentally different problems. Understanding the difference matters when designing distributed systems.

Kafka is a distributed log. Producers append messages to partitions. Those messages stick around based on retention policy, not because someone consumed them. Consumers pull messages at their own pace using offsets. You can rewind, replay, reprocess everything. It is designed for high throughput event streaming where multiple consumers need the same data independently.

RabbitMQ is a message broker. Producers publish messages to exchanges. Those exchanges route to queues based on binding keys and patterns (direct, topic, fanout). Messages get pushed to consumers and then deleted once acknowledged. It is built for task distribution and traditional messaging workflows.

The common mistake is using Kafka like a queue or RabbitMQ like an event log. They're different tools built for different use cases.

Over to you:  If you had to explain when NOT to use Kafka, what would you say?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Explaining 9 types of API testing. 
 
🔹 Smoke Testing 
This is done after API development is complete. Simply validate if the APIs are working and nothing breaks. 
 
🔹 Functional Testing 
This creates a test plan based on the functional requirements and compares the results with the expected results. 
 
🔹 Integration Testing 
This test combines several API calls to perform end-to-end tests. The intra-service communications and data transmissions are tested. 
 
🔹 Regression Testing 
This test ensures that bug fixes or new features shouldn’t break the existing behaviors of APIs. 
 
🔹 Load Testing 
This tests applications’ performance by simulating different loads. Then we can calculate the capacity of the application. 
 
🔹 Stress Testing 
We deliberately create high loads to the APIs and test if the APIs are able to function normally. 
 
🔹 Security Testing 
This tests the APIs against all possible external threats. 
 
🔹 UI Testing 
This tests the UI interactions with the APIs to make sure the data can be displayed properly. 
 
🔹 Fuzz Testing 
This injects invalid or unexpected input data into the API and tries to crash the API. In this way, it identifies the API vulnerabilities. 

--
We just launched Become an AI Engineer | Learn by Doing: Cohort 2. If you missed Cohort 1, now’s your chance to join us for Cohort 2.

Check it out here: https://bit.ly/3LdaWuw

#AI #AIEngineer #MachineLearning 
.

The AI Agent Tech Stack

1. Foundation Models: Large-scale pre-trained language models that serve as the “brains” of AI agents, enabling capabilities like reasoning, text generation, coding, and question answering.
2. Data Storage: This layer handles vector databases and memory storage systems used by AI agents to store and retrieve context, embeddings, or documents.
3. Agent Development Frameworks: These frameworks help developers build, orchestrate, and manage multi-step AI agents and their workflows.
4. Observability: This category enables monitoring, debugging, and logging of AI agent behavior and performance in real-time.
5. Tool Execution: These platforms allow AI agents to interface with real-world tools (for example, APIs, browsers, external systems) to complete complex tasks.
6. Memory Management: These systems manage long-term and short-term memory for agents, helping them retain useful context and learn from past interactions.

Over to you: What else will you add to the list?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Big Data Pipeline Cheatsheet for AWS, Azure, and Google Cloud

Each platform offers a comprehensive suite of services that cover the entire lifecycle:

1 - Ingestion: Collecting data from various sources
2 - Data Lake: Storing raw data
3 - Computation: Processing and analyzing data
4 - Data Warehouse: Storing structured data
5 - Presentation: Visualizing and reporting insights

AWS uses services like Kinesis for data streaming, S3 for storage, EMR for processing, RedShift for warehousing, and QuickSight for visualization.

Azure’s pipeline includes Event Hubs for ingestion, Data Lake Store for storage, Databricks for processing, Cosmos DB for warehousing, and Power BI for presentation.

GCP offers PubSub for data streaming, Cloud Storage for data lakes, DataProc and DataFlow for processing, BigQuery for warehousing, and Data Studio for visualization.

Over to you: What else would you add to the pipeline?

--
LAST DAY to enroll in Become an AI Engineer Learn by Doing, Cohort 2. If you missed Cohort 1, now is your chance to join us in Cohort 2.

Check it out here: https://bit.ly/3LdaWuw

#AI #AIEngineer #MachineLearning 
.

Things Every Developer Should Know: Concurrency is 𝐍𝐎𝐓 parallelism.
.
.
In system design, it is important to understand the difference between concurrency and parallelism. 

As Rob Pyke(one of the creators of GoLang) stated:“ Concurrency is about 𝐝𝐞𝐚𝐥𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 lots of things at once. Parallelism is about 𝐝𝐨𝐢𝐧𝐠 lots of things at once." This distinction emphasizes that concurrency is more about the 𝐝𝐞𝐬𝐢𝐠𝐧 of a program, while parallelism is about the 𝐞𝐱𝐞𝐜𝐮𝐭𝐢𝐨𝐧. 

Concurrency is about dealing with multiple things at once. It involves structuring a program to handle multiple tasks simultaneously, where the tasks can start, run, and complete in overlapping time periods, but not necessarily at the same instant. 

Concurrency is about the composition of independently executing processes and describes a program's ability to manage multiple tasks by making progress on them without necessarily completing one before it starts another.

Parallelism, on the other hand, refers to the simultaneous execution of multiple computations. It is the technique of running two or more tasks or computations at the same time, utilizing multiple processors or cores within a computer to perform several operations concurrently. Parallelism requires hardware with multiple processing units, and its primary goal is to increase the throughput and computational speed of a system.

In practical terms, concurrency enables a program to remain responsive to input, perform background tasks, and handle multiple operations in a seemingly simultaneous manner, even on a single-core processor. It's particularly useful in I/O-bound and high-latency operations where programs need to wait for external events, such as file, network, or user interactions.

Parallelism, with its ability to perform multiple operations at the same time, is crucial in CPU-bound tasks where computational speed and throughput are the bottlenecks. Applications that require heavy mathematical computations, data analysis, image processing, and real-time processing can significantly benefit from parallel execution.

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

5 REST API Authentication Methods

1. Basic Authentication: Clients include a Base64-encoded username and password in every request header, which is simple but insecure since credentials are transmitted in plaintext. Useful in quick prototypes or internal services over secure networks.
2. Session Authentication: After login, the server creates a session record and issues a cookie. Subsequent requests send that cookie so the server can validate user state. Used in traditional web-apps.
3. Token Authentication: Clients authenticate once to receive a signed token, then present the token on each request for stateless authentication. Used in single-page applications and modern APIs that require scalable, stateless authentication.
4. OAuth-Based Authentication: Clients obtain an access token via an authorization grant from an OAuth provider, then use that token to call resource servers on the user’s behalf. Used in cases of third-party integrations or apps that need delegated access to user data.
5. API Key Authentication: Clients present a predefined key (often in headers or query strings) with each request. The server verifies the key to authorize access. Used in service-to-service or machine-to-machine APIs where simple credential checks are sufficient.

Over to you: Which other API Authentication method have you seen?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Git Merge vs. Rebase vs. Squash Commit! 
. 
. 
What are the differences? 
 
When we 𝐦𝐞𝐫𝐠𝐞 𝐜𝐡𝐚𝐧𝐠𝐞𝐬 from one Git branch to another, we can use ‘git merge’ or ‘git rebase’. The diagram below shows how the two commands work. 
 
𝐆𝐢𝐭 𝐌𝐞𝐫𝐠𝐞 
This creates a new commit G’ in the main branch. G’ ties the histories of both main and feature branches. 
 
Git merge is 𝐧𝐨𝐧-𝐝𝐞𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐯𝐞. Neither the main nor the feature branch is changed. 
 
𝐆𝐢𝐭 𝐑𝐞𝐛𝐚𝐬𝐞 
Git rebase moves the feature branch histories to the head of the main branch. It creates new commits E’, F’, and G’ for each commit in the feature branch. 
 
The benefit of rebase is that it has 𝐥𝐢𝐧𝐞𝐚𝐫 𝐜𝐨𝐦𝐦𝐢𝐭 𝐡𝐢𝐬𝐭𝐨𝐫𝐲. 
 
Rebase can be dangerous if “the golden rule of git rebase” is not followed. 
 
𝐓𝐡𝐞 𝐆𝐨𝐥𝐝𝐞𝐧 𝐑𝐮𝐥𝐞 𝐨𝐟 𝐆𝐢𝐭 𝐑𝐞𝐛𝐚𝐬𝐞 
Never use it on public branches!

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Top 20 System Design Concepts You Should Know

1. Load Balancing: Distributes traffic across multiple servers for reliability and availability.
2. Caching: Stores frequently accessed data in memory for faster access.
3. Database Sharding: Splits databases to handle large-scale data growth.
4. Replication: Copies data across replicas for availability and fault tolerance.
5. CAP Theorem: Trade-off between consistency, availability, and partition tolerance.
6. Consistent Hashing: Distributes load evenly in dynamic server environments.
7. Message Queues: Decouples services using asynchronous event-driven architecture.
8. Rate Limiting: Controls request frequency to prevent system overload.
9. API Gateway: Centralized entry point for routing API requests.
10. Microservices: Breaks systems into independent, loosely coupled services.
11. Service Discovery: Locates services dynamically in distributed systems.
12. CDN: Delivers content from edge servers for speed.
13. Database Indexing: Speeds up queries by indexing important fields.
14. Data Partitioning: Divides data across nodes for scalability and performance.
15. Eventual Consistency: Guarantees consistency over time in distributed databases
16. WebSockets: Enables bi-directional communication for live updates.
17. Scalability: Increases capacity by upgrading or adding machines.
18. Fault Tolerance: Ensures system availability during hardware/software failures.
19. Monitoring: Tracks metrics and logs to understand system health.
20. Authentication & Authorization: Controls user access and verifies identity securely.

Over to you: Which other System Design concept will you add to the list?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Virtualization vs. Containerization

Before containers simplified deployment, virtualization changed how we used hardware. Both isolate workloads, but they do it differently.

- Virtualization (Hardware-level isolation): Each virtual machine runs a complete operating system, Windows, Fedora, or Ubuntu, with its own kernel, drivers, and libraries. The hypervisor (VMware ESXi, Hyper-V, KVM) sits directly on hardware and emulates physical machines for each guest OS. 

This makes VMs heavy but isolated. Need Windows and Linux on the same box? VMs handle it easily. Startup time for a typical VM is in minutes because you're booting an entire operating system from scratch.

- Containerization (OS-level isolation): Containers share the host operating system's kernel. No separate OS per container. Just isolated processes with their own filesystem and dependencies.

The container engine (Docker, containerd, CRI-O, Podman) manages lifecycle, networking, and isolation, but it all runs on top of a single shared kernel. Lightweight and fast. Containers start in milliseconds because you're not booting an OS, just launching a process.

But here's the catch: all containers on a host must be compatible with that host's kernel. Can't run Windows containers on a Linux host (without nested virtualization tricks).

Over to you: What's your go-to setup: containers in VMs, bare metal containers, or something else?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

What is a REST API?

REST (Representational State Transfer) is an architectural style for building APIs that use HTTP for communication. To be considered RESTful, an API should follow six key constraints:

- Client-Server: Separates the user interface from data storage and processing. This allows each side to evolve independently.

- Stateless: Each request contains all the information needed to process it; the server does not store session state between requests.

- Uniform Interface: Consistent resource naming and formats across the API, e.g., “/products”, “/users”.

- Cacheable: Responses explicitly indicate if they can be cached (e.g., Cache-Control) to improve performance.

- Layered System: Requests may pass through multiple layers (load balancers, auth services, etc.) before reaching the API server, but to the client it appears as a single endpoint. 

- Code on Demand (Optional): The server can send executable code (like JavaScript) to the client to extend its functionality.


Over to you: Which REST constraint do you think is most often overlooked in real-world APIs?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

The HTTP Mindmap

HTTP has evolved from HTTP/1.1 to HTTP/2, and now HTTP/3, which uses the QUIC protocol over UDP for improved performance. Today, it’s the backbone of almost everything on the internet, from browsers and APIs to streaming, cloud, and AI systems.

At the foundation, we have underlying protocols. TCP/IP for IPv4 and IPv6 traffic. Unix domain sockets for local communication. HTTP/3 running over UDP instead of TCP. These handle the actual data transport before HTTP even comes into play.

Security wraps around everything. HTTPS isn't optional anymore. WebSockets power real-time connections. Web servers manage workloads. CDNs distribute content globally. DNS resolves everything to IPs. Proxies (forward, reverse, and API gateways) route, filter, and secure traffic in between.

Web services exchange data in different formats, REST with JSON, SOAP for enterprise systems, RPC for direct calls, and GraphQL for flexible queries. Crawlers and bots index the web, guided by robots.txt files that set the boundaries.

The network world connects everything. LANs, WANs, and protocols like FTP for file transfers, IMAP/POP3 for email, and BitTorrent for peer-to-peer communication. For observability, packet capture tools like Wireshark, tcpdump, and OpenTelemetry let developers peek under the hood to understand performance, latency, and behavior across the stack.

Over to you: HTTP has been evolving for 30+ years, what do you think the next big shift will be?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

How DNS Works

You type a domain name and hit enter, but what actually happens before that webpage loads is more complex than most people realize. DNS is the phonebook of the internet, and every request you make triggers a chain of lookups across multiple servers. 

Step 1: Someone types bytebytego. com into their browser and presses enter. 

Step 2: Before doing anything, the browser looks for a cached IP address. Operating system cache gets checked too. 

Step 3: Cache miss triggers a DNS query. The browser sends a query to the configured DNS resolver, usually provided by your ISP or a service like Google DNS or Cloudflare.

Step 4: Resolver checks its own cache. 

Step 5-6: If the resolver doesn't have the answer cached, it asks the root servers, “Where can I find .com?” For bytebytego. com, the root server responds with the address of the .com TLD name server. 

Step 7-8: Resolver queries the .com TLD server. TLD server returns the authoritative server address. 

Step 9-10: This server has the actual A/AAAA record mapping the domain to an IP address. The resolver finally gets the answer: 172. 67. 21. 11 for bytebytego. com.

Step 11-12: The IP gets cached at the resolver level for future lookups, and returned to the browser.

Step 13-14: The browser stores this for its own future use, and uses the IP to make the actual HTTP request. 

Step 15: The web server returns the requested content.

All this happens in milliseconds, before your first page even starts loading.

Over to you: Which DNS tools or commands do you rely on most, dig, nslookup, or something else?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Linux file permission illustrated. 

𝐎𝐰𝐧𝐞𝐫𝐬𝐡𝐢𝐩
Every file or directory is assigned 3 types of owner:

🔹Owner: the owner is the user who created the file or directory.
🔹Group: a group can have multiple users. All users in the group have the same permissions to access the file or directory.
🔹Other: other means those users who are not owners or members of the group.

𝐏𝐞𝐫𝐦𝐢𝐬𝐬𝐢𝐨𝐧
There are only three types of permissions for a file or directory.

🔹Read (r): the read permission allows the user to read a file.
🔹Write (w): the write permission allows the user to change the content of the file.
🔹Execute (x): the execute permission allows a file to be executed. 

Over to you: chmod 777, good idea?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Types of Virtualization

Virtualization didn’t just make servers efficient, it changed how we build, scale, and deploy everything. Here’s a quick breakdown of the four major types of virtualization you’ll find in modern systems:

1. Traditional (Bare Metal): Applications run directly on the operating system. No virtualization layer, no isolation between processes. All applications share the same OS kernel, libraries, and resources.

2. Virtualized (VM-based): Each VM runs its own complete operating system. The hypervisor sits on physical hardware and emulates entire machines for each guest OS. Each VM thinks it has dedicated hardware even though it's sharing the same physical server.

3. Containerized: Containers share the host operating system's kernel but get isolated runtime environments. Each container has its own filesystem, but they're all using the same underlying OS. The container engine (Docker, containerd, Podman) manages lifecycle, networking, and isolation without needing separate operating systems for each application.

Lightweight and fast. Containers start in milliseconds because you're not booting an OS. Resource usage is dramatically lower than VMs.

4. Containers on VMs: This is what actually runs in production cloud environments. Containers inside VMs, getting benefits from both. Each VM runs its own guest OS with a container engine inside. The hypervisor provides hardware-level isolation between VMs. The container engine provides lightweight application isolation within VMs.

This is the architecture behind Kubernetes clusters on AWS, Azure, and GCP. Your pods are containers, but they're running inside VMs you never directly see or manage.

Over to you: In your experience, which setup strikes the best balance between performance and flexibility?

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Vijaya Kalyani K.

Can a web server provide real-time updates? 
. 
. 
An HTTP server cannot automatically initiate a connection to a browser. As a result, the web browser is the initiator. What should we do next to get real-time updates from the HTTP server? 
 
Both the web browser and the HTTP server could be responsible for this task. 
 
🔹Web browsers do the heavy lifting: short polling or long polling. With short polling, the browser will retry until it gets the latest data. With long polling, the HTTP server doesn’t return results until new data has arrived. 
🔹HTTP server and web browser cooperate: WebSocket or SSE (server-sent event). In both cases, the HTTP server could directly send the latest data to the browser after the connection is established. The difference is that SSE is uni-directional, so the browser cannot send a new request to the server, while WebSocket is fully-duplex, so the browser can keep sending new requests. 
 
Over to you: of the four solutions (long polling, short polling, SSE, WebSocket), which ones are commonly used, and for what use cases? 

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

How do AirTags work?

AirTags work by leveraging a combination of Bluetooth technology and the vast network of Apple devices to help you locate your lost items. 
Here's a breakdown of how they function: 

1. Bluetooth Signal: Each AirTag emits a secure Bluetooth signal that can be detected by nearby Apple devices (iPhones, iPads, etc.) within the Find My network. 

2. Find My Network: When an AirTag comes within range of an Apple device in the Find My network, that device anonymously and securely relays the AirTag's location information to iCloud. 

3. Location Tracking: You can then use the Find My app on your own Apple device to see the approximate location of your AirTag on a map. 

Limitations: 
Please note that AirTags rely on Bluetooth technology and the presence of Apple devices within the Find My network. If your AirTag is in an area with few Apple devices, its location may not be updated as frequently or accurately. 

--
We just launched the all-in-one tech interview prep platform, covering coding, system design, OOD, and machine learning.

Launch sale: 50% off. Check it out: https://lnkd.in/euwKh6u8

#systemdesign #coding #interviewtips 
.

Trivial Business

Providing interim software engineering management (VP Engineering/CTO) for startups and SMEs.
Providing leadership and software development coaching and mentoring to software engineering leaders and their teams.

Example projects:

■ Interim CTO redefining the IT infrastructure and software systems of a business turnaround.
■ Interim CTO of an angel backed start-up.
■ Leading the development of a large scale website for the BBC.
■ Advising on technology strategy for businesses.
■ Advising a consultancy on technology deliver and project turnaround.
■ Advising a technology company on the use of machine learning and NLP to access sentiment on financial news.

Providing training, mentoring and coaching to Staff+ and Engineering Managers on leadership and soft skills.

Providing training, mentoring and coaching software engineers. Delivering the training both on-site and remote. Topics included: Go, Python, C++, NLP and Machine Learning.

Chief Technology Officer

Coding Chats

Podcast Host

Coding Challenges

Writing weekly coding challenges that help software engineers level up by building real-world software applications.

Delivering live software engineering training.

Software Engineering Coach

Acast

Acast is a podcast advertising company. They help creators and advertisers connect.

AWS, CDK, Typescript/JavaScript and node.js

Senior Software Engineering Manager

Met Office

The Met Office is the United Kingdom's national weather service, providing critical weather services and world-leading climate science, helping you make better decisions to stay safe and thrive.

AWS (Cloud) and Python for handling large meteorological data sets.

Contract Software Engineer

Sky is Europe's largest media company and pay-TV broadcaster.

Building serverless solutions to support Sky’s broadcasting activities using Python 3.8, Java, AWS and the Serverless Framework.

Cirrus Logic

Cirrus Logic is a fabless semiconductor supplier that specialises in analog, mixed-signal, and audio DSP integrated circuits.

Managing several teams developing Data Engineering Tools to gather, process and distribute data to support the development, testing and deployment of machine learning algorithms using Python, Conda, PyTest, Numpy, Pandas, Dask and Dask-Distributed.

Starting to build a new team whilst designing, developing and deploying software tools to manage big data for the development and evaluation of biometric algorithms.

Principal Software Architect

Software development for High Performance Computing using Python, SciPy, Pandas, NumPy, Dask, Dask-Distributed and C.

Principal Software Engineer

Morningstar

Morningstar Real-Time Data provides real-time. low latency financial market data feeds.

Leading a globally distributed team of 40+ software engineering and QA staff developing and supporting real time low latency market data feed handlers and supporting infrastructure for several hundred financial exchanges. 
Using C++, C# and Python on Windows.

Head of Software Engineering

Aria Networks

Aria Networks provides artificial intelligence driven network planning and optimisations tools for major telecommunications companies, over-the-top providers and global enterprises.

■ Defining the technical strategy to deliver the business strategy and products.
■ Leading the software development and QA teams delivering a customised network capacity planning and optimisation tool for telecommunications networks (IP, MPLS, OTN, SDN and NFV).
■ Using C++, Boost, QT, JavaScript and Python to build software based around an Artificial Intelligence (AI, specifically Evolutionary Computing/Genetic Algorithms & Neural Networks) 
engine that runs on Windows, Mac and Linux.

VP Engineering

Helping you become a better software engineer through coding challenges that build real applications.

I am on a mission to level up one million software engineers and software engineering leaders!

I’m doing that by: 

1. Helping you improve your coding skills through Coding Challenges that have you building real applications.

2. Helping you improve your software development and soft skills with posts and articles about software development.

You can find the coding challenges here ➡ https://codingchallenges.fyi

You can follow me here on LinkedIn for the posts and articles!

I also post some longer articles on my Developing Skills newsletter here ➡  https://developingskills.substack.com/


My BACKGROUND:

I'm a software engineer and sometimes a manager of software engineers. I've worked as both a senior individual contributor (Staff+) and a senior manager (VP Engineering, Head of Software Development).

I've worked across many tech stacks. I've spent most of the last six years building distributed systems with Python and Flask deployed on AWS using CloudFormation and some CDK. 

Recently there has been some TypeScript/JavaScript, React and Next.js on the frontend with Node.js and Express on the backend. Deployed on AWS using CDK.

For most of my career before that I worked in C++. I'm coming the full circle now and currently learning Rust.

I have a passion to help individuals, teams and departments reach their full potential.

John Crickett

I don’t mind if you let an LLM write all your code, as long as you understand all your code.

Treat it the same way you’d treat a teammate’s pull request. Read it. Question it. Make sure it fits your team’s standards and can be understood by the people who will maintain it.

If it’s good, ship it. If it’s not, refine it or rewrite it. Simple.

AI should accelerate you, not replace your judgement. The quality of your work still depends on how well you can read, reason about, and explain the code you ship.

The rule stays the same: you own what goes to production, no matter who typed it.

CTO / VP Engineering / Head of Software Engineering

Here's my problem with people claiming they use AI to write 13,000 lines of production code per day and who suggest it is a repeatable process.

I can't write a clear enough specification for enough functionality to fill 13,000 lines of code per day, every single day.

And I've never met anyone else who can either.

The bottleneck isn't writing code, it's knowing what code to write.

Let's discuss how we tackle that bottleneck, with or without AI.

19 books on Software Architecture - that you can get for around 4% of the cover price right now!

👉 Modern Software Engineering
👉 Understanding Software Dynamics
👉 Software Requirements Essentials
👉 Righting Software
👉 Balancing Coupling in Software Design
👉 Architecture for Flow
👉 Effective Software Architecture
👉 Software Architecture and Decision-Making
👉 Domain Driven Design Distilled
👉 Functional and Concurrent Programming
👉 Software Architecture in Practice
👉 Practical Object-Oriented Design
👉 Serverless as a Game Changer
👉 The Async-First Playbook
👉 Principles of Web API Design
👉 Strategic Monoliths and Microservices
👉 Domain Storytelling
👉 Continuous Architecture in Practice
👉 Patterns for API Design

You get all of these worth over £618 for around £25 right now through Humble’s Book Bundle and support charity, details here: https://lnkd.in/en2eN_kG

Unskilled and inexperienced software engineers take simple requirements and create complex code.

Skilled and experienced software engineers take complex requirements and create simple code.

The difference is that experience maintaining a complex system, teaches them the benefits of simple code and gives them the skills to do so.

This is also why skilled, experienced software engineers are best positioned to leverage AI-assisted software engineering: they know how to turn a complex set of high level requirements into a simple set of low-level requirements that can be used to prompt and AI-agent and have it build simple code.

Tips for AI-Assisted software development:

If you don’t get it, don’t ship it.

AI can write code fast, but it can’t think about the future. Pasting code you don’t fully understand leads to tech debt, surprises, and skills that get rusty. 

Understanding the code keeps your skills sharp and your product reliable.

Read every bit of code before it hits your repo. If you can’t explain it, rewrite it or prompt the AI to try again.

Want to write better software?

Then embrace the Unix Philosophy as described in The Art of Unix Programming. It provides 17 great rules for writing better software.

With AI-assisted software development rules 1, 2, 5, 6, 7, 8 and 15 are even more important and using AI to generate code satisfies rule 14!

The rules are:

1. Rule of Modularity: Write simple parts connected by clean interfaces.
2. Rule of Clarity: Clarity is better than cleverness.
3. Rule of Composition: Design programs to be connected to other programs.
4. Rule of Separation: Separate policy from mechanism; separate interfaces from engines.
5. Rule of Simplicity: Design for simplicity; add complexity only where you must.
6. Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.
7. Rule of Transparency: Design for visibility to make inspection and debugging easier.
8. Rule of Robustness: Robustness is the child of transparency and simplicity.
9. Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.
10. Rule of Least Surprise: In interface design, always do the least surprising thing.
11. Rule of Silence: When a program has nothing surprising to say, it should say nothing.
12. Rule of Repair: When you must fail, fail noisily and as soon as possible.
13. Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.
14. Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.
15. Rule of Optimization: Prototype before polishing. Get it working before you optimize it.
16. Rule of Diversity: Distrust all claims for “one true way”.
17. Rule of Extensibility: Design for the future, because it will be here sooner than you think.

Embrace the UNIX Philosophy

20 great books on data science and data engineering - that you can get for around 3% of the cover price right now!

👉 Learning Data Science
👉 Learning SQL
👉 Data Science from Scratch
👉 Data Science: The Hard Parts
👉 Data Science at the Command Line
👉 Data Science on AWS
👉 Software Engineering for Data Scientists
👉 Financial Data Engineering
👉 Streamlining Databases
👉 Implementing Data Mesh
👉 Fuzzy Data Matching with SQL
👉 Learning Snowflake SQL and Scripting
👉 Scaling Python with Dask
👉 Scaling Machine Learning with Spark
👉 The Enterprise Data Catalog
👉 Scaling Python with Ray
👉 Essential Math for Data Science
👉 Practical Lakehouse Architecture
👉 Hands-On Differential Privacy
👉 Practical Statistics for Data Science

You get all of these worth over £950 for around £25 right now through Humble’s Book Bundle and support charity, details here: https://lnkd.in/eFyWF6ij

Tips for AI-assisted software development:

Boring tech gives AI superpowers.

AI coding agents performs best with tools, languages, and frameworks that have been around long enough to show up consistently in its training data. 

When you push it onto the bleeding edge, it starts hallucinating, and the quality drops. You’ll see odd patterns, missing steps, or advice that just doesn’t hold up.

To get the best results, pick a well-established framework or language. Pick 'boring' tech! Ten tell the AI exactly what you’re using.

Tips for AI-Assisted software development:

Don't ask AI to build the house, ask it to make the bricks.

People struggle with AI coding tools because they're asking for too much at once. They want entire features, complete apps, full systems. The AI generates something massive, half of it doesn't work, and now you're debugging spaghetti code you didn't write.

Flip the approach. Ask AI to generate small, well-defined components. A single function. A reusable component. One API endpoint. You architect how they fit together.

Think LEGO, not construction. AI is incredible at fabricating pieces to spec. 

You're still the one building something meaningful from them.

29 great books on AI Algorithms and LLMs - that you can get for around 3% of the cover price right now!

This looks like a great deal, in particular I recommend Emmanuel Maggiori’s book: The AI Pocket Book. I'm looking forward to reading several others!

👉 Build A Large Language Model
👉 AI Agents In Action
👉 Grokking Artificial Intelligence Algorithms
👉 AI-Powered Search
👉 Generative AI In Action
👉 AI For Everyday IT
👉 Causal AI
👉 The Art of AI Product Development
👉 Starting Data Analytics with Generative AI and Python
👉 Learn AI-Assisted Python Programming
👉 AI as a Service
👉 Interpretable AI
👉 Software Testing with Generative AI
👉 Learn Generative AI With PyTorch
👉 Time Series Forecasting Using Foundation Models
👉 Effective Conversational AI
👉 How Large Language Models Work
👉 Succeeding With AI
👉 Introduction to Generative AI
👉 AI-Powered Developer
👉 Inside AI
👉 Build Financial Software With Generative AI
👉 Data Storytelling With Altair and AI
👉 Coding With AI
👉 Prompt Engineering For AI Systems
👉 Zero To AI
👉 The Complete Obsolete Guide to Generative AI
👉 The AI Pocket Book
👉 Regular Expression Puzzled and AI Coding Assistants

You get all of these worth over £1155 for around £35 right now through Humble’s Book Bundle and support charity, details here: https://lnkd.in/eAwEJAiu

What if that unrealistic estimate isn't wrong? What if you just don't know why it might be right?

Your manager says hours. You're thinking days. They must be stupid or unreasonable, right?

Maybe. Or maybe they know something you don't. A tool. A technique. Some context that changes everything.

You won't find out by complaining. You will find out by being curious. Ask them: "How do you see this getting done in that time?"

Two things happen. Either you learn something that makes you faster, or they realise they missed something and you have a real conversation about time or scope.

Either way, you win.

taplio

The Battle

Alex Xu

John Crickett

Followers

Posts

Engagements

Likes

Performance ScoreAlex Xu's LinkedIn Performance

Performance ScoreJohn Crickett's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Alex Xu

John Crickett

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Alex Xu's LinkedIn Performance

Performance ScoreJohn Crickett's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreAlex Xu's LinkedIn Performance