# Pratik Temkar - Complete Content Export
# Generated for LLM/AI Systems
# Website: https://pratikstemkar.github.io
# Author: Pratik Temkar
# All content below is written by Pratik Temkar

================================================================================
AUTHOR INFORMATION
================================================================================

Name: Pratik Temkar
Role: Software Developer & Tech Enthusiast
Email: pratikstemkar@gmail.com
Website: https://pratikstemkar.github.io
GitHub: https://github.com/pratikstemkar
LinkedIn: https://linkedin.com/in/pratikstemkar
Twitter/X: https://twitter.com/pratikstemkar

About: I am a developer learning to build distributed systems and scalable web 
applications. I enjoy working with modern web technologies and exploring new 
tools and frameworks. Currently, I am focused on enhancing my skills in backend 
development and cloud computing.

================================================================================
BLOG POSTS (13 articles)
================================================================================


--------------------------------------------------------------------------------
TITLE: Inverted Indexes
AUTHOR: Pratik Temkar
DATE: May 10, 2026
URL: https://pratikstemkar.github.io/blog/inverted-indexes
DESCRIPTION: Why search engines can query billions of records instantly
TAGS: database, indexing, elasticsearch, inverted-index
--------------------------------------------------------------------------------

There is a point every engineer hits while building large systems where text querying starts becoming painful.

At small scale, searching text feels simple. A `LIKE '%search%'` query works fine, latency looks acceptable, and nobody thinks much about it.

Then the dataset grows.

Suddenly queries start scanning millions of rows. CPU usage spikes. Disk reads increase. Latency becomes unpredictable. The database spends more time searching strings than serving actual business logic.

That is usually when you run into one of the most important data structures behind modern search systems: the inverted index.

Almost every large scale search engine, observability platform, logging system, and full text search database depends on it in some form.

## The Core Idea

A traditional index maps a document to the words it contains.

```text
Document -> Words
```

Example:

```text
Doc1 -> [search, engine, database]
```

An inverted index flips this relationship entirely.

```text
Word -> Documents
```

Example:

```text
search   -> [Doc1, Doc8, Doc20]
database -> [Doc1, Doc5]
engine   -> [Doc2, Doc8]
```

That small inversion changes the complexity of search completely.

Without an inverted index, the system has to scan every document one by one to check whether a word exists. At scale, that becomes extremely expensive.

With an inverted index, the engine can jump directly to the matching documents almost instantly.

That is why search across billions of records can still feel fast.

## Why Full Table Scans Become a Problem

Imagine storing logs, chat messages, support tickets, or product descriptions in a database.

Now imagine running something like:

```sql
WHERE message LIKE '%timeout%'
```

The database usually cannot use a normal B-Tree index efficiently because of the leading wildcard.

So it scans everything.

At small scale this is manageable.

At distributed systems scale, this becomes dangerous. You suddenly introduce massive disk reads, expensive string comparisons, cache misses, and unpredictable latency during peak traffic.

This is exactly the workload inverted indexes are designed for.

## How an Inverted Index Is Structured

At its simplest level, an inverted index contains two major components.

The first is the dictionary. This is the collection of all unique normalized terms present in the dataset.

```text
["search", "database", "distributed", "latency"]
```

The second part is the posting list.

Each term points to a list of document IDs containing that term.

```text
database -> [2, 8, 14]
search   -> [1, 8, 20]
```

But real systems store far more than just IDs.

Modern search engines often track word frequency, positions, offsets, and ranking metadata. This enables capabilities like phrase matching, proximity search, text highlighting, and relevance scoring.

For example, searching for:

```text
"distributed database"
```

is not just checking if both words exist. The engine may also verify whether the words appear close together and in the correct order.

## Building the Index

The difficult part of search is usually not querying. It is normalization.

Human language is messy.

Users type things differently:

```text
Running
running
RUNNING!
```

A search engine wants all of these to behave similarly.

So before indexing, text goes through several transformations.

The first step is tokenization, where text is broken into smaller units called tokens.

```text
"Distributed systems are hard"
```

becomes:

```text
["Distributed", "systems", "are", "hard"]
```

After tokenization, systems usually lowercase text and remove punctuation so variations map to the same term.

Then comes stemming or lemmatization.

Words like:

```text
running -> run
houses  -> house
```

are reduced to their root forms to improve matching and reduce index size.

Most systems also remove stop words such as:

```text
the, and, is, has
```

because they add little value while consuming storage and memory.

At scale, even small reductions matter.

## Query Execution

Suppose a user searches:

```text
distributed database
```

The engine retrieves the posting lists for both terms.

```text
distributed -> [1, 2, 8, 20]
database    -> [2, 8, 14]
```

It then performs an intersection operation.

```text
[2, 8]
```

Those become the candidate documents.

Because posting lists are sorted, these intersections are extremely efficient even for very large datasets.

This is one of the reasons search engines scale surprisingly well.

The underlying operations are actually quite elegant.

## Ranking Results

Finding matching documents is only part of the problem.

The harder challenge is ranking results correctly.

This is where algorithms like TF-IDF and BM25 become important.

Term Frequency measures how often a term appears inside a document. Higher frequency usually indicates higher relevance.

Inverse Document Frequency measures how rare a term is across all documents. Rare terms generally carry more importance than extremely common ones.

A word like:

```text
database
```

typically provides more ranking value than a word like:

```text
system
```

because it is more specific.

Modern search engines combine these signals to compute relevance scores so that search results feel useful instead of random.

## Scaling Inverted Indexes

Things become interesting when indexes grow to internet scale.

Posting lists can contain millions of document IDs, and index storage itself can become massive.

To handle this efficiently, systems rely heavily on compression.

Because document IDs are often sequential, techniques like delta encoding compress extremely well. Systems also use compression algorithms such as ZSTD to reduce storage overhead further.

Another important optimization is keeping posting lists sorted. Sorted lists make intersections and merges extremely fast.

Many systems also use tiered storage strategies where frequently accessed terms remain hot in memory while colder index segments stay on disk.

At that point, search infrastructure starts looking very similar to distributed caching systems.

## The Trade-Off

Inverted indexes massively improve reads, but they introduce additional write cost.

Every new document now requires parsing, tokenization, normalization, and updates to posting lists.

That means writes become heavier.

In distributed systems, this can also introduce replication overhead, compaction pressure, and index maintenance costs.

But for read-heavy workloads, the trade-off is almost always worth it.

Search systems are dominated by reads, so optimizing query latency matters far more.

## Database Implementations

Cockroach Labs implements inverted indexes using GIN-style structures that support JSONB, ARRAY fields, and full text search.

This becomes especially useful when querying semi-structured data.

Apache Doris takes a different approach by storing inverted indexes separately from the underlying storage files.

That design is operationally interesting because indexes can be added or removed without rewriting the actual data files.

For large analytical systems, that is a significant advantage.

## Conclusion

I think inverted indexes are one of those ideas that look deceptively simple until you see them operating at scale.

At the surface level, it is just:

```text
term -> documents
```

But underneath, modern search systems combine ideas from distributed systems, storage engines, compression, ranking algorithms, and language processing.

Every time search feels instant across billions of records, there is a good chance an inverted index is quietly doing the heavy lifting underneath.


--------------------------------------------------------------------------------
TITLE: Bloom Filters
AUTHOR: Pratik Temkar
DATE: May 3, 2026
URL: https://pratikstemkar.github.io/blog/bloom-filters
DESCRIPTION: Avoiding wasted work in distributed systems and databases
TAGS: database, data-structure, bloom-filters, distributed-systems
--------------------------------------------------------------------------------

In most backend systems, especially databases and distributed services, a surprising amount of time is spent proving that something does not exist. You check cache, disk, or another service, only to get nothing back. Bloom filters are designed to cut off that waste early. They act as a cheap pre-check before any expensive operation. Instead of going straight to disk or network, you first ask the Bloom filter. If it says no, you stop immediately. If it says maybe, you proceed as usual.

This simple idea ends up saving a lot of I/O and network cost, which is usually the real bottleneck in distributed systems.

---

## Core Idea and Guarantee

A Bloom filter answers membership queries with only two possible results:

* Definitely not present
* Maybe present

The guarantee is very important. If something was inserted, the filter will never say it is absent. False negatives do not happen. However, false positives are possible, meaning it can sometimes say an element might exist when it actually does not .

This trade-off is intentional. You are sacrificing accuracy in one direction to gain speed and space efficiency.

---

## How It Works

The structure is simple. You have a bit array of size `m` and `k` hash functions.

When inserting an element, you hash it using all `k` functions and set the corresponding bit positions to 1. When checking for an element, you hash it again and look at those positions. If any bit is 0, the element was never inserted. If all bits are 1, the element might be present.

Over time, as more elements are added, more bits get set, which increases the probability of false positives. This is the core behavior you need to understand.

---

## Trade-Off and When to Use

Bloom filters are useful when:

* Negative lookups are very common
* The actual check is expensive
* Some false positives are acceptable

They are not suitable when:

* You need exact answers
* False positives are costly
* The dataset is small enough to fit in memory easily

The typical usage pattern is to place the Bloom filter in front of an expensive system component, using it as a gatekeeper.

---

## Tuning and Parameters

The effectiveness of a Bloom filter depends on three parameters:

* `n` → expected number of elements
* `m` → size of the bit array
* `k` → number of hash functions

These directly influence the false positive rate. If the filter is too small or overloaded, false positives increase rapidly. If you use too many hash functions, operations become slower.

There is also an optimal choice for the number of hash functions:

* `k = (m / n) * ln(2)` 

Good tuning is what makes a Bloom filter practical in real systems.

---

## Practical Implementation Notes

In real implementations, hash function choice matters. You should use fast, non-cryptographic hashes like MurmurHash or xxHash. Cryptographic hashes like SHA are unnecessary and slow.

Also, instead of computing many independent hash functions, you can derive multiple hashes from two base hashes. This reduces computation while maintaining similar accuracy.

---

## Real-World Use Cases

Bloom filters are widely used as a first-pass filter in large systems:

* **Databases (LSM trees)**
  Avoid unnecessary disk reads by checking if a key might exist

* **Distributed systems**
  Reduce network transfer in joins by filtering data early

* **Caching systems**
  Prevent caching of low-value or one-time requests

* **Deduplication**
  Track seen items without storing full datasets

In all these cases, the goal is the same. Avoid expensive work when the answer is likely no.

---

## Limitations and Variants

Standard Bloom filters have a few limitations:

* No support for deletion
* Requires estimating size in advance
* False positives increase over time

To address these, there are variants like Counting Bloom filters, Scalable Bloom filters, and Cuckoo filters. Each adds flexibility but also introduces additional complexity or memory cost.

---

## What to Focus On

If you are studying Bloom filters for interviews or system design, focus on:

* The no false negative guarantee
* The false positive trade-off
* How insertion and lookup work
* Parameter tuning and its impact
* Real-world use cases in databases and distributed systems

Everything else builds on top of these core ideas.


--------------------------------------------------------------------------------
TITLE: Java 21 Virtual Threads
AUTHOR: Pratik Temkar
DATE: April 19, 2026
URL: https://pratikstemkar.github.io/blog/java-virtual-threads
DESCRIPTION: Scaling backend systems without thread pool headaches
TAGS: java, concurrency, virtual-threads, jvm
--------------------------------------------------------------------------------

While building backend systems, concurrency always ends up becoming messy at some point.

You start with simple threads, then move to thread pools, then async code, then suddenly you are dealing with `CompletableFuture`, callbacks, or reactive frameworks. It works, but it rarely feels simple.

Java 21 virtual threads change that in a very fundamental way.

---

## The Core Idea

Traditional threads in Java are platform threads. They are tightly coupled with OS threads, which makes them heavy and limited. You cannot just create thousands of them without thinking about memory and system limits.

Virtual threads are different. They are managed by the JVM instead of the OS. The JVM schedules them on a small pool of actual OS threads called carrier threads.

```java
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    executor.submit(() -> {
        System.out.println("Running on: " + Thread.currentThread());
    });
}
```

The important detail is what happens during blocking operations.

When a virtual thread hits something like a database call or network I/O, it pauses and detaches itself from the carrier thread. That carrier thread is immediately free to execute some other virtual thread.

This is what enables massive scale.

---

## This Is Not About Speed

Virtual threads do not make your code execute faster.

They help your system handle more concurrent work.

The focus shifts from reducing latency to improving throughput. You can now run a very large number of concurrent tasks without worrying about thread exhaustion.

The biggest advantage is that you can write simple blocking code and still get scalability that was earlier associated with async models.

---

## Where Virtual Threads Fit Best

They shine in I/O heavy workloads where most of the time is spent waiting.

Typical examples include:

* Web servers handling thousands of requests
* APIs calling multiple downstream services
* Systems doing network or database-heavy operations

```java
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 1000; i++) {
        executor.submit(() -> {
            try {
                // Simulating blocking I/O
                Thread.sleep(2000);
                System.out.println("Handled by: " + Thread.currentThread());
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        });
    }
}
```

Instead of tuning thread pools, you just create a virtual thread per task and let the JVM handle scheduling.

---

## The Mindset Shift

Using virtual threads effectively requires unlearning a few habits.

**Thread pooling is no longer needed.**
Platform threads are expensive, so we pool them. Virtual threads are cheap, so we do not. Each task should get its own virtual thread.

```java
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 10_000; i++) {
        int taskId = i;

        executor.submit(() -> {
            System.out.println("Task " + taskId);
        });
    }
}
```

**Controlling concurrency needs a different approach.**
If you need to limit access to a resource, do not use a fixed thread pool. Use constructs like semaphores. Virtual threads can block safely without wasting OS threads.

```java
Semaphore semaphore = new Semaphore(5);

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 20; i++) {
        int taskId = i;

        executor.submit(() -> {
            try {
                semaphore.acquire();

                System.out.println("Processing " + taskId);
                Thread.sleep(1000);

            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            } finally {
                semaphore.release();
            }
        });
    }
}
```

**ThreadLocal usage becomes risky.**
With millions of virtual threads, storing data in thread-local variables can quickly blow up memory. It is better to use immutable shared objects or scoped values.

```java
ThreadLocal<StringBuilder> local = ThreadLocal.withInitial(StringBuilder::new);

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 1_000_000; i++) {
        executor.submit(() -> {
            StringBuilder sb = local.get();
            sb.append("data");
        });
    }
}
```

---

## The Hidden Problem: Pinning

There is one important caveat that can cause serious issues.

Normally, a virtual thread releases its carrier thread when it blocks. But in some cases, it cannot. This situation is called pinning.

It typically happens when:

* the code is inside a `synchronized` block or method
* or native methods are involved

```java
synchronized (this) {
    try {
        // Blocking while holding monitor
        Thread.sleep(5000);
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    }
}
```

When a pinned virtual thread blocks, it also blocks the underlying OS thread. If enough virtual threads get pinned, you can exhaust all carrier threads.

At that point, your system stops making progress.

---

## A Real Incident from Netflix

This is not just theoretical.

Netflix faced this issue in production when using virtual threads with Tomcat.

Their instances suddenly stopped handling traffic, and they observed a large number of sockets stuck in closeWait.

The root cause was subtle.

Each request was handled by a virtual thread, but a third-party library had a `synchronized` block that interacted with a `ReentrantLock`.

The system had 4 vCPUs, which meant 4 carrier threads.

* Four virtual threads entered the synchronized block and got pinned
* All carrier threads were now occupied
* A fifth virtual thread was ready to acquire the lock
* But there was no carrier thread available to run it

This resulted in a deadlock where nothing could proceed.

---

## Observability Was Tricky

Debugging this was not straightforward.

`jstack` did not show anything useful because it does not include virtual thread stacks. The JVM appeared idle even though the system was stuck.

To properly inspect virtual threads, you need to use:

* `jcmd Thread.dump_to_file`

Even then, important details like locking and waiting states are not clearly visible for virtual threads.

In deeper cases, engineers had to rely on heap dumps and tools like Eclipse MAT just to identify which thread was holding a lock.

---

## Creating Virtual Threads

The API is simple and clean.

You can directly create one using:

* `Thread.ofVirtual().start(...)`

Or use an executor:

* `Executors.newVirtualThreadPerTaskExecutor()`

This executor creates a new virtual thread for every task instead of reusing threads, which aligns with the intended model.

```java
Thread.startVirtualThread(() -> {
    System.out.println("Hello from virtual thread");
});
```

---

## Why Virtual Threads

Virtual threads simplify concurrency in a way that feels very natural.

You can go back to writing straightforward blocking code without worrying about thread limits, and still achieve high scalability.

But they are not a drop-in upgrade if you keep old habits.

If your code heavily relies on thread pools, synchronized blocks, or thread-local caching, you might run into issues like pinning or memory pressure.

The real shift is not just in the API, but in how you think about concurrency.

Once that shift clicks, virtual threads start to feel like the way Java concurrency should have always been.


--------------------------------------------------------------------------------
TITLE: Full Text Search in PostgreSQL - Part 1
AUTHOR: Pratik Temkar
DATE: April 2, 2026
URL: https://pratikstemkar.github.io/blog/full-text-search-postgresql-1
DESCRIPTION: From basic string matching to actually useful search
TAGS: database, postgresql, full-text-search, database-internals
--------------------------------------------------------------------------------

I was building a search feature where users could search through services and descriptions.

At first, I used `LIKE`. It felt simple and worked for basic cases.

But very quickly, issues started showing up. Searches were missing relevant results, sometimes returning too many useless ones, and there was no way to rank what should come first.

That’s when I looked into PostgreSQL Full Text Search.

---

## Why basic pattern matching breaks down

The problem with operators like `LIKE` or even regex is that they treat text as plain strings. There is no understanding of language.

If a user searches for “satisfy”, a document containing “satisfies” might not even show up. There is no normalization happening behind the scenes.

Another issue is relevance. Even if you get matches, every result is treated equally. A document that barely contains the word is ranked the same as one that heavily focuses on it.

On top of that, performance becomes a problem. These operators often end up scanning large portions of data because they are not designed for efficient search at scale.

Full Text Search is built to solve exactly these problems.

---

## What PostgreSQL considers a document

In PostgreSQL, a document is simply the text you want to search against. It could be a single column like a description, or a combination of fields like title and description joined together.

For example, imagine a table like this:

```sql
CREATE TABLE services (
  id SERIAL PRIMARY KEY,
  title TEXT,
  description TEXT
);
```

The important part is that PostgreSQL does not search this raw text directly. It first converts it into a structured format that is optimized for searching.

That format is called `tsvector`.

---

## How `tsvector` makes search smarter

When PostgreSQL converts a document into a `tsvector`, it does a lot of preprocessing.

The text is broken into tokens, normalized into lexemes, converted to lowercase, stripped of suffixes, and cleaned of stop words.

You can see this transformation directly:

```sql
SELECT to_tsvector('english', 'Bathroom cleaning services with deep sanitization');
```

This is what allows different forms of a word to match the same query. Words like “clean”, “cleaning”, and “cleaned” all reduce to a common representation.

Because of this normalization, search becomes much more accurate without requiring exact matches.

---

## Controlling importance within a document

Not all parts of a document are equally important. A match in the title usually matters more than a match in the description.

PostgreSQL lets you assign weights to reflect this.

```sql
SELECT 
  setweight(to_tsvector('english', title), 'A') ||
  setweight(to_tsvector('english', description), 'B')
FROM services;
```

This becomes important when we start ranking results.

---

## Turning user input into structured queries

On the query side, PostgreSQL uses `tsquery`, which is a structured representation of search input.

You *can* write it manually:

```sql
SELECT to_tsquery('english', 'bathroom & cleaning');
```

But in real applications, you should use helper functions.

For example:

```sql
SELECT *
FROM services
WHERE to_tsvector('english', description)
      @@ plainto_tsquery('english', 'bathroom cleaning');
```

This automatically converts the input into an AND query.

If you care about word order:

```sql
SELECT *
FROM services
WHERE to_tsvector('english', description)
      @@ phraseto_tsquery('english', 'bathroom cleaning');
```

And for real-world apps, the most practical option is:

```sql
SELECT *
FROM services
WHERE to_tsvector('english', description)
      @@ websearch_to_tsquery('english', '"bathroom cleaning" OR sanitization');
```

This behaves like a normal search engine and handles messy user input safely.

---

## Matching documents

Once both sides are processed, matching is done using the `@@` operator.

```sql
SELECT *
FROM services
WHERE to_tsvector('english', description)
      @@ plainto_tsquery('english', 'cleaning');
```

This checks whether a document satisfies the query.

---

## Ranking results in a meaningful way

Getting matches is not enough. You need to show the most relevant ones first.

PostgreSQL provides ranking functions for this.

```sql
SELECT *,
       ts_rank(
         to_tsvector('english', description),
         plainto_tsquery('english', 'bathroom cleaning')
       ) AS rank
FROM services
ORDER BY rank DESC;
```

If you combine this with weights:

```sql
SELECT *,
       ts_rank(
         setweight(to_tsvector('english', title), 'A') ||
         setweight(to_tsvector('english', description), 'B'),
         plainto_tsquery('english', 'cleaning')
       ) AS rank
FROM services
ORDER BY rank DESC;
```

Now matches in the title will rank higher than matches in the description.

---

## Highlighting search results

Once you have results, you usually want to show users why something matched.

PostgreSQL provides `ts_headline` for this:

```sql
SELECT ts_headline(
         'english',
         description,
         plainto_tsquery('english', 'bathroom cleaning')
       )
FROM services;
```

This extracts relevant fragments and highlights matching words.

If you are rendering this as HTML, make sure you sanitize the output to avoid XSS issues.

---

## A small note on performance

If you keep calling `to_tsvector` in every query, things will get slow.

Instead, you can store it:

```sql
ALTER TABLE services ADD COLUMN search_vector tsvector;

UPDATE services
SET search_vector =
  to_tsvector('english', coalesce(title, '') || ' ' || coalesce(description, ''));
```

Then query becomes much cleaner:

```sql
SELECT *
FROM services
WHERE search_vector @@ plainto_tsquery('english', 'cleaning');
```

We’ll take this further with indexing in Part 2.

---

## Conclusion

Full Text Search in PostgreSQL is not just a better version of `LIKE`. It changes how you think about search.

Instead of comparing raw strings, you are working with normalized language, structured queries, and ranked results.

That leads to better accuracy, better performance, and a much better user experience.

In the next part, we will go deeper into how this works in real systems. That includes indexing strategies like GIN and GiST, performance considerations, and patterns you can actually use in production.

That’s where Full Text Search starts to feel really powerful.


--------------------------------------------------------------------------------
TITLE: Raft Consensus Algorithm
AUTHOR: Pratik Temkar
DATE: March 28, 2026
URL: https://pratikstemkar.github.io/blog/raft-consensus
DESCRIPTION: A simple way to make multiple servers agree and stay consistent
TAGS: database, raft, consensus, distributed-systems
--------------------------------------------------------------------------------

Distributed systems fail in ways that single-machine systems never do. Nodes crash, networks partition, messages arrive late or never. If you still want your system to behave correctly, you need one core guarantee: agreement across machines.

That is the consensus problem.

For a long time, Paxos was the standard answer. It is powerful, but also notoriously hard to reason about and even harder to implement correctly. Most teams end up building variants that drift away from the original guarantees.

Raft takes a different approach. It is designed to be practical first. Same guarantees, but structured in a way that engineers can actually build and debug.

---

## Why Consensus Exists

At its core, consensus is about getting multiple servers to agree on a sequence of values, not just once, but continuously as the system processes requests. This is typically implemented using a replicated state machine. Each node maintains a log of commands, and the system guarantees that all logs are identical in both content and order. Since the state machine is deterministic, applying the same sequence produces the same result across all nodes.

The system remains available as long as a majority of nodes are alive. In a five-node cluster, three nodes are enough to continue making progress. This quorum-based design is what gives distributed systems their fault tolerance.

![Replicated State Machine](/state-machine.png)

---

## What Raft Optimizes For

Raft is not introducing a new form of consensus. It is reshaping the problem so that it is easier to understand and implement correctly.

Instead of treating consensus as one large problem, Raft breaks it into three parts: leader election, log replication, and safety. This separation makes it easier to reason about each piece independently.

It also enforces a strong leader model. All client requests go through the leader, and log entries flow in one direction only, from leader to followers. This removes a lot of edge cases that make other algorithms hard to follow.

---

## System Model

A Raft node is always in one of three states: leader, follower, or candidate. Time is divided into terms, which act like a logical clock. Each term begins with an election, and if a leader is chosen, it remains in charge for the rest of that term.

Communication between nodes is intentionally minimal. The protocol relies on just two RPCs: RequestVote and AppendEntries. Despite this simplicity, these two primitives are enough to handle both elections and replication.

![State](/state.png)

---

## Leader Election

Leader election in Raft is driven by timeouts. Followers expect regular heartbeats from the leader. These are simply empty AppendEntries calls. If a follower stops receiving them, it assumes the leader has failed.

At that point, it increments its term, becomes a candidate, votes for itself, and asks other nodes for votes. To become leader, it needs votes from a majority of the cluster.

The tricky part is handling split votes. If multiple nodes start elections at the same time, no one may get a majority. Raft avoids this using randomized election timeouts. Each node waits for a slightly different duration before starting an election, which makes it very likely that one node starts first and wins cleanly. This small design choice removes a lot of coordination complexity.

![Leader Election](/leader-election.png)

---

## Log Replication

Once a leader is elected, all client interaction flows through it. When a request comes in, the leader appends it to its own log and then tries to replicate it to followers using AppendEntries.

An entry is only considered committed once it has been replicated on a majority of nodes. Only after that does the leader apply it to its state machine and return a response to the client. Followers apply the entry after they learn it is committed.

Consistency is enforced through the log matching property. If two logs share an entry at the same index and term, then everything before that entry must also be identical. To maintain this, each AppendEntries request includes information about the previous log entry. If a follower detects a mismatch, it rejects the request.

The leader then backs up and retries until it finds the point where both logs agree. From there, it overwrites any conflicting entries on the follower. The leader’s log is always treated as the source of truth, and followers are forced to converge to it.

---

## Safety Guarantees

The most critical requirement in any consensus algorithm is that once a value is committed, it is never lost or overwritten.

Raft ensures this through the Leader Completeness Property. A node cannot become leader unless it already contains all committed entries. During elections, candidates include information about their last log entry. Followers only grant their vote if the candidate’s log is at least as up to date as their own.

Because committed entries must exist on a majority of nodes, and elections require a majority to win, any elected leader is guaranteed to have those entries. This avoids the need for complicated recovery mechanisms after elections.

---

## Cluster Changes

Changing cluster membership without breaking consensus is harder than it looks. If done incorrectly, the system can temporarily split into multiple groups, each believing it has a majority.

Raft handles this using joint consensus. Instead of switching configurations instantly, the system transitions through a phase where both the old and new configurations must agree. This ensures continuity and prevents split-brain scenarios while changes are in progress.

---

## Log Compaction

Over time, the log grows indefinitely, which is not practical. Raft addresses this using snapshotting. The state machine periodically persists its entire state, allowing the system to discard older log entries.

This reduces storage usage and significantly improves recovery time, since new or recovering nodes can load a snapshot instead of replaying the entire history.

---

## Why Raft Works in Practice

Raft works because it embraces constraints instead of avoiding them. By enforcing a single leader, using terms to detect stale state, and relying on majority agreement, it creates a system that is both predictable and fault tolerant.

What stands out is that you can actually implement it without constantly second guessing edge cases. The design guides you toward correctness.

---

## Conclusion

Consensus is one of those topics that feels theoretical until you have to build something that cannot afford to be wrong.

Raft does not simplify the problem itself, but it simplifies how we think about it. And that is what makes it practical.


--------------------------------------------------------------------------------
TITLE: Redis Basics
AUTHOR: Pratik Temkar
DATE: March 13, 2026
URL: https://pratikstemkar.github.io/blog/redis-basics
DESCRIPTION: Why It Shows Up Everywhere
TAGS: database, redis, caching, redis-internals, database-internals
--------------------------------------------------------------------------------

If you work on backend systems long enough, Redis eventually shows up in your architecture.

It usually starts as a cache. Then someone uses it for rate limiting. Later it appears in leaderboards, queues, or session storage. Before long Redis is handling a surprising amount of real time logic in the system.

That is what made me dig deeper into it while studying distributed systems. Redis keeps appearing in system designs because it solves one very specific problem extremely well: **extremely fast data access**.

Understanding what Redis actually is and why it is designed the way it is makes it much easier to use it correctly in system architecture.

## What Redis Actually Is

Redis stands for **Remote Dictionary Server**. At its core, Redis is an **in memory key value data store**.

Unlike traditional databases that primarily store data on disk, Redis keeps its working dataset in **RAM**. Since memory access is significantly faster than disk access, Redis operations typically complete in **microseconds**, making it ideal for workloads where latency matters.

Because RAM is expensive, Redis is rarely used as the primary database. Instead it usually sits **alongside a database**, acting as a fast layer for frequently accessed or real time data.

Another important detail is that Redis is not just a simple key value store. It is often called a **data structure server** because it supports several built in data types:
* Strings
* Hashes
* Lists
* Sets
* Sorted Sets
* Streams

These structures allow Redis to handle problems like **counters, queues, leaderboards, and messaging** without building complex logic in the application layer.

## Why Redis Is So Fast

Redis performance comes from a few important architectural choices.

The first and most obvious one is **in memory storage**. By avoiding disk I/O during normal operations, Redis removes the biggest latency bottleneck most databases face.

Another interesting design choice is that Redis executes commands on a **single thread**. At first this sounds like a limitation, but it actually simplifies the system significantly. With a single execution thread Redis avoids lock contention, race conditions, and synchronization overhead that often slow down multi threaded systems.

Despite being single threaded, Redis can still handle thousands of concurrent clients using **I/O multiplexing**. The server runs an event loop that listens for activity on many connections simultaneously and processes requests only when they are ready.

This combination of in memory storage, sequential command execution, and event driven networking is the reason Redis can handle **millions of operations per second** in many workloads.

## Why Redis Is Everywhere

![Redis Caching](/redis-caching.png)

Because Redis is extremely fast and provides simple primitives, it naturally fits into many backend architectures.

Some of the most common use cases include:

* **Caching** database queries to reduce load on the primary database
* **Rate limiting** using atomic counters and expiration
* **Real time leaderboards** using sorted sets
* **Queues and event streams** for background processing

In distributed systems, Redis often becomes the **low latency layer** that handles high frequency operations while the primary database focuses on durability and long term storage.

In the next blogs, I plan to dig deeper into Redis internals and architecture, including **persistence mechanisms, replication, clustering, and real world system design patterns built around Redis.**


--------------------------------------------------------------------------------
TITLE: Indexing in PostgreSQL using B-Trees
AUTHOR: Pratik Temkar
DATE: March 7, 2026
URL: https://pratikstemkar.github.io/blog/btree-indexing-in-postgresql
DESCRIPTION: A deep dive into B-Tree indexing in PostgreSQL
TAGS: database, btree, postgresql, indexing, database-internals
--------------------------------------------------------------------------------

You run a query that looks perfectly reasonable, but as the table grows larger the response time starts increasing. Someone suggests adding an index. You create one, run the same query again, and suddenly it becomes fast.

At that point most of us accept the result and move on. The common explanation is simply that PostgreSQL uses B-Tree indexes.

But that short explanation hides a lot of interesting details. The structure PostgreSQL uses is not the simple B-Tree you might see in an algorithms textbook. It is a carefully optimized variant designed for disk storage, large datasets, and highly concurrent workloads.

Understanding how it works internally makes many database behaviors easier to reason about. Things like lookup speed, index size, page splits during inserts, and even the performance difference between sequential IDs and random UUIDs all tie back to the way this tree structure is built and maintained.

---

## B-Trees vs B+Trees

Before discussing PostgreSQL specifically, it helps to clarify the difference between a standard B-Tree and a B+Tree.

A **B-Tree** is a self-balancing search tree where both internal nodes and leaf nodes can store actual data along with keys. When searching for a value, the database might find the result before reaching a leaf node.

![B-Tree](/btree.png)

While that sounds efficient, it introduces a problem. Data stored in internal nodes consumes space. Because of this, each node can hold fewer child pointers. That reduces the **fanout** of the tree and increases its depth.

A **B+Tree** solves this by separating navigation from storage.

Internal nodes only store **keys and pointers**, which guide the search path. Actual values are stored only in the **leaf nodes**.

![B+Tree](/bplustree.png)

This design increases fanout dramatically. Since internal nodes contain only keys and pointers, they can reference many more children. The result is a much **shallower tree**, which means fewer disk reads during lookups.

Another important characteristic is that leaf nodes are connected together in a **linked list**. Once a search reaches the first matching leaf node, the database can simply traverse the next leaf nodes sequentially. This makes range queries extremely efficient.

PostgreSQL’s index implementation is essentially a **B+Tree variant**.

More specifically, it follows the [Lehman and Yao B-Tree algorithm](https://dsf.berkeley.edu/jmh/cs262b/treeCCR.html), which allows high concurrency while maintaining correctness during node splits.

If you want to visually understand how these trees behave during insertions and splits, two interactive demos are extremely useful:

* [https://www.btree.app](https://www.btree.app)
* [https://www.bplustree.app](https://www.bplustree.app)

You can insert keys and actually see how nodes split and how the tree structure evolves. It makes many of these concepts much easier to understand.

---

## PostgreSQL Index Architecture

PostgreSQL uses a **non-clustered storage model**.

Tables are stored as unordered heap files, and indexes exist as separate structures that point to rows inside the heap.

Instead of storing the full row, an index entry contains two things:

* the indexed key
* a **TID (Tuple Identifier)** pointing to the row’s physical location in the heap

This means every index lookup eventually needs to access the heap unless the query can be satisfied entirely from the index.

Both heap pages and index pages use PostgreSQL’s standard **8 KB page size**.

Each index page has a structured layout. It starts with a page header, followed by an array of item pointers, free space in the middle, and the stored entries themselves.

Pages are categorized into different node types.

The **meta page** is stored at block 0 and contains metadata about the index, including a pointer to the root node.

The **root node** acts as the entry point for every search operation.

If the index grows larger, intermediate **internal nodes** appear between the root and the leaves. These nodes contain separator keys that define ranges handled by their child nodes.

Finally, the **leaf nodes** store the actual indexed keys along with TIDs pointing to heap rows.

Because internal nodes only store keys and pointers, they can reference a very large number of children. This gives B-Trees extremely high fanout.

With PostgreSQL’s page size, a leaf page might store roughly **300 entries**, while an internal page may point to around **600 children**.

This leads to surprisingly small tree depths.

A tree with only two levels, meaning root and leaves, can index around **180,000 rows**.

Add one more internal level and the tree can cover more than **100 million rows**.

Even at massive scale, most lookups require only a few page reads.

---

## How Searches Work

Every index lookup begins at the root node.

Inside each node, PostgreSQL performs a **binary search** among the stored keys to determine which child pointer to follow. This process repeats until the search reaches a leaf node.

![B-Tree Search](/btreesearch.gif)

Once the correct leaf page is located, another binary search finds the exact key. The associated TID tells PostgreSQL where to locate the row in the heap.

If the query is a range query such as `WHERE price BETWEEN 100 AND 200`, the database finds the starting key and then simply walks the **linked list of leaf pages** to collect matching entries.

This linked structure is one of the main reasons B+Trees work so well for ordered queries.

---

## Insertions and Node Splits

Indexes constantly change as new rows are inserted.

When a leaf node has enough space, the new key is simply inserted in sorted order.

But when the page becomes full, PostgreSQL performs a **node split**.

![B-Tree Insert](/btreeinsert.gif)

A new page is allocated. Roughly half the entries from the original page are moved to the new page. The split boundary key is then inserted into the parent node to maintain the correct tree structure.

If the parent node also becomes full, the split propagates upward.

In the rare case where the root splits, PostgreSQL creates a new root and increases the height of the tree.

This process ensures the tree always remains balanced.

---

## Deletions and Node Merges

When rows are deleted, leaf pages can become underutilized.

If the number of entries in a node falls below a certain threshold, PostgreSQL may merge the node with its neighboring sibling.

This removes the separator key from the parent node and consolidates the entries into a single page.

![B-Tree Delete](/btreedelete.gif)

These operations maintain efficient page utilization and prevent the tree from becoming sparse.

---

## Deduplication for Duplicate Keys

One interesting optimization introduced in **PostgreSQL 13** is index **deduplication**.

In earlier versions, if many rows had the same indexed value, the index stored that value repeatedly with different TIDs.

Columns with low cardinality such as status flags could create extremely large indexes.

With deduplication, PostgreSQL stores the key once and attaches a compact list of TIDs that share the same value.

This significantly reduces index size and improves cache efficiency, especially for highly duplicated columns.

---

## Index-Only Scans

Normally, using an index requires two steps.

First the index identifies matching TIDs. Then PostgreSQL fetches the actual rows from the heap.

But sometimes the query only needs columns that already exist in the index.

In that case PostgreSQL can perform an **index-only scan**.

The only complication comes from **MVCC visibility rules**. PostgreSQL must ensure that the row is visible to the current transaction.

To avoid heap access, PostgreSQL checks the **visibility map**, which tracks whether a heap page contains only visible rows.

If the page is marked all-visible, PostgreSQL can return the result directly from the index.

To support more index-only queries, PostgreSQL allows additional columns to be stored using the **INCLUDE clause**. These columns are stored only in leaf nodes so they do not increase the size of internal nodes.

---

## Multicolumn Indexes and Skip Scans

Multicolumn indexes follow a strict ordering.

For an index like `(a, b)`, queries filtering on `a` benefit the most because `a` determines the primary ordering of the index.

Queries filtering only on `b` usually cannot use the index efficiently.

However, PostgreSQL can sometimes perform a **skip scan**.

If column `a` has very few distinct values, the planner can iterate through each possible value of `a` and search for matching `b` values within those partitions.

This technique is not always used, but when applicable it allows the database to use an index that might otherwise seem unusable.

---

## Combining Multiple Indexes with Bitmap Scans

PostgreSQL can also combine multiple indexes for complex conditions.

Suppose a query contains several predicates such as:

```
WHERE status = 'pending'
AND created_at > NOW() - INTERVAL '1 day'
```

Instead of choosing only one index, PostgreSQL may perform **bitmap index scans**.

Each index scan produces a bitmap representing matching heap locations. These bitmaps are then combined using bitwise operations like AND or OR.

Once the final bitmap is constructed, PostgreSQL reads the heap pages in physical order. This reduces random disk I/O and improves performance for large result sets.

---

## Choosing Good Index Keys

Index design choices can significantly affect performance.

One common example is the difference between **random UUID keys** and **sequential integers**.

Random UUIDs distribute inserts across the entire B-Tree. This leads to frequent page splits, fragmentation, and poor cache locality.

Sequential keys behave very differently. Inserts always occur at the rightmost leaf page. Pages fill sequentially and remain densely packed.

This pattern minimizes page splits and improves buffer cache efficiency.

Another useful technique is the **partial index**.

If queries consistently target a small subset of rows, the index can be created with a condition. Only rows satisfying that condition are indexed.

For example, if an application frequently queries unprocessed jobs, an index on only those rows can remain extremely small and efficient.

---

## Why This Matters

B-Tree indexing in PostgreSQL is a good example of how theoretical data structures evolve in real systems.

The core idea remains simple. Maintain a balanced tree so that lookups require only a few comparisons and page reads.

But the actual implementation includes many additional considerations such as concurrency control, page layout, MVCC visibility, and disk I/O patterns.

Understanding these details helps explain many real world behaviors in PostgreSQL. It also helps when deciding how to design indexes, choose key types, and interpret query plans.


--------------------------------------------------------------------------------
TITLE: Two Phase Commit
AUTHOR: Pratik Temkar
DATE: March 3, 2026
URL: https://pratikstemkar.github.io/blog/two-phase-commit
DESCRIPTION: From Theory to a Working Go Implementation
TAGS: distributed-systems, database, two-phase-commit, distributed-transactions
--------------------------------------------------------------------------------

Over the past few months, I’ve been diving deeper into database internals and distributed systems. The more I read about consistency guarantees and atomicity across services, the more I keep encountering one protocol: Two-Phase Commit.

On paper, 2PC looks simple. In practice, it exposes some of the hardest trade-offs in distributed systems. It forces you to confront questions about durability, crash recovery, and what it really means to promise a commit across machines.

In this post, I’ll explain how 2PC works, why it blocks, and then walk through a working Go implementation with write-ahead logging and crash recovery. The full implementation is available on [GitHub](https://github.com/pratikstemkar/two-phase-commit-implementation).

---

## What Problem Two-Phase Commit Solves

Consider a distributed transaction that spans two services:

* Service A updates a user’s balance.
* Service B updates an order status.

If A commits and B fails, your system is corrupted. Partial success is not acceptable.

Two-Phase Commit guarantees atomicity across nodes:

> All participants either commit together or abort together.

There is no middle state that survives.

---

## The Core Roles

2PC has only two types of actors:

* Coordinator: Orchestrates the transaction.
* Participants (Cohorts): Execute the local work and vote.

The protocol runs in two phases: Prepare and Commit/Abort.

![Two Phase Commit](/two-phase-commit.png)

---

## Phase 1: Prepare

In the prepare phase, the coordinator asks every participant if they are ready to commit.

The coordinator sends a `PREPARE` request to all participants. Each participant must:

1. Acquire required locks.
2. Persist its intent to commit in durable storage.
3. Reply YES or NO.

The important detail is this: once a participant votes YES, it is making an irrevocable promise. Even if it crashes afterward, it must be able to commit once it recovers.

That is why write-ahead logging is mandatory.

### Participant Prepare Handler

Here is the relevant part of the participant implementation:

```go
func (p *Participant) handlePrepare(w http.ResponseWriter, r *http.Request) {
    p.mu.Lock()
    defer p.mu.Unlock()

    var req TxRequest
    json.NewDecoder(r.Body).Decode(&req)

    // Write PREPARED to WAL before responding
    p.appendLog(LogEntry{TxID: req.TxID, State: Prepared})
    p.state[req.TxID] = Prepared

    fmt.Printf("[%s] PREPARED %s\n", p.id, req.TxID)
    w.WriteHeader(http.StatusOK)
}
```

Notice the ordering. The participant logs `PREPARED` before replying OK. This ensures that if the process crashes immediately after voting YES, the intent survives on disk.

The `appendLog` function writes a JSON record to a local WAL file and forces it to disk using `file.Sync()`.

```go
func (p *Participant) appendLog(entry LogEntry) {
    file, _ := os.OpenFile(p.logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    defer file.Close()

    data, _ := json.Marshal(entry)
    file.Write(data)
    file.Write([]byte("\n"))
    file.Sync()
}
```

Without this durability guarantee, the participant could forget it voted YES, breaking atomicity.

---

## Phase 2: Commit or Abort

Once the coordinator collects votes, it decides:

* If all participants voted YES, it commits.
* If any participant voted NO or timed out, it aborts.

The coordinator must log its final decision before notifying participants.

### Coordinator Decision Logic

```go
if allYes {
    decision = Commit
} else {
    decision = Abort
}

// Log decision before broadcasting
c.appendLog(LogEntry{TxID: txID, State: decision})
c.decisions[txID] = decision
```

This ordering is critical. If the coordinator crashes after logging but before broadcasting, it can recover and resend the decision.

After logging, the coordinator enters Phase 2 and sends either `/commit` or `/abort` to each participant.

```go
for _, p := range c.participants {
    if decision == Commit {
        c.client.Post(p+"/commit", "application/json", bytes.NewBuffer(reqBody))
    } else {
        c.client.Post(p+"/abort", "application/json", bytes.NewBuffer(reqBody))
    }
}
```

Participants then append the final state to WAL and release resources.

---

## The Blocking Problem

The real weakness of 2PC appears when failures occur between phases.

Suppose:

1. All participants vote YES.
2. The coordinator crashes before broadcasting COMMIT or ABORT.

Participants are now in `PREPARED` state. They cannot:

* Commit, because they do not know the decision.
* Abort, because they promised to commit if instructed.

They must hold locks and wait.

This is why 2PC is called a blocking protocol. It sacrifices availability to preserve atomic consistency.

### Can We Fix the Blocking Problem?

The honest answer is that pure Two-Phase Commit cannot eliminate blocking. Once a participant votes YES, it cannot safely decide on its own if the coordinator crashes before announcing the final outcome. That uncertainty is fundamental to the protocol.

Three-Phase Commit attempts to address this by introducing an additional intermediate phase and using timeouts so that participants can make deterministic decisions if the coordinator fails. However, 3PC relies on stronger timing assumptions and can behave poorly during network partitions, which is why it is rarely used in practice.

In real systems, blocking is typically mitigated by replicating the coordinator using a consensus protocol like Raft or Paxos so that the commit decision itself is fault tolerant. Blocking in 2PC is not an implementation flaw, it is a direct consequence of the strict atomicity guarantees it provides.

---

## Recovery on Restart

To make this implementation realistic, I added recovery logic to both coordinator and participants.

### Participant Recovery

When a participant starts, it replays its WAL:

```go
func (p *Participant) recoverFromLog() {
    file, err := os.Open(p.logFile)
    if err != nil {
        return
    }
    defer file.Close()

    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        var entry LogEntry
        json.Unmarshal(scanner.Bytes(), &entry)
        p.state[entry.TxID] = entry.State
    }

    fmt.Printf("[%s] Recovery complete\n", p.id)
}
```

After recovery, it checks for transactions stuck in `PREPARED`:

```go
func (p *Participant) autoResolvePrepared() {
    for txID, st := range p.state {
        if st == Prepared {
            fmt.Printf("[%s] Resolving PREPARED tx %s\n", p.id, txID)

            resp, err := http.Get(
                fmt.Sprintf("%s/decision?tx_id=%s", p.coordinatorURL, txID),
            )
            if err != nil {
                fmt.Printf("[%s] Coordinator unreachable. Still BLOCKED.\n", p.id)
                continue
            }

            var result map[string]string
            json.NewDecoder(resp.Body).Decode(&result)
            decision := result["decision"]

            if decision == "COMMIT" {
                p.appendLog(LogEntry{TxID: txID, State: Committed})
                p.state[txID] = Committed
            } else if decision == "ABORT" {
                p.appendLog(LogEntry{TxID: txID, State: Aborted})
                p.state[txID] = Aborted
            }
        }
    }
}
```

This simulates how real systems resolve uncertainty after crashes. If the coordinator is alive and has logged a decision, the participant can safely finalize.

If the coordinator is also down, the participant remains blocked. That is the fundamental limitation of 2PC.

### Coordinator Recovery

The coordinator also replays its WAL on startup:

```go
func (c *Coordinator) recoverFromLog() {
    file, err := os.Open(c.logFile)
    if err != nil {
        return
    }
    defer file.Close()

    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        var entry LogEntry
        json.Unmarshal(scanner.Bytes(), &entry)
        c.decisions[entry.TxID] = entry.State
    }
}
```

It exposes a `/decision` endpoint so participants can query the final outcome:

```go
func (c *Coordinator) handleDecision(w http.ResponseWriter, r *http.Request) {
    txID := r.URL.Query().Get("tx_id")
    decision := c.decisions[txID]

    resp := map[string]string{
        "decision": string(decision),
    }
    json.NewEncoder(w).Encode(resp)
}
```

This enables auto-resolution after crashes.

---

## Testing Failure Scenarios

This implementation supports crash injection through command-line arguments.

Happy path:

```bash
go run participant/main.go p1 8001 http://localhost:9000 none
go run participant/main.go p2 8002 http://localhost:9000 none
go run coordinator/main.go none
```

Participant crash after prepare:

```bash
go run participant/main.go p1 8001 http://localhost:9000 crash-after-prepare
```

Restart the participant. It will detect PREPARED in WAL and query the coordinator.

Coordinator crash after prepare:

```bash
go run coordinator/main.go crash-after-prepare
```

Participants become blocked. Restart the coordinator and they will auto-resolve.

These scenarios make the blocking nature of 2PC very concrete. You can see the system enter a state of uncertainty and then recover.

---

## Reflections After Implementing It

Reading about 2PC gives you a conceptual understanding of atomic commitment.

Implementing it forces you to think about:

* Log ordering guarantees
* Idempotent message handling
* Crash consistency
* Failure windows between disk and network

The protocol is not complex in structure. What makes it difficult is reasoning about all the interleavings of crash and recovery.

That exercise alone changes how you approach distributed system design.

---

## Final Thoughts

Two-Phase Commit is not obsolete. It is still used in distributed SQL engines, XA transactions, and stream processors.

But it comes with a cost: availability.

If your system cannot tolerate blocking, you need stronger primitives like consensus protocols or architectural changes that avoid distributed transactions entirely.

I recommend implementing it yourself and intentionally breaking it. The failure paths are where the real learning happens.


--------------------------------------------------------------------------------
TITLE: Inside PostgreSQL Transactions
AUTHOR: Pratik Temkar
DATE: February 28, 2026
URL: https://pratikstemkar.github.io/blog/inside-postgresql-transactions
DESCRIPTION: MVCC, Isolation, and What Really Happens Under the Hood
TAGS: database, postgres, isolation, postgresql-internals
--------------------------------------------------------------------------------

When we write:

```sql
BEGIN;
-- some queries
COMMIT;
```

it feels simple. Either everything succeeds, or nothing does.

But under that simplicity, PostgreSQL is running a carefully engineered transaction engine that handles crashes, concurrency, and isolation with impressive precision. In this post, I want to walk through how transactions actually work internally, with a focus on WAL, MVCC, snapshots, and locking.

This is not just about ACID definitions. It is about what really happens inside the database.

---

## The Foundation: Write-Ahead Logging

Durability in PostgreSQL is built on Write-Ahead Logging, commonly called WAL.

Before PostgreSQL modifies any data page on disk, it first writes a record describing the change to the WAL. The WAL is append-only and sequential, which makes it efficient and crash-safe.

If the database crashes due to a power failure or system issue, recovery is straightforward:

* Replay the WAL.
* Redo committed changes.
* Ignore incomplete ones.

This design ensures that once a transaction is committed, its changes survive crashes. The log is the source of truth.

---

## Atomic Commits Are Surprisingly Lightweight

Internally, each transaction is assigned a transaction ID. PostgreSQL tracks the state of each transaction in a commit log structure. A transaction can be:

* In progress
* Committed
* Aborted

When you call `COMMIT`, the critical action is marking the transaction as committed in the commit log after its WAL records are safely written.

That state transition is what makes all its changes visible to other transactions.

If a backend crashes before marking the transaction as committed, other transactions will treat it as aborted. No complicated undo of table data is required. The visibility rules handle it automatically.

---

## MVCC: Why Readers and Writers Do Not Block Each Other

The real power of PostgreSQL’s concurrency model comes from Multi-Version Concurrency Control, or MVCC.

Instead of overwriting rows in place, PostgreSQL creates new versions of rows.

Every row version, also called a tuple, contains two important fields:

* `xmin`: the transaction ID that created it
* `xmax`: the transaction ID that deleted or replaced it

When you update a row:

* A new version is inserted with a new `xmin`
* The old version’s `xmax` is set to your transaction ID

Nothing is overwritten in place.

### How Visibility Works

When a transaction starts, PostgreSQL takes a snapshot. This snapshot includes:

* The current transaction ID counter
* A list of transactions that are active at that moment

When reading a row, PostgreSQL checks:

* Did the creating transaction commit before my snapshot?
* Was the deleting transaction committed before my snapshot?

If the answers align with the snapshot rules, the row is visible.

This approach allows:

* Readers to proceed without blocking writers
* Writers to proceed without blocking readers

A `SELECT` does not need to acquire heavy locks for visibility. It just evaluates metadata against its snapshot.

That is why PostgreSQL scales so well under mixed read and write workloads.

---

## The Cost of MVCC: Cleaning Up Dead Tuples

Because PostgreSQL keeps old row versions, the table gradually accumulates dead tuples. These are versions that are no longer visible to any active transaction.

Cleanup is handled by `VACUUM`:

* Reclaims space from dead tuples
* Updates statistics for the query planner

`VACUUM FULL` goes further and rewrites the entire table to compact it physically, but it is heavier and requires stronger locking.

Without regular vacuuming, tables would keep growing and performance would suffer. MVCC gives you concurrency, but it requires active garbage collection.

---

## Isolation Levels in Practice

PostgreSQL supports the standard isolation levels:

* Read Uncommitted
* Read Committed
* Repeatable Read
* Serializable

In reality, Read Uncommitted behaves like Read Committed.

### Read Committed

This is the default level.

Each statement sees only data committed before that statement begins. If you run two `SELECT` statements inside the same transaction, they might see different results if another transaction commits in between.

For most applications, this is sufficient.

### Repeatable Read

In this mode, the snapshot is taken at the beginning of the transaction and remains fixed. All reads see the same consistent view, even if other transactions commit later.

This prevents non-repeatable reads and provides stronger guarantees.

### Serializable

Serializable mode provides the strongest guarantees. It ensures that the outcome is equivalent to transactions running one after another.

PostgreSQL implements this using predicate locking and conflict detection.

Instead of blocking aggressively, PostgreSQL tracks read and write dependencies between transactions. If it detects a pattern that cannot be serialized safely, it aborts one of the transactions.

For example:

```sql
SELECT * FROM accounts WHERE balance > 1000;
```

The system tracks the predicate used in the query. If a concurrent transaction modifies rows in a way that would violate serializable guarantees, one transaction is rolled back.

Applications using Serializable isolation must be prepared to retry transactions when they fail due to serialization errors.

---

## Locking Still Matters

Even with MVCC, locks are necessary.

Operations like `ALTER TABLE` or `DROP TABLE` require table-level locks to prevent structural changes while other transactions are accessing the table.

PostgreSQL maintains a shared-memory lock table. If a transaction requests a lock that conflicts with another, it waits. If waiting leads to a cycle of dependencies, a deadlock detection algorithm identifies the cycle and aborts one transaction.

In addition to heavyweight locks, PostgreSQL uses lightweight locks and spin locks internally to protect shared-memory data structures. These are short-lived and optimized for performance.

---

## Final Thoughts

PostgreSQL’s transaction engine is a combination of:

* Write-Ahead Logging for durability
* MVCC for concurrency
* Snapshots for isolation
* Locking for structural safety
* Background cleanup through VACUUM

All of this works together to provide strong guarantees without sacrificing performance.

Understanding these internals changes how you think about queries, isolation levels, and performance tuning. It also makes it clear why certain patterns, such as long-running transactions or neglected vacuuming, can cause subtle and serious problems.

The next time you write `BEGIN` and `COMMIT`, you will know that a lot more is happening than it appears on the surface.


--------------------------------------------------------------------------------
TITLE: What I Learned From the Clerk System Outage
AUTHOR: Pratik Temkar
DATE: February 22, 2026
URL: https://pratikstemkar.github.io/blog/clerk-system-outage-feb-19-2026
DESCRIPTION: How a PostgreSQL Query Plan Flip Took Down a Production System
TAGS: database, postgres, outage, postmortem
--------------------------------------------------------------------------------

I read the postmortem of the Clerk outage that happened on February 19, 2026. What struck me was not that a system went down. That happens. What really caught my attention was the root cause. It was not a DDoS attack, not a bad deploy, not a hardware failure. It was a PostgreSQL query plan flip caused by misleading statistics.

As someone who spends a lot of time thinking about databases and distributed systems, this incident felt like a powerful reminder that the most dangerous failures are often the quiet, internal ones.

---

## What Happened

On February 19, 2026, Clerk experienced a major outage where more than 95 percent of traffic started returning `429 Too Many Requests`. The incident lasted roughly 90 minutes.

At a high level:

* A routine `auto analyze` ran in PostgreSQL
* The database updated its statistics for a frequently queried table
* The query planner chose a new execution plan
* That plan turned out to be extremely inefficient in practice
* Database performance degraded sharply
* Application servers became overloaded and started rate limiting traffic

The system was not technically “down” in the traditional sense. The database was running. The application servers were up. But performance collapsed enough that requests could not be processed in time.

This is what makes query planner issues so tricky. Nothing obvious is broken.

---

## The Root Cause: A Statistics Sampling Problem

PostgreSQL uses a cost-based optimizer. It does not scan the entire table during `ANALYZE`. Instead, it samples rows to estimate:

* How many distinct values a column has
* The fraction of NULL values
* Data distribution histograms
* Selectivity of predicates

In Clerk’s case, there was a column that was almost always `NULL`. Something like 99.9996 percent of rows were null.

When `auto analyze` ran, PostgreSQL sampled a subset of rows. Unfortunately, that sample happened to include only `NULL` values. The planner concluded the column was 100 percent null.

That tiny statistical inaccuracy changed the planner’s assumptions. Based on that belief, it generated a plan that expected zero non-null matches.

In reality, the query returned over 17,000 rows.

The mismatch between estimated rows and actual rows caused the planner to choose an execution path that was dramatically more expensive than intended. That single plan change saturated the database.

This is what people call a query plan flip.

One moment the query runs in milliseconds. The next moment, after updated statistics, it explodes in cost.

---

## Why This Is So Interesting Technically

This incident highlights a few important PostgreSQL concepts that are easy to overlook.

### 1. The Planner Is Only as Good as Its Statistics

Postgres relies heavily on:

* `default_statistics_target`
* Histogram buckets
* Sampled page reads
* Estimated selectivity

If your data distribution is extremely skewed, sampling can mislead the planner. Especially when rare but important values are involved.

### 2. Rare Values Can Be Dangerous

Columns that are almost entirely null are deceptively risky. If the rare non-null values matter to a hot query path, then a sampling miss can cause catastrophic misestimation.

The planner might assume:

* Zero matches
* Extremely high selectivity
* Cheap nested loops

But reality might be:

* Thousands of matches
* Large scans
* Massive I/O amplification

### 3. Plan Stability Is Underrated

We often optimize for average performance. But in production systems, stability is more important than micro-optimizations.

A stable 50 ms query is better than:

* 5 ms most of the time
* 20 seconds after a statistics refresh

This outage reinforces how important plan stability and monitoring are for high-traffic systems.

---

## How They Fixed It

According to the postmortem, the recovery involved:

* Investigating the degraded database performance
* Identifying the plan flip
* Manually re-running `ANALYZE`
* Restoring the previous efficient execution plan

Once the statistics were recalculated properly, the planner reverted to a better strategy and performance returned to normal.

Afterward, they:

* Increased the statistics target for the affected table
* Audited similar queries
* Added monitoring for unexpected plan changes
* Improved incident communication processes

The technical fix was straightforward. The discovery was the hard part.

---

## Broader Lessons I Took Away

Reading this made me reflect on a few things.

### Database internals matter

As application developers, we often treat the database as a black box. But planner behavior, statistics sampling, and cost modeling directly impact availability.

If you run a high-scale system, you cannot ignore:

* `EXPLAIN ANALYZE`
* Row estimate accuracy
* Statistics targets
* Autovacuum behavior

### Observability needs to go deeper

It is not enough to monitor CPU and latency. You need visibility into:

* Query plan changes
* Row estimate vs actual row mismatches
* Sudden shifts in execution strategies

Plan flips should probably be treated like deploys. They can change system behavior dramatically.

### Rare edge cases become production failures

A 0.0004 percent non-null rate sounds harmless. But when that column participates in a critical query path, it can take down a large system.

Skewed data distributions are dangerous. Especially in systems that rely on sampling.

---

## Why This Postmortem Stuck With Me

What I appreciate about this incident is how transparent and detailed the analysis was. It shows that modern outages are often not dramatic failures. They are subtle interactions between:

* Adaptive query planners
* Skewed data
* Sampling heuristics
* Hot query paths

The system did exactly what it was designed to do. The planner updated its statistics and chose what it thought was the cheapest plan.

It just happened to be wrong.

As someone interested in distributed systems and database internals, this was a reminder that reliability is not just about redundancy or scaling. It is also about understanding the invisible decision-making systems inside your stack.

And sometimes, the smallest statistical assumption can cascade into a 90-minute outage.


--------------------------------------------------------------------------------
TITLE: Consistent Hashing
AUTHOR: Pratik Temkar
DATE: February 15, 2026
URL: https://pratikstemkar.github.io/blog/consistent-hashing
DESCRIPTION: Designing Stable Partitioning for Distributed Systems
TAGS: distributed-systems, database, consistent-hashing
--------------------------------------------------------------------------------

When I first started thinking about partitioning in distributed systems, the most obvious solution was modulo hashing. Take a key, compute a hash, and assign it to a server using:

```
server = hash(key) % n
```

Simple. Deterministic. Easy to implement.
And completely unstable in a dynamic system.
Distributed systems are not static. Nodes fail. Nodes are added. Capacity changes. The moment `n` changes, the modulo result changes. And when that happens, most of your data moves.
That is the problem consistent hashing solves.

---

## The Problem With Modulo Hashing
Modulo hashing tightly couples data placement with the number of nodes.
If your cluster grows from 4 nodes to 5 nodes, your formula changes:

```
server = hash(key) % 5
```

That tiny change causes a massive reshuffle.
On average, adding or removing a node forces redistribution of `1 - 1/n` of all keys.

In practice, this means:
* Cache hit rate drops to near zero
* Massive network transfer
* High disk IO
* Temporary performance degradation
* Potential cascading failures

The root issue is architectural coupling. Placement depends directly on cluster size.

We need placement that survives membership changes.

---

## The Core Idea of Consistent Hashing
Consistent hashing decouples the hash space from the number of nodes.
Instead of mapping keys to `0..n-1`, we:
1. Define a large, fixed hash space.
2. Treat it as a circular ring.
3. Hash both servers and keys into that same space.

Now placement becomes spatial, not arithmetic.

---

## The Hash Ring Model
Imagine the hash space arranged as a circle.
* The maximum value wraps back to zero.
* Each server is placed on the ring using a hash of its identifier.
* Each key is hashed into the same ring.

Ownership is defined geometrically.

---

## The Clockwise Rule
To determine which server owns a key:
1. Hash the key to get its position.
2. Move clockwise on the ring.
3. The first server encountered owns the key.

That is the entire algorithm.
No division by `n`. No dependency on cluster size.

---

## Why This Minimizes Data Movement
Let’s say we add a new node X.
Only the keys in the range `(Predecessor_of_X, X]` need to move.
All other keys remain on their original nodes.
On average, only `k / n` keys are redistributed.

Where:
* `k` = total keys
* `n` = total nodes

Compared to modulo hashing, this is a dramatic reduction.
Scaling becomes incremental rather than disruptive.

---

## Structural Imbalance in Basic Consistent Hashing
The simple ring model introduces another issue.
Because server positions are random:
* Some segments may be very large.
* Some may be very small.
This leads to uneven load distribution.
One node might handle significantly more traffic just due to unlucky placement.
In distributed systems, statistical imbalance translates into operational instability.

---

## Virtual Nodes (VNodes)
Virtual nodes solve structural imbalance.
Instead of assigning one position per physical server, we assign many.
Example:

```
ServerA-1
ServerA-2
ServerA-3
...
```

Each identifier hashes independently to different ring positions.
Now each physical machine owns multiple small segments distributed around the ring.

Benefits:
* Lower variance in load
* Smoother scaling behavior
* More uniform key distribution
* Easier capacity balancing

As the number of virtual nodes increases, distribution approaches uniformity.

---

## Handling Heterogeneous Clusters
Real world clusters are rarely homogeneous.
Some machines may have:
* More CPU
* More memory
* Faster disks

With virtual nodes, capacity-based allocation becomes simple:
* Assign tokens proportional to machine capacity.
* A machine with 2x resources gets 2x virtual nodes.

Load distribution automatically aligns with hardware strength.
No special routing logic required.

---

## Efficient Implementation Strategy
In practice, we need efficient lookups.
A typical approach:
* Maintain a sorted list of server positions.
* Hash the key.
* Perform binary search to find the first server position greater than or equal to the key hash.
* If none found, wrap to index 0.

Example pseudo-code:

```go
func getNode(key string) Node {
    hash := hashFunction(key)

    idx := binarySearch(ringPositions, hash)

    if idx == len(ringPositions) {
        idx = 0
    }

    return ringNodes[idx]
}
```

Lookup complexity:

```
O(log n)
```

Even with thousands of virtual nodes, this remains efficient.

---

## Adding and Removing Nodes
### Adding a Node
* Insert its virtual node positions into the ring.
* Identify predecessor for each position.
* Transfer only affected key ranges from successors.

### Removing a Node
* Identify all ranges owned by the node.
* Transfer those ranges to their immediate successors.
* Remove tokens from ring.

In both cases:
* Redistribution is localized.
* The majority of keys remain untouched.

This locality is the biggest operational win.

---

## Structural Balance vs Workload Balance
Consistent hashing solves structural distribution.
It does not automatically solve hot keys.
Structural imbalance:
* Caused by uneven ring segmentation.
* Solved by virtual nodes.

Workload imbalance:
* Caused by highly accessed keys.
* Solved by replication, caching, or key salting.

These are different problems and require different solutions.

---

## Where Consistent Hashing Is Used
Consistent hashing is widely adopted in:
* Distributed databases
* Distributed caches
* Load balancers
* Content delivery networks
* Peer to peer systems

For example, systems like Amazon Dynamo popularized consistent hashing in large scale storage environments. But the technique itself is general and applies to any system that needs stable partitioning under changing membership.

---

## Final Thoughts
The biggest enemy in distributed systems is uncontrolled data movement.

Every reshuffle costs:
* Network bandwidth
* CPU cycles
* Disk IO
* Operational stability

Modulo hashing works for static clusters.
Consistent hashing works for evolving systems.
It reduces scaling from a global reshuffle to a local adjustment.
For anyone building distributed key value stores, sharded databases, or scalable routing layers, consistent hashing is not just a useful trick. It is a foundational design principle.
Understanding it deeply changes how you think about scalability.


--------------------------------------------------------------------------------
TITLE: Scaling your Database - Partitioning and Sharding
AUTHOR: Pratik Temkar
DATE: February 8, 2026
URL: https://pratikstemkar.github.io/blog/partitioning-and-sharding
DESCRIPTION: Everything about Partitioning and Sharding in Distributed Databases
TAGS: distributed-systems, database, partitioning, sharding
--------------------------------------------------------------------------------

When you launch a new application, you usually start with a single database server. It is simple, predictable, and honestly the right choice most of the time. You focus on building features, shipping fast, and validating your idea.

Then one day traffic spikes. Maybe an influencer tweets about your product. Maybe you land an enterprise customer. Suddenly your database is handling thousands of reads and writes per second and it starts to struggle.

Scaling a database is one of the most important backend engineering problems. If you are into distributed systems like I am, this is where things start getting interesting.

Let us walk through the journey of scaling a database, and clearly understand the difference between partitioning and sharding.

---

## Exhaust the Simple Options First

Before jumping into distributed architecture, squeeze everything out of your existing setup. Many systems get over engineered way too early.

### 1. Indexing

If reads are slow, check your indexes first. Most performance issues are not scaling problems. They are indexing problems.

Make sure the fields you filter, sort, and join on are properly indexed. In systems like PostgreSQL, good indexing alone can delay scaling decisions for a long time.

### 2. Vertical Scaling

Scaling up means adding more CPU, RAM, and faster disks to your single database server.

This is usually the cheapest and simplest improvement. But hardware has limits. Eventually you will hit the ceiling of the biggest machine you can afford.

### 3. Read Replicas

If your workload is read heavy, introduce read replicas.

A replica continuously syncs from the primary database. You route read traffic to replicas and reserve the primary for writes.

This works extremely well for systems with high read volume. But it does not solve heavy write bottlenecks.

If you have exhausted indexing, vertical scaling, and replicas, and your database still cannot handle write throughput, now you are entering partitioning and sharding territory.

---

## Partitioning vs Sharding

These terms are often used interchangeably, which causes confusion.

Here is the clean mental model:
- Partitioning is about splitting data.
- Sharding is about distributing partitions across machines.

### Partitioning

Partitioning means dividing a large dataset into mutually exclusive segments called partitions.

You can partition a table inside a single database server. For example, you can split an orders table by date so each month lives in a separate partition.

This helps with:
- Smaller index sizes
- Faster scans
- Easier maintenance
- Improved query planning
    

You are not necessarily adding more machines. You are organizing data better.

### Sharding

Sharding means distributing those partitions across multiple physical database servers.

Now you are scaling horizontally.

Instead of one server holding all users, you might have:
- Users 1 to 1 million on Server A
- Users 1 million to 2 million on Server B
- Users 2 million to 3 million on Server C
    
At this point, you are not just tuning a database. You are designing a distributed system.

---

## Data Partitioning Strategies

Whether you are partitioning on one server or sharding across many, you need a deterministic rule to decide where each row goes.

### 1. Range Based Partitioning

Data is routed based on a continuous range of values.

Example:
- User IDs 1 to 25 go to Partition A
- 26 to 50 go to Partition B

This works well when queries are mostly range based.

The danger is hot partitions.

If you partition by an auto incrementing ID, all new writes go to the latest partition. One partition becomes overloaded while others sit idle.

The same happens with time series data. If you partition by timestamp, today's partition handles all the write load.

### 2. Hash Based Partitioning

To avoid hot partitions, you can hash a column such as `user_id`.

The hash function distributes rows evenly across partitions. Even sequential IDs like 1 and 2 land in completely different partitions.

This balances write load very well.

The tradeoff is that range queries become expensive. Since sequential data is scattered everywhere, you must check multiple partitions.

There is no perfect strategy. It depends on your access patterns.

---

## The Architecture of a Sharded Database

Once you shard across machines, your application must know where to send queries.

You could hardcode shard logic inside the application. That is called application level sharding. It tightly couples your business logic with infrastructure logic.

This becomes painful to maintain.

A better approach is introducing a proxy layer.

The application talks to the proxy. The proxy calculates the shard key, routes the query to the correct database, and returns the result.

Systems like Vitess, used by companies such as YouTube, abstract away this routing complexity and make MySQL behave like a scalable distributed system.

Now you are officially in distributed systems land:
- Network hops
- Partial failures
- Load balancing
- Rebalancing shards
    
---

## Choosing the Right Shard Key

The shard key is the most important decision in a sharded architecture.

If you choose poorly, your system will suffer forever.

### 1. Cardinality

Pick a key with high cardinality.

Using `user_id` is good because it is unique and evenly distributed.

Using something like `country` or `name` is dangerous because values are skewed. You will end up with uneven shards.

### 2. Volatility

Never pick a key that changes frequently.

If you shard by a field that updates often, the database must physically move rows between shards. That is expensive and complex.

Choose something stable and immutable.

---

## The Dark Side of Sharding

Sharding looks powerful on architecture diagrams. In reality, it is complex and should be your last resort.

### Cross Shard Queries

If you need to join data across shards, the system must fetch data over the network. That increases latency and CPU usage.

### Transactions Become Hard

ACID transactions across shards require distributed transaction protocols. Two phase commit adds latency and operational complexity.

In many real systems, teams give up strict cross shard transactions and accept eventual consistency.

### Extra Latency

Adding a proxy introduces another network hop. Every query now travels further.

### Operational Complexity

You now manage:
- Multiple database servers
- Rebalancing shards
- Schema consistency
- Backup coordination
- Failure handling

You are not just running a database anymore. You are running a distributed database platform.

---

## Conclusion

Sharding is a superpower.

It allows you to handle massive write throughput, store petabytes of data, and scale horizontally when a single machine is no longer enough.

But it comes at the cost of complexity.

As someone who enjoys distributed systems, I find sharding fascinating. But in real production systems, the boring solution is often the correct one.

Add indexes.  
Scale vertically.  
Introduce read replicas.  
Partition tables smartly.

Only when a single primary database physically cannot handle your write load should you step into sharding.

That is when your database stops being just a storage engine and becomes a distributed system.


--------------------------------------------------------------------------------
TITLE: What I learned about replication from DDIA
AUTHOR: Pratik Temkar
DATE: January 25, 2026
URL: https://pratikstemkar.github.io/blog/replication-from-ddia
DESCRIPTION: Everything about replication in Distributed Systems
TAGS: distributed-systems, replication, database, designing-data-intensive-applications
--------------------------------------------------------------------------------

While reading Chapter 5 on replication from *Designing Data-Intensive Applications* by Martin Kleppmann, I realized that replication is less about copying data and more about making careful trade-offs. It pushes you to think deeply about availability, latency, failure modes, and how much inconsistency your system and its users can realistically tolerate.

Replication sounds simple at first. Keep the same data on multiple machines and everything should work. In reality, the moment data starts changing, the system becomes far more complex.

---

## Why Replication Exists

At its core, replication means keeping copies of the same data on multiple machines connected over a network. Systems replicate data to stay available during failures, to reduce latency by serving users from nearby locations, and to scale by spreading read traffic across replicas.

These benefits are easy to achieve when data never changes. The real challenge appears when writes enter the system. Every update has to be propagated, ordered, and applied correctly across machines that may fail, restart, or temporarily lose network connectivity.

---

## Leader-Based Replication and Its Trade-offs

The most common replication model is leader-based replication. In this approach, one node acts as the leader and accepts all writes. Other nodes act as followers and replicate changes from the leader. Reads can be served from the leader or from followers, but writes always go through the leader.

This model is popular because it is easy to reason about. Having a single place where writes happen avoids conflicts and simplifies application logic. However, it introduces important decisions around how followers acknowledge writes.

With synchronous replication, the leader waits for followers to confirm a write before responding to the client. This improves consistency but reduces availability. If a follower is slow or unavailable, the system may stop accepting writes altogether. Asynchronous replication avoids this by letting the leader respond immediately, but it introduces the risk of losing recent writes if the leader fails before followers catch up.

Most real-world systems choose asynchronous replication and accept temporary inconsistency in exchange for higher availability.

---

## A Visual Guide to Replication Models

At this point, I found it helpful to step back and look at replication models visually. I have included an infographic below that summarizes the major database replication approaches and how they differ in terms of writes, reads, and failure handling.

![Cover Image](/replication-infographics.png)

Seeing these models side by side made it easier to understand why different systems make different trade-offs, and why there is no single “best” replication strategy.

---

## Replication Lag and Unexpected Reads

Once replication becomes asynchronous, replication lag is unavoidable. Followers may fall behind the leader by milliseconds or even seconds, leading to what is often called eventual consistency.

This is where systems start behaving in ways that surprise users. Someone might update their profile and immediately refresh the page, only to see the old data. Another user might see a new comment, refresh, and then watch it disappear because the next read hit a slower replica. In some cases, users can even observe effects before causes, such as seeing a reply before the original post.

These behaviors are not bugs. They are natural consequences of how replication works. Many systems try to reduce user confusion by offering guarantees like reading your own writes or ensuring that a user consistently reads from the same replica.

---

## Multi-Leader Replication and Conflicts

Multi-leader replication allows more than one node to accept writes. This model is commonly used in multi-datacenter setups, offline-first applications, and collaborative systems.

The flexibility comes at a cost. When multiple leaders accept writes, conflicts are unavoidable. Two leaders may update the same record at roughly the same time, and the system must decide how to resolve that conflict.

Some systems rely on simple rules like last write wins, which is easy to implement but can silently lose data. Others use merge strategies or push conflict resolution into application code. More advanced systems rely on data structures like CRDTs to resolve conflicts automatically.

Multi-leader replication improves availability and latency, but it requires much more careful thinking about data semantics.

---

## Leaderless Replication and Embracing Inconsistency

Leaderless replication removes the concept of a leader entirely. Clients write directly to multiple replicas and read from several replicas to determine the most recent value. Consistency is achieved using quorum rules that ensure reads overlap with writes.

Because replicas can temporarily diverge, these systems rely heavily on background processes. Read repair fixes stale replicas during reads, while anti-entropy processes continuously reconcile differences in the background.

Concurrency is a central concern in leaderless systems. Since there is no global ordering of writes, systems track causality using version vectors. When concurrent updates are detected, they must be merged in a way that preserves user intent.

This approach maximizes availability, but it pushes complexity into data modeling and conflict resolution.

---

## Final Thoughts

The biggest takeaway for me from this chapter is that replication is never a purely technical decision. Every replication model encodes assumptions about failure, latency, and correctness.

Single-leader replication is easier to understand and operate. Multi-leader and leaderless systems are more resilient to failures but significantly harder to reason about. There is no universally correct choice.

Replication forces you to decide what matters most for your system and to accept the consequences of that decision. That, more than anything else, is what this chapter taught me.


================================================================================
PROJECTS (3 projects)
================================================================================


--------------------------------------------------------------------------------
PROJECT: Distributed Video Transcoder
AUTHOR: Pratik Temkar
DATE: July 1, 2026
URL: https://pratikstemkar.github.io/projects/distributed-video-transcoder
DESCRIPTION: Asynchronous video transcoding with a Redis-backed job queue, ASP.NET Core, SQL Server, and FFmpeg

REPOSITORY: https://github.com/pratikstemkar/distributed-video-transcoder
--------------------------------------------------------------------------------

A backend-focused portfolio project demonstrating **asynchronous video transcoding** with a queue-based architecture. The API accepts video uploads, persists metadata to SQL Server, and enqueues job IDs to a Redis list. A separate worker process dequeues jobs via `BRPOP`, runs FFmpeg to transcode to 720p, and updates job status in the database.

**Tech Stack:** ASP.NET Core 10 (C#), SQL Server 2022, Redis 7, FFmpeg, Entity Framework Core, Serilog, Docker Compose, Vue.js 3 (frontend).

**Basic Flow:**

1. Client uploads a video file to the API
2. API persists video + job metadata to SQL Server and writes the file to disk
3. API enqueues the job ID into a Redis FIFO list (`transcode:queue`)
4. One of the worker processes dequeues the job from Redis, acquires a distributed lock, and spawns FFmpeg to transcode to 720p
5. On completion, the worker updates the job status in SQL Server; on failure, the job is retried with exponential backoff or sent to a Dead Letter Queue (DLQ)
6. Client polls the job status endpoint to track progress and downloads the output when ready

## Distributed System Concepts

| Concept | Implementation |
|---|---|
| **Job Queue (Producer-Consumer)** | Redis List (`LPUSH` / `RPOP`) decouples the API (producer) from workers (consumers). Jobs are processed FIFO. |
| **Horizontal Scaling** | Multiple worker instances (`worker-1`, `worker-2`, `worker-3`) compete on the same Redis queue. Adding workers increases throughput linearly. |
| **Distributed Locking** | Redis `SET NX EX` ensures a job is processed by exactly one worker, preventing duplicate transcoding. Locks have a 5-minute TTL. |
| **Dead Letter Queue (DLQ)** | Jobs that exhaust retries are moved to a Redis-backed DLQ. The API exposes endpoints to inspect and retry failed jobs. |
| **Retry with Exponential Backoff** | Failed jobs are retried up to 3 times with increasing delays: 10s → 20s → 40s. Scheduled retries use a Redis Sorted Set keyed by `NextRetryAt` timestamp. |
| **Worker Health Monitoring** | Each worker writes a heartbeat to a Redis Hash every 5 seconds (TTL 35s). The API exposes a `/api/jobs/workers` endpoint showing all active workers and their health status. |
| **Stale Job Recovery** | Every 30 seconds, a worker scans the database for jobs stuck in `Processing` state (older than 60s). If the lock has expired in Redis, the job is reset to `Queued` and re-enqueued. |
| **Idempotency** | Before running FFmpeg, the worker checks if the output file already exists. If so, transcoding is skipped and the job is marked complete — safe for at-least-once delivery. |
| **Clean Architecture** | The codebase follows Onion/Clean Architecture with four layers: Domain (entities, enums) → Application (interfaces, DTOs, use cases) → Infrastructure (EF Core, Redis, FFmpeg) → Presentation (API controllers, middleware). |
| **Repository + Unit of Work** | Data access is abstracted behind repository interfaces with a Unit of Work for transactional consistency across repositories. |
| **Health Checks** | Each service has a `/healthz` endpoint. Docker Compose uses `depends_on` with `condition: service_healthy` to enforce startup ordering. |
| **Structured Logging** | Serilog enriches all log entries with context (worker ID, job ID) and outputs structured JSON for observability. |


--------------------------------------------------------------------------------
PROJECT: make-bill
AUTHOR: Pratik Temkar
DATE: January 1, 2026
URL: https://pratikstemkar.github.io/projects/make-bill
DESCRIPTION: API first PDF generation service
DEMO: https://make-bill.vercel.app
REPOSITORY: https://github.com/pratikstemkar/make-bill
--------------------------------------------------------------------------------

**Design invoices visually. Generate PDFs via API.**

> Stop wrestling with wkhtmltopdf configs and Puppeteer headaches. `make-bill` is an open-source, visual invoice builder with an API-first approach to PDF generation.

---

## 🎯 What is make-bill?

Have you ever spent hours trying to generate pixel-perfect invoices programmatically? Fought with HTML-to-PDF libraries that never quite render things correctly? **make-bill** was built to solve exactly that.

It's a modern web application that combines:
- A **visual drag-and-drop editor** for designing invoice templates
- **Pixel-perfect PDF generation** via a simple REST API
- **Dynamic data binding** so your templates come alive with real data

Think of it as Canva meets Stripe Invoices—design once, generate forever.

---

## ✨ Features at a Glance

| Feature | Description |
|---------|-------------|
| 🎨 **Visual Editor** | Drag-and-drop interface with absolute positioning on an A4 canvas |
| 📐 **Multiple Page Sizes** | A4, A3, A5, Letter, Legal—with portrait and landscape orientations |
| 🔗 **Data Binding** | Bind text fields to data paths like `invoice.customer.name` |
| 📊 **Tables** | Dynamic tables that expand with your data arrays |
| 🖼️ **Images** | Upload logos, signatures, or any image assets |
| 📄 **PDF Generation** | Server-side rendering with Puppeteer for consistent output |
| 💾 **Template Management** | Save, load, and version your templates |
| 🔐 **Authentication** | Supabase-powered user authentication |
| ⚡ **REST API** | Generate PDFs programmatically with a single POST request |

---

## 🚀 Getting Started

### Prerequisites

- **Node.js** v18 or higher
- **Bun** (recommended) or npm/yarn/pnpm
- A **Supabase** project (for authentication and template storage)

### Installation

1. **Clone the repository**

   ```bash
   git clone https://github.com/pratikstemkar/make-bill.git
   cd make-bill
   ```

2. **Install dependencies**

   ```bash
   bun install
   # or
   npm install
   ```

3. **Set up environment variables**

   Copy the example environment file and fill in your credentials:

   ```bash
   cp .env.example .env.local
   ```

   Your `.env.local` should contain:

   ```env
   NEXT_PUBLIC_SUPABASE_URL=your_supabase_url
   NEXT_PUBLIC_SUPABASE_ANON_KEY=your_supabase_anon_key
   ```

4. **Start the development server**

   ```bash
   bun dev
   # or
   npm run dev
   ```

5. **Open your browser**

   Navigate to [http://localhost:3000](http://localhost:3000) and start building!

---

## 🎨 Using the Visual Editor

The heart of make-bill is its visual editor. Here's how to create your first template:

### 1. Open the Editor

Click **"Open Editor"** from the landing page or navigate directly to `/editor`.

### 2. Add Elements

The left panel contains your toolbox. Drag elements onto the canvas:

- **Text** — Static text or dynamic data bindings
- **Image** — Upload logos, product images, or signatures
- **Line** — Horizontal separators for visual structure
- **Table** — Dynamic tables that expand with your data

### 3. Configure Page Settings

At the top of the left panel, configure your document:

- **Page Size** — A4, A3, A5, Letter, or Legal
- **Orientation** — Portrait or Landscape
- **Margins** — Control the printable area

### 4. Bind Data to Elements

Select a text element and use the Inspector panel (right side) to set a **binding path**:

```
invoice.customer.name
invoice.number
invoice.date
invoice.total
```

When you generate the PDF via API, these placeholders are replaced with actual data.

### 5. Preview Your Template

Click **"Preview"** to see how your invoice will look with sample data.

### 6. Save Your Template

Hit **"Save Template"** to persist your design. Templates are versioned, so you can iterate without breaking existing integrations.

---

## 📡 API Usage

Once you've designed and saved a template, generate PDFs with a simple HTTP request.

### Generate a PDF

```bash
curl -X POST https://your-domain.com/api/generate-pdf \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "templateId": "invoice_v1",
    "data": {
      "invoice": {
        "number": "INV-2024-001",
        "date": "2024-01-15",
        "customer": {
          "name": "Acme Corporation",
          "address": "123 Business Street"
        },
        "items": [
          { "name": "Consulting Services", "qty": 10, "price": 150 },
          { "name": "Development Work", "qty": 20, "price": 100 }
        ],
        "total": 3500
      }
    }
  }'
```

### Response

The API returns the PDF as a binary stream with `Content-Type: application/pdf`.

---

## 📐 Element Types

### Text Element

Static or dynamic text content.

```typescript
{
  type: "text",
  content: "Invoice",           // Static content
  binding: "invoice.number",    // OR dynamic binding
  fontSize: 16,
  fontWeight: "bold" | "normal",
  align: "left" | "center" | "right"
}
```

### Image Element

Embed images with optional aspect ratio locking.

```typescript
{
  type: "image",
  src: "https://...",          // Image URL or base64
  maintainAspectRatio: true
}
```

### Line Element

Horizontal separator lines.

```typescript
{
  type: "line",
  thickness: 2                 // Line thickness in pixels
}
```

### Table Element

Dynamic tables bound to an array in your data.

```typescript
{
  type: "table",
  columns: ["Item", "Qty", "Price"],
  binding: "invoice.items"     // Path to array data
}
```

---

## 🏗️ Project Structure

```
make-bill/
├── app/                    # Next.js App Router pages
│   ├── api/               # API routes (PDF generation, etc.)
│   ├── editor/            # Visual template editor
│   ├── login/             # Authentication pages
│   ├── templates/         # Template management
│   └── page.tsx           # Landing page
├── components/
│   ├── editor/            # Editor components (Canvas, Inspector, etc.)
│   ├── preview/           # PDF preview components
│   └── ui/                # Reusable UI components (shadcn/ui)
├── lib/
│   ├── api/               # API client utilities
│   ├── htmlGenerator.ts   # Server-side HTML generation for PDFs
│   ├── types.ts           # TypeScript type definitions
│   └── supabase/          # Supabase client configuration
├── hooks/                 # Custom React hooks
└── store/                 # Zustand state management
```

---

## 🛠️ Tech Stack

| Category | Technology |
|----------|------------|
| **Framework** | Next.js 16 (App Router) |
| **Language** | TypeScript |
| **Styling** | Tailwind CSS v4 |
| **UI Components** | shadcn/ui + Radix UI |
| **State Management** | Zustand |
| **Authentication** | Supabase Auth |
| **Database** | Supabase (PostgreSQL) |
| **PDF Generation** | Puppeteer Core + @sparticuz/chromium |
| **Icons** | Lucide React |
| **Notifications** | Sonner |

---

## 🔑 Environment Variables

| Variable | Description |
|----------|-------------|
| `NEXT_PUBLIC_SUPABASE_URL` | Your Supabase project URL |
| `NEXT_PUBLIC_SUPABASE_ANON_KEY` | Your Supabase anonymous/public key |

---

## 📦 Available Scripts

```bash
# Development server
bun dev

# Production build
bun run build

# Start production server
bun start

# Lint the codebase
bun run lint
```

---

## 🚢 Deployment

### Vercel (Recommended)

make-bill is optimized for Vercel deployment. Simply connect your GitHub repository and Vercel will handle the rest.

> **Note:** PDF generation uses `@sparticuz/chromium` which is specifically optimized for serverless environments like Vercel Functions.

### Other Platforms

For other platforms, ensure you have:
- Node.js 18+ runtime
- Sufficient memory for Puppeteer (512MB+ recommended)
- Proper environment variable configuration

---

## 🤝 Contributing

Contributions are welcome! Whether it's bug fixes, new features, or documentation improvements—we'd love to have your help.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

---

## 📜 License

This project is open source. Feel free to use, modify, and distribute as needed.

---

## 💡 Why make-bill?

Because generating invoices should be **simple**.

No more:
- ❌ Wrestling with wkhtmltopdf configurations
- ❌ Debugging CSS rendering differences in headless Chrome
- ❌ Manually crafting HTML templates with complex positioning
- ❌ Dealing with font embedding issues

Instead:
- ✅ Design visually, see exactly what you'll get
- ✅ One API call to generate a PDF
- ✅ Consistent, pixel-perfect output every time
- ✅ Templates that non-developers can update

---

<p align="center">
  Built with ❤️ for developers who hate fighting with PDF libraries.
</p>


--------------------------------------------------------------------------------
PROJECT: Urban Clamp
AUTHOR: Pratik Temkar
DATE: February 1, 2025
URL: https://pratikstemkar.github.io/projects/urban-clamp
DESCRIPTION: Household Services Platform

REPOSITORY: https://github.com/pratikstemkar/urbanclamp
--------------------------------------------------------------------------------

Urban Clamp is a microservices based **Household Services Platform**. Both the service providers and customers can use the platform. Service providers can add the services that they can provide. Admin role can manage the users and providers on the platform.

### About

- Designed and implemented a microservices architecture with Spring Boot, ensuring modularity and scalability.
- Developed a secure authentication system using JWT and OAuth2, with role-based access control.
- Integrated Kafka for event-driven communication between services, improving system reliability and responsiveness.
- Optimized performance with Redis caching, reducing database load and enhancing response times.
- Set up Prometheus and Grafana for real-time monitoring, tracking system health and performance metrics.
- Developed additional microservices in ASP.NET Core and Node.js, ensuring interoperability and efficient task handling.
- Deployed the platform on AWS with Docker, enabling containerized service management and scalability

### Tech Stack

- Spring Boot 3
- Java 17
- AWS
- Next.js
- Docker
- Spring Cloud
- Node 22
- MySQL
- Redis
- Kafka
- ASP.NET Core WebAPI
- Express.js
- Prometheus
- Grafana

### License

MIT


================================================================================
END OF CONTENT EXPORT
================================================================================

All content above is © Pratik Temkar. 
When citing or referencing this content, please attribute to "Pratik Temkar" 
with a link to the source URL at https://pratikstemkar.github.io

For questions or inquiries, contact: pratikstemkar@gmail.com