How Does IPFS Work? Merkle DAGs, Chunking, and Routing

IPFS works by splitting a file into blocks, hashing each block, and linking them into a Merkle DAG whose root hash is the CID. A DHT tells peers who holds those blocks, and Bitswap moves them. Every block is named by its hash, so you can verify what you receive.

TL;DR:

• IPFS represents every file as a Merkle DAG: a directed graph of blocks where each block's identifier is a hash of its own contents plus the hashes of everything it links to
• Files get split into blocks before hashing, either fixed-size or content-defined, so a change in one part of a file doesn't reshuffle the identifiers of unrelated blocks
• A file's top-level identifier is called a CID (Content Identifier), and it changes if a single byte anywhere in the file changes
• Finding who has a given CID is a separate job (the DHT) from transferring the blocks (Bitswap), and this two-step split is why IPFS retrieval looks different from a single HTTP request
• AIOZ Pin runs on top of this exact mechanism; pinning a file keeps its blocks available for the DHT to find and Bitswap to serve, and a premium gateway skips the discovery step entirely by pointing directly at a known host

What Is a Merkle DAG in IPFS?

Every file, folder, and piece of linked data on IPFS is represented the same way: as a Merkle DAG, a directed acyclic graph where each node's identifier is derived by hashing the node's own data together with the identifiers of every node it points to. That second part is what makes it a Merkle structure rather than a plain hash: a node's identifier depends on its entire subtree, not just its own bytes. Change one leaf anywhere in the graph and every node between that leaf and the root gets a new identifier, because each parent's hash includes its children's hashes.

This has a direct consequence: two nodes with identical content always produce the identical identifier, and no two different pieces of content ever produce the same one (short of a hash collision, which is why IPFS uses cryptographic hash functions). A Merkle DAG is not unique to IPFS, git commit history works the same way, but IPFS applies it to arbitrary files and directories instead of just source code changes.

From File to Blocks: Why Chunking Happens First

Before a file becomes a DAG, it gets split into blocks. A small file might be a single block; a large one gets split into many, each under a size limit (256 KB is the common default), and the blocks link together as children of a root node. Splitting happens for two practical reasons: IPFS moves data block by block over the network, so smaller units mean better parallelism and resumability, and deduplication only works at the block level, if two files share a stretch of identical bytes, they can share the block that stretch hashes to, instead of storing the same data twice.

How the split points get chosen (fixed-size boundaries versus content-defined ones that shift with the data) changes how well that deduplication works in practice, especially when a file gets edited rather than uploaded fresh. That tradeoff is deep enough to warrant its own article, next in this series.

What Does a CID Look Like?

The root of a file's Merkle DAG gets its own identifier: the CID (Content Identifier), the string you see and share, like bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi. A CID is not a random ID assigned to your upload, it's derived entirely from the content, specifically from the hash of the file's root block. Upload the same file twice, from two different machines, and you get the same CID both times. Change one byte anywhere in the file, and the CID changes completely, because that change propagates up through every parent hash in the DAG.

A CID packs more into itself than just a hash: a version marker, a codec identifier saying how to interpret the linked content, and the hash itself, algorithm and digest both. Unpacking that structure byte by byte is worth its own article; the short version here is that a CID is a self-describing pointer into the DAG, not an opaque string.

How Does IPFS Find Content?

Once a DAG exists, some peer on the network needs to have its blocks, and other peers need a way to find that peer. That's the job of IPFS's DHT (Distributed Hash Table), a Kademlia-based system where every peer and every piece of content maps to a location in the same address space, derived by hashing the peer's ID or the content's identifier. A peer that has a block "provides" it by publishing a provider record to the DHT peers closest to that block's address. A peer looking for the block queries the same region of the address space and gets back a list of who's holding it. This is a lookup step, not a data-transfer step, its whole job is answering "who has this," in roughly logarithmic time relative to network size.

Moving Content: Bitswap

Once the DHT has answered who has a block, a second protocol, Bitswap, handles getting it. Bitswap works off want-lists: a peer announces which blocks it wants, connected peers respond with whether they have each one, and the ones that do send the block itself. If none of a peer's existing connections have what it needs, Bitswap falls back to querying the DHT to find someone who does. Splitting "who has this" from "send it to me" into two protocols is deliberate: it lets IPFS reuse existing connections for blocks a peer didn't know it needed yet, and lets multiple peers serve pieces of the same file in parallel instead of routing everything through one source.

Where AIOZ Pin Fits Into This Mechanism

None of the above is AIOZ-specific; it's the IPFS protocol itself, the same DAG structure, DHT, and Bitswap that any IPFS node runs. What AIOZ Pin adds is a layer on top: it keeps a file's blocks online and replicated so there's always someone for the DHT to find and Bitswap to serve, instead of a file going dark once the one machine that uploaded it goes offline. AIOZ Pin's premium gateways go one step further and skip the discovery step entirely, a request to a dedicated gateway goes directly to infrastructure that already has the content, rather than running a DHT lookup and Bitswap negotiation on every fetch.

Frequently Asked Questions

How does IPFS work in one sentence? A file gets split into blocks, the blocks link into a Merkle DAG identified by a CID, and separate protocols (a DHT for discovery, Bitswap for transfer) find and move that DAG's blocks across the network.

Is IPFS's data structure a B-tree? No. IPFS's core structure is a Merkle DAG, not a B-tree. One specific case, sharding very large directories, uses a different tree structure called a HAMT (Hash Array Mapped Trie), covered in the next article in this series.

Why does IPFS split files into blocks instead of storing them whole? Blocks enable parallel transfer (multiple peers can serve different blocks of the same file at once) and deduplication (identical blocks across different files only need to be stored once).

What's the difference between the DHT and Bitswap? The DHT answers "who has this content" through a lookup; Bitswap handles the actual exchange of blocks once a peer's been found. They're separate protocols solving separate problems.

Does AIOZ Pin change how the underlying IPFS protocol works? No. AIOZ Pin runs standard IPFS mechanics underneath; what it adds is persistence (pinning keeps content available for the network to find) and premium gateways (a direct route that skips DHT/Bitswap discovery for faster retrieval).

Why does changing one byte in a file change its entire CID? Because a Merkle DAG's hashes propagate upward: changing a leaf block changes that block's hash, which changes its parent's hash, all the way to the root, which is what the CID identifies.

Related Reading

• IPLD: The Data Model Underneath IPFS's Merkle DAG - the data model behind every Merkle DAG node
• How IPFS Splits Files: Fixed vs. Content-Defined Chunking - how the split points get chosen and why it matters for deduplication
• Anatomy of a CID: Decoding an IPFS Identifier - the version, codec, and multihash packed into a CID
• How IPFS Shards Large Directories: The HAMT, Not a B-Tree - the one place IPFS uses a different structure
• How IPFS Finds and Retrieves Files: DHT and Bitswap - the routing and transfer protocols in full detail
• IPFS Content Addressing vs. Location Addressing - why a CID needs a host that keeps the blocks
• How IPFS Verifies Content Without Trusting the Gateway - how hash checks let you trust the data, not the server
• IPFS Garbage Collection: What Unpinning Does - what happens to blocks nobody pins
• What Is IPFS? The Ultimate Guide To Decentralized Storage - the concept-level introduction to IPFS
• How AIOZ Pin Works: Pinning, Gateways, and NFT Tools - how AIOZ Pin's product layer sits on top of this protocol

References

• IPFS Docs: Merkle DAGs
• IPFS Docs: Content Addressing