How IPFS Finds and Retrieves Files: DHT and Bitswap

Request a file over HTTP and one server answers, at one address, end of story. Request a file over IPFS and there is no fixed address to ask, only a CID, so the network has to solve "who has this" before it can solve "send it to me." IPFS splits those into two separate protocols: a DHT (Distributed Hash Table) for the first question, Bitswap for the second. This closes out the series on how IPFS actually works by covering exactly how each step runs, and where a dedicated gateway on AIOZ Pin skips both entirely.

TL;DR:

• IPFS has no central directory of "which server has which file." Content routing is peer-to-peer: a Kademlia-based DHT lets peers advertise and locate content by CID
• Advertising content means publishing a provider record to the K=20 peers whose ID is numerically closest to the content's hash; looking it up means querying that same region of the address space
• Bitswap handles the actual transfer once a peer is found: it sends a want-have message, gets back a have or dont-have response, then requests the block itself with want-block
• If none of a peer's existing connections have a block, Bitswap falls back to querying the DHT to find someone who does, chaining the two protocols together
• AIOZ Pin's premium gateways skip this entire negotiation: a request goes straight to infrastructure that already has the content, instead of running a DHT lookup and Bitswap exchange on every fetch

Why Content Needs a Routing Step at All

A URL points at a server. A CID points at content, wherever it happens to live, which could be one peer, many peers, or temporarily nobody if nothing currently has it pinned and online. Before any data can move, something has to answer "which peers currently have this CID," and IPFS answers that with a distributed, peer-to-peer lookup rather than a central registry, since a central registry would reintroduce exactly the kind of single point of control content addressing is meant to avoid.

The DHT: Kademlia-Based Content Routing

IPFS's DHT is built on Kademlia, a peer-to-peer lookup design where every peer gets an ID, and both peers and pieces of content get placed into the same address space by hashing: a peer's position comes from SHA256(PeerID), content's position comes from SHA256(multihash). Distance in that address space is what determines who is responsible for storing information about what.

Advertising content (a "provider PUT") works like this: a peer that has a block runs a Kademlia lookup to find the K=20 peers whose IDs are numerically closest to that block's hashed address, then stores a provider record, a mapping from the content's identifier to itself, at each of those peers (and keeps it locally too). Looking content up (a "provider GET") is the mirror image: query that same region of the address space, asking each peer both for closer peers (in case the lookup needs another hop) and for any provider records they are holding, accumulating results until enough are found or the closest known peers have all been consulted.

Keeping 20 redundant peers per address-space region, rather than one authoritative peer, exists specifically to tolerate churn: peers join and leave the network constantly, and one peer going offline should not silently erase every provider record it was holding. The lookup itself runs in roughly logarithmic time relative to network size, since each hop in a Kademlia lookup narrows the remaining distance by roughly half.

One structural rule worth noting: a peer can only advertise content it actually has. Nothing in the protocol lets one peer claim another peer has content on its behalf, which keeps provider records honest at the protocol level rather than relying on peers to police each other.

Bitswap: Getting the Actual Blocks

Once the DHT has answered "who has this," Bitswap takes over the actual transfer, and it works off want-lists rather than a single request/response. A peer looking for a block sends a want-have message, essentially "do you have this CID," to its connected peers. Peers that have the block respond with have and get added to a session for that request; peers that do not respond with dont-have. Once a peer has established which of its connections can actually serve the block, it sends want-block to request the data itself, and the responding peer sends the block back.

If none of a peer's current connections respond with have, meaning nobody it is already talking to has the block, Bitswap falls back to querying the DHT to find a new peer that does, then repeats the want-have/want-block exchange with that peer. This is the point where the two protocols chain together: Bitswap tries the cheap path first (ask peers you are already connected to) and only pays the cost of a DHT lookup when that cheap path comes up empty.

Why Split Into Two Protocols At All

Splitting discovery from transfer is not an accident of implementation, it lets each protocol specialize. The DHT is built for infrequent, network-wide lookups; running a full lookup for every single block of every single file would be wasteful when a peer already has an open connection to someone holding most of what it needs. Bitswap is built for making the most of connections that already exist, and for letting multiple peers serve different pieces of the same file in parallel, since a want-list can be satisfied by whichever connected peer answers first. Chaining them, cheap path first, DHT fallback second, is what makes that combination work without paying full lookup cost on every request.

Where AIOZ Pin's Gateways Skip This Entirely

Everything above describes peer-to-peer retrieval: a client running its own IPFS node, doing its own DHT lookups and Bitswap exchanges. AIOZ Pin's public and premium gateways sit in front of that process. A request to a gateway is a plain HTTP GET to infrastructure that already holds the content, no DHT lookup to find a provider, no Bitswap want-list negotiation to fetch it, because the gateway already is the provider. This is a meaningful part of why a premium gateway (dedicated, not subject to the standard public-gateway rate limits) can retrieve content faster and more predictably than relying on the open network to find and serve it fresh on every request: it is not doing less work by accident, it is skipping a negotiation that a direct route does not need.

Frequently Asked Questions

How does IPFS find content without a central server? Through a Kademlia-based DHT: peers and content both map into the same hashed address space, and a piece of content's provider record gets stored at the peers numerically closest to its hashed address.

What is a provider record? A record mapping a content identifier to a peer that has advertised holding that content, stored at the K=20 DHT peers closest to that content's hashed address.

What is Bitswap? The protocol that handles actual block transfer once a peer has been found: a want-have/have/want-block exchange between peers, falling back to a DHT query if no connected peer has the requested block.

Why does IPFS need both a DHT and Bitswap? They solve different problems. The DHT answers "who has this content" through a network-wide lookup; Bitswap handles the actual exchange of blocks, prioritizing peers already connected before falling back to a fresh DHT lookup.

Can a peer falsely claim to have content it does not? No, not within the protocol's own rules. A peer can only publish provider records for content it actually holds; nothing in the DHT lets one peer advertise on another's behalf.

Why is a dedicated AIOZ Pin gateway faster than a public one? A dedicated gateway serves content directly, skipping the DHT-lookup-plus-Bitswap-negotiation a plain peer-to-peer fetch requires, and is not subject to the rate limits applied to the shared public gateway.

Related Reading

• How IPFS Really Works: Merkle DAGs, Chunking, and Routing - the overview this article closes out
• Why IPFS Provider Records Expire and Get Renewed - what keeps a provider record findable after the initial advertisement above
• libp2p: The Networking Layer Underneath IPFS - the connection layer both protocols on this page actually run over
• Delegated Routing: How Light Clients Skip Running a DHT - the lighter-weight HTTP alternative to the DHT lookup described above
• IPFS Bitswap Ledgers: How Peers Decide Who to Trust - how Bitswap discourages freeloading between the peers exchanging blocks here
• AIOZ Pin Dedicated Gateways: Simple, High-Performance IPFS Access - the public vs. premium gateway distinction this article's product section builds on
• How AIOZ Pin Actually Keeps Your Files Online: Replication, CIDs, and Proof-of-Storage - what makes a peer worth finding in the first place

References

• IPFS Docs: The DHT
• IPFS Docs: Bitswap