IPLD: The Data Model Underneath IPFS's Merkle DAG

IPLD is the layer that explains why IPFS's Merkle DAG isn't actually limited to files at all. This series' hub article introduced the DAG as a hash-linked graph of blocks, and the UnixFS article showed one specific way to interpret that graph as files and directories. IPLD is the general framework underneath both: an abstract data model for linked, hash-addressed data, with UnixFS as one of several concrete serializations (codecs) built on top of it, not the only one possible.

TL;DR:

• IPLD (InterPlanetary Linked Data) separates "what kind of data this is" (the Data Model: maps, lists, scalars, links) from "how it's serialized into bytes" (a codec) from "how those bytes move and persist" (storage and transport)
• UnixFS, the file-and-directory format covered earlier in this series, is one codec-and-schema combination built on IPLD, not the whole of what IPLD supports
• Different codecs make different tradeoffs: DAG-PB implements only a small, limited subset of the Data Model and is what UnixFS itself is built on; DAG-CBOR and DAG-JSON implement the Data Model far more completely and are recommended for general-purpose linked data
• A CID's multicodec field, covered in this series' CID-anatomy article, is what tells a reader which codec to use when decoding a given block, so the same underlying addressing and verification machinery works regardless of which codec produced the data
• AIOZ Pin's own product surface touches this directly: ERC-8004 Registration Files, JSON documents referenced by ipfs://{cid}, are exactly the kind of structured, non-file data IPLD's broader model exists to support, distinct from the file-and-folder data UnixFS is built for

Why the DAG Needed a General Model

The Merkle DAG structure at the center of this series is powerful specifically because it's generic: nodes, links, hashes, no assumptions about what the data inside a node represents. UnixFS is one interpretation of that generic structure, built specifically for files and directories, using its own protobuf schema and the DAG-PB codec to encode it. But nothing about the underlying DAG requires that interpretation. IPLD is the name for the layer that makes this explicit: a data model general enough to describe files, but just as capable of describing arbitrary structured data, linked databases, or application-specific schemas that have nothing to do with filesystems at all.

Three Layers: Data Model, Codec, Storage

IPLD separates concerns into three distinct layers. The Data Model is an abstract description of the kinds of data that can exist, maps, lists, strings, numbers, bytes, and links, independent of how any of it gets written to disk or sent over a wire. A codec is what actually serializes that abstract model into concrete bytes, DAG-PB, DAG-CBOR, and DAG-JSON are all codecs, each making different tradeoffs. Storage and transport is the final layer, how those serialized bytes actually move between peers and get held on disk, the DHT and Bitswap mechanics this series already covers in depth. Keeping these separate is what lets the exact same logical data structure be serialized differently, moved over different transports, and processed by different programming languages, while staying semantically the same thing underneath.

The Codecs: DAG-PB vs. DAG-CBOR and DAG-JSON

DAG-PB, the codec UnixFS is built on, implements only a small, deliberately limited subset of the full IPLD Data Model: it can represent raw bytes and named links, and essentially nothing else. That limitation isn't a flaw, it's exactly matched to what a file or directory node actually needs, data plus a set of named children, no more. DAG-CBOR and DAG-JSON, by contrast, implement the Data Model far more completely, supporting the full range of maps, lists, and scalar types, which makes them the recommended choice for general-purpose linked data that isn't specifically a file or folder. DAG-CBOR in particular is binary and length-delimited, which makes it efficient to parse, while DAG-JSON trades some of that efficiency for being human-readable.

How a CID Knows Which Codec to Use

This series' CID-anatomy article already covered the multicodec field inside a CID, the part of the identifier that says how to interpret the linked content, using values like dag-pb (0x70), raw (0x55), and dag-cbor (0x71). IPLD is the reason that field has to exist at all: since the same underlying DAG structure can hold data serialized with any of several codecs, something in the address itself has to say which one applies, so a reader knows whether to run DAG-PB's limited decoder or DAG-CBOR's fuller one before it can make sense of a block's bytes.

Beyond Files: What IPLD Actually Unlocks

Because the Data Model doesn't assume anything about files or directories, arbitrary structured data, a JSON-like document, a linked database record, an application-specific schema, can be represented as an IPLD DAG the same way a file can, addressed by CID, verified the same trustless way, and linked to other IPLD data across completely different applications. This is the property that lets IPFS's addressing, verification, and transport machinery serve as general infrastructure for linked data generally, not a mechanism that only happens to work for files because files were the first thing built on top of it.

Where This Shows Up in an AIOZ Pin Workflow

This series' ERC-8004 coverage elsewhere on this site already touches IPLD territory without naming it directly: an ERC-8004 Registration File is a structured JSON document, referenced directly as ipfs://{cid} per the standard's own convention, not a UnixFS file wrapping arbitrary bytes in the filesystem sense. Pinning that document on AIOZ Pin protects and replicates it the same way any pinned content is protected, the pinning and persistence mechanics don't change based on which codec produced the data. What does change is how a client interprets the bytes once retrieved, and that's exactly the layer IPLD, and the multicodec field pointing at the right codec, is responsible for.

Frequently Asked Questions

What is IPLD in one sentence? The general data model underneath IPFS's Merkle DAG, separating what kind of data something is from how it's serialized (codec) and how it moves and persists (storage/transport).

Is UnixFS part of IPLD or something separate? UnixFS is one specific schema built on top of IPLD, using the DAG-PB codec. IPLD itself is broader and supports other codecs and data shapes beyond files and directories.

What's the difference between DAG-PB and DAG-CBOR? DAG-PB implements a small, limited subset of the IPLD Data Model, just enough for files and named links. DAG-CBOR implements the full Data Model and is recommended for general-purpose structured data.

How does a reader know which codec to use for a given block? The multicodec field inside the block's CID specifies the codec, so decoding always knows which serialization rules to apply before interpreting the bytes.

Does IPLD only work for files? No. Its Data Model is general enough to represent any structured, linked data, files are just the first and most common use case built on top of it.

Is AIOZ Pin's ERC-8004 support built on IPLD? ERC-8004 Registration Files are structured JSON documents referenced by CID, the same kind of non-file, structured data IPLD's broader model exists to support, distinct from UnixFS's file-and-directory schema.

Related Reading

• How IPFS Really Works: Merkle DAGs, Chunking, and Content Routing Explained - the generic DAG structure IPLD's Data Model formalizes
• UnixFS: How IPFS Represents Files and Directories - the specific file-and-directory schema built on the DAG-PB codec
• Anatomy of a CID: Decoding an IPFS Content Identifier Byte by Byte - where the multicodec field this article relies on gets decoded in detail
• IPLD Selectors: Fetching Part of a DAG, Not All of It - how to traverse just part of the Data Model this article describes
• AIOZ Pin for AI Agents: Storing ERC-8004 Files on IPFS - a real, structured, non-UnixFS document type pinned on AIOZ Pin

References

• IPLD Documentation
• IPLD Codec Docs: DAG-PB
• IPLD Codec Docs: DAG-CBOR