File Vectors
File Vectors turns a pinned text file into vector embeddings, so you can search your files by meaning (semantic search) instead of by exact keywords. You can create the vectors while you pin a file, or later for a file you already pinned. A query returns your pin objects directly, so you don't need to keep your own table that maps vectors back to files. The original file stays on IPFS and is not changed. AIOZ Pin makes the embeddings with the BAAI/bge-m3 model, which supports more than 100 languages.
Pin a text file and create its vectors in one request:
curl --location --request POST 'https://api.aiozpin.network/api/pinning/' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET' \
--form 'file=@"./notes.md"' \
--form 'metadata="{\"name\": \"notes.md\"}"' \
--form 'options="{\"vectorize\": true}"'When the file's vectorize_status is completed, query your file vectors with plain text:
curl --location --request POST 'https://api.aiozpin.network/api/v2/vectorize/query' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET' \
--header 'Content-Type: application/json' \
--data-raw '{"text": "how do I store files on IPFS"}'Limits
Only UTF-8 text files can be vectorized. The file type must be one of:
text/*(for exampletext/plain,text/markdown,text/csv,text/html)application/json,application/xml,application/yaml,application/x-yamlapplication/javascript,application/typescript,application/csv- any type ending with
+jsonor+xml
Rich documents (PDF, DOCX, images, audio, video), CAR files, and folders are not supported. A request for an unsupported file returns 400 Unsupported file type for vectorize: <mime type>.
When you upload a file, AIOZ Pin detects its type from the content, not from the file name. Most text files, including Markdown, CSV, and JSON, are stored as text/plain; charset=utf-8. When you pin by CID, the type comes from the IPFS gateway.
The text is embedded as it is. HTML and XML tags are not removed, and bytes that are not valid UTF-8 are dropped.
Other limits:
- Each account has one vector index, and a query only searches files in your account.
- Long files are split into parts of about 2,000 characters. A file ranks by its best-matching part and appears once in the results.
- A query returns up to 100 files, 10 by default.
- Results have no similarity score and no matched text.
Permissions
All File Vectors endpoints require your pinning API key and secret key, and the key needs the admin scope or the pin_file_to_ipfs scope. Otherwise the API returns 401: API Key don't have permissions to perform this action.
When you vectorize on upload or on pin by CID, the pin endpoint's own scope applies: pin_file_to_ipfs for a file upload and pin_by_hash for pin by CID.
Vector Status
Each file has one vector job. Pin objects show its state in vectorize_status. Read it with Get Pin Details by pinID or List Pins of User.
| Status | Meaning |
|---|---|
not_requested | No vectors were requested for this file. |
queued | Waiting to run. A file pinned with vectorize: true stays here until pinning finishes. |
processing | The file is being read, embedded, and stored. |
completed | The file appears in query results. |
failed | The last attempt failed. The retry rules are below this table. |
deleted | The file's vectors were deleted with Delete File Vectors. After they are cleaned up, usually within a day, the status goes back to not_requested. |
When an attempt fails, the API retries the file in the background, up to 5 attempts in total. The wait between attempts doubles each time, starting at 30 seconds. If the file is still failed after that, call Vectorize a File to try again. A file with no text, such as an empty file, always fails.
Vectorizing Files
There are three ways to vectorize a file:
- Vectorize on Upload with the
vectorizepin option. - Vectorize on Pin by CID with the
pin_options.vectorizeoption. - Vectorize a File that is already in your account.
Then you can query your file vectors and delete the vectors of a file.
With the first two, the response contains vectorize_status: "queued", and the file is vectorized in the background once it is pinned. Check progress with Get Pin Details by pinID until vectorize_status is completed.
Vectorize on Upload
Send the options form field to POST /api/pinning/:
{
"cid_version": 1,
"wrap_with_directory": false,
"vectorize": true
}curl --location --request POST 'https://api.aiozpin.network/api/pinning/' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET' \
--form 'file=@"./notes.md"' \
--form 'metadata="{\"name\": \"notes.md\"}"' \
--form 'options="{\"vectorize\": true}"'Response:
{
"data": {
"id": "string",
"root_hash": "string",
"cid": "string",
"size": number,
"user_id": "string",
"pinned": true,
"is_pin_by_hash": false,
"is_dir": false,
"metadata": {
"name": "notes.md",
"type": "text/plain; charset=utf-8"
},
"status": "string",
"vectorize_status": "queued"
},
"status": "success"
}Rules when vectorize is true:
- Upload one file only. Multiple files or
wrap_with_directory: truereturns400 Vectorize only supports a single text file upload. - The file type is checked before pinning. A non-text file returns
400 Unsupported file type for vectorize. - CAR files return
400 CAR files are not supported for vectorize. - Chunked upload (
/api/v2/pinning/completeUpload?vectorize=true) returns400 Vectorize is not supported for chunked upload completion. Pin the file first, then call Vectorize a File. - If this file is already pinned in your account, the API returns
409 Pin with that hash already exists. Use Vectorize a File instead.
Vectorize on Pin by CID
Add pin_options.vectorize to the pinByHash body:
{
"hash_to_pin": "CID",
"metadata": {
"name": "string"
},
"pin_options": {
"vectorize": true
}
}curl --location --request POST 'https://api.aiozpin.network/api/pinning/pinByHash/' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET' \
--header 'Content-Type: application/json' \
--data-raw '{
"hash_to_pin": "Qmc1135ziMvmFG534i75E8HpJoLqzLzgKBxfjBV9cBsMAs",
"metadata": { "name": "name-ipfs-hash" },
"pin_options": { "vectorize": true }
}'Before pinning, the API reads the file type from the IPFS gateway. This check can be slow if the CID is hard to find on the network.
- An invalid CID returns
400 Invalid cid. - An unsupported type returns
400 Unsupported file type for vectorize. - If the gateway cannot return the file, the request fails with
500and the CID is not pinned.
Pinning by CID can take time, so vectorize_status stays queued until the file is pinned. Vectorizing starts after that.
Vectorize a File
Generate embeddings for a file that is already pinned. The file is identified by its CID and must belong to your account.
Parameters
Headers
Response
The request has no body. The call waits until the whole file is read, embedded, and stored, so a large file can take minutes. If the request times out, check vectorize_status with Get Pin Details by pinID. A failed attempt is retried in the background.
Behavior to know:
- The file must be fully pinned (
pinned: trueandstatusispinned). Otherwise the API returns400 File is not pinned yet. - A CID that is not in your account returns
404 Pin not found. - It is safe to call again. If the file is already
completedwith the same CID, the API returns the existing job without doing the work again. - If vectorizing fails, the response is
500with the messageSomething went wrong, andvectorize_statusbecomesfailed. See Vector Status for retries. - Calling it on a file whose vectors you deleted with Delete File Vectors adds the file back to query results.
Response fields:
| Field | Description |
|---|---|
id | Vector job ID. |
pin_id | ID of the pin that was vectorized. |
user_id | Your user ID. |
cid | CID of the file. |
index_id | Your vector index. Each account has its own index. |
status | Always completed in a successful response. |
model, model_version | Embedding model, BAAI/bge-m3. |
dimensions | Length of each vector (1024). |
chunk_count | Number of text parts stored for the file. Long files are split into parts of about 2,000 characters. |
content_hash | SHA-256 hash of the file content. |
attempt_count | Number of times the file has been processed. |
created_at, started_at, completed_at | Job timestamps. |
curl --location --request POST 'https://api.aiozpin.network/api/v2/vectorize/files/Qmc1135ziMvmFG534i75E8HpJoLqzLzgKBxfjBV9cBsMAs' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET'Query File Vectors
Search your vectorized files with a natural language query. Results are ranked by how close their meaning is to the query.
Headers
Body
Response
Behavior to know:
- Results are pin objects, ordered from the most similar to the least similar. Long files are split into parts of about 2,000 characters, and a file ranks by its best-matching part. A file appears once in the results. Use the
cidto fetch or download the file. - The response has no similarity score and no matched text. Search returns the closest files even when none of them is about the query.
countcan be lower thanlimiteven when you have more vectorized files, because several of the best-matching parts can come from the same file.- Only files with
vectorize_status: completedare returned. A file whose vectors you deleted with Delete File Vectors disappears from the results right away. - Unpinning a file does not delete its vectors, so results can include files with
pinned: false. Call Delete File Vectors to remove them. filtermatches the metadata stored with each vector, which is onlypin_id.{"pin_id": "<pin id>"}limits the search to that one file. Filters on the filenameorkeyvaluesreturn no results.textis required and cannot be blank.limitmust be 100 or less. A value above100returns400withlimit must not exceed 100. Iflimitis omitted or0or below, the default10is used.
curl --location --request POST 'https://api.aiozpin.network/api/v2/vectorize/query' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET' \
--header 'Content-Type: application/json' \
--data-raw '{
"text": "how do I store files on IPFS",
"limit": 5
}'Delete File Vectors
Delete the vectors of a file, which removes it from query results. This does not unpin or delete the file. To remove the file itself, see Remove Files (Unpin).
Parameters
Headers
Response
The file no longer appears in Query File Vectors results right after this call. Its vectorize_status becomes deleted, and the vectors are deleted in the background, usually within a day. To add the file back, call Vectorize a File again.
The call returns 200 even if the file was never vectorized. A CID that is not in your account returns 404.
curl --location --request DELETE 'https://api.aiozpin.network/api/v2/vectorize/files/Qmc1135ziMvmFG534i75E8HpJoLqzLzgKBxfjBV9cBsMAs' \
--header 'pinning_api_key: KEY' \
--header 'pinning_secret_key: SECRET'