API Reference
Submit URLs
POST /api/v1/knowledge-bases/:knowledgeBaseId/urls — crawl web pages into a Knowledge Base.
Crawls one page, a site from a root URL, or a list of pages, and indexes what it finds. Crawling runs in the background; follow it with List URL sources. See Web ingestion for how crawling works.
POST /api/v1/knowledge-bases/{knowledgeBaseId}/urlsRequest body
| Field | Type | Required | Description |
|---|---|---|---|
type | single | root | list | Yes | One page, a site crawled from this root, or the exact pages in urls. |
url | string | For single, root | The page or the crawl root. |
urls | string[] | For list | The pages to fetch. |
maxPages | integer | No | Crawl limit for root. |
allowSubdomains | boolean | No | For root: also follow links to subdomains. Default false. |
ignoreLinks | boolean | No | Do not follow links from the fetched pages. Default false. |
Response
202 Accepted
{ "sources": [{ "sourceId": "c3a9…", "url": "https://example.com/docs" }] }Errors
| Status | error | When |
|---|---|---|
400 | validation error | url missing for single/root, or urls missing for list. |
400 | invalid_request | The service refused a URL, for example a private or internal address. |
403 | forbidden | Your key is scoped to a different project. |
404 | not_found | No such Knowledge Base in your organization. |