Pages, captures, and the corpus
Understand what you’re querying and what stays available.
A page is not a capture
A page identifies a URL. A capture records an observation of that page. One page can have many captures over time, including observations with identical content.
documents exposes the content of each capture. HTML has a parsed node tree;
JSON keeps its lexical payload for native SQL field extraction and array expansion.
json_values provides field-local reads with exact scalar tokens and array positions.
Periplus does not assign application-specific fields or meanings to either format.
See Query JSON documents.
| Identity | Meaning |
|---|---|
| page_id | A page identified by its canonical URL |
| capture_id | One observation of a page |
| document_id | The exact document bytes; content can be shared across captures |
| capture_id + node_index | A node within the HTML of one capture, at the queried publication |
example.com/product
Capture 2: Price: €45
Document B · Captures 2 and 3 share identical content, but remain separate observations.
Shared data, organization-owned work
The web corpus is shared. Captures and crawls you request, pins, schedules, saved queries, and usage belong to your organization. Collecting a source does not make its captures private to your organization.
You can pin any available capture in the corpus, even if your organization did not collect it.
Keep the sources you need
Every capture is kept for seven days after collection. After that, it is scheduled for deletion unless a pin keeps it. Removal may happen later. Viewing or querying a capture does not restart the seven days.
A pin keeps specific captures for a chosen period. It does not freeze the whole corpus or automatically include later captures of the same page. Once a capture has been deleted, pinning cannot bring it back.