API Guide

HTTP surface reference for automated / programmatic clients. Base URL: http://reg-download.marklo.de. Machine-readable schema: /openapi.json · interactive docs: /docs · /redoc.

Auth model (split surface). All read endpoints (JSON GETs, the bundle ZIP, the Atom feed, SSE) are fully public — no auth. Mutating (admin) endpoints are gated by a single shared token. The token is passed as a form field named token in an application/x-www-form-urlencoded POST body — NOT as a header or bearer. Header-based clients silently fail the comparison and get a 303 redirect (?err=invalid_token), never a 401. If the admin token is unset, mutating routes are disabled (?err=sync_disabled).

Read endpoints (no auth)

Method / Path	Description
`GET /api/updates`	Paginated updates feed. Params: `lang, q, jurisdiction, regulator, source, limit (1..200), offset`. Returns `{updates, total, next_offset}`.
`GET /api/documents`	Global document list. Params: `lang, q, jurisdiction, regulator, collection, source, status, limit (1..200), offset`.
`GET /api/sources/{source_code}/documents`	Per-source documents. Params: `lang, q, entity_type, content_type, sektor, subsektor, jenis, tahun, limit, offset`.
`GET /api/omnibox`	Combined source + document search. Params: `q` (min 2 chars), `lang, limit_sources=8, limit_documents=12`.
`GET /api/download/bundle.zip`	Streams a ZIP of successfully stored artifacts. Filters: `source, jurisdiction, regulator, collection, year, q, ids=12,34`. `404` if no match. Record-only / external references are skipped.
`GET /feeds/updates.atom`	Atom feed of recent updates. Same filter params as `/api/updates`; `limit` up to 500.
`GET /api/sse/sources/{source_code}` `GET /api/sse/jobs` `GET /api/sse/jobs/progress`	Server-Sent Events live progress streams.

Mutating endpoints (admin token in form body)

All POST, application/x-www-form-urlencoded, with the admin token in the token field. These mostly return 303 redirects to UI pages (with ?ok= / ?err= query flags), not JSON — parse the Location header, not the body.

Method / Path	Form fields (besides `token`)
`POST /sources/{source_code}/sync`	`max_pages, page_size, limit, download, only_missing, force_download, confirm_large, lang`. `only_missing` and `force_download` are mutually exclusive. Concurrent runs for the same source are blocked.
`POST /sources/{source_code}/health-check`	`sample_count, lang`.
`POST /sources/{source_code}/backfill`	`limit, lang`. OJK sources only (`ID.OJK.*`).
`POST /sources/{source_code}/retry-last-failed`	—
`POST /runs/{run_id}/rerun` `POST /runs/{run_id}/retry-failed` `POST /runs/{run_id}/diagnostics`	—
`POST /maintenance/rebuild-fts` `POST /maintenance/backfill-tags` `POST /maintenance/bulk-health-check` `POST /maintenance/prune-artifacts-preview` `POST /maintenance/prune-artifacts-apply` `POST /maintenance/scheduler/update` `POST /maintenance/scheduler/run-now` `POST /maintenance/proxy/update`	Task-specific fields. Always run prune preview before apply.

curl examples

BASE=http://reg-download.marklo.de

# Read — no auth.
curl -s "$BASE/api/updates?limit=20&jurisdiction=HK"
curl -s "$BASE/api/omnibox?q=cyber"
curl -s "$BASE/api/sources/HK.IA.circulars_reg_matters_en/documents?limit=50"

# Bundle a whole regulator's PDFs into a local knowledge base.
curl -s -o ia.zip "$BASE/api/download/bundle.zip?jurisdiction=HK®ulator=IA&year=2020"

# Atom feed + SSE.
curl -s "$BASE/feeds/updates.atom"
curl -sN "$BASE/api/sse/jobs/progress"

# Mutating — token goes in the FORM BODY, not a header.
curl -s -X POST "$BASE/sources/HK.IA.circulars_reg_matters_en/sync" \
  -d "token=$ADMIN_TOKEN&max_pages=2&download=true&only_missing=true"
curl -s -X POST "$BASE/maintenance/rebuild-fts" -d "token=$ADMIN_TOKEN"

Limits & gotchas

limit is clamped to 1..200 on the list endpoints; offset is floored at 0. Omnibox needs q length ≥ 2.
The admin token is a form field, not a header — wrong/missing token yields a 303 redirect, never a 401.
bundle.zip uses ZIP_STORED (PDFs are already compressed) and only includes documents with a successful stored artifact.
Ingestion is de-duplicated at the DB layer; prune has a preview/apply pair — always preview first.