app.services.ingest

The shared ingest path used by poll and push intake.

One document goes through exactly these steps, whoever delivered it:

  1. parse and validate (app.domain.records.parse_record());

  2. compute the canonical digest (app.domain.hashing.content_digest()) – any client-supplied digest is ignored;

  3. store the immutable snapshot (put_revision) of the canonical text;

  4. render the Markdown body (and, for an erratum, its original, corrected and notes sections) with markdown-it-py and sanitize the HTML with nh3;

  5. upsert the published row, or the candidate row (replace or create-only), with its per-record validation issues.

prepare_document() (steps 1, 2 and 4) is pure, so push intake can inspect the result (e.g. compare the document UUID with the request) before writing anything; ingest_document() adds steps 3 and 5.

Ingest never touches decision_event: approval history is keyed by identity and digest and is never rewritten by intake. Callers run ingest inside lock_registration().

Relationship issues (missing_target, ambiguous_target, duplicate_uuid) are not stored by ingest. They depend on other records – also those of other registrations, which can change without this record changing – so they are computed at read time from list_relationship_summaries() with app.domain.relationships.RelationshipIndex (phase 05, catalog service). Stored issues therefore hold only the per-document findings of parse_record().

Public API: IncomingDocument, PreparedDocument, IngestResult, render_markdown(), render_erratum(), prepare_document(), ingest_document().

class app.services.ingest.IncomingDocument(*, source_id, project, path, ref, content, change_token=None, source_url=None, author=None)

Bases: object

One delivered document, independent of the intake path.

Parameters:

  • source_id (str) – Owning registration id.

  • project (str) – Owning project path.

  • path (str) – File path inside the project.

  • ref (str) – Branch/ref the content came from.

  • content (bytes) – The exact bytes fetched or pushed.

  • change_token (str | None) – Source change token (None for pushes).

  • source_url (str | None) – Link to the file at the source, if any.

  • author (str | None) – Push or commit author, if known.

source_id: str
project: str
path: str
ref: str
content: bytes
change_token: str | None
source_url: str | None
author: str | None
class app.services.ingest.PreparedDocument(*, parsed, digest, text, record)

Bases: object

A parsed, hashed and rendered document, ready to be stored.

Parameters:

  • parsed (ParsedRecord) – Parse and validation result.

  • digest (str) – Canonical digest string.

  • text (str) – Canonical text stored as the revision snapshot.

  • record (RecordWrite) – The row to write.

parsed: ParsedRecord
digest: str
text: str
record: RecordWrite
class app.services.ingest.IngestResult(*, record_id, kind, digest, created, digest_changed, previous_digest, record_key, issues)

Bases: object

Outcome of ingesting one document.

Parameters:

  • record_id (UUID) – Stored row id.

  • kind (RecordState) – Published or candidate.

  • digest (str) – Canonical digest string.

  • created (bool) – True if a new row was inserted.

  • digest_changed (bool) – True if an existing row got new content.

  • previous_digest (str | None) – Digest before this write, if any.

  • record_key (str) – Identity key of the row.

  • issues (tuple[Issue, ...]) – Per-document validation issues.

record_id: UUID
kind: RecordState
digest: str
created: bool
digest_changed: bool
previous_digest: str | None
record_key: str
issues: tuple[Issue, ...]
app.services.ingest.render_markdown(body)

Render a Markdown body to sanitized HTML.

Parameters:

body (str) – Markdown without frontmatter.

Return type:

str

Returns:

HTML safe to embed (scripts, event handlers and unsafe URLs removed; links get rel="noopener noreferrer").

app.services.ingest.render_erratum(content)

Render the sections of an erratum to sanitized HTML.

Parameters:

content (ErratumContent | None) – Parsed erratum content, None for a decision.

Return type:

StoredErratum | None

Returns:

The content to store (missing optional sections stay None), or None for a decision.

app.services.ingest.prepare_document(document)

Parse, validate, hash and render a document (pure).

Never raises for content problems.

Parameters:

document (IncomingDocument) – The delivered document.

Return type:

PreparedDocument

Returns:

The prepared document.

app.services.ingest.ingest_document(store, prepared, kind, *, create_only=False)

Store the snapshot and upsert the row of a prepared document.

Parameters:
Return type:

IngestResult

Returns:

The outcome.

Raises:

app.errors.CandidateExistsError – If create_only and the candidate exists.