Document ingest (document-ingest)

Upload a PDF or scanned image, wait for the LiteParse worker job to parse it, and read the extracted text back over HTTP. The recipe is the composition itself: document-ingest-foundation (kumiko-framework) owns the upload trigger and the documentExtract entity; document-ingest-provider-liteparse (kumiko-enterprise) is the worker-lane job that actually parses the bytes.

What it shows

• Why this lives in kumiko-enterprise, not kumiko-framework — document-ingest-foundation is public/BUSL and must never import a private enterprise package (same constraint as the ai-text-demo sample). The full pipeline can only be wired together from the enterprise side.
• Event-triggered, not job-triggered — the foundation’s fileRef.created MSP validates mime/size and appends documentIngest.requested; the provider’s r.job reacts to that event on the worker lane, so OCR/parse never runs inline in the upload transaction.
• The read side neither upstream package ships — both only write documentExtract (the foundation via r.entity, the provider via its job). This recipe adds the missing document-extract:list / document-extract:detail query handlers, gated to TenantAdmin/Admin/SystemAdmin only — a row carries the extracted text of whatever any tenant member uploaded, with no per-uploader scoping, so it’s broader than the foundation’s User-readable config keys and stays admin-only here.
• documentIngestSampleFeatures is the feature list — still needs an explicit file-storage provider, the jobs feature, and jobs: { consumerLane: "worker" } so the LiteParse job actually runs (see Wiring below; includeBundled alone is not enough).

When to reach for it

Any app that needs searchable/readable text out of uploaded PDFs or scans — invoice intake, document review queues, compliance archives. The cap/truncation and OCR-routing behavior live in document-ingest-provider-liteparse’s own integration test; this recipe’s test only proves the composition plus the HTTP read path.

Wiring

documentIngestSampleFeatures deliberately does not include tenant / compliance-profiles / tenant-lifecycle — composeFeatures only dedupes its own bundled list, so spreading a second copy here would crash any app that already mounts tenant-lifecycle itself (Duplicate feature). They ARE required though, same as config/jobs below — the consumer wires them:

• createTenantFeature() / createComplianceProfilesFeature() / createTenantLifecycleFeature() — required because documentExtract.pages is tenant-subject ciphertext (kumiko-framework#1621): the foundation registers an EXT_TENANT_DATA destroy hook, tenant-lifecycle hosts that extension point, and without it the registry refuses to boot.
• A configured subject KMS (configurePiiSubjectKms(...) from @cosmicdrift/kumiko-framework/crypto) — without it the PII layer is off and pages lands in Postgres as plaintext, and tenant-destroy shreds nothing (no key to destroy). This sample’s own test uses createTestEnvelopeCipher()/configureEntityFieldEncryption as its KMS stand-in; a real app needs its own KMS adapter configured before mounting this feature list.
• An explicit file storageProvider, createJobsFeature(), and jobs: { consumerLane: "worker" } — includeBundled alone does not start the LiteParse worker.

composeFeatures(..., { includeBundled: true }) only adds config / user / tenant / auth-email-password — not any of the above. Without them, uploads return HTTP 201 but documentIngest.requested is never consumed (no extract, no error), or pages ends up unencrypted. Match what the integration test uses:

import { createComplianceProfilesFeature } from "@cosmicdrift/kumiko-bundled-features/compliance-profiles";
import { createConfigFeature } from "@cosmicdrift/kumiko-bundled-features/config";
import { createJobsFeature } from "@cosmicdrift/kumiko-bundled-features/jobs";
import { createTenantFeature } from "@cosmicdrift/kumiko-bundled-features/tenant";
import { createTenantLifecycleFeature } from "@cosmicdrift/kumiko-bundled-features/tenant-lifecycle";
import { configurePiiSubjectKms } from "@cosmicdrift/kumiko-framework/crypto";
import { createInMemoryFileProvider } from "@cosmicdrift/kumiko-framework/files";
import { documentIngestSampleFeatures } from "./feature";

configurePiiSubjectKms(yourKmsAdapter); // required — see above

const features = [
  createConfigFeature(),
  createJobsFeature(),
  createTenantFeature(),
  createComplianceProfilesFeature(),
  createTenantLifecycleFeature(),
  ...documentIngestSampleFeatures,
  ...appFeatures,
];

// composeFeatures only merges the feature list (+ optional includeBundled).
// Storage + worker lane are runtime options on setupTestStack / the server:
await setupTestStack({
  features,
  files: { storageProvider: createInMemoryFileProvider() },
  jobs: { consumerLane: "worker" },
});

bun --env-file=../.env test --config=bunfig.integration.toml samples/recipes/document-ingest

---

Source code

The feature entry point — embedded straight from the source file, so the code here is exactly what runs. Multi-file samples keep their remaining files next to it on GitHub (link below):

// document-ingest sample — the composition IS the recipe.
//
// document-ingest-foundation (kumiko-framework, public/BUSL) owns the
// documentExtract entity and the fileRef.created → documentIngest.requested
// trigger. document-ingest-provider-liteparse (kumiko-enterprise, private)
// is the worker-lane job that actually parses the bytes. Neither package
// may depend on the other directly across the publish boundary — framework
// samples must never import a private enterprise package (see
// samples/apps/styleguide's ai-text-demo for the same constraint) — so an
// app that wants the full pipeline can only wire them together here, in
// kumiko-enterprise.
//
// The one piece neither package ships: a way to read `documentExtract` back
// over HTTP. Both upstream features only write it (the foundation via
// r.entity, the provider via its job) — this feature adds the read side.

import {
  documentExtractEntity,
  documentIngestFoundationFeature,
} from "@cosmicdrift/kumiko-bundled-features/document-ingest-foundation";
import {
  defineEntityDetailHandler,
  defineEntityListHandler,
  defineFeature,
} from "@cosmicdrift/kumiko-framework/engine";
import { documentIngestProviderLiteparseFeature } from "@cosmicdriftgamestudio/kumiko-document-ingest-provider-liteparse";

// Admin-only, not the foundation's config-key gate (which also allows
// "User"): a documentExtract row carries the full extracted text of
// whatever ANY tenant member uploaded — list/detail have no per-uploader
// scoping (crossTenant defaults to false, meaning tenant-wide, not
// uploader-wide), so this is a broader exposure than a config scalar. A
// consuming app that wants per-uploader visibility needs its own
// ownership-filtered query handler, not this one widened to "User".
const readAccess = {
  access: { roles: ["TenantAdmin", "Admin", "SystemAdmin"] },
} as const;

export const documentIngestQueryFeature = defineFeature("document-ingest-sample-query", (r) => {
  r.requires(documentIngestFoundationFeature.name);

  r.queryHandler(defineEntityListHandler("documentExtract", documentExtractEntity, readAccess));
  r.queryHandler(defineEntityDetailHandler("documentExtract", documentExtractEntity, readAccess));
});

// Feature list for upload → parse → readable extract. Callers still must
// pass a file storageProvider, createJobsFeature(), and
// jobs: { consumerLane: "worker" } — includeBundled alone does not start
// the LiteParse worker (see feature.integration.test.ts / README Wiring).
//
// Deliberately does NOT include tenant / compliance-profiles /
// tenant-lifecycle: those are infrastructure a real app almost always
// already mounts itself (e.g. for GDPR tenant-destroy), and
// `composeFeatures` only dedupes its own bundled list — spreading a second
// copy of tenant-lifecycle here would crash an app that already mounts it
// ("Duplicate feature"). The consumer wires those three (README Wiring),
// same as config/jobs above. They ARE required though: documentExtract.pages
// is tenant-subject ciphertext since kumiko-framework#1621 — the foundation
// registers an EXT_TENANT_DATA destroy hook, and tenant-lifecycle hosts that
// extension point. Without it the registry refuses to boot. And without a
// configured subject KMS (`configurePiiSubjectKms`, README Wiring), the PII
// layer is off and `pages` lands in plaintext with no crypto-shredding.
export const documentIngestSampleFeatures = [
  documentIngestFoundationFeature,
  documentIngestProviderLiteparseFeature,
  documentIngestQueryFeature,
];

Enterprise recipe — samples/recipes/document-ingest/src/feature.ts in the private kumiko-enterprise workspace.