Architecture map

This page orients a contributor in the codebase: the process model, the request flow, and where each concern lives. For the runtime concepts behind it, see Architecture and the rest of the Concepts section.

One process, four sockets

The manager is a single uber-jar that exposes four sockets:

Socket	Default	What it is
Manager REST + React UI	:20900	Tapir + HTTP4s Ember. Endpoints under ondemand/api/*Handlers.scala; the SPA at /ui/* is served from src/main/resources/ui.
Arrow FlightSQL edge	:31338	The query surface for driver-based clients. TLS on by default, cert auto-generated under certs/.
Native Quack front door	:9494	The query surface for DuckDB clients (ATTACH 'quack:host:9494'). edge/quack/QuackFrontDoorServer.scala; relays each statement to a node byte for byte. Plain HTTP by default, see TLS.
Quack nodes	:21900-22500	Child DuckDB Quack processes (local mode) or pods (Kubernetes), each serving a /quack HTTP endpoint.

The FlightSQL request flow

A query traverses these components in order:

client
  → FlightProducerImpl        (Arrow Flight wire surface)
  → AuthenticationService     (Basic / JWT / OIDC)
  → FlightSqlRouter.execute   (the routing core)
  → StatementValidator        (per-statement ACL gate)
  → StatementClassifier        (READ / WRITE / DDL bucket)
  → Router.pick               (least-loaded node for the kind)
  → QuackHttpAdapter          (HTTP call to the chosen node's /quack)
  → child node                (DuckDB over DuckLake)
  → Arrow stream back through Flight

The routing core (FlightSqlRouter, Router, StatementClassifier, RoleMatcher) is deliberately separable from the Flight wire, so RouterSpec, StatementClassifierSpec, and FlightSqlRouterSpec exercise it without a live Flight connection.

Package layout

Package	Responsibility
ai.starlake.quack (Main.scala)	Wiring: load config, pick the state store, build the auth service and ACL validator, select the runtime backend, mount endpoints.
ai.starlake.quack.edge	The FlightSQL edge: FlightEdgeServer, FlightProducerImpl, FlightSqlRouter, sessions, the Quack HTTP adapter.
ai.starlake.quack.edge.auth	Authentication providers (database, JWT, OIDC, OAuth), AuthScope realm tagging, TenantOidcRegistry for per-tenant OAuth client overrides, role extraction.
ai.starlake.quack.secrets	SecretRefResolver -- shared env: / aws-sm: / etc. reference resolution used by per-tenant OIDC clientSecretRef (and reusable elsewhere).
ai.starlake.quack.edge.config / edge.catalog	Auth/ACL/session config types and the DuckLake catalog resolver.
ai.starlake.quack.route	The routing core: Router, RoleMatcher, StatementClassifier.
ai.starlake.quack.ondemand	The control plane: ManagerServer, PoolSupervisor, the REST handlers, state stores, RBAC, manifest, runtime backends.
ai.starlake.quack.ondemand.runtime	The QuackBackend strategy and its local / Kubernetes implementations.
ai.starlake.acl	The SQL parser used by the ACL validator to extract table accesses.
ai.starlake.quack.observability	Micrometer metrics and the sink selection.

State and storage

Main.scala wires the Postgres-only control-plane store: the normalized qodstate_* tables managed by Liquibase (the legacy single-blob file store and its stateStorage / statePath config keys were dropped 2026-06-12). See State storage. Each managed tenant-db is a separate Postgres database holding its DuckLake catalog; see DuckLake catalogs.

Extending

The two seams a contributor is most likely to extend, the runtime backend and the auth providers, have their own guide: Extending the manager.