Operations

Index CRD

An Index represents one namespace exposed through the gateway. It declares which upstream namespace to use, snapshot policy, cache posture, and consistency mode. The backend connection itself lives in a VectorStore.

apiVersion: hevlayer.com/v1
kind: Index
metadata:
  name: products
  namespace: layer
spec:
  backend:
    storeRef: turbopuffer-default
    namespace: products
    distanceMetric: cosine_distance
  embedding:
    model: voyage-3-large@v1
    outputDim: 1024
    normalization: l2
  metadata:
    labels:
      app: shop
    tags:
      - catalog
  snapshot:
    interval: 5m
    retention: never
    facetFields:
      - category
      - brand
  search:
    fullText: true
  scan:
    threads: 8
  cache:
    ttl: 24h
    capGiB: 64
    mode: standard
  consistency: strong

Backend

Field	Purpose
`backend.storeRef`	Optional `VectorStore` name in the same namespace. The gateway routes requests for this upstream namespace to that store. Defaults to the namespace’s default store.
`backend.namespace`	Optional upstream namespace override. Defaults to the Index name.
`backend.distanceMetric`	Vector metric, default `cosine_distance`.

For kind: search stores, the operator accepts only the metrics the backend serves directly today: L2-style metrics for single-vector namespaces and cosine_distance for multivector namespaces. Unsupported values put the Index in Ready=False with reason MetricMismatch.

Embedding

spec.embedding declares the embedding identity of this namespace’s vectors. It is optional for a single-namespace query, where distances are only ever compared within the namespace. It is required to include a namespace in a federated vector query: the gateway merges those by distance, which is only meaningful when every namespace in the set shares one embedding space.

Field	Purpose
`embedding.model`	Model identity and version, e.g. `voyage-3-large@v1`. Treated as an opaque token compared for equality across a namespace set.
`embedding.outputDim`	Vector dimensionality. Part of the identity because a model truncated to a smaller dimension (Matryoshka) is not comparable to its full-width output. Cross-checked against the namespace schema.
`embedding.normalization`	Vector normalization, e.g. `l2` or `none`.

Together with backend.distanceMetric, these form the embedding profile the gateway compares across a fan-out. Two namespaces are distance-comparable only when all four match; otherwise a fused vector query over them falls back to rank-interleave (or is rejected under strict).

spec.embedding declares an already-computed embedding’s identity for comparison purposes; it does not compute anything. To have the gateway compute vectors for you, declare embed: on a schema attribute instead — see below.

Schema-attribute embedding

A schema attribute can declare embed: so the gateway computes its vector for you instead of you sending one. This is Turbopuffer’s native-embeddings wire (embed on a schema attribute, private beta at turbopuffer.com/docs/embedding), matched verbatim and backed by Layer’s own runtime rather than a fixed managed-model menu. A request written against tpuf’s embedding docs works against Layer unchanged, on both the hev search and Turbopuffer backends — breaking changes to Layer’s older EmbeddingProfile-only CRD surface above are in scope where they were in tension with this wire.

// simple form — tpuf-compatible
"schema": {
  "text": { "type": "string", "embed": "Snowflake/snowflake-arctic-embed-m-v1.5" }
}

// extended form — tpuf fields plus Layer extensions
"schema": {
  "text": {
    "type": "string",
    "embed": {
      "model": "Snowflake/snowflake-arctic-embed-m-v1.5",
      "revision": "main",                  // Layer extension: pinned HF revision
      "dims": 768,                         // tpuf-compatible: Matryoshka truncation
      "instructions": {                    // Layer extension: asymmetric models
        "query": "Represent this sentence for searching relevant passages: "
      },
      "chunk": { "strategy": "section", "maxTokens": 512, "overlap": 64 }  // Layer extension: see below
    }
  },
  "image": {
    "type": "string",                      // URL or base64, tpuf-compatible encoding
    "embed": { "model": "openai/clip-vit-base-patch32", "modality": "image" }  // Layer extension: CLIP
  }
}

Field	tpuf-compatible	Purpose
`embed` (string)	✅	Simple form. The model id, an HF repo id (e.g. `Snowflake/snowflake-arctic-embed-m-v1.5`).
`embed.model`	✅	Extended form’s model id. Same namespace as the string form.
`embed.dims`	✅	Output dimensionality via Matryoshka truncation.
`embed.revision`	Layer extension	HF revision, pinned at profile creation and frozen — the model never silently changes under a live index. Defaults to the repo’s default branch.
`embed.instructions.query` / `.document`	Layer extension	Prefix text for asymmetric models (e.g. arctic-embed) applied at write and query time.
`embed.chunk`	Layer extension	Declarative fan-out to N chunk rows per document; see Chunking below.
`embed.modality`	Layer extension	`"image"` routes the attribute through a checkpoint’s image tower (CLIP-family models); see Multimodal below.

This section describes the schema-attribute wire settled by this doc; it is not yet reflected in the EmbeddingProfile Rust struct (apps/layer-gateway/src/index_config.rs:57, today {model, output_dim, distance_metric, normalization}) or the operator. Per the embed-wire RFC, EmbeddingProfile grows to carry revision, modality, instructions, chunk, and a runtime hint, and the embed: schema attribute becomes the wire spelling of that struct.

A derived vector column named embed_<attr> is created and maintained by the gateway — queryable as an ordinary vector attribute, with the source attribute stored alongside it. Writes may omit vectors entirely; the row is durable and acknowledged immediately, and the vector lands asynchronously. Readiness is the existing is_stable / {col}_v version-marker signal, reported per namespace. The gateway never forwards embed: upstream to tpuf’s managed embedding. Layer’s runtime computes every vector, on every backend, so cost is visible in gateway cost accounting and behavior is identical regardless of which VectorStore backs the namespace.

Changing embed.model (or revision, or dims) never requires you to re-send data — the source attribute is already stored, so the gateway re-embeds from what it holds. On the hev search backend this is a native vector rewrite. On a Turbopuffer-backed namespace, where vector columns are immutable post-creation and patch_columns rejects vector fields, a same-dims model or revision change is a rolling in-place re-embed (each row is fetched, re-embedded, and re-upserted as its {col}_v marker goes stale); a dims change, or adding embed: to a namespace with no matching vector column, runs an orchestrated shadow-namespace rebuild that cuts the query path over once stable. Either way, you send a profile change, not a re-ingest.

Chunking

embed.chunk composes the RFC 0056 / RFC 0079 declarative chunking block — same strategies (fixed-window, section, structured splitters), same document/chunk model — as a field on the embedded attribute instead of a separate Pipeline stage. A row whose attribute exceeds the chunk window becomes one document row plus N chunk rows carrying the standard document id / ordinal / span linkage; chunk rows hold the embed_<attr> vectors, the document row holds the source text. ANN over embed_<attr> matches chunks; the existing chunk-aware query echo groups hits under their document with spans, so callers see documents with located passages, not orphaned fragments — no new response shape. Omitting chunk is exactly tpuf’s behavior: one row, one vector, and an input over the model’s window is a 422 naming the token count.

Multimodal

embed: {model, modality: "image"} on a string attribute treats its value as an image — a URL (fetched by the runtime under the namespace’s egress policy) or base64, encoding-compatible with tpuf’s documented workaround. The runtime routes the value through the checkpoint’s image tower; a text Embed at query time against the same column uses the text tower of the same checkpoint, giving cross-modal text-to-image search with no application code. Only CLIP-family checkpoints are supported for modality: "image" today.

Any Hugging Face model

The model registry is the Hub, not a fixed menu: embed.model accepts any sentence-transformers/fastembed-loadable text encoder or CLIP-family checkpoint’s HF repo id. Profile creation validates loadability eagerly and rejects with the loader error rather than accepting writes that can never embed. Models run under the hevlayer-inference GPU base image on Layer’s compute pools with scale-to-zero; small text models default to the CPU pool for interactive latency.

Model-serving policy

Per model, embed.serving.prefer chooses how vectors are computed. This field’s name and shape are settled by this doc — there is no corresponding code today (no serving field on EmbeddingProfile, no scheduler for it).

`prefer`	Behavior
`native`	Serve from the gateway’s own always-on model pool (the CPU-tier fastembed class, or a warm GPU replica for larger/CLIP models). Lowest, most predictable latency; the pool’s steady-state cost is fixed regardless of query volume.
`autoscaler`	Serve from the RFC 0094 GPU-autoscaled Pipeline/Function path, scale-to-zero when idle. Lowest idle cost; the first request after a scale-to-zero pays a cold-start latency penalty (`warmWindowSeconds` amortizes this under sustained traffic).
`blended` (default)	Serve every request from `native` immediately; when sustained query volume crosses the autoscaler’s scale-up threshold, the gateway starts routing new requests to the warmed autoscaled replica and lets the native pool’s share fall off. Named for what it is, not how it’s scheduled: nobody observes a cold-start latency spike, and steady high-volume traffic still lands on the cheaper autoscaled path once it’s warm. Under `blended`, expect `native`-shaped latency during the shift window and `autoscaler`-shaped cost once traffic has moved.

blended is this doc’s answer to a question the RFC’s “best-of-both-worlds policy” section left as observable behavior only, described from the operator’s viewpoint (latency, cost) rather than the scheduler internals that produce it — the specific shift threshold and window are implementation, not part of the doc’s contract.

Snapshot policy

Field	Default	Purpose
`snapshot.facetFields`	`[]`	Fields the gateway materializes into durable facet snapshots. Empty disables the automatic writer.
`snapshot.interval`	`5m`	Minimum spacing between automatic snapshot writes after upstream-stable advances.
`snapshot.retention`	`never`	`never` keeps all snapshot bodies; a duration such as `30d` prunes older bodies while keeping the latest.

Search backend policy

spec.search applies when the Index targets a kind: search VectorStore. The operator uses it to drive the backend’s explicit index lifecycle.

Field	Default	Purpose
`search.fullText`	`false`	Build the backend’s BM25 index for the namespace’s `text` column. Enable this for lexical, FTS, or hybrid-text namespaces.

Scan policy

scan.threads sets the per-namespace default for origin scan fan-out: the maximum concurrent upstream requests one scan may issue during scatter/gather. It defaults to 8 and is clamped by the gateway’s server cap and the active shard count. Request-level threads overrides this default for one scan.

Cache policy

Aerospike remains an ephemeral cache; durable snapshot history stays in S3. Cache warming uses the same scan fan-out policy as other origin scans.

Status

The operator reports observed generation, metadata sync state, and conditions. status.snapshot.lastRun and lastSuccess are reserved for the gateway history bridge.