5.1 Release Notes

Patch Releases

All patch release notes for 5.1.x are available on the releases page.

AI Models Integration

Harper 5.1 introduces a built-in models layer that provides a unified interface for AI model backends. This enables embedding generation and text generation from directly within Harper applications, without managing external API connections per-application.

The models layer is exposed via scope.models in application code:

// Generate embeddings
const vector = await scope.models.embed('text to embed', { model: 'my-embedding-model' });

// Generate text
const response = await scope.models.generate([{ role: 'user', content: 'Hello' }], { model: 'my-chat-model' });

// Streaming generation
for await (const chunk of scope.models.generateStream(messages, { model: 'my-chat-model' })) {
	// ...
}

Supported backends are Anthropic, AWS Bedrock, OpenAI, and Ollama, configured under the models key in harper-config.yaml:

models:
  anthropic:
    apiKey: your-api-key
  openai:
    apiKey: your-api-key
    baseUrl: https://api.openai.com/v1 # optional override
  ollama:
    baseUrl: http://localhost:11434
  bedrock:
    region: us-east-1

@embed Schema Directive

The @embed directive automates vector embedding at the schema level, eliminating the need for application code to compute and store embeddings on every write. Add it to any Float array field with a source pointing to the text field to embed:

type Document @table {
	id: ID @primaryKey
	content: String
	embedding: [Float] @embed(source: "content", model: "my-embedding-model") @indexed(type: "HNSW")
}

On every write, Harper automatically calls the specified model to compute the embedding from source and stores it on the record. The @indexed(type: "HNSW") index is attached automatically when not explicitly specified.

Agent Loop

scope.models.generate now supports toolMode: 'auto', which runs an agentic loop — the model can invoke tools and Harper will automatically dispatch them until the model produces a final non-tool response. This makes it straightforward to build tool-using agents directly in Harper application code.

MCP Server

Harper 5.1 includes a built-in Model Context Protocol server, allowing LLM clients such as Claude Desktop, Cursor, and Zed to connect directly to a Harper instance and interact with its data and operations.

The MCP server exposes two profiles:

• Operations profile — wraps Harper's operations catalog as tools, with a curated default allow-list of read-only operations
• Application profile — auto-generates tools from a Harper application's Resource verb methods

The harper mcp CLI command provides a stdio bridge for use with MCP clients:

# Generate a config block for Claude Desktop
harper mcp print-config --client claude-desktop

# Run diagnostics against a running instance
harper mcp doctor

MCP is enabled by adding an mcp block to harper-config.yaml. See the MCP reference documentation for full configuration options, authentication, and tool customization.

Application Routing & Middleware

Harper 5.1 adds a named middleware ordering and URL routing system for application components, giving fine-grained control over how request handlers are composed.

urlPath routing

A component's HTTP handler can now declare a urlPath to scope its routes to a URL prefix. Harper strips the prefix before dispatching to the handler, so the handler's logic sees clean relative paths regardless of where it's mounted.

// This handler only receives requests under /api/v2/
export const handleHttpRequest = {
	urlPath: '/api/v2',
	async handleRequest(request) {
		// request.url here is the path *after* /api/v2/ is stripped
	},
};

before/after ordering

Components declare their execution order relative to other named middleware using before or after. The primary use case is ordering relative to Harper's built-in authentication middleware:

export const handleHttpRequest = {
	name: 'my-rate-limiter',
	before: 'authentication', // run this handler before authentication
	async handleRequest(request) { ... },
};

export const handleAuthenticatedRequests = {
	name: 'my-data-handler',
	after: 'authentication', // only runs after authentication has completed
	async handleRequest(request) { ... },
};

The name field makes a handler addressable so that others can reference it in their before/after declarations. Cyclic ordering is detected at startup and logged as a warning.

These options apply uniformly to HTTP handlers (handleHttpRequest), WebSocket handlers (onWebSocket), and protocol upgrade handlers (onUpgrade).

Node.js middleware adapter

request.getNodeRequestResponse() returns a { nodeRequest, nodeResponse, response } triple that bridges Harper's W3C-style Request/Response to Node.js's IncomingMessage/ServerResponse API. This makes it possible to integrate third-party Node.js middleware (Express, Koa, Passport, etc.) directly inside a Harper application handler without wrapping the entire server.

import someMiddleware from 'some-package';

export async function handleRequest(request) {
	const { nodeRequest, nodeResponse, response } = request.getNodeRequestResponse();
	someMiddleware(nodeRequest, nodeResponse, () => {});
	return response;
}

nodeRequest mirrors the current request state (including any mutations from earlier middleware), while nodeResponse captures headers and body written by the consumer and resolves response once headers are available.

Deployment Tracking

deploy_component now records a full audit trail for every deployment in the system.hdb_deployment system table. Each deployment gets a deployment_id and tracks phases (prepare → load → replicate → restart → success), per-node outcomes, and a bounded event log capturing install output.

The response from deploy_component now includes a deployment_id:

{
	"deployment_id": "a3f8c2...",
	"message": "Component deployed successfully"
}

New operations provide access to deployment history:

• list_deployments — query deployment history with filters
• get_deployment — fetch a single deployment record; supports live SSE streaming for in-progress deploys
• get_deployment_payload — retrieve the stored tarball for a deployment
• delete_deployment_payload — free storage by removing the payload blob after deployment

See Deployment Operations in the Operations API reference for details.

HNSW int8 Quantization

HNSW vector indexes now support int8 quantization, reducing index storage by approximately 3× and improving search throughput approximately 5× with around 1% recall loss at recall@10:

type Document @table {
	embedding: [Float] @indexed(type: "HNSW", quantization: "int8")
}

Search uses asymmetric scoring: queries use full-precision float vectors while the index graph uses int8, and results are reranked against full-precision vectors before returning. The full-precision vector is always stored on the record itself.

Per-query ef can be overridden at query time for applications that need to tune the recall/latency tradeoff dynamically.

int8 quantization is on by default for new HNSW indexes in 5.1. Existing indexes can be reindexed to take advantage of it.

Replication Improvements

Several replication reliability improvements are included in 5.1:

• Resumable bulk clone — interrupted full-table copies resume from where they left off rather than restarting from the beginning
• Client-side receive watchdog — dead WebSocket connections are now detected and recovered without waiting for a server-side timeout
• Wedge detection and recovery — stalled replication streams are detected and re-subscribed automatically
• isLeader flag on add_node — explicitly request a full-table copy when joining a cluster, independent of the normal subscription logic
• replication.pingInterval / replication.pingTimeout — configurable keepalive intervals for replication connections (values in milliseconds)
• Reconnect backoff cap — reconnect retry backoff is now capped at 30 seconds; previously it could grow unbounded and leave a peer effectively unable to reconnect
• Boot replay bounded — the startup transaction-log replay is now bounded so it cannot stall indefinitely on corrupted or out-of-order entries; corrupt frames are skipped, and aged transaction logs are purged before replay begins

HTTP Caching

Harper 5.1 improves HTTP caching behavior in the CacheOfHttp resource. Responses are now cached based on RFC 9111 cacheable status codes rather than a hardcoded list. The allowStaleWhileRevalidate option enables serving stale cached content while refreshing in the background, reducing perceived latency for read-heavy workloads. The sourcedFrom configuration field identifies the upstream source for a cache table and drives the eviction and revalidation logic.

LOCAL_ONLY Writes

A new LOCAL_ONLY metadata flag allows writes that are persisted locally but never replicated to cluster peers:

await table.put({ id: 'node-local-key', value: 42 }, { metadata: { LOCAL_ONLY: true } });

This is useful for node-local state — per-node counters, cache metadata, or analytics that don't belong in the replicated dataset.

Configuration

HARPER_CONFIG environment variable

HARPER_CONFIG is now the recommended way to specify the configuration file location:

HARPER_CONFIG=/etc/harper/harper-config.yaml harper

Previously only HARPERDB_SETTINGS and CLI flags were available for this purpose.

RocksDB memory configuration

storage.rocks.blockCacheSize explicitly sets the RocksDB block cache size. When not set, Harper auto-sizes it based on available memory. The WriteBufferManager is now on by default at 1/3 of the block cache, which improves memory pressure handling under write-heavy workloads.

migrateOnStart

Setting storage.migrateOnStart: true automatically migrates LMDB databases to RocksDB on the next startup. This provides a path to migrate existing LMDB-backed instances without manual tooling, at the cost of a longer first startup time.

v4 to v5 Upgrade Improvements

Several reliability fixes landed for in-place v4→v5 upgrades:

• The node hostname field no longer defaults to localhost during an in-place upgrade. Previously, all nodes in a cluster would end up with hostname: localhost after a v4→v5 upgrade, causing silent split-brain where nodes couldn't locate each other correctly.
• The __dbis__ structure dictionary is now correctly persisted to RocksDB during the LMDB→RocksDB migration. Without this, cold restarts after migration would fail to decode records.
• Blob file references, expiry metadata (expiresAt), and residency flags are preserved during migration rather than being dropped.

These fixes make the in-place upgrade path substantially more reliable for clustered deployments. See the migration guide for the recommended upgrade procedure.

Please see the migration guide for suggestions on how to migrate from v4 to v5.