Skip to main content

5.2 Release Notes

Patch Releases​

All patch release notes for 5.2.x are available on the releases page.

Backup & Restore​

New managed-backup operations for RocksDB databases: create_backup, list_backups, verify_backup, delete_backup, purge_backups, and restore_backup. Managed backups are a server-side repository under the configured backup root (storage.backupPath, default <rootPath>/backup); the RocksDB data files are shared and incremental across backups, while the audit/transaction log — and, for databases with file-backed blobs, the blobs (skip with exclude_blobs) — are captured in full on every backup. verify_backup checks the RocksDB files (checksums with verify_checksum) and the transaction-log snapshot; it does not verify blobs. restore_backup restores a user database in place without stopping Harper, as long as no loaded component is holding that database open; a database held open by a component — and always the system database — is restored offline by running restore_backup from the CLI with the server stopped. Each of these operations can be run from the CLI under its operation name (e.g. harper create_backup database=data): while Harper is running the operation is forwarded to the server, and while it is stopped the command operates directly on the files. See Backups and Backup Operations.

get_backup now understands RocksDB: it streams a full-snapshot tar of the database — including any file-backed blobs (pass exclude_blobs: true to omit them), gzipped by default (pass gzip: false for a plain tar) — instead of failing. The LMDB behavior (streaming the .mdb, with table/tables/include_audit) is unchanged.

Transaction Log Deletion​

delete_transaction_logs_before no longer accepts a table scope on RocksDB (the default storage engine): all tables in a database share one transaction log, and the previous behavior silently ignored table and purged the entire database's log while reporting success (harper#2049). Three behavior changes may affect scheduled retention jobs on upgrade:

  • On RocksDB, a request naming a table now fails with an error directing you to the database-wide form (database + timestamp only).
  • The deprecated delete_audit_logs_before operation requires table, so on RocksDB it now fails for an existing table with the same guidance (a nonexistent table fails with the not-found error described below); it remains usable on LMDB.
  • On either engine, a table that does not exist now fails with a not-found error (previously a silent no-op on LMDB, and the database-wide purge on RocksDB).

See Transaction Logging for the updated operation reference.

Querying​

Filtered Vector Search (Predicate-Aware HNSW Traversal)​

Vector searches combined with filters now evaluate the filter during HNSW graph traversal, so the query keeps exploring until it has enough matching nearest neighbors instead of post-filtering a fixed candidate set (which under-filled results under selective filters). Filters can come from query conditions or JavaScript vectorFilter and rowFilter functions. rowFilter is an explicit query/subscription predicate that applications attach after admitting the operation; it also filters normal searches and subscription delivery. Very selective conditions automatically use an exact scan instead of graph traversal, and a filterExpansion visit budget bounds traversal cost. See Vector Indexing.

Authorization​

The legacy allowRead, allowUpdate, allowCreate, and allowDelete hooks remain one-time operation gates and are deprecated in favor of operation overrides, which receive the complete target and can access the request context through getContext(). This restores the v5.1 operation-scoped contract for standard instance-mode collection reads: admission occurs before query planning or scanning. It also closes legacy loadAsInstance = false gaps in built-in table handlers: search and subscribe use one allowRead verdict, array PUT uses one allowUpdate, query DELETE uses one allowDelete, and publish uses one allowCreate. Custom false-mode handlers remain responsible for authorization unless they delegate to the built-in handlers. See Row-Level Access Control with Explicit Filters for intentional row and subscription-event narrowing.

Components​

Scheduler: Recurring Jobs from Component Config​

Components can now declare recurring jobs in their configuration with a new built-in scheduler plugin. Jobs run on a five-field cron expression or a simple interval (90s, 5m, 1h), invoking a designated export from the component. In a cluster, execution is leader-coordinated - under normal operation each occurrence runs once, on an automatically elected leader node - with heartbeat-based failover, catch-up for missed occurrences, and per-job run state recorded in a replicated system table (handlers should be idempotent, as failover can occasionally deliver an occurrence twice; conversely, catch-up only backfills the single most recent missed occurrence, not a full backlog). See Scheduler.

Deploying by Git Reference (5.2.3)​

harper deploy by_ref=true builds a package reference from the local git repository, so a deploy records which commit it came from instead of shipping an anonymous snapshot of the working directory. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one.

A pinned SHA fixes the source revision rather than the built artifact, since the cluster still builds from source on each node. Tags and branches are resolved to a full commit SHA before the deploy is sent — from the local checkout when it has the ref, and from the remote when it doesn't, as a shallow CI clone usually doesn't. A ref that can't be resolved either way stops the deploy rather than being sent by name, because peers resolve the package independently and a name that moves mid-deploy would leave nodes on different code. The CLI warns when the working tree is dirty and when the commit isn't on any remote branch. Under GitHub Actions on a pull_request run it deploys the pull request's head commit, not the merge commit the runner checks out. See Deploying by Reference.

Sealed Deploy Credentials (5.2.3)​

harper deploy setup=true provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an enc:v1: envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. It handles the two private-source kinds it supports — a private GitHub repository and a private npm registry — and prints the credentials entry the deploy should use. For a git-by-reference deploy that entry is attached by harper deploy credential=true, which derives the host from the package so the credential always matches the clone it authenticates; an npm-registry credential is attached by passing the printed entry, which names the registry rather than a host. See Provisioning a Deploy Credential.

Configuration​

Replicated set_configuration​

The set_configuration operation now accepts "replicated": true to apply a configuration change to all cluster nodes in a single Operations API call, with per-node outcomes reported in the response's replicated array. Only cluster-appropriate parameters should be replicated — see Configuration Operations.

Analytics​

Transaction queue depth and commit time metrics​

New write-transaction-queue-depth and read-transaction-queue-depth metrics report per-thread write-commit and open-read-transaction concurrency against the storage engine, as a throughput/concurrency signal — see Transaction Queue Depth Metrics. A new transaction-commit-time metric records per-commit submit-to-settle duration, a leading indicator of gradual slowdowns toward the write-queue HTTP 503 rejection — see Storage Metrics.

CLI​

Explicit Authentication for Operations API Commands​

CLI Operations API commands now accept dedicated auth_username= and auth_password= parameters, allowing commands such as add_user and alter_user to authenticate as an administrator while keeping the affected user's username= and password= in the operation payload. Environment-variable credentials and saved harper login tokens now take precedence over the legacy username= and password= authentication fallback. Credential pairs are also resolved within one environment-variable namespace, preventing a username from HARPER_CLI_* from being combined with a password from legacy CLI_TARGET_* variables. See CLI Authentication.

Token Credentials for CI/CD​

A pipeline no longer needs an admin password. HARPER_CLI_REFRESH_TOKEN supplies a long-lived token that the CLI trades for a fresh, short-lived operation token on every run, and HARPER_CLI_OPERATION_TOKEN supplies one directly for callers that mint their own. Both rank above a saved harper login token and above the legacy username=/password= fallback, so a configured CI identity is authoritative on a runner that also has a developer's login — as long as the token resolves. It does not always fail closed when it doesn't: see Token credentials for CI/CD. A token refreshed from an environment variable is held in memory for that invocation only.

harper login --for-ci provisions them: it prints HARPER_CLI_TARGET and HARPER_CLI_REFRESH_TOKEN to stdout in .env format and nothing else, so harper login --for-ci | gh secret set --env-file - stores both without the token being displayed.

Because Harper keeps one refresh-token hash per user, issuing a token revokes that user's previous one — run --for-ci as a dedicated CI user rather than your own account. See Token credentials for CI/CD.

HTTP​

Middleware routing and ordering​

Applications can now be routed by virtual hostname, URL prefix, or both, with no dispatch code. Declare host and urlPath on the application's entry in the root harper-config.yaml — or pass them to deploy_component — and every handler the application registers is served under that hostname and path. Because the routing lives in the root config, the same application can be mounted differently per environment without editing it. A plugin's own urlPath still positions it within the application, and the application's mount is prefixed onto it.

Components can also pass host and urlPath directly to server.http(), server.ws(), and server.upgrade(). The new name, before, and after options provide explicit middleware ordering. See HTTP middleware routing and HttpOptions.

Security​

Web Application Firewall​

Harper Pro now includes a Web Application Firewall that evaluates rule-based IP/CIDR, method, path, header, and query conditions before authentication and application routing. Rules support block, log, and score actions; cluster-wide monitor and off modes; per-rule shadowing; node activation gates; live replicated updates; and RE2-backed regular expressions. See Web Application Firewall.

Server-Side SSH Key Generation (5.2.4)​

add_ssh_key accepts generate: true to have Harper mint an ed25519 keypair itself and return only the public half, so a deploy key's private half is never carried in a request body, shell history, or CI log. Generation happens in process, requiring no ssh-keygen binary on the host. The public key is returned only on the generating call — Harper does not retain it — and is commented harper:<name> so it is identifiable in the git host's key list. See Server-side key generation.

Replication​

Replicating the System Database with a Constrained Topology​

Controlled-flow replication can now include the system database while keeping a constrained (non-mesh) topology.

Previously, replicating system was discouraged when using controlled replication flow: because the node registry (hdb_nodes) lives in system and each node advertised itself as a full-mesh participant, replicating system caused every node to discover and directly connect to every other node — defeating the point of a constrained topology.

As of 5.2, a node with directional routes advertises a directional registry record derived from those routes (the neighbors it sends to / receives from) instead of a blanket "connect to everyone." A discovered non-neighbor that advertises such a record therefore is not subscribed to and does not receive a replication connection. This lets central configuration — users, roles, and schemas — propagate transitively across the entire cluster while user-database connections stay on the routes you configured.

For example, in a roadside → middle → core aggregation tree, replicating system now lets a role created on a roadside node reach the core (through the middle tier) without the core ever opening a direct replication subscription to a roadside node.

Behavior notes:

  • This applies only to nodes that have directional routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected. The constraint comes from the discovered node's advertised record, so in a mixed cluster a directional node still subscribes to a discovered node that advertises the legacy record; the no-direct-connection guarantee needs every discoverable participant to advertise directional routes.
  • This constrains replication subscriptions only; on-demand residency/retrieval connections (e.g. sharded or invalidated-cache reads) are a separate mechanism, governed by data residency, and are unaffected by this record.
  • add_node / set_node have accepted database-scoped sendsTo / receivesFrom entries since 5.1, but they only gate replication as documented from 5.2 on. Before 5.2 the entries were written to the added node's registry record alongside a blanket "sends everything" flag, which authorized receiving every database from that node and left sendsTo's database scoping inert, while the send-authority check honored a receivesFrom entry only when it named both this node and the database. As of 5.2 the blanket flag is gone — so the entries restrict replication with the added node to the databases they name — and an entry that omits the peer or the database is treated as a wildcard (see Add Node). Re-check any existing add_node / set_node scoping before upgrading — it can change in either direction: databases that were replicating only because of the blanket flag stop replicating on 5.2, and a peer-agnostic receivesFrom entry that the 5.1 send check ignored becomes an active wildcard that newly authorizes any subscribing peer for the database it names. Only the peer-agnostic object form ({ database?, excludeTables? }, with no target/source peer reference) is reliable today: Harper's reciprocal add_node_back registration to the added peer doesn't rewrite bare peer-name strings or target/source fields for the new direction, so attempts to scope an entry to one peer can silently fail to authorize replication (harper-pro#710 - add_node_back does not rewrite peer-qualified sendsTo/receivesFrom entries). These describe the added node's perspective, the opposite direction from a config route's replicates.sendsTo / replicates.receivesFrom, and (unlike config routes) aren't confined to the connection that created them. They also do not make the local node advertise a directional (non-mesh) hdb_nodes self-record, which is derived solely from the node's own config routes. Constraining system replication cluster-wide still requires directional routes in harper-config.yaml — which are also what makes the constraint durable, since every node rewrites its own registry row from its config routes on restart or component reload, superseding entries an add_node call wrote onto that row.
  • Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its hdb_nodes registry. This does not open a connection either way.

See Controlling Replication Flow for configuration details.