Transaction Logging

Harper provides two complementary mechanisms for recording a history of data changes on a table: the audit log and the transaction log. Both are available at the table level and serve different use cases.

Feature	Audit Log	Transaction Log
Storage	Standard Harper table (per-table)	Clustering streams (per-table)
Requires clustering	No	Yes
Available since	v4.1.0	v4.1.0
Stores original record values	Yes	No
Query by username	Yes	No
Query by primary key	Yes	No
Used for real-time messaging	Yes (required)	No

Audit Log

Available since: v4.1.0

The audit log is a data store that tracks every transaction across all tables in a database. Harper automatically creates and maintains a single audit log per database. The audit log captures the operation type, the user who made the change, the timestamp, and both the new and original record values.

The audit log is enabled by default. To disable it, set logging.auditLog to false in harper-config.yaml and restart Harper.

The audit log is required for real-time messaging (WebSocket and MQTT subscriptions) and replication. Do not disable it if real-time features or replication are in use.

Audit Log Operations

read_audit_log

Queries the audit log for a specific table. Supports filtering by timestamp, username, or primary key value.

By timestamp:

{
	"operation": "read_audit_log",
	"schema": "dev",
	"table": "dog",
	"search_type": "timestamp",
	"search_values": [1660585740558]
}

Timestamp behavior:

search_values	Result
[]	All records for the table
[timestamp]	All records after the provided timestamp
[from, to]	Records between the two timestamps

By username:

{
	"operation": "read_audit_log",
	"schema": "dev",
	"table": "dog",
	"search_type": "username",
	"search_values": ["admin"]
}

By primary key:

{
	"operation": "read_audit_log",
	"schema": "dev",
	"table": "dog",
	"search_type": "hash_value",
	"search_values": [318]
}

Response example:

{
	"operation": "update",
	"user_name": "HDB_ADMIN",
	"timestamp": 1607035559122.277,
	"hash_values": [1, 2],
	"records": [
		{
			"id": 1,
			"breed": "Muttzilla",
			"age": 6,
			"__updatedtime__": 1607035559122
		}
	],
	"original_records": [
		{
			"__createdtime__": 1607035556801,
			"__updatedtime__": 1607035556801,
			"age": 5,
			"breed": "Mutt",
			"id": 1,
			"name": "Harper"
		}
	]
}

The original_records field contains the record state before the operation was applied.

delete_audit_logs_before

Deletes audit log entries older than the specified timestamp. Deprecated in favor of delete_transaction_logs_before.

Changed in: v4.3.0 — Audit log cleanup improved to reduce resource consumption during scheduled cleanups

Changed in: v4.5.0 — Storage reclamation: Harper automatically evicts older audit log entries when free storage drops below a configurable threshold

Changed in: v5.2.0 — This operation requires table, but on the RocksDB storage engine (the default) history cannot be deleted for a single table because all tables in a database share one transaction log. On RocksDB this operation now always returns an error directing you to delete_transaction_logs_before; it remains usable on LMDB.

{
	"operation": "delete_audit_logs_before",
	"schema": "dev",
	"table": "dog",
	"timestamp": 1598290282817
}

Transaction Log Operations

delete_transaction_logs_before

Deletes transaction log entries older than the specified timestamp.

Engines: RocksDB, LMDB

On RocksDB (the default storage engine), deletion is database-wide: all tables in a database share one transaction log, so omit table and pass only database (or schema) and timestamp.

Changed in: v5.2.0 — On RocksDB, a request that includes table is now rejected with a 400 error; previously the table scope was silently ignored and the entire database's transaction log was purged. On either engine, a table that does not exist now returns a 404 (previously this was a silent no-op on LMDB). On LMDB, a valid table continues to scope the deletion to that table's history, unchanged.

On LMDB only, the optional cleanup_deleted_records (boolean) parameter additionally removes leftover tombstone entries for records deleted before the timestamp — a repair step for tombstones that normal audit log cleanup should already have removed. It is ignored on RocksDB.

{
	"operation": "delete_transaction_logs_before",
	"database": "dev",
	"timestamp": 1598290282817
}

---

Enabling Audit Log Per Table

You can enable or disable the audit log for individual tables using the @table directive's audit argument in your schema:

type Dog @table(audit: true) {
	id: Long @primaryKey
	name: String
}

This overrides the logging.auditLog global configuration for that specific table.

Related Documentation

• Logging — Application and system logging (separate from transaction/audit logging)
• Replication — Clustering setup required for transaction logs
• Logging Configuration — Global audit log configuration (logging.auditLog)
• Operations API — Sending operations to Harper