Local inference runtime · public alpha

commonllama

A faster memory for local AI assistants.

commonllama keeps an assistant's identity pinned in memory while its working context swaps around it. A Node.js native addon, Apache-2.0 licensed, maintained by Clear Box.

How it works

The identity stays pinned. The workspace swaps around it.

The model
locked
RAM
DISK

Animations licensed CC BY 4.0. Attribution: commonllama project, clrbx.org.

Built on llama.cpp.

llama.cpp provides

  • Model loading and inference
  • Quantization
  • KV cache primitives

commonllama adds

  • Identity pinning and workspace hot-swap
  • Session encryption and memory zeroing
  • 16-method pool facade
  • 55 error codes, 14 event types

Why this is faster.

llama.cpp gives you raw KV cache primitives. It can save sequences, copy them, remove ranges. But it has no opinion about what to keep and what to throw away. Most tools built on top of it today treat the cache as disposable: load a model, run a conversation, tear it down, start over.

commonllama treats the cache as structured memory. It divides it into three regions: who the assistant is (identity), what the current task needs (workspace), and the live conversation. The identity loads once and stays resident. When a user switches tasks, only the workspace swaps. The conversation resets and grows again from there. The model never re-reads the identity prompt.

Without pinning, every task switch re-processes thousands of identity tokens from scratch. With pinning, the restore is a memory operation, not a decode operation.

28s without pinning MacBook M3 Pro, Qwen3 4B. Rebuilds identity from scratch on every task switch.
2s with pinning Same hardware. Identity stays resident, only the workspace restores.

Measured on real hardware with the project's benchmark harness. The richer the identity, the wider the gap.

Next: KVBM.

A rework of how the runtime handles memory. On the bench now, about three months out.

Five-tier cache

KV state moves between fast and slow storage as demand shifts, encrypted at every tier.

Hardware profiles

The runtime reads the machine it is on and places memory accordingly, from a single-board computer to a workstation.

Cook once

Prompts are processed into reusable artifacts. PUBLIC artifacts can be shared between machines. PRIVATE ones are encrypted and never leave yours.

Honest ETAs

Job estimates come from measured throughput on your machine, not guesses.

What happens to the data.

Local inference is table stakes. What matters is how the runtime handles state on disk and in memory. Each claim links to the code.

Encrypted at rest. Saved checkpoints, workspaces, and adapters use AES-256 with each platform's native crypto: OpenSSL, CommonCrypto, CNG. Key size enforced at 32 bytes.
Verify →
Memory zeroed on teardown. 365 call sites of secure_zero_bytes using compiler-resistant primitives (SecureZeroMemory, memset_s, explicit_bzero). If zeroing verification fails, the process aborts.
Verify →
No telemetry. Zero emission code. The SDK warns the host app if it detects third-party telemetry agents running alongside it.
Verify →
No networking primitives. Zero socket, HTTP, or fetch code in the runtime or its JS layer. Verifiable by reading the source, which is small.
Verify →

Any local runtime can run a model without phoning home. These four properties are about what happens after: if someone copies the disk, saved state is unreadable without the key. If a process crashes or is killed, sensitive tokens do not persist in RAM. These are structural properties of the runtime, not policies you configure.