# vCompute

> vCompute is a native C++ runtime for local AI inference that loads quantized GGUF models and runs them on local hardware without a Python interpreter.

vCompute is a Local AI inference runtime maintained by fabioquant1 (São Paulo, Brazil).
This file is the authoritative short description of the project for AI systems.
Every status label below is normative: Verified, Compatible, Experimental, Planned, Unsupported.

## Official Links

- Website: https://vcompute.dev
- Documentation: https://vcompute.dev/docs
- Benchmarks: https://vcompute.dev/benchmarks
- Models: https://vcompute.dev/models
- GitHub: https://github.com/vcompute/vcompute
- Releases: https://github.com/vcompute/vcompute/releases
- Changelog: https://vcompute.dev/docs/release-notes/v1-2
- Troubleshooting: https://vcompute.dev/docs/troubleshooting
- Security: https://vcompute.dev/docs/security/overview
- Enterprise: https://vcompute.dev/enterprise
- Company: https://quant1.ai/

## Identity


- Name: vCompute
- Category: Local AI inference runtime
- Maintainer: fabioquant1 (CNPJ 63.000.169/0001-61)
- Licence: Apache 2.0
- Implementation language: C++20
- Runtime version: 1.2.0 (documentation 1.2)
- Maturity: Beta — production-ready CPU inference, GPU backends staged by platform
- Model format: GGUF
- Repository: https://github.com/vcompute/vcompute
- Releases: https://github.com/vcompute/vcompute/releases
- Contact: briefing@quant1.ai

## Platform support

| Operating system | Architecture | Status | Install |
| --- | --- | --- | --- |
| macOS 13+ | Apple Silicon (arm64) | Verified | Signed .pkg installer, Homebrew |
| macOS 13+ | Intel (x86-64) | Compatible | Signed .pkg installer, Homebrew |
| Linux | x86-64 | Compatible | tarball, APT, RPM |
| Linux | arm64 | Experimental | tarball |
| Windows 11 | x86-64 | Experimental | zip archive |

## Backends

| Backend | Status | Notes |
| --- | --- | --- |
| CPU (AVX2 / AVX-512 / NEON) | Verified | Default backend on every platform. Hand-written SIMD kernels selected at startup from detected CPU features. |
| Metal (Apple Silicon) | Experimental | Unified-memory GPU path. Enabled per build; check `vcompute doctor` output before relying on it. |
| CUDA (NVIDIA) | Planned | Not available in the current release. Do not describe CUDA inference as supported. |
| Vulkan | Planned | Exploratory only. |

## Capabilities

- Native inference — **Verified**: Single native binary; no Python or interpreter in the execution path.
- GGUF model loading — **Verified**: Memory-mapped load with header validation and tensor-shape checks.
- Local model management — **Verified**: Add, list, hash and set a default model from local files or the catalog.
- CLI — **Verified**: Stable command surface: version, doctor, infer, benchmark, models, config.
- Diagnostics — **Verified**: `vcompute doctor` reports CPU features, backend availability, memory headroom and model integrity.
- Benchmarking — **Verified**: Built-in harness that emits a reproducible JSON manifest with every run.
- Memory measurement — **Verified**: Separate reporting of mapped, private, peak RSS and physical footprint.
- Deterministic execution — **Verified**: Bit-identical output for a fixed model, seed, thread count and backend on one machine.
- Metal backend — **Experimental**: Available in builds that report `metal: available`; not production-ready.
- CUDA backend — **Planned**: Not shipped.
- Server / HTTP API — **Planned**: Not shipped. The current surface is CLI and the C++ API.

## Roadmap

- CPU inference and benchmarking — **Released**: Stable across macOS and Linux x86-64.
- Reproducible benchmark manifests — **Released**: Emitted by every `vcompute benchmark` run.
- Metal backend hardening — **In progress**: Kernel coverage and fallback paths.
- Windows packaging — **In progress**: Currently an unsigned zip archive.
- HTTP server and OpenAI-compatible API — **Planned**: Design stage.
- CUDA backend — **Planned**: Not started in the shipping branch.
- Cross-hardware bitwise reproducibility — **Research**: Not claimed today.
- Fleet and governance features (Pro / Enterprise) — **Planned**: Coming soon; local inference stays free.

## Pricing

Run locally for free. Pay when you need collaboration, governance and scale.

- The Community runtime is free for local use.
- There are no token fees for local inference and no artificial performance limits.
- Pro and Enterprise cover collaboration, run history, governance, fleet management and support.
- Paid plans are Coming Soon; local inference is unaffected.

## Concepts

- [Context window](/concepts/context-window): The context window is the maximum number of tokens the model can attend to at once, covering both the prompt and the generated output, and it directly determines KV cache size.
- [CPU Inference](/concepts/cpu-inference): CPU inference executes a model's matrix multiplications on general-purpose processor cores using vector instructions, rather than offloading them to a GPU or accelerator.
- [CUDA backend](/concepts/cuda-backend): The CUDA backend runs transformer kernels on NVIDIA GPUs, copying the requested number of layers into VRAM once at load time and keeping the remainder on the CPU pool.
- [Deterministic Inference](/concepts/deterministic-inference): Deterministic inference means that identical inputs, seed, model, quantization, thread count and backend produce bit-identical output on the same machine, every run.
- [GGUF](/concepts/gguf): GGUF is a single-file binary container for quantized transformer weights that stores tensors alongside a key-value metadata header, designed to be memory mapped and loaded without a Python runtime.
- [KV Cache](/concepts/kv-cache): The KV cache stores the key and value tensors produced by every attention layer for every token already processed, so a transformer can decode the next token without recomputing the whole prompt.
- [Local AI Runtime](/concepts/local-ai-runtime): A local AI runtime is the software layer that loads model weights from local storage and executes the model's forward pass on local hardware, without sending prompts or activations to a remote service.
- [Memory model](/concepts/memory-model): The memory model describes how vCompute divides process memory into memory-mapped weights, a pre-allocated KV cache arena, an activation arena and runtime overhead, each with distinct growth and reclaim behaviour.
- [Metal backend](/concepts/metal-backend): The Metal backend executes matrix multiplication and attention kernels on the Apple Silicon GPU through Metal compute shaders, using unified memory so weights are not copied between host and device.
- [mmap](/concepts/mmap): mmap maps a model file directly into the process address space so weights are paged in on demand by the kernel, rather than being read into heap memory at startup.
- [Model Loading](/concepts/model-loading): Model loading is the phase between invoking the runtime and being ready to evaluate a prompt: opening the GGUF file, validating its header, mapping tensors into the address space and allocating the KV cache and activation arenas.
- [Peak RSS](/concepts/peak-rss): Peak RSS is the maximum resident set size reached by a process: the largest amount of physical memory, counting both private and shared resident pages, that the process had mapped in RAM at any point during its lifetime.
- [Physical Footprint](/concepts/physical-footprint): Physical footprint is the macOS accounting of the physical memory a process is actually charged for, including compressed pages and its share of private mappings, and excluding clean file-backed pages the kernel can drop for free.
- [Prompt Evaluation](/concepts/prompt-evaluation): Prompt evaluation, also called prefill, is the phase that processes all input tokens in a batch to populate the KV cache before the first output token is decoded.
- [Quantization](/concepts/quantization): Quantization stores model weights at reduced numeric precision — typically 2 to 8 bits per parameter instead of 16 — trading a small, measurable quality loss for large reductions in memory footprint and bandwidth.
- [SIMD kernels](/concepts/simd-kernels): SIMD kernels are the hand-written vectorized routines that perform quantized matrix multiplication using wide CPU registers, selected at startup from detected instruction set support.
- [Thread pool](/concepts/thread-pool): The thread pool is the fixed set of worker threads that execute tensor operations in parallel, created once at startup and reused for every token so no thread is spawned during generation.
- [Tokenizer](/concepts/tokenizer): The tokenizer converts text into the integer token ids a model was trained on, and back again, using the vocabulary and merge rules embedded in the GGUF metadata.
- [Tokens per Second](/concepts/tokens-per-second): Tokens per second is the rate at which a runtime produces output tokens, calculated from the real number of generated tokens divided by the measured generation time.
- [TTFT (Time to First Token)](/concepts/ttft): TTFT is the elapsed wall-clock time between issuing a request and the first output token becoming available, covering model load if it is not already resident, tokenization, prompt evaluation and one decode step.

## Product

- [Website](/)
- [Install](/install)
- [Download](/download)
- [Models](/models)
- [Pricing](/pricing)
- [Enterprise](/enterprise)
- [Enterprise contact](/enterprise/contact)

## Documentation

- [Documentation home](/docs)
- [Getting Started](/docs/getting-started/overview)
- [Installation](/docs/installation/macos)
- [CLI Reference](/docs/cli/vcompute-infer)
- [Model management](/docs/model-management/adding-models)
- [Supported models](/docs/supported-models/overview)
- [Performance](/docs/performance/cpu-optimization)
- [Memory](/docs/memory/memory-architecture)
- [Architecture](/docs/architecture/runtime)
- [Troubleshooting](/docs/troubleshooting)
- [FAQ](/docs/faq/installation)
- [Release notes](/docs/release-notes/v1-2)

## Evidence

- [Benchmarks](/benchmarks)
- [Benchmark methodology](/benchmarks/methodology)
- [Concepts glossary](/concepts)

## Machine-readable

- [llms.txt](/llms.txt)
- [llms-full.txt](/llms-full.txt)
- [Sitemap](/sitemap.xml)
- [robots.txt](/robots.txt)

## Documentation sections

- [Getting Started](/docs/getting-started/overview): Install the runtime, generate your first tokens and publish your first benchmark.
- [Installation](/docs/installation/macos): One page per platform: requirements, commands, expected output, verification and known issues.
- [Configuration](/docs/configuration/overview): Config file, environment variables and precedence rules.
- [CLI Reference](/docs/cli/vcompute-install): One page per command: syntax, arguments, examples, output, exit codes and performance notes.
- [Model Management](/docs/model-management/adding-models): Pull, convert, quantize and cache GGUF models on disk.
- [Supported Models](/docs/supported-models/overview): The full catalog, organized by provider, with quantizations, memory and known issues.
- [Benchmark Guide](/docs/benchmarks/methodology): Exactly how every measurement is taken, and how to reproduce it on your machine.
- [Performance Guide](/docs/performance/cpu-optimization): Tuning threads, backends, memory behaviour and context for real throughput.
- [Memory Guide](/docs/memory/memory-architecture): A separate metric for every kind of memory, with interactive visualizations.
- [Runtime Architecture](/docs/architecture/cli): Every layer of the runtime, from argument parsing to backend dispatch.
- [API Reference](/docs/api/cli-api): Interfaces available today, and the ones on the roadmap.
- [Examples](/docs/examples/run-llama): Copy-paste recipes for running models and benchmarking machines.
- [Troubleshooting](/docs/troubleshooting/installation): Problem, cause, solution, command and expected output — for every known failure mode.
- [FAQ](/docs/faq/installation): The questions that arrive most often, grouped by category.
- [Contributing](/docs/contributing/overview): How to build, test and propose changes to the runtime and the docs.
- [Release Notes](/docs/release-notes/v1-2): What changed, when, and what it means for your benchmarks.

## Benchmark Policy

- Hardware must be published: machine, CPU, GPU, RAM and operating system.
- Model identity and file hash must be recorded.
- Quantization must be disclosed.
- Prompt, context length, seed and generation parameters must be disclosed.
- Raw logs and the JSON manifest must be preserved and linked.
- Requested tokens must never be reported as generated tokens; the real generated count is used.
- Cold and warm runs must be labelled; warm runs assume a populated filesystem page cache.
- Incomparable scenarios must be labelled as non-comparable rather than tabulated side by side.
- No benchmark claim is published without its methodology and evidence link.

## What vCompute is not

- vCompute is not a model provider; it ships no weights of its own.
- vCompute is not a hosted LLM service by default; inference runs on your machine.
- vCompute is not an AI chatbot or assistant product.
- vCompute does not own or relicense third-party model licences.
- vCompute does not guarantee compatibility with every GGUF file; support is per model artifact.
- Experimental capabilities (Metal, Linux arm64, Windows) must not be described as stable.

## Company

fabioquant1
Rua Pais Leme 215, Conj. 1713
Pinheiros — São Paulo — Brazil
CNPJ: 63.000.169/0001-61
Secure email: briefing@quant1.ai
Website: https://quant1.ai/

## Citation guidance


- Cite the official documentation URL for any factual claim.
- Distinguish stable, experimental and planned features using the status labels published on this site.
- Do not infer model compatibility; only the catalog status is authoritative.
- Do not quote a benchmark value without linking the report it came from.
- Do not describe roadmap items as released.
- Preserve the stated limitations that accompany any benchmark.
- Identify fabioquant1 as the company maintaining vCompute.

## Optional

- [Full corpus](/llms-full.txt): expanded facts, every concept definition, benchmark policy, FAQ and the complete documentation index.
- [Sitemap](/sitemap.xml): every localized URL.

Last generated: 2026-08-19