AI Layer

Local SLM summaries — no cloud dependency

When the --ai flag is used, primer runs a quantized small language model locally to generate a plain-English summary of each CVE finding. The model runs entirely on your machine — no data leaves your system. If no model is present, primer falls back to displaying raw CVE JSON with no crash.

Default model

PropertyValue
ModelSmolLM2-135M-Instruct
FormatGGUF Q4_K_M (~80 MB)
SourceHuggingFace Hub (public, no account required)
Inference enginecandle (Rust-native, no Python)
Hardware accelerationApple Metal (M-series), NVIDIA CUDA, AVX-512

Quick start

primer model add                   # download default model
primer scan requests --ecosystem PyPI --ai   # scan with AI summary

Model management

Add a model

# Download the default model from HuggingFace Hub
primer model add

# Download a specific model from HF Hub
primer model add --repo HuggingFaceTB/SmolLM2-135M-Instruct-GGUF --file smollm2-135m-instruct-q4_k_m.gguf

# Import a local GGUF without a network request
primer model add --from /path/to/model.gguf

List models

primer model list

Shows each registered model with its absolute path, file size, and whether it is the active model.

Set the active model

# Local GGUF
primer model set ~/.primer/models/smollm2.gguf

# Ollama backend
primer model set ollama:llama3.2
primer model set ollama:mistral

When the backend is ollama, primer skips candle inference and sends the CVE summary prompt to http://localhost:11434/api/generate. Ollama must be running separately.

Remove models

primer model remove smollm2.gguf          # remove one model by name
primer model remove                        # interactive select list
primer model remove --all                  # remove all models and clear config

External paths: files outside ~/.primer/models/ (registered via model set <path>) are never deleted — only the config pointer is cleared. Ollama targets have no file on disk; only the config entry is removed.

Using AI in scans

primer scan pillow --ecosystem PyPI --ai

Appends a ≤3-sentence summary citing the CVE ID before the decision prompt. The system prompt enforces strict CVE citation and JSON output — no hallucination of package names or severity scores.

Disabling AI

PRIMER_AI=0 primer scan pillow --ecosystem PyPI

Set PRIMER_AI=0 in CI pipelines to suppress AI summaries and reduce log noise. Findings are still reported; only the summary is omitted.

Config written by model commands

# ~/.primer/config.toml after primer model set

# local GGUF:
[ai]
backend   = "local"
model     = "/Users/you/.primer/models/smollm2.gguf"
tokenizer = "/Users/you/.primer/models/tokenizer.json"

# Ollama:
[ai]
backend = "ollama"
model   = "llama3.2"

Build with AI support

Release binaries ship with --features ai compiled in. For development builds:

cargo build --release --features ai      # enables candle + Metal/CUDA flags
cargo build --release --features ai,cuda # explicit CUDA acceleration
cargo build --release --features ai,metal # explicit Metal acceleration