local ai benchmarks

1. Current snapshot

This page tracks measured local AI runs. It is not a vendor leaderboard. Each row should keep the date, hardware, runtime, model, quantization, context, pass rate, and exact benchmark conditions together so progress is visible over time. Cloud profiles are tracked as comparison lanes, not local hardware lanes. Comparable result tables are sorted by best performer first.

2. How to read the numbers

The benchmark separates three layers. Raw tokens per second is useful, but local coding agents should be judged on task success first.

Promote a profile only when it improves correctness, tool-call validity, and stall behavior. A fast chat response is not enough. Do not compare Gemini token/s directly against ds4 or Spark CUDA as if all three were local hardware runtimes.

3. Run registry

Individual measured profile rows are ordered by current benchmark standing. Aggregate run rows stay last because they describe a task set, not one performer.

4. Daily-agent task results

Median wall time below excludes the long-context stress task. Rows are sorted by total successful daily-agent attempts, then lower wall time.

5. Practical suites

The expanded harness now runs four suites: smoke, code, question, and wiki. Rows are sorted by overall pass count, then lower median wall time.

6. Cloud comparison lane

Gemini belongs in this registry as an explicit cloud lane beside the local stack. Run it from the M5, keep Spark focused on local CUDA serving, and publish cloud metrics separately from local token-throughput metrics.

7. Spark token rates

Rows in each runtime table are sorted by median generation rate. Ollama response metrics use qwen3-coder:30b with OLLAMA_NUM_CTX=32768.

llama.cpp response metrics for the same model bits copied into qwen3-coder-30b.gguf with 32K context.

8. M5 Max 128GB reference data

The home AI wiki has a separate M5 Max 128GB thread. Keep these rows separate from the daily-agent registry because they mix local measurements, community MLX runs, and vendor claims.

Local measured ds4 on M5 Max 128GB

Measured ds4 rows are sorted by generation rate; tiny-output overhead rows are last.

Community MLX / GGUF data for M5 Max 128GB

Community rows are sorted by reported generation throughput. They differ in model, context, and source, so they should stay labeled rather than merged into the local harness ranking.

Vendor / planning data

External references for the non-local rows: Apple MLX M5 research, Ollama MLX preview, llama.cpp Apple Silicon benchmark thread, and oMLX community runs go1vd8aj, m1wd0ucw, r9m8lvr3.

9. Long-context status

Long-context behavior is not resolved for Spark Ollama yet. A long-prefill task timed out at 180 seconds with OLLAMA_NUM_CTX=32768; the expanded practical run timed out at 240 seconds. Before setting num_ctx, the model loaded with a 262144-token context and also stalled.

Spark llama.cpp completed the standalone long-prefill stress task in 5596 ms with 2753.8 prefill t/s. Apple ds4 completed the same stress task in the 20 second range, but only passed 1/3 because some outputs were too terse for the scoring rule.

{
  "context_length": 32768,
  "size": 21889382400,
  "size_vram": 0
}

Verify Spark GPU placement directly before drawing conclusions from this result:

docker exec ollama ollama ps
nvidia-smi

10. Next runs

11. Public scripts

The public bundle includes the benchmark runner, wrappers, tunnel examples, and a redaction check. It intentionally omits raw run logs, SSH known-hosts files, LAN addresses, MAC addresses, hostnames, usernames, and private paths.

12. Reproduce

Download the public bundle into a bench-local-ai folder, then run the same dated task set. Keep the generated summary public and run the redaction check before publishing raw records.

mkdir -p bench-local-ai
cd bench-local-ai
base='https://learntoprompt.org/downloads/bench-local-ai'
for file in \
  bench.py pull-ollama.py \
          run-ds4.sh run-spark.sh run-llama.sh run-both.sh \
  run-practical-all.sh \
  spark-tunnel.example.sh spark-llama-tunnel.example.sh \
  spark-llama-cpp-server.sh redaction-check.sh; do
  curl -fsSLO "$base/$file"
done
chmod +x *.py *.sh

# Spark Ollama daily-agent run
SPARK_MODEL='qwen3-coder:30b' ./run-spark.sh 3 --exclude-kind long-context --timeout 90

# Spark candidate run
SPARK_MODEL='qwen3-coder-next:q4_K_M' ./run-spark.sh 3 --exclude-kind long-context --timeout 180

# Spark llama.cpp current default
LLAMA_MODEL='qwen3-coder-30b.gguf' ./run-llama.sh 3 --exclude-kind long-context --timeout 180

# Full practical suite
LLAMA_MODEL='qwen3-coder-30b.gguf' ./run-practical-all.sh 1 --timeout 240

# Apple ds4 run
./run-ds4.sh 3 --timeout 240

# Check generated files before publishing
./redaction-check.sh results

For every new run, publish the date, hardware, runtime, exact model tag, quantization, context, thinking mode, task set, pass rate, median wall time, and the reason the profile was promoted or rejected. For cloud comparison runs, also publish first-token latency, cost per task, and whether credentials were loaded only from the environment or provider auth.