ext-infer: Local LLM Inference, Natively in PHP

ext-infer is a PHP 8.3+ extension that loads a GGUF model and runs LLM inference inside the PHP process. Chat completions, raw completions, and embeddings — no Python sidecar, no inference daemon, no remote API. It’s written in Rust on top of ext-php-rs and the llama-cpp-2 bindings to llama.cpp.

Best yet, it’s MIT-licensed with binaries installable via PIE.

Five minutes to first inference

Grab a model — Qwen3-0.6B is a great starting point at under a gigabyte:

mkdir -p models
curl -L -o models/Qwen3-0.6B-Q8_0.gguf \
    https://huggingface.co/Qwen/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf

Then talk to it:

<?php
use Displace\Infer\Model;
use Displace\Infer\Prompt;

$model    = Model::load('models/Qwen3-0.6B-Q8_0.gguf');
$response = $model->chat(
    Prompt::system('You are a helpful assistant.')
        ->withUser('What is 2+2?'),
    maxTokens: 256,
    temperature: 0.0,
);

echo $response->answer(), PHP_EOL;
$model->close();

That’s a chatbot in a dozen lines of PHP, running on your CPU, with no network in the loop.

The API is deliberately PHP-shaped

If you’ve worked with llama.cpp directly, you’ve met the chat-template problem: every model family expects its own special-token framing, and getting it wrong degrades output silently. ext-infer refuses to make that your problem. The Prompt builder is immutable and role-aware — Prompt::system(...)->withUser(...)->withAssistant(...) — and renders through the template embedded in the model file itself.

You write roles; the extension writes tokens.

Reasoning models get the same treatment. Qwen3 and R1-style models emit <think>…</think> blocks before their answer; the Response object splits them automatically. $response->answer() is the answer, $response->reasoning() is the chain of thought — log it, display it, or ignore it.

Embeddings, in-process

The same extension covers deeper workloads, like semantic search:

<?php
$model = Model::load('models/Qwen3-Embedding-0.6B-Q8_0.gguf');

$embedding = $model->embed('feeling overwhelmed by life');

$embedding->dimensions();          // 1024
$embedding->normalize();           // unit length
$embedding->cosineSimilarity($b);  // compare against another embedding

Every semantic search pattern can now run without an external API key. Embedding a query takes single-digit milliseconds on commodity hardware — less than the network round trip to a cloud endpoint costs before the remote model even starts working.

Engineering notes

• In-process means in-process. No subprocess fork, no IPC, no daemon to babysit. The model memory-maps into your PHP process; the only latency is decode time.
• Thread-safe by design. The llama backend is a Sync-guarded singleton and each call builds its own context, so ZTS PHP with parallel works.
• Acceleration is opt-in. Portable CPU is the default; build with FEATURES=metal for Apple Silicon GPU offload.
• Typed exceptions, PHPT suite, CI across PHP 8.3/8.4/8.5 on macOS arm64 and Linux x86_64/arm64.

What comes next

The sweet spots today are CLI tools, queue workers, and long-running processes — anywhere a loaded model can serve many requests. Per-request model loading in classic FPM works but pays the cost of reloading the model into memory each time; pair the extension with a worker runtime for hot paths.

Full docs are live at infer.displace.tech.

Inference is only half of a local RAG stack. The other half is storing and searching the vectors you generate. Stay tuned for tomorrow …