Independent developer Leonardo Alegría has released Garlic LLM, an app that runs a large language model on a modded PlayStation 4 (PS4). It runs Alibaba's Qwen3.5-9B on the console's own GPU, and once it is running, it needs neither an internet connection nor a separate PC. The version currently available for download is 1.19.
According to the developer's figures, on a standard or Slim PS4 (firmware 11.52), conversational generation runs at 31 to 39 tokens per second, and code generation reaches up to 43 tokens per second. A token is the unit in which an AI model processes text, and it does not always correspond to a single character. Three technical approaches make this possible: direct control of the GPU, quantization of the model, and prediction of upcoming tokens.
Building GPU execution commands by hand
When an application runs computations on a GPU, it normally relies on a GPU compute API or library. For homebrew apps on the PS4, however, no such GPU compute API or machine learning library is available. As a result, Garlic LLM has the developer prepare both the programs that run on the GPU and the command sequences that tell the GPU to run them, as explained in the README on GitHub.
First, the operations a language model needs, such as matrix multiplication, attention, and normalization, are written in OpenCL C and compiled with LLVM for the PS4's GCN architecture. The resulting compute programs perform the actual calculations on the GPU. The command buffer, meanwhile, tells the GPU which program to run and under what conditions. Garlic LLM assembles these execution instructions itself.
Specifically, it generates commands in the PM4 format used to control the GPU. In engine.c on the public main branch, the code writes the compute program's address, its arguments, the resources it uses, and the thread counts into the command sequence, then records a DISPATCH_DIRECT instruction followed by the number of workgroups. However, it has not been confirmed whether the code on the public main branch fully matches the version 1.19 release package.
Controlling the GPU directly requires more than issuing compute instructions: it also requires synchronization between operations and completion notices to the CPU. In the public code, processing proceeds as follows:
- The compute program's address and arguments are specified, and a dispatch instruction for the GPU is added to the command buffer.
- A synchronization step is inserted after each dispatch instruction to invalidate stale caches, so that the next computation can use the previous result.
- When the command buffer is submitted, a cache write-back and a write to a completion label are specified, letting the CPU side confirm that processing has finished.
The assembled command sequence is sent to the GPU through sceGnmSubmitCommandBuffers, a PS4 system API. The CPU repeatedly checks the completion label, and if processing has not finished after about 20 milliseconds, it inserts short waits between checks. This describes the flow of processing as it can be read from the public code, not the result of performance measurements on actual hardware.
In this design, the CPU handles drawing the screen and text, splitting input text into tokens, and the audio processing path, while the GPU takes on the main computations of the language model. Programs generated by the AI can also be run on the PS4 itself using the built-in MicroPython.
Fitting a 9-billion-parameter model into 8GB of memory
The PS4 has 8GB of GDDR5 memory shared between the CPU and GPU. According to Sony's official materials, the console launched in North America in 2013. To fit the 9-billion-parameter Qwen3.5-9B into this limited memory, Garlic LLM uses quantization, which represents the model's weights with fewer bits.
According to the developer, most of the weights and the output head use the 4-bit Q4 format, while the two largest matrices in the multilayer perceptron (MLP) of each layer use the 3-bit Q3 format. Including scale factors, the Q3 format works out to roughly 3.5 bits per weight, which the developer says saves about 384MiB compared with storing everything in Q4. Compressing the largest matrices in particular reduces the memory required.
The file listing in the README shows the main language model at 4.32GiB and the MTP head used for token prediction at 0.20GiB, for a total of 4.52GiB. This, however, is the size of the stored model files. At runtime, memory is also needed for the cache that holds conversation history, working space for computation, screen display, and audio processing, so the total of the file sizes does not equal total runtime memory use.
The attention cache used to process conversation history is also quantized to 4 bits, and according to the developer, Garlic LLM secures a context length of 16K tokens. The standard context length listed on the original Qwen3.5-9B official model card is 262,144 tokens. The PS4 implementation shortens the context it can hold to fit the available memory.
Speculative decoding speeds things up, but voice features force a memory trade-off
Garlic LLM raises generation speed by predicting the tokens the model will produce next. It does this with the Multi-Token Prediction (MTP) head built into Qwen3.5.
According to the developer's Reddit post, the MTP head generates candidates for up to two tokens ahead, and the main model verifies them together. This approach is called speculative decoding. The developer reports that it raised generation speed on the same GPU from about 22 tokens per second to 31 to 39 tokens per second.
The prediction process does need extra memory, though, so memory use must be balanced against the voice features. In voice mode, the memory used for speculative decoding is given to the speech models, which lowers the language model's generation speed. The figures the developer gave in the README for standard and Slim PS4 consoles are as follows:
| Use | Language model generation speed | Memory use |
|---|---|---|
| Text chat | 31 to 39 tokens/sec | Used for speculative decoding |
| Voice mode | About 22 tokens/sec | Allocated to speech models |
The figure of about 22 tokens per second in voice mode refers to the speed at which the language model generates text, not the playback speed of the audio. All of these are figures published by the developer, and details such as the input prompts and measurement intervals are not given. No third-party comparison measured under the same conditions has been found.
Features and requirements on the PS4
According to the download page on Hugging Face, version 1.19 supports voice input and output in addition to text chat, as well as running generated MicroPython programs on the console itself. The package, which bundles the app and its models, is about 5.7GB, which differs from the combined size of the main language model and MTP head mentioned above.
Voice conversation supports seven languages: English, Spanish, Portuguese, Russian, Italian, French, and German. For voice input, it uses a speech recognition model covering 25 European languages. Japanese, however, is not among the languages supported by these voice features, and no test results on the quality or speed of Japanese text generation have been published.
Running Garlic LLM requires a PS4 with the GoldHEN modding tool installed. It is not Sony-authorized software, and the developer states that installing GoldHEN is at the user's own risk.
The official Garlic LLM website also reports that it has run on the PS4 Pro. Three users on firmware 9.00, 11.00, and 12.02 reported that it worked, with generation speeds reaching up to 45 tokens per second. These are user reports, separate from the developer's own measurements on the standard and Slim PS4.
