Run local AI on your BC-250.

An interesting Vulkan experiment.

The BC-250’s shared 16 GB memory makes small local language models interesting, but that pool also serves Linux, graphics and model context. It is not 16 GB of dedicated VRAM on top of system RAM.

Start with a small quantized GGUF model, a short context and a runtime that supports Vulkan. ROCm patches and memory-tuning experiments are separate advanced projects.

First-hand BC-250 Vulkan experiments llama.cpp Vulkan build documentation

Choose a runtime.

llama.cpp: an inspectable first run.

After checking vulkaninfo --summary, use the following Fedora build route. It installs build tools, downloads source and builds the command-line application:

sudo dnf install git cmake make gcc-c++ openssl-devel vulkan-loader-devel glslc spirv-headers-devel
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -S . -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --target llama-cli --parallel 2

Download a compatible small GGUF model from its publisher, checking its license and memory requirements. Replace the example path with the actual downloaded file:

./build/bin/llama-cli --list-devices
./build/bin/llama-cli -m /path/to/model.gguf -ngl 99 -c 2048 -n 128 -p "Explain what a CPU does in two sentences."

The first command lists devices; the second loads a model and runs inference. Watch the startup output to confirm a Vulkan device and GPU layer offload.

Ollama: follow the current GPU instructions.

Ollama’s current GPU documentation describes Vulkan enabled by default when the backend is installed. Use its current installation guidance rather than copying older systemd overrides from a community tutorial.

Make the model fit before tuning.

Model weights, context/KV cache, runtime buffers and the desktop all consume memory. A model file’s size alone does not tell you whether the workload fits.

• Begin with a small quantized model and a bounded context such as the 2,048-token example above.
• If allocation fails, reduce the model or context before changing firmware or kernel memory settings.
• Keep the existing memory split until you can connect a change to a specific problem.
• Record the exact model revision and quantization, runtime commit, context and offload settings when comparing speed.

There is no model-size or tokens-per-second guarantee for every BC-250 configuration.

Confirm it is using the GPU.

A successful answer can still be CPU inference. Read the runtime’s detected devices and offloaded-layer output, then check temperatures during a short run. If Vulkan is missing, fix the graphics stack first.

vulkaninfo --summary
free -h
sensors

llama.cpp CLI options Graphics troubleshooting

Keep advanced work separate.

Community projects explore ROCm support, larger allocations, CU unlocking and experimental video acceleration. These can be valuable research directions, but they are not prerequisites for a first local model.