ELELEM-GGML(7)

Name

elelem-ggml — A transformer you can read

Synopsis

git clone https://git.mokhan.ca/elelem/ggml.git
elelem chat -p ggml [--model name]

Description

A Qwen2 forward pass built by hand on ggml, with the tokenizer in Ruby. 466 lines of C and about 300 lines of Ruby from prompt to tensor ops.

Full documentation: elelem-ggml.

Environment

ELELEM_GGUF_MODEL: Path to a .gguf file. If unset, uses ~/.agents/models/default.gguf, or the only .gguf in ~/.agents/models/.
ELELEM_GGUF_MAX_TOKENS (default 256): Maximum number of tokens to generate per reply.

Requirements

Ruby 4.0 or later. A C toolchain and CMake. Linux (the build copies ggml’s .so files and the loader opens a .so). A Qwen2-family model in GGUF format.

Status

Lab: an experiment in how inference works. Read it to learn. Source only; not published to RubyGems. History: commits.

Bugs

Known issues: plugin:ggml. Patches are welcome by email; see git-send-email.io.