ELELEM-LLAMA2(7)

Name

elelem-llama2 — llama2.c, wired into elelem

Synopsis

git clone https://git.mokhan.ca/elelem/llama2.git
elelem chat -p llama2 [--model name]

Description

Runs Andrej Karpathy's llama2.c inside elelem. One C file does the whole forward pass; a 161-line C++ shim streams tokens back to Ruby.

Full documentation: elelem-llama2.

Environment

ELELEM_LLAMA2_CHECKPOINT

Required. Path to a llama2.c .bin checkpoint.

ELELEM_LLAMA2_TOKENIZER (default vendor/llama2.c/tokenizer.bin)

Path to a llama2.c tokenizer.bin. The default is the one in the vendored submodule.

ELELEM_LLAMA2_STEPS (default 0)

Total positions to run, prompt included. 0 means the model’s seq_len.

ELELEM_LLAMA2_TEMP (default 1.0)

Sampling temperature.

ELELEM_LLAMA2_TOPP (default 0.9)

Top-p (nucleus) sampling threshold.

ELELEM_LLAMA2_SEED (default -1)

Random seed. Negative uses the current time in nanoseconds.

Requirements

Ruby 4.0 or later. A C and C++ compiler (cc and c++). A llama2.c .bin checkpoint (not GGUF).

Status

Lab: an experiment in how inference works. Read it to learn. Source only; not published to RubyGems. History: commits.

Bugs

Known issues: plugin:llama2. Patches are welcome by email; see git-send-email.io.

See also

elelem(1), elelem-ggml(7), elelem-llama(7), documentation, source