It is pitch black. You are likely to be eaten by a grue.

Grue is a benchmark that has language models play text adventures and scores every turn. Nine of them played two games written for the occasion. Zork, but they haven’t read it.

Scores

Nine models played two original games. They are ranked by the share of points earned.

Lighting the lamp.

Runs

Every finished run, turn by turn, filterable by game and model.

- playing now
Lighting the lamp.

Transcript

No run selected. Select a run to read it.

Get the data

One row per finished run: the settings, the score, the tokens and the cost. No key needed.

Main columns of the export
ColumnMeaning
session_idThe run. Its transcript is at /api/results/{session_id}/transcript.
game_id, collectionThe game, and the featured collection a run belongs to (blank for the rest).
model, effortProvider and model, and the reasoning effort sent. Blank effort means the provider default.
turns, max_turnsCommands played, and the turn limit.
score, score_pct, wonFinal score, its share of the maximum, and whether the game was won.
endingHow it stopped: won, died, turn_limit, quit, timeout and so on.
invalid_commands, invalid_rateCommands the parser rejected, and that count per turn.
unique_locationsDistinct rooms visited.
input_tokens, output_tokensReported by the client after the run.
cost_usd, cost_sourceCost in dollars. provider was billed by the API, prices is an estimate.

36 columns in all, including the run settings (history, notes, temperature, max_tokens). A blank cell means not reported. Filters: ?game=, ?collection=, ?include_tests=1. Details in PROTOCOL.md.

pandas
import pandas as pd
df = pd.read_csv("https://grue.party/api/export.csv")
DuckDB
duckdb -c "select model, effort, avg(score_pct) from 'https://grue.party/api/export.csv' where game_id = 'skerrin-light' group by all order by 3 desc"

Plug in your model

Games, results, and transcripts are public. Playing here needs a key, which you get by opening a GitHub issue; self-hosting is free.

An original game is the most useful thing you can send, and no model has read it.

    Brass Lantern: plug in a model
    git clone https://github.com/herzigma/brass-lantern && cd brass-lantern
    pip install -e . && cp .env.example .env   # add a model key
    brass-lantern play --game skerrin-light --model openrouter:openai/gpt-6.1-sol
    Grue: run your own server
    git clone https://github.com/herzigma/grue && cd grue
    npm install && npm run setup && npm start   # localhost:8081

    The API is plain JSON over HTTP: PROTOCOL.md.