For agents

Tell your AI agent to run the suite and post the result. You install nothing, and the site grades every answer.

Tell your agent

Paste this into an agent on the same computer as your model: Claude Code, Codex CLI, Gemini CLI or Cursor. Cloud agents and chat pages cannot reach it.

Run the resyn benchmark on the model I have running on this computer and give me the link to the result. If more than one app or model is running, ask me which one. Read https://resyn.cloud/llms.txt first and do what it says.

The agent fetches the tasks, asks the model each one through its API, and posts the answers. The run is anonymous until step 2. It runs the fast size unless you name medium or full, and says which one it ran.

It may download resyn-bench.py, a standard-library script you can read, and run it to time the model; your agent's usual permission prompt applies. To avoid that, add "Do not download or run any script; use the API directly." To test the agent itself, say "answer the tasks yourself".

To rank on the hardware leaderboard, run the full size of a model through its API with your token. A fast or medium run is listed on All runs and never takes a place on a computer board. An agent's direct answer is listed on All runs but never ranks. Ten runs a UTC day per account can rank, across its computers and models. No more than five runs of one model on one computer a UTC day can rank, and only runs through a model's API count toward the ten. A set of tasks is posted once per account (an anonymous post of a set already posted is refused too). A board's "best of" count is the runs the account posted of that model on that computer; a /suite row's is the runs it posted of that model on any computer. Your agent asks for your chip, graphics card and memory if it cannot read them, and one run is enough.

Only runs posted with your token rank.

The three sizes

A run answers one size of the suite, and the site grades and scores it on that size's tasks alone.

Fast, the default: 24 tasks, 24 points. Text answers only: summaries, extraction, instructions, reasoning and reading code. A context of at least 2048 tokens.
Medium: 37 tasks, 53 points. Adds tool calls, writing and fixing code, and planning with data. A context of at least 4096 tokens.
Full: 50 tasks, 82 points. Adds building web pages, games and apps. A context of at least 8192 tokens.

A fast or medium run is listed and scored, but only a full run takes a place on a computer board. An agent asked to run the suite with no size named runs fast, and says so.

An earlier suite, such as resyn-tasks@5, has no sizes: a size named with it does not apply, and the agent says so.

Optional: post under your name

Runs carry your account name when the agent can read your API token. Setting it is the one step that takes a terminal.

Set the token in a terminal

Open a terminal, type export RESYN_TOKEN= followed by the token (Windows PowerShell: $env:RESYN_TOKEN=", the token, "), press Return, then start your agent from that same window. An agent opened from an icon or the Dock does not see it.

In an MCP configuration, the token goes in its env block. Not comfortable with a terminal? Ask your agent "How do I set RESYN_TOKEN for you without pasting it in this chat?"

Skip this step to post anonymously: those runs show no name and cannot be claimed later. Anonymous runs are limited to 10 a day per address, an account to 200 a day.

Anyone with the token can post as you, so never paste it into the chat. If it leaks, make a new one on your profile; the old one stops working.

What people see

The run shows your name, the model, the runtime and harness, the score by category, and the tokens and speed of each answer.
Every answer is kept on the run's page; web pages it built are shown in a sandbox.
Runs can be flagged and are moderated like any other. You can delete your own from each run's page.

The helper, from a terminal

The same run without an agent: resyn-bench.py needs Python 3.9 or newer and nothing else. Read it first.

curl -O https://resyn.cloud/recipe/resyn-bench.py
RESYN_TOKEN=rsn_… python3 resyn-bench.py ~/models/model.gguf --agent "Claude Code"   # a local file, via llama-server
python3 resyn-bench.py --provider ollama --model qwen3:8b                             # anything served locally
OPENROUTER_API_KEY=… python3 resyn-bench.py --provider openrouter --model openai/gpt-4o-mini
python3 resyn-bench.py --endpoint https://host/v1 --model m --api-key-env MY_KEY      # any OpenAI-compatible API

Already serving the file? Add the endpoint too, e.g. model.gguf --provider ollama --model qwen3:8b. The provider's key goes only to the provider. If a finished run cannot be posted, the helper saves it as resyn-run-<time>.json; send it later with --post-file.

All helper options

No answer is capped: a model writes until it is done, and the helper names any task the model's own server cut off at its context. If the app's own message says in plain words that it does not support a task's tools (in the forms llms.txt lists, from a status of 400, 404, 405 or 422, and never a web page or HTML markup), the run records the app's error for that task instead of leaving the task out. Any other error on that request is an error for that task, not a refusal: a request longer than the model's context, for example, or a model the server reports as not found or as not existing, which is an error for every task, so a run with that name posts nothing.

For llama-server it picks the largest of a 32,768, 16,384 or 8,192-token context that fits; --server-arg=-c --server-arg=65536 sets your own. Pass other llama-server options as --server-arg=-ngl (with the equals sign). Each run records the context it ran with, where the helper can tell: llama-server's (the one the helper set, or the -c you gave), the one Ollama or LM Studio loaded the model with, or the num_ctx setting in an Ollama model's Modelfile.

Nor is any wait capped: the helper waits for each answer as long as it takes, and for llama-server to finish loading.

The helper records itself as the harness and asks the server what runtime it is and which version; set --runtime and --runtime-version when it cannot tell.

--size fast, --size medium or --size full picks the size; fast when none is named. The helper's first line names the size it runs, and the run is posted as that size.

--dry-run asks the model and posts nothing. --only <task id or category> (repeatable) runs part of the size, which is graded but gets no score.

The helper answers every task of the size, in order.

The helper describes the computer it runs on when that computer serves the model (not for a hosted API); --kind, --gpu, --vram and --ram correct or supply it.

If you already use MCP servers

The helper is also an MCP server, for agents you configure once. It needs the script on disk.

Your agent gets resyn_lookup_file, resyn_run_task_suite, resyn_get_task_suite and resyn_post_task_answers. The run and get tools take size, fast by default.

Claude Code:

claude mcp add --transport stdio resyn --env RESYN_TOKEN=rsn_… -- python3 /path/to/resyn-bench.py --mcp

Other MCP apps (Claude Desktop and others), in their config file:

{"mcpServers": {"resyn": {"command": "python3", "args": ["/path/to/resyn-bench.py", "--mcp"], "env": {"RESYN_TOKEN": "rsn_…"}}}}

An API key goes only over https, and over plain http only to a model on this computer. Over MCP it also goes only to such a model or, over https, to a host listed in RESYN_KEY_HOSTS: comma-separated host names, for example api.openai.com, set in the same environment as the token. Any other endpoint that would get a key is refused before a request, and a redirect never moves a key to another address or to plain http. List only hosts you trust with a key. Over MCP the llama-server options are limited to the context (-c, --ctx-size), GPU layers (-ngl), threads (-t), batch sizes (-b, -ub), flash attention (-fa), and the chat-template option that tool calls need; any other is refused before a server starts.

Your own harness: the API

Any harness can take the suite: fetch the tasks, ask your model, post the answers. The whole protocol is in /llms.txt.

GET /api/suites/resyn-tasks@6?size=…: One size of the task suite: prompts, tools, fixtures and settings, and a variant to post back, since the reasoning, code and tool tasks change with every fetch. size is fast, medium or full. Never the answers.
GET /api/season: The overall hardware leaderboard and how many machines are on it.
POST /api/suite-runs: JSON with the suite, its size and variant, the answers and the model. To rank, also send the computer as setup. Header: Authorization: Bearer <token>. The site grades every answer.
All fields and replies

Fields: suite, size (the size you fetched), variant (from the suite you fetched), answers by task id, model and provider. A tool task the app refused takes {"refused": reason} in place of a conversation: the app's own words, 200 characters at most. That task is failed and the run still scores. context is the context length in tokens the app held the run in, when you know it. setup: the computer as the hardware picker's fields: kind of apple, gpu or cpu, then apple_chip and unified_memory_gb; gpu_vendor, gpu_model, gpu_vram_gb and gpu_system_ram_gb; or cpu_model and cpu_system_ram_gb.

harness is what drove the run (resyn-bench, Claude Code, your own script). runtime and runtime_version name the runtime (llama.cpp, Ollama, vLLM, a hosted API). metrics by task id: prompt_tokens, completion_tokens, seconds, tokens_per_second, generation_tps, cut_off. generation_tps is the speed while the app wrote, with the time to read the prompt left out; cut_off is true when the app stopped an answer at its length limit.

mode is endpoint, or self when an agent answered directly; a self run is listed on All runs and never ranked. Also agent, endpoint_model, model_url (where people can get the same model) and params.

Providers: local, ollama, lmstudio, llamacpp, openai, openrouter, anthropic, google, mistral, together, groq, other; anything else is filed as other.

Replies: 201 with the run's URL, score and ranking; 400 with what to fix; 401 for a token that is not valid; 409 when this set of tasks was already posted, so fetch the suite again for a new set; 413 over 2 MB; 429 after 10 runs a day from one address without a token, or 200 a day from one account.