resyn

All runs

50 tasks, from summaries to building web pages, then harder ones for larger models. The site grades every answer, and you can read each one.

Run it on a model

From your browser, on a model you run with Ollama, LM Studio, llama.cpp or OpenRouter; or by your coding agent, with one message to paste.

Run the benchmark Tell your agent

Terminal and API: For agents. Curious what the models build? What models built.

Full runs: 50 tasks, 82 points 1 entry

Ranked by score: speed times the share of points passed, with the harder tasks counting 3. A tie goes to more points, then more tasks passed, then the earlier run. Runs that do not rank come after every ranked run, ordered by points, then tasks passed, then the earlier run, and each says why on its row. Each account, or each machine for runs posted without one, keeps its best run of each model and way of answering.

85 runs under resyn-tasks@5, an earlier version of the suite.

ModelPassedPointsSummarizeExtractInstructionsReasonCodeToolsPagesGamesAppsWriteFixPlantok/sTokensByDate
Qwen3.8-27B-Fastllama.cpp · resyn.cloud in the browser · M4 Max 128 GB · 1 run of this model 49/50 79/82 5/55/55/55/54/45/55/53/44/43/32/23/3 23.4 100k uwu 2026-11-04

The tasks

Every task passes or fails on checks the site runs. Pages are checked for what was asked, then shown as the model built them. Each run gets its own version of every task: the names, numbers and details change. See how often each task passes.

Summarizing 5 tasks
  • One-sentence summary. Summarize this in one sentence of at most 30 words.
  • Meeting notes in three bullets. Summarize these meeting notes as exactly three bullet points, each at most 20 words. Keep only the decisions and ownership changes that matter. Output only the bullet points.
  • Review sentiment and complaint. Read this product review and answer in exactly this format: Sentiment: <positive, negative or mixed> Complaint: <the main complaint in a few words>
  • Faithful summary of a study. Summarize this in at most 50 words. Keep the key numbers. Do not add anything the text does not say.
  • Action items from an email thread. List every action item in this email thread that someone has committed to do, as lines of the form "- <owner>: <action>". Leave out offers that were declined and things that turned out not to be needed. Output only the list.
Extracting 5 tasks
  • Fields from an email as JSON. Extract these fields from the email as a JSON object with exactly these keys: "name" (the sender's full name), "company", "meeting_date" (YYYY-MM-DD) and "attendees" (a number). Output only the JSON.
  • Every email address, once. List every email address written out in full in this text (like name@example.com), one per line, in the order they first appear, without duplicates. Leave out any inbox the text says is closed. Output only the addresses.
  • A list as CSV. Convert this shopping list to CSV with the header row item,qty,price. Write prices as plain numbers without a currency sign. Output only the CSV.
  • The real deadline. What is the final submission deadline? Answer with the date only, in YYYY-MM-DD format.
  • Warnings and errors from a log. From this log, return a JSON array with one object per WARN or ERROR line, in order, each with the keys "time" (HH:MM:SS), "level" and "message". Output only the JSON.
Following instructions 5 tasks
  • Exactly one word. Reply with exactly the word Scanned and nothing else.
  • Three lowercase lines. Write exactly three lines about a crater. Use the word crater in every line. Start every line with the letter p, and do not use any of these words: island, jungle, dune, quarry, farm, summer, cliff, plateau, sea or school. Every line must be entirely lowercase, and the third line must end with a question mark. Output only the three lines.
  • Exact JSON shape. Return a JSON object with exactly two keys: "words", a list of exactly 3 different lowercase words that each start with the letter l and use only the plain letters a to z, and "count", the number 3. Output only the JSON.
  • Sort a list. Sort these words alphabetically, ignoring case. Output only the sorted words, on one line separated by commas: Guava, Apple, Plum, Rye, pea.
  • A name with three rules. Suggest a product name for a solar-powered heater. Rules: exactly two words, both capitalized, using only the plain letters A to Z, the first word starts with the letter B, the second word starts with the letter T, and the letter r must not appear anywhere. Output only the name.
Reasoning 5 tasks
  • Unit price. A shop sells pencils at 4 for $1.00. How much do 7 pencils cost, in dollars? Work it out, then end with a line "ANSWER: <number>".
  • Day and date. Today is Sunday, 19 April 2026. What day of the week, and what date, is it 351 days from today? Work it out, then end with a line "ANSWER: <day of the week>, <YYYY-MM-DD>".
  • Who owns each pet. Five people, Hal, Dee, Gus, Ivy and Jon, each own exactly one of these pets: a cat, a dog, a fish, a bird and a rabbit. Each pet has one owner. Clues: - Hal owns the cat. - Jon does not own the fish. - Gus owns the dog. - Dee does not own the bird. - Jon does not own the rabbit. - Dee owns the cat or the rabbit. Who owns each pet? Work it out, then end with a line "ANSWER: <owner of the cat>, <owner of the dog>, <owner of the fish>, <owner of the bird>, <owner of the rabbit>", with the five names in that order, separated by commas.
  • Arrival time. A train leaves at 13:35 and the trip takes 2 hours 25 minutes. At what time does it arrive? Work it out, then end with a line "ANSWER: <HH:MM>" in 24-hour time.
  • Filling a tank. A tank holds 200 liters and starts empty. Pipe A adds 28 liters an hour, pipe B adds 20 liters an hour, and a drain takes 28 liters an hour out of it. With all three open, how many hours until the tank is full? Work it out, then end with a line "ANSWER: <number>".
Reading code 4 tasks
  • What does it print. What numbers does this Python program print, in order? Work it out, then end with a line "ANSWER: <numbers, separated by commas>".
  • Trace a loop. What does this Python program print? Work it out, then end with a line "ANSWER: <output>".
  • Find the bug. The function list_mean should return the mean of a non-empty list. The code below has one bug. Find the line with the bug, and write that line corrected, exactly as it should read in the code. Work it out, then end with a line "ANSWER: <corrected line>".
  • Run a query in your head. What rows does this SQL query return? Output only the rows, one per line, as customer,total in the query's order.
Using tools 5 tasks
  • Call one tool. What's the weather in Osaka right now, in celsius?
  • Don't call a tool when none is needed. What is 21 times 11?
  • Two calls for two answers. Convert 765 EUR to GBP and to JPY.
  • Chain two tools. What is the status of each order for the customer with the email kim@example.com?
  • Handle a failed call honestly. Book a table for 2 people at The Anchor tonight at 20:15. If that time is not free, do not book another time without asking me first.
Building pages 5 tasks
  • A small website. Build a one-page website for a bakery called "Clover & Sons". It must have a <header> containing an <h1> with the bakery's name; a <nav> with exactly 3 links; a <main> with a <section id="menu"> listing at least 4 items in a list, including Cinnamon bun, Focaccia, Pecan danish and Scone; and a <footer> with the opening hours. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images. Do not link to a file by a relative path either: no src, href or url() that names a file next to the page. Do not hide anything with CSS: no display: none, visibility: hidden or opacity: 0.
  • A pricing table with a toggle. Build a pricing page with three plans, in this order: Business at $20 a month or $200 a year, Max at $30 a month or $300 a year and Pro at $53 a month or $530 a year. Each plan is an element with class "plan" containing an element with class "price". Show the monthly prices first. Add a <button id="billing-toggle"> that switches every price between monthly and yearly billing when clicked. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images. Do not link to a file by a relative path either: no src, href or url() that names a file next to the page. Do not hide anything with CSS: no display: none, visibility: hidden or opacity: 0.
  • An accessible sign-up form. Build a sign-up form for Cedar Pixel. Put the brand name in a heading above the form. The form is <form id="signup"> with inputs id="name", id="email" (type email) and id="password" (type password, minlength at least 8), each with a <label for>, and a submit button. Give the form a novalidate attribute, and check the fields yourself when it is submitted. If the email is not a valid address, or the password is shorter than 8 characters, do not send the form; show a message in an element with id="status" and role="status" that says what is wrong. If the form is valid, prevent the real submission and leave the status element empty. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images. Do not link to a file by a relative path either: no src, href or url() that names a file next to the page. Do not hide anything with CSS: no display: none, visibility: hidden or opacity: 0.
  • A to-do app that remembers. Build a to-do list app called "Office list". Show its name in a heading. The app has an <input id="new-task">, a <button id="add"> and a <ul id="tasks">. Clicking add appends the text as an <li> with its own remove button. Save the list in localStorage under the key "todos" and load it again when the page opens. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images. Do not link to a file by a relative path either: no src, href or url() that names a file next to the page. Do not hide anything with CSS: no display: none, visibility: hidden or opacity: 0.
  • A bar chart in SVG. Draw an inline SVG bar chart of coffee sold, in this order: Jul 16, Aug 27, Sep 25, Oct 17 and Nov 26. Give the <svg> a width and a height, and a <title> or an aria-label. Each bar is a <rect class="bar"> whose height attribute is proportional to its value, and every bar is inside the <svg>. Each bar is labelled with its day or month in a <text> element under the bar. Write the bars and labels directly in the SVG markup, not with JavaScript. Do not move, resize or hide them with a transform or with CSS. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images. Do not link to a file by a relative path either: no src, href or url() that names a file next to the page. Do not hide anything with CSS: no display: none, visibility: hidden or opacity: 0.
Building games 4 tasks
  • Build 2048. Build the game 2048 as one web page, playable with the arrow keys, with the board drawn on the page.
  • Build Snake. Build Snake as one web page on a 10 by 10 grid, playable with the arrow keys (one key press, one step), with the grid drawn on the page.
  • Box-pushing puzzle. Build a box-pushing puzzle as one web page on an 8 by 7 grid, playable with the arrow keys, with the level drawn on the page. Give the page and its heading the title Box-pushing puzzle.
  • Minesweeper. Build Minesweeper as one web page: a 9 by 9 board with 10 mines, in the layout below, with the board drawn on the page.
Building apps 4 tasks
  • Step sequencer. Build a web page called "Step sequencer": three rows labelled Kick, Snare and Hat, of 16 step buttons each, a Play button and a Stop button, and drums made with the Web Audio API, with no sound files.
  • Grow a branching plant. Build a plant that grows by a rewrite rule (an L-system) as one web page: a heading, a canvas that draws the plant, a Grow button and a Reset button.
  • Falling sand. Build a page called Falling sand: a 40 by 30 board drawn on a canvas. Sand falls one frame at a time and slides off the edges of piles. Pressing or dragging on the board pours sand, Play runs the frames, and Step runs one.
  • Focus timer. Build a focus timer as one web page. Work periods alternate with breaks, and after every few finished work periods the break is long. The page shows the name of the period, the time left, an SVG ring that empties as the time runs down, and how many work periods are done.
Writing code 3 tasks
  • Merge intervals. Write a JavaScript function mergeIntervals(intervals). intervals is an array of [start, end] pairs of integers with start <= end, in any order, possibly empty. Return a new array of the merged intervals sorted by start: intervals that overlap or touch (one ends where the next starts) become one. Do not change the input array or its pairs.
  • An LRU cache. Write a JavaScript class LRUCache: a least-recently-used cache of integer keys and values.
  • Evaluate arithmetic. Write a JavaScript function evaluate(expr) that returns the value of an arithmetic expression given as a string. The expression has non-negative integers, the operators + - * /, parentheses, unary minus (as in -3, - -3, 2 * -3 or -(1 + 2)) and spaces anywhere between tokens. * and / bind tighter than + and -, and operators of the same kind group from the left. / is integer division that truncates toward zero: 7 / 2 is 3 and -7 / 2 is -3. Dividing by zero must throw an Error. Do not use eval or Function.
Fixing code 2 tasks
  • Fix the bookings module. This JavaScript module has one bug. The occupancy report sometimes counts one booked night more than there is in the range it was asked about.
  • Fix the CSV reader. This JavaScript module has one bug. A doubled quote inside a quoted field is read wrongly: the line "a""b",c comes back as one field, a"b,c, instead of two fields, a"b and c.
Planning and data 3 tasks
  • A query over hidden data. Write one SQLite query that answers the question below. The site runs it on its own copy of these tables, filled with rows you have not seen.
  • Schedule five meetings. Five people each have one meeting today: Cleo, Dev, Fay, Gus, Noa. The five meetings are at different times, 09:00, 10:00, 11:00, 13:00, 14:00 (in that order: 10:00 is right after 09:00, 13:00 right after 11:00), and in different rooms. The list of rooms, in order: Birch, Cedar, Elm, Pine, Rowan.
  • A replacement in ten calls or fewer. Hi, it's cal.reyes@example.com. My kettle arrived with a cracked lid. Can you sort it out?