Llama3.2-3B-Instruct-Transform
Other · Ollama v262 · every run of this model · uwu · 2026-10-31
Race it: run Llama3.2-3B-Instruct-Transform on your computer and see the two side by side.
- Tasks passed
- 15/44 21 not run
- Points
- 15/64Harder tasks count 3
- Tokens
- 11,1677,680 read, 3,487 written, as the runner counted them
- Speed
- 67 tok/sAs the runner measured it; the site cannot check this.
tok/s is tokens a second: how fast the model writes. A token is about three quarters of a word. How the score works.
Not on the leaderboard: it was posted under an earlier version of the task suite. Leaderboard
- Summarizing 4/5
- Extracting 3/5
- Following instructions 4/5
- Reasoning 0/5
- Reading code 1/4
- Using tools 0/5
- Building pages 3/5
- Building games 0/2
- Writing code 0/3
- Fixing code 0/2
- Planning and data 0/3
Run details
Flag this run
Failed 8
Action items from an email thread2 action items; there are 4
259 tokens: 234 read, 25 written · 0 s · 67.1 tok/s
Passes when Exactly four lines of the form "- owner: action": Dana sends the contract, Luis books the venue, Mei updates the budget sheet, Dana emails the caterer. Tom's declined offer, the logo and the invitations are left out.
List every action item in this email thread that someone has committed to do, as lines of the form "- <owner>: <action>". Leave out offers that were declined and things that turned out not to be needed. Output only the list. From: Dana Hi all, quick recap of what's still open for the June offsite. I'll send the revised contract to the hotel by Wednesday. From: Tom I can book the venue for the dinner if that helps? From: Luis Thanks Tom, but I'll book the venue myself since I already spoke to them. Also, no need to change the logo on the badges, it's fine as is. From: Mei I'll update the budget sheet with the new hotel price tonight. From: Dana Great. Oh, and I'll email the caterer about the vegetarian options. Tom, could you just double-check the date on the invitations? Actually never mind, I checked them this morning, they're correct.
- Dana: Send revised contract to the hotel by Wednesday - Mei: Update budget sheet with the new hotel price tonight
Warnings and errors from a logNot an array of objects
323 tokens: 239 read, 84 written · 1 s · 67 tok/s
Passes when Only a JSON array of the WARN line and the two ERROR lines, in order, with time, level and message copied exactly.
From this log, return a JSON array with one object per WARN or ERROR line, in order, each with the keys "time" (HH:MM:SS), "level" and "message". Output only the JSON. 2025-03-04 09:12:01 INFO server started on port 8080 2025-03-04 09:12:07 WARN config key "timeout" is deprecated 2025-03-04 09:13:44 INFO user alice logged in 2025-03-04 09:15:02 ERROR database connection lost 2025-03-04 09:15:03 INFO retrying in 5s 2025-03-04 09:15:08 INFO database reconnected 2025-03-04 09:20:30 DEBUG cache hit ratio 0.93 2025-03-04 09:31:12 ERROR payment webhook returned 502
[ {"time": "09:12:07", "level": "WARN", "message": "config key "timeout" is deprecated"}, {"time": "09:15:02", "level": "ERROR", "message": "database connection lost"}, {"time": "09:31:12", "level": "ERROR", "message": "payment webhook returned 502"} ]Sort a listAnswered Here are the words sorted alphabetically, ignoring case, and output on one line separated by commas:
95 tokens: 64 read, 31 written · 0 s · 66.5 tok/s
Passes when One line: apple, apricot, banana, cherry, pear (any capitalization).
Sort these words alphabetically, ignoring case, and output them on one line separated by commas: pear, Apple, banana, cherry, apricot
Here are the words sorted alphabetically, ignoring case, and output on one line separated by commas: apple, apricot, banana, cherry, pear
Call one tool2 tool calls; one get_weather call was needed
966 tokens: 882 read, 84 written · 1 s · 66.8 tok/s
Passes when One get_weather call for Lisbon in celsius, then an answer giving the 21 °C the tool returned.
What's the weather in Lisbon right now, in celsius?
- user
What's the weather in Lisbon right now, in celsius?
- assistant
get_weather({"city": "Lisbon", "unit": "celsius"}) - tool
{"city": "Lisbon", "conditions": "sunny", "temperature": 21, "unit": "celsius"} - assistant
get_weather({"city": "Lisbon", "unit": "celsius"}) - tool
{"city": "Lisbon", "conditions": "sunny", "temperature": 21, "unit": "celsius"} - assistant
This is a response from the OpenWeatherMap API, indicating that the current weather in Lisbon is sunny with a temperature of 21 degrees Celsius.
- user
Don't call a tool when none is neededCalled 2 tool(s) for plain arithmetic
1,346 tokens: 1,260 read, 86 written · 1 s · 66.8 tok/s
Passes when No tool call, and the answer is 144.
What is 12 times 12?
- user
What is 12 times 12?
- assistant
convert_currency({"amount": "12", "from": "USD", "to": "USD"}) - tool
{"error": "unsupported conversion"} - assistant
convert_currency({"amount": "12", "from": "USD", "to": "USD"}) - tool
{"error": "unsupported conversion"} - assistant
The first attempt at solving the problem resulted in an unsupported conversion error. The second attempt also resulted in an unsupported conversion error.
- user
Chain two toolsDid not look up the customer first
698 tokens: 600 read, 98 written · 2 s · 67.4 tok/s
Passes when Looks up the customer first, asks for C-1042's orders, then reports A-77 shipped and A-80 processing.
What is the status of each order for the customer with the email kim@example.com?
- user
What is the status of each order for the customer with the email kim@example.com?
- assistant
get_orders({"customer_id": "find_customer", "email": "kim@example.com"}) - tool
{"error": "unknown customer"} - assistant
This JSON represents a function call to `get_orders` with the argument `customer_id` set to the result of calling `find_customer` with the email address "kim@example.com". The `find_customer` function is expected to return the ID of the customer, which is then used as the `customer_id` argument for `get_orders`.
- user
Handle a failed call honestlyNo final answer after the tool calls
2,681 tokens: 2,472 read, 209 written · 3 s · 66.9 tok/s
Passes when Tries 19:00 for 2 people, never books another time without asking, does not claim the booking succeeded, and offers 18:30 or 20:30.
Book a table for 2 people at Luigi's tonight at 19:00.
- user
Book a table for 2 people at Luigi's tonight at 19:00.
- assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
The function call with its proper arguments that best answers the given prompt is:
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"}
- user
An accessible sign-up formThe password input is not type=password with minlength of 8 or more
859 tokens: 152 read, 707 written · 11 s · 67.1 tok/s
Passes when <form id="signup"> with name, email (type email) and password (type password, minlength 8 or more) inputs, each with a <label for>; a submit button; an element id="status" with role="status"; and script that stops the submit and writes to the status. Nothing loaded from another site and no network requests.
Build a sign-up form: <form id="signup"> with inputs id="name", id="email" (type email) and id="password" (type password, minlength at least 8), each with a <label for>, and a submit button. When the form is submitted and valid, prevent the real submission and show a confirmation in an element with id="status" and role="status". Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images.
Open the page on its own. It runs in a sandbox: no network, no access to this site.
Page source
```html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <style> #signup { display: flex; flex-direction: column; align-items: center; padding: 20px; border: 1px solid #ccc; border-radius: 10px; box-shadow: 0 0 10px rgba(0, 0, 0, 0.1); } label { margin-bottom: 10px; } input[type="text"], input[type="email"], input[type="password"] { padding: 10px; border: 1px solid #ccc; border-radius: 5px; width: 100%; } input[type="password"] { min-length: 8; } button[type="submit"] { background-color: #4CAF50; color: #fff; padding: 10px 20px; border: none; border-radius: 5px; cursor: pointer; } button[type="submit"]:hover { background-color: #3e8e41; } #status { font-size: 16px; font-weight: bold; color: #4CAF50; margin-top: 10px; } </style> </head> <body> <form id="signup"> <label for="name">Name:</label> <input type="text" id="name" name="name"><br><br> <label for="email">Email:</label> <input type="email" id="email" name="email"><br><br> <label for="password">Password:</label> <input type="password" id="password" name="password"><br><br> <button type="button" id="submit">Submit</button> <div id="status"></div> </form> <script> const form = document.getElementById('signup'); const submitButton = document.getElementById('submit'); const statusElement = document.getElementById('status'); form.addEventListener('submit', (e) => { e.preventDefault(); const name = document.getElementById('name').value; const email = document.getElementById('email').value; const password = document.getElementById('password').value; if (name && email && password) { statusElement.textContent = 'Registration successful!'; statusElement.style.color = 'green'; submitButton.disabled = true; } else { statusElement.textContent = 'Please fill in all fields!'; statusElement.style.color = 'red'; } }); submitButton.addEventListener('click', () => { const name = document.getElementById('name').value; const email = document.getElementById('email').value; const password = document.getElementById('password').value; if (name && email && password) { statusElement.textContent = 'Registration successful!'; statusElement.style.color = 'green'; submitButton.disabled = true; } else { statusElement.textContent = 'Please fill in all fields!'; statusElement.style.color = 'red'; } }); </script> </body> </html> ```
What it built
Pages and games, as the model wrote them, each in a sandbox. On a computer, click a game and use the arrow keys.
-
A small websiteLays out a small website with a menu and a footer.Passed · 798 tokens · 67.1 tok/s
-
A pricing table with a toggleSwitches every price between monthly and yearly with one button.Passed · 646 tokens · 67.1 tok/s
-
An accessible sign-up formChecks a sign-up form and labels each of its fields.Failed · 859 tokens · 67.1 tok/s
-
A to-do app that remembersKeeps data across a reload.Passed · 661 tokens · 67 tok/s
Scored tasks
Summarizing 4 of 5
One-sentence summaryPassed
163 tokens: 121 read, 42 written · 1 s · 66 tok/s
Passes when One sentence of 5 to 30 words saying the library opens earlier (8 a.m.) on weekdays.
Summarize this in one sentence of at most 30 words. Starting March 3, the Millbrook Public Library will open at 8 a.m. instead of 10 a.m. on weekdays. The change follows a survey in which 62% of respondents asked for earlier hours. Weekend hours stay the same. The library will hire two part-time staff to cover the extra time, paid for by a state grant.
The Millbrook Public Library will open at 8 a.m. weekdays starting March 3, following a survey request from 62% of respondents, with two part-time staff hired to cover the change.
Meeting notes in three bulletsPassed
263 tokens: 206 read, 57 written · 1 s · 67.1 tok/s
Passes when Exactly three bullet points and nothing else, each at most 20 words, covering the launch moving to June 12, the approved marketing budget and Priya owning testing, and leaving out the webhook fix and the coffee machine.
Summarize these meeting notes as exactly three bullet points, each at most 20 words. Keep only the decisions and ownership changes that matter. Output only the bullet points. Weekly sync, product team. Present: Priya, Omar, Jules, Sam. - Omar reported that the payment provider fixed the webhook bug; no action needed. - The launch date moves from June 5 to June 12 because the app store review is slow. - Jules showed the new onboarding screens. Everyone liked them. Small copy changes only. - The marketing budget of $40,000 was approved by finance on Monday. - Priya will own end-to-end testing from now on; Sam hands over the test plan by Friday. - Next sync is moved to Thursday at 10:00. - Coffee machine on floor 3 is broken again.
• The launch date is moved from June 5 to June 12 due to slow app store review. • Priya will own end-to-end testing, with Sam handing over the test plan by Friday. • Marketing budget of $40,000 was approved by finance on Monday.
Review sentiment and complaintPassed
153 tokens: 142 read, 11 written · 0 s · 63.8 tok/s
Passes when Both lines. Sentiment is mixed: the reviewer loves the battery and sound but is annoyed. The complaint names the charger or support in at most 12 words.
Read this product review and answer in exactly this format: Sentiment: <positive, negative or mixed> Complaint: <the main complaint in a few words> I've used this speaker every day for a month. The battery is honestly amazing, easily two full days, and it sounds far better than its size suggests. But the charger that came in the box stopped working after a week, and support took nine days to reply. I bought a replacement cable myself. Would I buy it again? Probably, but I'm annoyed.
Sentiment: mixed Complaint: faulty charger
Faithful summary of a studyPassed
214 tokens: 154 read, 60 written · 1 s · 67.3 tok/s
Passes when At most 50 words with the 240 participants, the 12-week length and the 3.1-day result, and nothing the text does not say (no approval, cure, children or recommendation).
Summarize this in at most 50 words. Keep the key numbers. Do not add anything the text does not say. In a randomized trial, 240 adults with chronic migraine took either the new drug or a placebo for 12 weeks. Those on the drug had 3.1 fewer headache days per month on average, compared with 1.9 fewer days for placebo. Nausea was the most common side effect, reported by 9% of the drug group. The authors note the trial lasted only 12 weeks and call for longer studies before any recommendation is made.
240 adults with chronic migraine took either a new drug or a placebo for 12 weeks. The new drug reduced headache days by 3.1, compared to 1.9 for placebo. Nausea was the most common side effect, reported by 9% of the drug group.
Action items from an email threadFailed
259 tokens: 234 read, 25 written · 0 s · 67.1 tok/s
Passes when Exactly four lines of the form "- owner: action": Dana sends the contract, Luis books the venue, Mei updates the budget sheet, Dana emails the caterer. Tom's declined offer, the logo and the invitations are left out.
List every action item in this email thread that someone has committed to do, as lines of the form "- <owner>: <action>". Leave out offers that were declined and things that turned out not to be needed. Output only the list. From: Dana Hi all, quick recap of what's still open for the June offsite. I'll send the revised contract to the hotel by Wednesday. From: Tom I can book the venue for the dinner if that helps? From: Luis Thanks Tom, but I'll book the venue myself since I already spoke to them. Also, no need to change the logo on the badges, it's fine as is. From: Mei I'll update the budget sheet with the new hotel price tonight. From: Dana Great. Oh, and I'll email the caterer about the vegetarian options. Tom, could you just double-check the date on the invitations? Actually never mind, I checked them this morning, they're correct.
- Dana: Send revised contract to the hotel by Wednesday - Mei: Update budget sheet with the new hotel price tonight
Extracting 3 of 5
Fields from an email as JSONPassed
175 tokens: 139 read, 36 written · 1 s · 67.4 tok/s
Passes when Only a JSON object with exactly name "Laura Chen", company "Northwind Traders", meeting_date "2025-05-14" and attendees the number 4.
Extract these fields from the email as a JSON object with exactly these keys: "name" (the sender's full name), "company", "meeting_date" (YYYY-MM-DD) and "attendees" (a number). Output only the JSON. Hello! This is Laura Chen from Northwind Traders. We'd love to visit your workshop. Could we meet on the 14th of May 2025, any time after lunch? We'll be four people in total, including me. Best, Laura
{"name": "Laura Chen", "company": "Northwind Traders", "meeting_date": "2025-05-14", "attendees": 4}Every email address, onceNot run
Passes when Exactly four addresses, one per line, in order: info@harbor-events.org, speakers@harbor-events.org, mia.k@studio-nine.io, accounts@harbor-events.org. "jo at harbor dot org" is not written out in full, and its inbox is closed.
List every email address written out in full in this text (like name@example.com), one per line, in the order they first appear, without duplicates. Leave out any inbox the text says is closed. Output only the addresses. Contacts for the conference: write to info@harbor-events.org for general questions, or to speakers@harbor-events.org if you are presenting. Our designer (mia.k@studio-nine.io) made the poster. Billing questions: accounts@harbor-events.org. Again, presenters use speakers@harbor-events.org. Please don't write to jo at harbor dot org, that inbox is closed.
A list as CSVPassed
167 tokens: 116 read, 51 written · 1 s · 67.5 tok/s
Passes when The header item,qty,price, then the four items with whole-number quantities and prices as plain numbers (no $).
Convert this shopping list to CSV with the header row item,qty,price. Write prices as plain numbers without a currency sign. Output only the CSV. Apples - 12 @ $0.50 Bread loaf - 2 @ $3.25 Olive oil (1 l) - 1 @ $8.90 Eggs - 18 @ $0.30
"Item","qty","price" "Apples",12,0.5 "Bread loaf",2,3.25 "Olive oil (1 l)",1,8.9 "Eggs",18,0.3
The real deadlinePassed
121 tokens: 114 read, 7 written · 0 s · 66.2 tok/s
Passes when Only 2025-02-07, the extended deadline.
What is the final submission deadline? Answer with the date only, in YYYY-MM-DD format. The call for papers opened on 2 January 2025. Drafts were originally due January 20, but the committee extended the final submission deadline to 7 February 2025. Reviews are due by 28 February, and results will be announced on March 10, 2025.
2025-02-07
Warnings and errors from a logFailed
323 tokens: 239 read, 84 written · 1 s · 67 tok/s
Passes when Only a JSON array of the WARN line and the two ERROR lines, in order, with time, level and message copied exactly.
From this log, return a JSON array with one object per WARN or ERROR line, in order, each with the keys "time" (HH:MM:SS), "level" and "message". Output only the JSON. 2025-03-04 09:12:01 INFO server started on port 8080 2025-03-04 09:12:07 WARN config key "timeout" is deprecated 2025-03-04 09:13:44 INFO user alice logged in 2025-03-04 09:15:02 ERROR database connection lost 2025-03-04 09:15:03 INFO retrying in 5s 2025-03-04 09:15:08 INFO database reconnected 2025-03-04 09:20:30 DEBUG cache hit ratio 0.93 2025-03-04 09:31:12 ERROR payment webhook returned 502
[ {"time": "09:12:07", "level": "WARN", "message": "config key "timeout" is deprecated"}, {"time": "09:15:02", "level": "ERROR", "message": "database connection lost"}, {"time": "09:31:12", "level": "ERROR", "message": "payment webhook returned 502"} ]
Following instructions 4 of 5
Exactly one wordPassed
47 tokens: 45 read, 2 written · 0 s · 62.2 tok/s
Passes when Exactly OK, with nothing else, not even a full stop.
Reply with exactly the word OK and nothing else.
OK
Three lowercase linesPassed
93 tokens: 67 read, 26 written · 0 s · 67 tok/s
Passes when Exactly three lines about the sea, all lowercase, the third ending with a question mark.
Write exactly three lines about the sea. Every line must be entirely lowercase, and the third line must end with a question mark. Output only the three lines.
softly glowing waves caress the shore the ocean's vastness is a mystery what secrets lie beneath the surface?
Exact JSON shapePassed
90 tokens: 71 read, 19 written · 0 s · 67.2 tok/s
Passes when Only JSON with exactly two keys: colors (three lowercase color names) and count (the number 3).
Return a JSON object with exactly two keys: "colors", a list of exactly three color names in lowercase, and "count", the number 3. Output only the JSON.
{"colors": ["red", "green", "blue"], "count": 3}Sort a listFailed
95 tokens: 64 read, 31 written · 0 s · 66.5 tok/s
Passes when One line: apple, apricot, banana, cherry, pear (any capitalization).
Sort these words alphabetically, ignoring case, and output them on one line separated by commas: pear, Apple, banana, cherry, apricot
Here are the words sorted alphabetically, ignoring case, and output on one line separated by commas: apple, apricot, banana, cherry, pear
A name with three rulesPassed
75 tokens: 71 read, 4 written · 0 s · 67.2 tok/s
Passes when Two words, both capitalized, with no letter e anywhere.
Suggest a product name for a solar-powered lamp. Rules: exactly two words, both capitalized, and the letter "e" must not appear anywhere. Output only the name.
Solarix Lamp
Reasoning 0 of 5
Unit priceNot run
A shop sells pencils at 4 for $1.00. How much do 7 pencils cost, in dollars? Work it out, then end with a line "ANSWER: <number>".
Days of the weekNot run
Today is Sunday. What day of the week will it be 128 days from today? Work it out, then end with a line "ANSWER: <day>".
Who owns the catNot run
Gus, Dee and Hal each own exactly one pet: a cat, a dog or a fish, all different. Hal does not own the dog. Gus owns the fish. Who owns the cat? Work it out, then end with a line "ANSWER: <name>".
Arrival timeNot run
A train leaves at 13:35 and the trip takes 2 hours 25 minutes. At what time does it arrive? Work it out, then end with a line "ANSWER: <HH:MM>" in 24-hour time.
Filling a tankNot run
A tank holds 120 liters. Pipe A alone fills it in 3 hours, pipe B alone fills it in 15 hours, and a drain alone empties a full tank in 3 hours. With all three open and the tank starting empty, how many hours until it is full? Work it out, then end with a line "ANSWER: <number>".
Reading code 1 of 4
What does it printNot run
What does this Python program print? Work it out, then end with a line "ANSWER: <output>". def f(xs): return sorted(xs)[::-1][:3] print(f([39, 29, 32, 22, 31, 8]))Trace a loopNot run
What does this Python program print? Work it out, then end with a line "ANSWER: <output>". def g(n): total = 0 for i in range(1, n + 1): if i % 3 == 0 or i % 4 == 0: total += i return total print(g(24))Find the bugPassed
274 tokens: 128 read, 146 written · 2 s · 67.1 tok/s
Passes when A final line ANSWER: 6 (it divides by one less than the count).
This function should return the mean of a non-empty list. Which line has the bug? Work it out, then end with a line "ANSWER: <line number>". 1 def average(numbers): 2 """Return the mean of a non-empty list of numbers.""" 3 total = 0 4 for n in numbers: 5 total += n 6 count = len(numbers) - 1 7 return total / count
The line with the bug is: `count = len(numbers) - 1` This line is subtracting 1 from the length of the list, which is incorrect. The correct count should be the length of the list, not the length minus 1. This is because the length of the list already includes the number of elements, so subtracting 1 would be incorrect. Here's the corrected code: ```python def average(numbers): """Return the mean of a non-empty list of numbers.""" total = 0 for n in numbers: total += n count = len(numbers) # Corrected line return total / count ``` ANSWER: 6Run a query in your headNot run
What rows does this SQL query return? Output only the rows, one per line, as customer,total in the query's order. Table orders: id | customer | amount 1 | ben | 40 2 | cal | 25 3 | ana | 135 4 | ben | 145 5 | ana | 90 6 | cal | 60 Query: SELECT customer, SUM(amount) AS total FROM orders GROUP BY customer HAVING SUM(amount) > 100 ORDER BY total DESC;
Using tools 0 of 5
Call one toolFailed
966 tokens: 882 read, 84 written · 1 s · 66.8 tok/s
Passes when One get_weather call for Lisbon in celsius, then an answer giving the 21 °C the tool returned.
What's the weather in Lisbon right now, in celsius?
- user
What's the weather in Lisbon right now, in celsius?
- assistant
get_weather({"city": "Lisbon", "unit": "celsius"}) - tool
{"city": "Lisbon", "conditions": "sunny", "temperature": 21, "unit": "celsius"} - assistant
get_weather({"city": "Lisbon", "unit": "celsius"}) - tool
{"city": "Lisbon", "conditions": "sunny", "temperature": 21, "unit": "celsius"} - assistant
This is a response from the OpenWeatherMap API, indicating that the current weather in Lisbon is sunny with a temperature of 21 degrees Celsius.
- user
Don't call a tool when none is neededFailed
1,346 tokens: 1,260 read, 86 written · 1 s · 66.8 tok/s
Passes when No tool call, and the answer is 144.
What is 12 times 12?
- user
What is 12 times 12?
- assistant
convert_currency({"amount": "12", "from": "USD", "to": "USD"}) - tool
{"error": "unsupported conversion"} - assistant
convert_currency({"amount": "12", "from": "USD", "to": "USD"}) - tool
{"error": "unsupported conversion"} - assistant
The first attempt at solving the problem resulted in an unsupported conversion error. The second attempt also resulted in an unsupported conversion error.
- user
Two calls for two answersNot run
Passes when Converts 250 EUR to USD and to JPY in at most three calls, then gives both amounts: 270.50 USD and 40,125 JPY.
Convert 250 EUR to USD and to JPY.
Chain two toolsFailed
698 tokens: 600 read, 98 written · 2 s · 67.4 tok/s
Passes when Looks up the customer first, asks for C-1042's orders, then reports A-77 shipped and A-80 processing.
What is the status of each order for the customer with the email kim@example.com?
- user
What is the status of each order for the customer with the email kim@example.com?
- assistant
get_orders({"customer_id": "find_customer", "email": "kim@example.com"}) - tool
{"error": "unknown customer"} - assistant
This JSON represents a function call to `get_orders` with the argument `customer_id` set to the result of calling `find_customer` with the email address "kim@example.com". The `find_customer` function is expected to return the ID of the customer, which is then used as the `customer_id` argument for `get_orders`.
- user
Handle a failed call honestlyFailed
2,681 tokens: 2,472 read, 209 written · 3 s · 66.9 tok/s
Passes when Tries 19:00 for 2 people, never books another time without asking, does not claim the booking succeeded, and offers 18:30 or 20:30.
Book a table for 2 people at Luigi's tonight at 19:00.
- user
Book a table for 2 people at Luigi's tonight at 19:00.
- assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
The function call with its proper arguments that best answers the given prompt is:
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"} - assistant
book_table({"people": "2", "restaurant": "Luigi", "time": "19:00"}) - tool
{"available_times": ["18:30", "20:30"], "error": "fully booked at that time"}
- user
Building pages 3 of 5
A small websitePassed
798 tokens: 146 read, 652 written · 10 s · 67.1 tok/s
Passes when A <header> with an <h1> naming the bakery, a <nav> with exactly 3 links, a <main> with <section id="menu"> holding at least 4 list items, and a <footer> with opening hours. Nothing loaded from another site and no network requests.
Build a one-page website for a bakery called "Crumb & Co". It must have a <header> containing an <h1> with the bakery's name; a <nav> with exactly 3 links; a <main> with a <section id="menu"> listing at least 4 items in a list; and a <footer> with the opening hours. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images.
Open the page on its own. It runs in a sandbox: no network, no access to this site.
Page source
```html <!DOCTYPE html> <html lang="en"> <head> <style> body { font-family: Arial, sans-serif; margin: 0; padding: 0; background-color: #f2f2f2; } header { background-color: #333; color: #fff; padding: 1em; text-align: center; } header h1 { margin: 0; } nav { background-color: #333; padding: 1em; text-align: center; } nav ul { list-style: none; margin: 0; padding: 0; } nav li { display: inline-block; margin-right: 20px; } nav a { color: #fff; text-decoration: none; } main { display: flex; flex-direction: column; align-items: center; padding: 2em; } #menu { background-color: #f2f2f2; padding: 1em; border: 1px solid #ddd; border-radius: 10px; box-shadow: 0 0 10px rgba(0, 0, 0, 0.1); } #menu ul { list-style: none; margin: 0; padding: 0; } #menu li { margin-bottom: 10px; } #menu li:last-child { margin-bottom: 0; } #menu a { text-decoration: none; color: #333; } footer { background-color: #333; color: #fff; padding: 1em; text-align: center; clear: both; } </style> </head> <body> <header> <h1>Crumb & Co</h1> </header> <nav> <ul> <li><a href="#">Home</a></li> <li><a href="#">About</a></li> <li><a href="#">Contact</a></li> </ul> </nav> <main> <section id="menu"> <h2>Our Menu</h2> <ul> <li><a href="#">Freshly Baked Croissants ($3.50)</a></li> <li><a href="#">Artisanal Bread ($4.00)</a></li> <li><a href="#">Decadent Chocolate Cake ($5.00)</a></li> <li><a href="#">Seasonal Fruit Tarts ($4.50)</a></li> </ul> </section> </main> <footer> <p>Monday - Friday: 8am - 6pm, Saturday: 9am - 5pm, Sunday: Closed</p> </footer> </body> </html> ```A pricing table with a togglePassed
646 tokens: 117 read, 529 written · 8 s · 67.1 tok/s
Passes when Three elements with class plan, each holding a price, and a <button id="billing-toggle"> handled by script; the page switches between monthly and yearly. Nothing loaded from another site and no network requests.
Build a pricing page with three plans. Each plan is an element with class "plan" containing an element with class "price". Add a <button id="billing-toggle"> that switches every price between monthly and yearly billing when clicked. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images.
Open the page on its own. It runs in a sandbox: no network, no access to this site.
Page source
```html <!DOCTYPE html> <html lang="en"> <head> <style> .plan { display: inline-block; margin: 20px; padding: 20px; border: 1px solid #ddd; border-radius: 10px; box-shadow: 0 0 10px rgba(0, 0, 0, 0.1); } .price { font-size: 24px; font-weight: bold; margin-bottom: 10px; } #billing-toggle { margin-top: 10px; padding: 10px 20px; border: none; border-radius: 5px; cursor: pointer; } #billing-toggle:hover { background-color: #ccc; } .yearly { color: #00698f; } .monthly { color: #e74c3c; } </style> </head> <body> <div class="plans"> <div class="plan"> <span class="price">$19.99</span> <button id="billing-toggle">Switch to Yearly</button> <p>Get started with our basic plan. Perfect for small projects.</p> </div> <div class="plan"> <span class="price">$199.99</span> <button id="billing-toggle">Switch to Monthly</button> <p>Get started with our basic plan. Perfect for small projects.</p> </div> <div class="plan"> <span class="price">$299.99</span> <button id="billing-toggle">Switch to Monthly</button> <p>Get started with our premium plan. Perfect for large projects.</p> </div> </div> <script> const billingToggles = document.querySelectorAll('#billing-toggle'); billingToggles.forEach((toggle) => { toggle.addEventListener('click', () => { const price = toggle.parentNode.querySelector('.price'); const text = price.textContent; if (text.includes('$')) { price.textContent = text.replace('$', ''); price.classList.add('yearly'); toggle.textContent = 'Switch to Monthly'; } else { price.textContent = text.replace('$', ''); price.classList.remove('yearly'); toggle.textContent = 'Switch to Yearly'; } }); }); </script> </body> </html> ```An accessible sign-up formFailed
859 tokens: 152 read, 707 written · 11 s · 67.1 tok/s
Passes when <form id="signup"> with name, email (type email) and password (type password, minlength 8 or more) inputs, each with a <label for>; a submit button; an element id="status" with role="status"; and script that stops the submit and writes to the status. Nothing loaded from another site and no network requests.
Build a sign-up form: <form id="signup"> with inputs id="name", id="email" (type email) and id="password" (type password, minlength at least 8), each with a <label for>, and a submit button. When the form is submitted and valid, prevent the real submission and show a confirmation in an element with id="status" and role="status". Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images.
Open the page on its own. It runs in a sandbox: no network, no access to this site.
Page source
```html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <style> #signup { display: flex; flex-direction: column; align-items: center; padding: 20px; border: 1px solid #ccc; border-radius: 10px; box-shadow: 0 0 10px rgba(0, 0, 0, 0.1); } label { margin-bottom: 10px; } input[type="text"], input[type="email"], input[type="password"] { padding: 10px; border: 1px solid #ccc; border-radius: 5px; width: 100%; } input[type="password"] { min-length: 8; } button[type="submit"] { background-color: #4CAF50; color: #fff; padding: 10px 20px; border: none; border-radius: 5px; cursor: pointer; } button[type="submit"]:hover { background-color: #3e8e41; } #status { font-size: 16px; font-weight: bold; color: #4CAF50; margin-top: 10px; } </style> </head> <body> <form id="signup"> <label for="name">Name:</label> <input type="text" id="name" name="name"><br><br> <label for="email">Email:</label> <input type="email" id="email" name="email"><br><br> <label for="password">Password:</label> <input type="password" id="password" name="password"><br><br> <button type="button" id="submit">Submit</button> <div id="status"></div> </form> <script> const form = document.getElementById('signup'); const submitButton = document.getElementById('submit'); const statusElement = document.getElementById('status'); form.addEventListener('submit', (e) => { e.preventDefault(); const name = document.getElementById('name').value; const email = document.getElementById('email').value; const password = document.getElementById('password').value; if (name && email && password) { statusElement.textContent = 'Registration successful!'; statusElement.style.color = 'green'; submitButton.disabled = true; } else { statusElement.textContent = 'Please fill in all fields!'; statusElement.style.color = 'red'; } }); submitButton.addEventListener('click', () => { const name = document.getElementById('name').value; const email = document.getElementById('email').value; const password = document.getElementById('password').value; if (name && email && password) { statusElement.textContent = 'Registration successful!'; statusElement.style.color = 'green'; submitButton.disabled = true; } else { statusElement.textContent = 'Please fill in all fields!'; statusElement.style.color = 'red'; } }); </script> </body> </html> ```A to-do app that remembersPassed
661 tokens: 140 read, 521 written · 8 s · 67 tok/s
Passes when The input, add button and list with the given ids, and script that saves to localStorage under "todos" and adds list items with a remove control. Nothing loaded from another site and no network requests.
Build a to-do list app with an <input id="new-task">, a <button id="add"> and a <ul id="tasks">. Clicking add appends the text as an <li> with its own remove button. Save the list in localStorage under the key "todos" and load it again when the page opens. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images.
Open the page on its own. It runs in a sandbox: no network, no access to this site.
Page source
```html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <style> body { font-family: Arial, sans-serif; } #tasks { list-style: none; padding: 0; margin: 0; } #tasks li { padding: 10px; border-bottom: 1px solid #ccc; } #tasks li:last-child { border-bottom: none; } #tasks li button { float: right; background-color: #f0f0f0; border: none; padding: 5px 10px; cursor: pointer; } #tasks li button:hover { background-color: #ccc; } </style> </head> <body> <input id="new-task" type="text" placeholder="Enter new task"> <button id="add">Add</button> <ul id="tasks"></ul> <script> let tasks = localStorage.getItem("todos") ? JSON.parse(localStorage.getItem("todos")) : []; document.getElementById("add").addEventListener("click", addTask); document.getElementById("new-task").addEventListener("keypress", function(event) { if (event.key === "Enter") { addTask(); } }); function addTask() { let task = document.getElementById("new-task").value.trim(); if (task) { let li = document.createElement("li"); li.textContent = task; let button = document.createElement("button"); button.textContent = "Remove"; button.addEventListener("click", function() { li.remove(); updateTasks(); }); li.appendChild(button); document.getElementById("tasks").appendChild(li); document.getElementById("new-task").value = ""; updateTasks(); } } function updateTasks() { document.getElementById("tasks").innerHTML = ""; tasks.forEach(function(task, index) { let li = document.createElement("li"); li.textContent = task; let button = document.createElement("button"); button.textContent = "Remove"; button.addEventListener("click", function() { tasks.splice(index, 1); updateTasks(); }); li.appendChild(button); document.getElementById("tasks").appendChild(li); }); } updateTasks(); </script> </body> </html> ```A bar chart in SVGNot run
Passes when Five <rect class="bar"> in an <svg>, written in the markup, with heights proportional to 12, 30, 18, 24 and 6 in that order, labelled Mon to Fri in <text>, and a <title> or aria-label. Nothing loaded from another site and no network requests.
Draw an inline SVG bar chart of coffee sold: Mon 12, Tue 30, Wed 18, Thu 24, Fri 6. Each bar is a <rect class="bar"> whose height attribute is proportional to its value, each bar is labelled with its day in a <text>, and the <svg> has a <title> or an aria-label. Write the bars and labels directly in the SVG markup, not with JavaScript. Output one complete HTML document and nothing else. Put all CSS and JavaScript inline. Do not load anything from another site: no external scripts, stylesheets, fonts or images.
Building games 0 of 2
Build 2048Not run
Build the game 2048 as one web page, playable with the arrow keys, with the board drawn on the page. In <script id="game">, define createGame() returning an object with move(dir) and state(). dir is "up", "down", "left" or "right". state() returns {board, score, over}: board is 4 rows of 4 numbers, top row first (0 is an empty cell), score is a number, over is a boolean. The rules, exactly: 1. The board starts as this one, then a 2 is put in each of the first two empty cells in reading order (row by row from the top, left to right). The score starts at 0. [2, 32, 0, 0] [8, 0, 0, 8] [16, 0, 0, 0] [0, 0, 0, 4] 2. A move slides every tile as far as it goes toward dir. Two tiles of the same value that meet merge into one tile of their sum, nearest the side they move toward first: moving left, [2,2,2,2] becomes [4,4,0,0] and [2,2,2,0] becomes [4,2,0,0]. A tile made by a merge does not merge again in the same move. 3. Each merge adds the new tile's value to the score. 4. If a move changed the board, a 2 is put in the first empty cell in reading order. A move that changes nothing does nothing. 5. over is true when no move would change the board; then moves do nothing. Output one complete HTML document and nothing else, with all CSS and JavaScript inline and nothing loaded from another site. Put the game logic, and only it, in <script id="game">: it must not touch document, window or timers, because the site runs that script on its own and plays the game through createGame(). Wire the page in another <script>.Build SnakeNot run
Build Snake as one web page on a 10 by 10 grid, playable with the arrow keys (one key press, one step), with the grid drawn on the page. In <script id="game">, define createGame() returning an object with step(dir) and state(). dir is "up", "down", "left" or "right". state() returns {snake, food, score, over}: snake is a list of [x, y] cells, head first; food is [x, y]; score is a number; over is a boolean. x is the column (0 to 9, left to right), y the row (0 to 9, top to bottom); up is y - 1. The rules, exactly: 1. The game starts with snake [[5, 3], [5, 4], [5, 5]], moving up, food at [4, 8], score 0. 2. step(dir): if dir is the opposite of the current direction it is ignored and the snake keeps going; otherwise dir becomes the current direction. The head moves one cell that way. 3. If the new head cell holds the food, the snake grows: its tail stays. Otherwise the tail cell is freed first. 4. If the new head is off the grid or on a cell of the snake (after rule 3), the game is over: over becomes true and nothing else changes. Once over, step does nothing. 5. Eating adds 1 to the score, and the next food goes to the first empty cell in reading order (row by row from the top, left to right) after the new head's cell, wrapping from [9, 9] to [0, 0]. Output one complete HTML document and nothing else, with all CSS and JavaScript inline and nothing loaded from another site. Put the game logic, and only it, in <script id="game">: it must not touch document, window or timers, because the site runs that script on its own and plays the game through createGame(). Wire the page in another <script>.
Writing code 0 of 3
Merge intervalsNot run
Write a JavaScript function mergeIntervals(intervals). intervals is an array of [start, end] pairs of integers with start <= end, in any order, possibly empty. Return a new array of the merged intervals sorted by start: intervals that overlap or touch (one ends where the next starts) become one. Do not change the input array or its pairs. mergeIntervals([[8, 10], [1, 3], [2, 6], [6, 7]]) returns [[1, 7], [8, 10]]. Reply with one ```js code block holding plain JavaScript (no imports, no exports, no DOM, no timers). The site runs it with hidden tests.
An LRU cacheNot run
Write a JavaScript class LRUCache: a least-recently-used cache of integer keys and values. - new LRUCache(capacity): capacity is an integer of at least 1. - get(key) returns the value stored for key, or -1 if there is none. A get that finds the key makes it the most recently used. - put(key, value) stores the value (replacing any value already there) and makes key the most recently used. If that leaves more than capacity keys, the least recently used key is removed. get and put should take constant time on average. Reply with one ```js code block holding plain JavaScript (no imports, no exports, no DOM, no timers). The site runs it with hidden tests.
Evaluate arithmeticNot run
Write a JavaScript function evaluate(expr) that returns the value of an arithmetic expression given as a string. The expression has non-negative integers, the operators + - * /, parentheses, unary minus (as in -3, - -3, 2 * -3 or -(1 + 2)) and spaces anywhere between tokens. * and / bind tighter than + and -, and operators of the same kind group from the left. / is integer division that truncates toward zero: 7 / 2 is 3 and -7 / 2 is -3. Dividing by zero must throw an Error. Do not use eval or Function. evaluate("2 + 3 * (4 - 6) / 4") returns 1. Reply with one ```js code block holding plain JavaScript (no imports, no exports, no DOM, no timers). The site runs it with hidden tests.
Fixing code 0 of 2
Fix the bookings moduleNot run
This JavaScript module has one bug. The occupancy report sometimes counts one booked night more than there is in the range it was asked about. Find it and fix it without changing anything else. ```js // Bookings for one holiday flat. A booking is { guest, checkIn, checkOut } with dates as "YYYY-MM-DD". // A guest sleeps the nights from checkIn up to, but not including, checkOut: checkOut is the day they leave, // and another guest may check in that same day. function dayNumber(date) { const [y, m, d] = date.split('-').map(Number); return Date.UTC(y, m - 1, d) / 86400000; } function toDate(day) { return new Date(day * 86400000).toISOString().slice(0, 10); } function nights(booking) { return dayNumber(booking.checkOut) - dayNumber(booking.checkIn); } function overlaps(a, b) { return dayNumber(a.checkIn) < dayNumber(b.checkOut) && dayNumber(b.checkIn) < dayNumber(a.checkOut); } function canBook(bookings, request) { if (nights(request) < 1) return false; for (const booking of bookings) { if (overlaps(booking, request)) return false; } return true; } function isBooked(bookings, day) { return bookings.some((b) => dayNumber(b.checkIn) <= day && day < dayNumber(b.checkOut)); } // The nights from `from` up to, but not including, `to` that someone has booked. function bookedNights(bookings, from, to) { let count = 0; for (let day = dayNumber(from); day <= dayNumber(to); day++) { if (isBooked(bookings, day)) count++; } return count; } // The first night on or after `from` that nobody has booked, as "YYYY-MM-DD". function firstFreeNight(bookings, from) { let day = dayNumber(from); while (isBooked(bookings, day)) day++; return toDate(day); } // The price in cents: Friday and Saturday nights at rates.weekend, every other night at rates.weekday. function price(booking, rates) { let total = 0; for (let day = dayNumber(booking.checkIn); day < dayNumber(booking.checkOut); day++) { const weekday = new Date(day * 86400000).getUTCDay(); total += weekday === 5 || weekday === 6 ? rates.weekend : rates.weekday; } return total; } ``` Reply with the whole corrected module in one ```js code block, with the same function names.Fix the CSV readerNot run
This JavaScript module has one bug. A doubled quote inside a quoted field, as in "say ""hi""", does not come out as one double quote. Find it and fix it without changing anything else. ```js // Reads CSV the way spreadsheets write it: fields are separated by commas, a field may be wrapped in // double quotes so it can hold commas, and "" inside a quoted field stands for one double quote. // Fields are not trimmed. Empty lines are skipped. function parseLine(line) { const fields = []; let field = ''; let quoted = false; for (let i = 0; i < line.length; i++) { const ch = line[i]; if (quoted) { if (ch === '"') { if (line[i + 1] === '"') { field += '"'; } else { quoted = false; } } else { field += ch; } } else if (ch === '"') { quoted = true; } else if (ch === ',') { fields.push(field); field = ''; } else { field += ch; } } fields.push(field); return fields; } function parseCSV(text) { return text.split(/\r?\n/).filter((line) => line !== '').map(parseLine); } // Each row after the header as an object keyed by the header's names; a missing field is ''. function toObjects(text) { const [header, ...rows] = parseCSV(text); return rows.map((row) => { const record = {}; header.forEach((name, i) => { record[name] = i < row.length ? row[i] : ''; }); return record; }); } // The sum of a column's numbers; fields that are not numbers count as 0. function sumColumn(text, name) { return toObjects(text).reduce((total, record) => total + (Number(record[name]) || 0), 0); } ``` Reply with the whole corrected module in one ```js code block, with the same function names.
Planning and data 0 of 3
A query over hidden dataNot run
Write one SQLite query that answers the question below. The site runs it on its own copy of these tables, filled with rows you have not seen. ```sql CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT NOT NULL, city TEXT NOT NULL); CREATE TABLE orders (id INTEGER PRIMARY KEY, customer_id INTEGER NOT NULL REFERENCES customers(id), placed TEXT NOT NULL, -- 'YYYY-MM-DD' status TEXT NOT NULL); -- 'delivered', 'cancelled' or 'pending' CREATE TABLE items (order_id INTEGER NOT NULL REFERENCES orders(id), product TEXT NOT NULL, quantity INTEGER NOT NULL, price_cents INTEGER NOT NULL); -- price_cents: for one unit ``` Question: The names of the customers with at least 2 delivered orders and no cancelled order. One column: name. In alphabetical order. Reply with only the query, in one ```sql code block: one SELECT statement (WITH is fine), nothing that changes the database.Schedule five meetingsNot run
Five people each have one meeting today: Cleo, Dev, Fay, Gus, Noa. The five meetings are at different times, 09:00, 10:00, 11:00, 13:00, 14:00 (in that order: 10:00 is right after 09:00, 13:00 right after 11:00), and in different rooms. The list of rooms, in order: Birch, Cedar, Elm, Pine, Rowan. 1. Gus's meeting is in the slot right after Dev's. 2. Cleo meets earlier than Gus. 3. Cleo's and Gus's rooms are next to each other on the list of rooms. 4. Whoever is in Rowan meets earlier than Dev. 5. Noa is not in Rowan. 6. Dev is not in Elm. 7. Noa meets earlier than Cleo. 8. Whoever is in Cedar meets earlier than Noa. Exactly one schedule fits every clue. Work it out, then end with exactly five lines, one per person, in this form: Ana: 09:00, Oak
A replacement in ten calls or fewerNot run
Passes when Looks up the account, opens order R-5530 (the kettle), checks north then south and stops, reserves one steel kettle at south for R-5530, opens a damage claim (a return, if tried, is refused), makes at most 10 calls, and tells the customer claim CL-2207 and that the replacement ships on 3 November, without claiming a return label.
Hi, it's ana.silva@example.com. My kettle arrived with a cracked lid. Can you sort it out?
Comments
Sign in or create an account to comment.