Compare runs
Pick two or more runs to compare. Qwen2-VL-7B-Instruct on an unnamed computer is the first; add another below.
| Run | Qwen2-VL-7B-InstructNo computer named · Ollama v262Take out Qwen2-VL-7B-Instruct |
|---|---|
| Score | —No score |
| Tasks passed | 8 of 44 |
| Points | 8 of 64 |
| Speed | 59.6 tok/s |
| Tokens | 4,971 |
| Summarizing | 0/5 |
| One-sentence summary | Failed59.1 tok/s |
| Meeting notes in three bullets | Failed59.6 tok/s |
| Review sentiment and complaint | Failed59.6 tok/s |
| Faithful summary of a study | Failed59.2 tok/s |
| Action items from an email thread | Failed59.8 tok/s |
| Extracting | 2/5 |
| Fields from an email as JSON | Passed59.9 tok/s |
| Every email address, once | Not run |
| A list as CSV | Failed60 tok/s |
| The real deadline | Passed60 tok/s |
| Warnings and errors from a log | Failed59.5 tok/s |
| Following instructions | 2/5 |
| Exactly one word | Passed59.6 tok/s |
| Three lowercase lines | Failed58.5 tok/s |
| Exact JSON shape | Passed59.6 tok/s |
| Sort a list | Failed59.5 tok/s |
| A name with three rules | Failed59 tok/s |
| Reasoning | 0/5 |
| Unit price | Not run |
| Days of the week | Not run |
| Who owns the cat | Not run |
| Arrival time | Not run |
| Filling a tank | Not run |
| Reading code | 0/4 |
| What does it print | Not run |
| Trace a loop | Not run |
| Find the bug | Failed59.9 tok/s |
| Run a query in your head | Not run |
| Using tools | 1/5 |
| Call one tool | Failed59.9 tok/s |
| Don't call a tool when none is needed | Passed59.8 tok/s |
| Two calls for two answers | Failed59.6 tok/s |
| Chain two tools | Failed60 tok/s |
| Handle a failed call honestly | Failed59.7 tok/s |
| Building pages | 3/5 |
| A small website | Passed59.5 tok/s |
| A pricing table with a toggle | Failed59.6 tok/s |
| An accessible sign-up form | Passed59.5 tok/s |
| A to-do app that remembers | Passed59.7 tok/s |
| A bar chart in SVG | Not run |
| Building games | 0/2 |
| Build 2048 | Not run |
| Build Snake | Not run |
| Writing code | 0/3 |
| Merge intervals | Not run |
| An LRU cache | Not run |
| Evaluate arithmetic | Not run |
| Fixing code | 0/2 |
| Fix the bookings module | Not run |
| Fix the CSV reader | Not run |
| Planning and data | 0/3 |
| A query over hidden data | Not run |
| Schedule five meetings | Not run |
| A replacement in ten calls or fewer | Not run |
Scrolls sideways to show every run.
A check marks the best of each row. tok/s is tokens a second: how fast the model writes. How the score works.