I have been playing around with setting up my Strix Halo laptop so it can run LLMs locally. I found a guide here that was helpful to get a reliable connection (for now using Vulkan, though I’d like to see ROCm work later). Then using the Ollama Model Lab I can compare some local models.
Below are results of one comparison:
Test Prompt
Write a short funny poem about a bicycle tour.
| Metric | gemma3:12b | gpt-oss:20b | llama3:8b | mistral-nemo:12b | phi4:14b | qwen2.5:32b |
| Total Time (s) | 9.75 | 19.46 | 10.09 | 12.36 | 25.87 | 43.57 |
| Load Time (s) | 3.64 | 3.90 | 3.40 | 4.25 | 7.25 | 10.43 |
| Prompt Tokens | 19 | 77 | 20 | 13 | 20 | 39 |
| Prompt Eval Time (ms) | 118.82 | 210.12 | 68.73 | 79.96 | 197.49 | 732.23 |
| Response Tokens | 118 | 627 | 245 | 203 | 260 | 212 |
| Response Time (s) | 5.88 | 14.98 | 6.26 | 7.75 | 18.12 | 32.04 |
gemma3:12b
The tour began with sunshine bright,
A cycling dream, a glorious sight!
But hills appeared, a cruel surprise,
And sweat dripped down before our eyes.
The guide kept saying, "Feel the breeze!"
While we were panting, on our knees.
Our snacks were gone, our spirits low,
Just wishing we were home, you know.
A flat tire struck, a grumpy sigh,
Then someone yelled, "I want a pie!"
But still we pedaled, slow and weak,
A cycling saga, slightly bleak!
gpt-oss:20b
**The Great (and Slightly Tired) Bicycle Tour**
We hit the road on a bright‑sunny morn,
Dreaming of speed, of the world beyond.
The bike whispered, “Let’s take a spin—
And maybe a nap, where do we begin?”
We pedaled up hills with hearts so bold,
Only to find the path was a slippery mold.
The chain gave a sigh, the tire did sigh,
“Is it just me, or did I just say bye?”
At the summit we sighed, the view was sublime,
But the bike’s bell rang off‑key, like a jumbled rhyme.
We laughed at the trouble, the creaks, the gear’s groan,
And vowed that next time, we’d bring a bike‑own—(own a bike, not just own a throne).
llama3:8b
We set out on two wheels so fine,
A bike tour, a plan divine.
To ride and to roam with glee,
And maybe arrive somewhat free.
But first we must fuel up tight,
With coffee and pastry in sight.
Then off we pedal with gleeful cheer,
Only to face the dreaded hill fear!
My chain broke, oh what a test!
My friend's seat came loose, what a mess!
We wobbled and we wavered too,
But still we rode, our spirits anew.
We saw sights, we met some fame,
Like a cow that gave us quite a claim.
To fame, that is, for mooing loud,
And making us laugh with an "udderly" proud crowd!
At last we reached the final bend,
And thought our ride would soon amend.
But no, not yet! A sign did say,
"Caution: Potholes ahead, go away!"
So we bunny-hopped and rode with flair,
And finished strong, without a single care!
Now that's a bike tour, don't you agree?
Full of fun, full of glee!
We'll do it again, come what may,
For a bicycle ride is the best way!
mistral-nemo:12b
In lycra bright and helmets tight,
A dozen cyclists took to sight.
Their bikes were sleek, their spirits high,
But oh, their plans did soon go awry.
First went the seat-post, with a ping!
Right off it flew, like Superman's sting.
Next, a tire blew like a popped cork,
The rider landed in the mud, like a pork.
A water bottle launched like a rocket,
Straight into Mrs. Harper's pocket.
She shrieked, they laughed, oh what a din,
Their tour was turning out quite sinful.
Punctures plagued them, chains broke with snaps,
They pushed their bikes through calf-deep craps.
Swearing like sailors, sunburnt red,
Their once grand tour was now for dead.
But on they pedaled, through the muck,
Each challenge met with a cheerful "uck!"
For even when it all went wrong,
There's no fun quite like a bike song.
phi4:14b
On our two-wheeled voyage, oh so grand,
We pedaled through the countryside land.
With helmets askew and maps in hand,
Our bicycles danced across the sand.
The first hill appeared with quite a sneer,
And I thought, "This climb's going to smear my jeans!"
But up we went with giddy cheer,
Only to crash into a patch of mean weeds!
Lunchtime came on a grassy knoll,
Where sandwiches flew like airborne frisbees.
We laughed at birds who mocked our stroll,
While crumbs became snacks for the pesky fleas.
A river crossed us, wide and blue,
I paddled my bike with great aplomb!
My friend screamed loud, "You're doing it wrong!"
But we made it safe – though a bit more numb.
As dusk fell softly on our tour’s delight,
Our bellies full, hearts alight.
We sang silly songs under the starry night,
"Next time," we vowed, “let's pack some lighter bites!”
And so ended our bicycle caper spree,
With memories made and laughter free!
For what better way to travel wide,
Than on a bike with friends by your side?
qwen2.5:32b
In the land of wheels and gears, we rolled,
On bikes that creaked like an old boat’s hull.
We pedaled past the cows and moles,
With our helmets askew, quite droll.
Our tires were thin as a fiddle's neck,
And our water bottles filled to brim.
We laughed at every little pebble's heck,
For we were out on this silly bike trip.
The hills were tall, the paths so twisty,
Like a rollercoaster without any cars.
With each steep climb, we'd get quite misty,
But down the slopes, our spirits soared like stars.
At night we camped beneath a sky full of twinkle,
Our bikes lay still as if in deep slumber.
Though our legs were sore and we had to wobble and fumble,
We dreamed of another day to come and muddle.
So here's to the adventure on two wheels so light,
Where every bump was just an excuse to hold tight!
In this grand tour, we found joy each night.
