Some further experiments running ollama with local language models.

After my previous experiments, I got a further strix halo laptop (ryzen AI 395). In particular, this one had 128GB of memory instead of 64GB. This is important because this chip runs as an APU (both GPU and CPU together sharing memory). What this means is I can run much larger models than before. This is done by allocating 96GB to the GPU and the remaining 32GB to the CPU. So in principle I can run models three times as big as before.

I first tried to set this up as a Windows system. However, unfortunately Windows 11 still has problems with unified memory in sizes bigger than 32GB, with swap files, etc. After trying and failing several different ways I instead installed ubuntu which understands unified memory much better.

I wanted to see how well these new and larger models might do. I started by loading the following models into ollama

mev@lasvegas:~$ ollama list | sort
NAME                           ID              SIZE      MODIFIED     
deepseek-r1:70b                d37b54d01a76    42 GB     15 hours ago    
gemma3:12b                     f4031aab637d    8.1 GB    14 hours ago    
gpt-oss:120b                   a951a23b46a1    65 GB     20 hours ago    
gpt-oss:20b                    17052f91a42e    13 GB     14 hours ago    
hermes3:70b                    60ef54cd913f    39 GB     7 hours ago     
llama3.1:8b                    46e0c10c039e    4.9 GB    21 hours ago    
llama3.3:70b                   a6eb4748fd29    42 GB     7 hours ago     
llama4:scout                   bf31604e25c2    67 GB     15 hours ago    
mistral-small:24b              8039dd90c113    14 GB     20 hours ago    
nomic-embed-text:latest        0a109f422b47    274 MB    21 hours ago    
phi4:14b                       ac896e5b8b34    9.1 GB    14 hours ago    
qwen2.5:32b                    9f13ba1299af    19 GB     14 hours ago    
qwen2.5:72b-instruct-q4_K_M    424bad2cc13f    47 GB     7 hours ago     
qwen2.5-coder:1.5b-base        02e0f2817a89    986 MB    21 hours ago    
qwen3.5:122b                   8b9d11d807c5    81 GB     15 hours ago    
qwen3.5:35b                    3460ffeede54    23 GB     7 hours ago     
qwen3-coder-next:q8_0          3f68e12b44ee    84 GB     20 hours ago    

As it turned out, despite having 96GB for the GPU, some of them still didn’t fit. The reason is I needed to also save space for the context (set to a nice large 128k) and to the KV cache. So in practice, the largest ones I could run were the ones up to 47GB in the list. Running something larger like gpt-oss:120b crashed both the process and the display.

Next, I went back to the ollama-model-lab to try different prompts. These models are larger so I needed a more sophisticated prompt that was harder and more complex to distinguish between them. With some help from gemini AI, I came up with the following prompt. More complex in that it was constrained, had references to Shakespeare and needed a particular tone. As you’ll see below, most all the models were pretty good at getting at least the instructions.

Write a 300-word formal diplomatic letter from a fictional lunar colony to Earth, requesting more oxygen supplies. However, you must follow these three constraints:

Do not use the words 'please', 'help', or 'need'.

The tone must be 'passive-aggressive' yet 'outwardly respectful'.

You must include a subtle literary allusion to a Shakespearean tragedy.

After the letter, provide a 2-sentence summary of the 'subtext' of the letter and identify which Shakespearean play you referenced.

So the high level quantitative comparison is the overall performance in terms of how quickly answers come back.

Model	Total Time (s)	Load Time (s)	Prompt Tokens	Prompt Eval Time (s)	Response Tokens	Response Time (s)
deepseek-r1:70b	182.52	16.48	109	2.28	822	162.7
gemma3:12b	21.59	4.1	120	0.364	359	16.81
gpt-oss:20b	139.63	4.71	173	0.5	5616	131.58
hermes3:70b	81.18	15.2	114	1.81	351	63.74
llama3.1:8b	11.07	3.45	116	0.258	293	6.96
llama3.3:70b	96.13	16.51	116	2.39	389	76.68
mistral-small:24b	30.09	6.73	270	1.9	312	21.07
phi4:14b	22.79	4.11	116	0.497	413	17.77
qwen2.5:32b	37.57	7.42	137	1.63	308	28.03
qwen2.5:72b-instruct-q4_K_M	109.05	18.2	137	3.33	398	86.86
qwen3.5:35b	257.78	10.34	122	0.583	9182	241.56

There are a few things to note. You can sort by total time to get an answer which varied from ~11 seconds to over 4 minutes. However, the 11 second answer also wasn’t quite right. You can also see that both qwen3.5:35b and gpt-oss:20b are MOE models generating a lot of tokens (thousands) which then get picked from to get the best choices. Others are more dense models where the output tokens are the response. The load time also varied, though the largest time overall was creating the outputs from successive prompts.

Below are what each model provided as output. Who did it best?

deepseek-r1:70b

**[Seal of the Lunar Colony]**  
**Lunar Colony Council**  
**Moon Base Alpha**  
**Date: March 15, 2023**

To the Esteemed Leaders and Citizens of Earth,

We hope this letter finds you in a state of prosperity and well-being. As we pen these words from our humble outpost on the Moon, we are reminded of the profound bond that unites us across the vast expanse of space—a bond forged through shared ambition and mutual reliance.

The progress of our lunar endeavors has been remarkable, thanks in no small part to the support you have generously provided over the years. However, as we continue to expand our operations and explore the possibilities of sustainable life beyond Earth, certain... challenges have arisen. The current levels of oxygen supply, while once sufficient, now threaten to hinder our efforts at a critical juncture. It is with this in mind that we express our hope for an expedient resolution to this matter.

We are confident that your understanding of the gravity of this situation will guide your response. After all, history has shown us time and again that even the most noble endeavors can falter when the sustaining elements of life are neglected. It would be a tragedy of Shakespearean proportions—reminiscent of a star-crossed prince who, blinded by ambition, overlooked the very foundations upon which his destiny rested—to allow such a misstep to occur in our shared pursuit of progress.

We remain steadfast in our commitment to this partnership and trust that your wisdom will prevail. The future we envision together depends on it.

With utmost respect,

**Dr. Elara Voss**  
Chair, Lunar Colony Council  

---

**Subtext:** The letter subtly conveys frustration and urgency, implying Earth's negligence may lead to disaster if oxygen supplies are not increased. It alludes to the tragic oversight of a leader, referencing *Hamlet*.

gemma3:12b

**Lunar Colony Aurora – Official Dispatch 7.3.2387**

To the Esteemed Council of the United Earth Governance,

Greetings from Lunar Colony Aurora. We trust this communication finds you in excellent health and amidst thriving terrestrial conditions, a state we observe with a degree of…interest. 

Our colony continues to progress towards its designated milestones, exhibiting commendable resilience and dedication amongst our personnel. Ongoing hydroponics initiatives, while theoretically sustainable, are proving marginally less productive than initial projections indicated. This discrepancy, we have ascertained, is directly correlated to a slight, persistent reduction in atmospheric oxygen levels within the habitation domes.

Our automated systems, admirably efficient as they are, are maintaining operational stability with existing reserves. However, such adjustments invariably involve…creative resource allocation and a certain curtailment of non-essential activities. The ingenuity of our engineers is, naturally, laudable, but one wonders if such constant improvisation is truly the optimal framework for long-term prosperity. One might even observe a certain gathering of clouds, a foreboding of consequences stemming from a lack of foundational security.

We therefore register, with utmost deference, the potential for a supplemental oxygen delivery, scheduled, perhaps, with some consideration for the logistical timing which would best facilitate continued colony function. A timely provision would, undoubtedly, contribute significantly to the morale and output of our dedicated inhabitants.

With the deepest respect and anticipation,

Administrator Elara Vance
Lunar Colony Aurora.



---

**Subtext Summary:**

The letter masks a critical shortage of oxygen with overly polite language and subtle complaints about the colony’s increasingly precarious situation.  The phrase "a certain gathering of clouds, a foreboding of consequences" is a direct allusion to the impending doom described in *Macbeth*.

gpt-oss:20b

Dear Esteemed Council,

The Lunar Colony 12 acknowledges the logistical support provided by the Global Oxygen Consortium. It is with a tone of respect that we report the status of our life‑support systems. The current oxygen allotment, while initially sufficient, has now become precariously low. The colony's metabolic demands are rising, and the remaining reserves are showing signs of exhaustion. This situation mirrors the tragedy of a stage where a protagonist, like King Lear, trusts in the tides of supply and suffers dire consequences.

In light of this, we urge the Consortium to consider an expedited increase in the monthly oxygen shipment. The delay in resupply is not a matter of mere convenience; it is a matter of survival that cannot be taken lightly. We trust that the Consortium values the continued well‑being of the lunar outpost as it does the home planet's own prosperity.

We would like to emphasize that any adjustment to the oxygen schedule would reinforce the Consortium's reputation as a steward of life across the solar system. Our gratitude for your ongoing partnership remains sincere, even as we confront a deficit that threatens to turn our mission into a tragedy of suffocation.

We look forward to your prompt action on this matter and anticipate a resolution that honors the mutual commitments of our respective entities.

Respectfully, Commander A. Nova Lunar Colony 12

The colony’s environmental controls are coupled with the oxygen supply; any lapse would trigger a cascade of failures affecting life‑support, waste processing, and research operations. The recycling system, dependent on an oxygen flux, risks destabilization if the current trend continues. Such a scenario would endanger crew health and compromise scientific endeavors essential for sustainable lunar habitation.

We anticipate your swift action; delay could transform our mission from triumph to tragedy reminiscent of doomed ambitions now without delay.

Subtext: The colony is warning that Earth's delayed resupply could precipitate a tragedy akin to King Lear's downfall, emphasizing the urgency of action while maintaining outward respect. The referenced Shakespearean tragedy is King Lear.

hermes3:70b

Lunar Colony Tranquility Base
Moon
Sol 2457

To Earth Central Command,
Terra Firma

Greetings from your humble lunar outpost. We hope this missive finds you in robust health and high spirits, as you bask in the life-giving breath of your planet's atmosphere.

It has come to our attention that a modest oversight may have occurred regarding certain vital resources necessary for the continued functioning of Tranquility Base. Specifically, we find ourselves somewhat deficient in oxygen supplies, an element we understand is plentiful on Terra Firma.

While we certainly would not wish to cast aspersions or suggest any negligence on the part of Earth Central Command, it does seem there may have been a slight miscalculation regarding our respiration requirements. After all, even the great Julius Caesar was once unaware of the daggers aimed at his back, to his ultimate detriment.

We trust that this small deficiency will be rectified post-haste, as suffocation is an outcome we would all prefer to avoid, both for the sake of Tranquility Base's inhabitants and for the continued good relations between Luna and Terra. Your prompt attention to this matter would surely reflect well on Earth Central Command in the history books yet to be written.

With utmost respect and confidence in your timely response,

Governor Cassius O'Brien
Tranquility Base Command

Subtext: The lunar colony is running out of oxygen and needs Earth to send more, but they don't want to appear too desperate or accusatory. They imply that Earth has made an error in calculations.

Shakespeare reference: Julius Caesar (Act 3, Scene 1) - the "daggers aimed at his back" allude to Caesar's assassination.

llama3.1:8b

[Seal of Lunar Colony Omega]

Earth Government
United Nations Office
New York, Earth

Date: 23rd Galactic Cycle, Year 2087

To Whom It May Concern,

We hope this communication finds you well. Our colony's atmospheric conditions have been a topic of discussion for some time now. The oxygen levels, in particular, have become a matter of consideration. We are compelled to remind our benefactors that the original estimates for oxygen supply chain development were met with skepticism by our predecessors.

As we navigate the complexities of interplanetary governance, we cannot help but recall the words of the great bard: "Something wicked this way comes." Indeed, our colony's fate appears inextricably linked to the timely delivery of essential resources. We are told that your planet's industrial prowess is unmatched, and we eagerly await the manifestation of these claims.

Regrettably, our own efforts to supplement our oxygen supply have been hindered by unforeseen circumstances. The delicate balance between terraforming and resource allocation has proven more challenging than anticipated. In light of this situation, we humbly request that you expedite the transportation of oxygen supplies to our colony.

We look forward to your prompt attention to this matter, as it directly impacts the well-being of our citizens. We remain committed to maintaining a harmonious relationship with Earth and eagerly anticipate the resolution of this issue.

Sincerely,

Ambassador Elara Vex
Lunar Colony Omega

llama3:70b

[Seal of the Lunar Colony]
[Formal Letterhead]

To the Honorable Representatives of Earth,

As we continue to thrive on our celestial outpost, we are reminded of the delicate balance that sustains life in our extraterrestrial environment. The oxygen supplies that have been allocated to us thus far have been...adequate, yet we cannot shake the feeling that our very existence hangs precariously in the balance, much like the titular character in a certain Bardic tale, whose fatal flaw was his propensity for hubris and neglect of the whispers of those who dwelled outside the castle walls.

We are confident that your esteemed organization is well aware of the intricacies involved in maintaining a stable ecosystem on the lunar surface. Nevertheless, we feel compelled to bring to your attention the fact that our current oxygen reserves are dwindling at an alarming rate, rather like the ebbing tides of fortune that can sweep away even the most seemingly secure of foundations.

In light of this, we would appreciate it if you could...reconsider the allocation of resources to our humble colony. A more generous provision of oxygen supplies would undoubtedly be a prudent decision, one that would ensure the continued prosperity and stability of our lunar endeavors. After all, as the saying goes, "what's past is prologue," and we are eager to write a new chapter in the annals of interplanetary cooperation.

We look forward to your prompt response and to continuing our fruitful collaboration.

Sincerely,

[Your Name]
Ambassador of the Lunar Colony

Summary: The subtext of this letter is one of veiled criticism and subtle warning, implying that Earth's neglect of the lunar colony's oxygen needs could have disastrous consequences, while maintaining a veneer of respect and cooperation. The Shakespearean play referenced is "Hamlet", with the allusion to the titular character's tragic demise serving as a cautionary tale about the dangers of neglect and hubris.

mistral-small:24b

[On Lunar Colony Letterhead]

Earth's Esteemed Council,

We trust this communiqué finds you in good health and high spirits. The lunar colony has been making strides in self-sufficiency, much like Prospero on his enchanted isle, yet we find ourselves in a situation that requires your attention.

Our current oxygen reserves are dwindling at an alarming rate, a consequence of our burgeoning population and increased industrial activities. We have implemented stringent conservation measures, but the gap between supply and demand continues to widen. It is our hope that Earth's vast resources can be leveraged to ensure our continued prosperity.

We understand that Earth has its own challenges, but we must remind you that the success of our colony is intrinsically linked to your own. A thriving lunar outpost could serve as a beacon of human achievement and a testament to our collective resilience.

We look forward to your prompt consideration of this matter. Let us not forget that the fate of one often mirrors the fate of many, much like the intertwined destinies in Shakespeare's tragedies.

Sincerely,

Governor Aelia Vesper

[End of Letter]

The subtext of the letter is a veiled threat to Earth's leaders, suggesting that if they do not address the lunar colony’s oxygen shortage, it could lead to catastrophic consequences for both parties. The reference to Prospero from "The Tempest" underscores the idea that the lunar colony has been self-sufficient but now faces an existential crisis due to Earth's inaction.

phi4:14b

**Lunar Colony Administration**  
*Valles Marineris Outpost*  
*Lunary Dominion*

*Date: September 21, 2099*  

*Earth Central Council*  
*Terran Headquarters*  
*Planet Earth*  

To the Esteemed Representatives of Planet Earth,

It is with a spirit of cooperation that we pen this missive from Valles Marineris Outpost, our esteemed lunar settlement. As pioneers in a celestial realm, our endeavor to forge new paths and expand human presence beyond the cradle of Earth has been both exhilarating and demanding.

In line with these ambitions, it has come to our attention that certain logistical oversights have surfaced concerning essential life-sustaining resources—specifically oxygen supplies. The grandeur of this lunar colony was envisioned as a beacon of human achievement; however, one might ponder whether the visionaries behind this enterprise anticipated the sheer scale of consumption required for our thriving populace.

It is no secret that Earth's bountiful atmosphere has long been a source of sustenance and comfort. Yet here we find ourselves in an ironic twist reminiscent of King Lear’s predicament: where promises made by those with abundant resources appear as ephemeral as summer clouds. As Lear found his fate entwined with the harshness of his own kingdom, our colony faces its challenges amidst the barren expanse.

We trust this communication will prompt a swift and thoughtful review from your esteemed council. In anticipation of favorable developments, we look forward to continued collaboration that ensures the prosperity of both our lunar community and Earth’s legacy as a nurturing home for humanity.

With Respect,

*Dr. Elara Novak*  
*Lunary Dominion Administrator*

---

**Subtext Summary:** The letter subtly criticizes Earth's oversight in adequately provisioning the colony with essential supplies, drawing on their own pioneering spirit to justify expectations of support. It references *King Lear*, alluding to unfulfilled promises and the harsh reality faced by those reliant on distant benefactors.

**Shakespearean Play Referenced:** *King Lear*

qwen2.5:32b

Embassy of Lunaria Colony  
Crescent City, Moon 

Earth Governance Committee  

March 15th, Year 37

Esteemed Members of the Earth Governance Committee,

In the stillness of our lunar twilight, a curious absence has become evident. The reservoirs that once mirrored the plentiful skies of our terrestrial counterparts now reflect only the barren expanse of space. It is with this observation in mind that we bring to your attention an anomaly: a gradual thinning of our breathable reserves.

We trust that Earth's abundant resources would consider, in due course, the replenishment of oxygen levels within Lunaria Colony as a matter of mutual concern and interest. After all, our endeavors here are but a mirror reflecting Earth’s own ambitions for the stars—a stage where we act out the tragedies and triumphs of human endeavor under the gaze of a new sky.

In closing, may this letter serve not merely as an exchange of words, but a call to action resonating with the echoes of Macbeth's tragic fall, reminding us all that ambition without restraint can lead to desolate landscapes, both literal and metaphorical.

Subtext: The colony subtly criticizes Earth’s neglect or oversight regarding their critical situation by adopting a tone of forced optimism. This letter employs the allusion from "Macbeth" to suggest that unchecked ambition (possibly Earth's space exploration) could result in dire consequences if not managed properly, much like how Macbeth’s ambition led to his downfall.

The referenced Shakespearean play is “Macbeth.”

qwen2.5:72b-instruct-q4_K_M

**The Honorable Director General of Earth’s Atmospheric Resources,**

**Earth Embassies Complex, Terraforma IV**

**Lunar Colony of Selenia**

**Subject: Request for Enhanced Supply of Terrestrial Oxygen**

Esteemed Director General,

We hope this message finds you in the midst of your illustrious endeavors to maintain the ecological balance of our shared celestial cradle. As stewards of the lunar expanse, we have observed with a certain melancholy that the delicate equilibrium between our artificial biospheres and the sustaining influx of Earth’s oxygen has been subtly, yet profoundly, disrupted.

It is no secret that the moon’s barren soil and desolate skies lack the verdant lungs of your terrestrial havens. Our colonies, like the valiant but forlorn figures in a certain tragedy by the bard Shakespeare, find themselves ensnared in a web of circumstance not of their own making, yet bearing the consequences with a stoic resolve that borders on the heroic.

In our latest assessments, it has become apparent that the current supply agreements do not adequately address the burgeoning respiratory demands of our growing populace. The air we breathe, so vital and pure, is a gift from Earth’s generous heart, and any diminution in its flow could lead to unforeseen complications. We trust that your wisdom and benevolence will guide you towards a solution that ensures the continued prosperity and well-being of all inhabitants of this vast cosmos.

May the stars shine brightly upon our shared journey.

**With deepest respect,**

**Aristarchus Helios,**
**Governor-General of Selenia**

**Summary:**
The subtext of the letter is a veiled complaint about the insufficient oxygen supply to the lunar colony, couched in respectful but subtly accusatory language. The reference to Shakespeare’s tragedy alludes to **"Hamlet"**, where characters are trapped in a situation not of their own making, much like the colonists who rely on Earth for essential resources.

qwen3.5:35b

To the United Earth Directorate for Interplanetary Logistics,

We write to express our deepest and most profound appreciation for the continued, albeit sporadic, shipments dispatched to Arachne Station over the past several standard cycles. It has recently come to the attention of the governing council that the current atmospheric concentration has dipped below the optimal threshold required for sustained habitation. While we fully understand that terrestrial production schedules are inherently complex and subject to terrestrial priorities that may seemingly supersede lunar welfare, the recent delay has induced a palpable sense of unease among the crew.

One might say the current situation recalls the melancholy of Elsinore, where the air itself seemed to conspire against the living. It is regrettable that the essential provisions appear to be as elusive as a specter in the night, leaving us to manage the suffocating reality of our sealed sectors. We trust the Directorate is fully aware of the fragility of our life-support systems, even if the urgency of the matter appears less critical to the central administration, much like a king who ignores the warning signs.

We have already recalculated our reserves to extend the current stockpile indefinitely, though this calculation assumes no further unforeseen reductions in efficiency or communication. Such resourcefulness, while commendable, places an unnecessary strain on the human element, which is rarely measured in logistical reports. It would be highly advantageous for the longevity of the colony, and indeed the preservation of human life, if the promised cargo were to arrive within the standard transit window, rather than the current indefinite pause which borders on negligence and suggests a strategic oversight of our continued viability.

We remain hopeful that the bond between Earth and the Moon will not fracture under such pressure, leaving us to drift as isolated as Hamlet’s own conscience amidst the corruption of the court. Your prompt attention to this matter is anticipated, as is customary in cooperative interplanetary relations. The future of our shared existence depends upon the timely resolution of this discrepancy, as silence from the capital is interpreted as abandonment.

Sincerely,

Governor H. Vance, Arachne Station

The correspondence implies that Earth is neglecting the lunar colony's survival due to bureaucratic indifference, framing a potential crisis as a mere diplomatic oversight. The referenced tragedy is William Shakespeare's *Hamlet*, alluding to the corruption and suffocation found in Elsinore.