[App] ToshLLM — local LLMs on Intel + AMD GPU (Metal, AMD‑patched llama.cpp, open source)

[jsl]

On 7/9/2026 at 6:23 PM, engeldlgado said:

Hey @mitch_de and @jsl

 

Can you guys test the latest build, that includes some patchets for Vega/GCN cards (RX-500 series), please test with and withow both Flash Attention enable and disable it... and start first with small models, and send me the logs... It should work, "should" because I still have no way of testing it directly myself; for the moment I'm going in blind and you're guiding me. It's just that, because of the earthquake, the person who was going to send me the card can't send it yet.

Thanks for your new version (0.81.62) in which working in my H97M with RX-6700XT, X299 with RX-6600XT and Z690 with RX-570.

I can confirm that there's performance enhancement with enabled Flash Attention.

Edited July 12 by jsl

[mitch_de]

Results RX 560M MACBOOKPro with latest Version (running with / without new TOSH_W64_PREFILL=1 - I didn't used that together with W64WAVE argument together)

Best generation BENCH  much faster compared to versions of last week (1.3 > 9.5 4B / 1.2 > 2.6 8B).  Also Promt BENCH faster but "only2 15% FASTER.

 

[engeldlgado]

Hello everyone, it's a pleasure to announce that I've updated the main post with new information to keep you up-to-date with the changes... also, I wanted to let you know that the app now has a website. 

 

https://toshllm.com

 

Thanks to everyone who has been testing the app since I first released it.