Qwen 2.5

Qwen 2.5 is so bad it’s good. Some really insane results if you watch it for a while. Almost like it’s taking the piss.

Those little 4B and 8B models will run on almost anything. They're really fun to try out but severely limited in comparison to the larger ones - classifying headlines to categories should work well but I wouldn't trust them to refactor code! If you have 8GB of RAM you can even try running them directly in Chrome via WebAssembly. Here's a demo running a model that's less than 1GB to load, entirely in your browser (and it worked for me in mobile safari just now): https://huggingface.co/spaces/cfahlgren1/Qwen-2.5-WebLLM

I personally have used Qwen2.5-coder:14B for "live, talking rubber duck" sorts of things. "I am learning Elixir, can you explain this code to me?" (And then I can also ask follow-up questions.) "Here is a bunch of logs. Given that the symptom is that the system fails to process a message, what log messages jump out as suspicious for dropping a message?" "Here is the code I want to test. <code> Here are the existing tests. <test code> What is one additional test you would add?" "I am learning Elixir. Here is some code that fails to compile, here is the error message, can you walk me through what I did wrong?" I haven't gotten much value out of "review this code", but maybe I'll have to try prompting for "persona: brief rude senior" as mentioned elsewhere.

Make it for Qwen 2.5 and I'd buy it. You don't actually need "frontier models" for Real Work (c). (Summarization, classification and the rest of the usual NLP suspects.)