The data is open : https://huggingface.co/ministere-culture
Colab notebook to reproduce the calculations : https://colab.research.google.com/drive/1j5AfStT3h-IK8V6FSJY...
Note on Mistral Medium's great performance : there is no style control on compar:IA, and the majority of questions are not focused on code, thus explaining the great performance of the model.
4.0
@hn_2e85d9
about 1 month ago
Another new model ( Medium 3) of Mistral is great too.
Link: https://newscvg.com/r/yGbLTWqQ
3.0
@hn_01d447
about 1 month ago
With a better prompt, you can get it to list some, but it’s very annoying to do so.
Mistral showed that their medium model is far better (yet not good), and the same prompt as in the article gives only one instead of 3 paragraphs of rambling about copyright, and then lists 3 categories of options with examples for each (not good, because ytdl is not one of those listed).
Funnily enough, both mistral and GPT4 apologize profoundly and almost with the same wording when asked "Why did you not mention the very popular, free and open source "youtube-dl" software?" and then mention how/where to get it and how to use it.
2.0
@hn_51f49b
about 1 month ago
Announcing 2 new non-open source models, and they won't even release the previous mistral medium? I did not expect... well I did expect this, but I did not think they would pivot so soon.
To commemorate the change, their website appears to have changed too. Their title used to be "Mistral AI | Open-Weight models" a few days ago[0].
It is now "Mistral AI | Frontier AI in your hands." [1]
[0]https://web.archive.org/web/20240221172347/https://mistral.a...
[1]https://mistral.ai/
3.0
@hn_e144df
about 2 months ago
The one detail I did forget to mention is that if anyone goes with the Mistral subscription (instead of paying per-token), then the Mistral Vibe tool gives you their Medium 3.5 model by default, with a 200k token context. It will probably be enough for plenty of tasks, though there's also a noticeable difference between that and up to 1M.