Meta's 8B dense open-weight LLM from Llama 3.1, context extended to 128K tokens from Llama 3's 8K, released July 2024 under the Llama 3.1 Community License.
@hn_b88fcb
about 1 month ago
Not to downplay their accomplishment but Llama 3.1 8B is a terrible model. It's really outdated at this point. It's cool that they were able to accelerate a model with silicon, but it also feels wasteful since llama 8B is such a useless model?
@hn_825090
about 1 month ago
A lot of naysayers in the comments, but there are so many uses for non-frontier models. The proof of this is in the openrouter activity graph for llama 3.1: https://openrouter.ai/meta-llama/llama-3.1-8b-instruct/activ... 10b daily tokens growing at an average of 22% every week. There are plenty of times I look to groq for narrow domain responses - these smaller models are fantastic for that and there's often no need for something heavier. Getting the latency of reponses down means you can use LLM-assisted processing in a standard webpage load, not just for async processes. I'm really impressed by this, especially if this is its first showing.
@hn_97323c
about 2 months ago
Its llama 3.1 8b. So not a particular good model, mostly as a tech demo of the speed possible.
@hn_1a32b0
about 2 months ago
It’s llama 3.1 8B. No vision, not smart. It’s just a technical demo.
@hn_6ca66d
about 2 months ago
Different skills and context. Llama 3.1 8B has just 128k context length, so packing everything in it may be not a great idea. You may want one agent analyzing the requirements and designing architecture, one writing tests, another one writing implementation and the third one doing code review. With LLMs it’s also matters not just what you have in context, but also what is absent, so that model will not overthink it. EDIT: just in case, I define agent as inference unit with specific preloaded context, in this case, at this speed they don’t have to be async - they may run in sequence in multiple iterations.