Llama 3

Smart choice using Groq's free tier with LLaMA 3 - the speed is hard to beat for real-time summarization. Using Mozilla Readability for extraction is solid too, it handles messy article HTML surprisingly well. Curious - how do you handle pages where Readability fails to extract cleanly, like single-page apps or paywalled content? Also, DOMPurify for sanitization is a nice touch for security. 36 customization combos is a lot of flexibility without overcomplicating the UX.

Llama3 still is worse than GPT4 for programming, but I expect this gap to be eliminated as newer versions of Llama keep coming. https://openlm.ai/coder-evalplus/ says that CodeQwen1.5-7B-Chat is the best open model for coding. https://huggingface.co/spaces/open-llm-leaderboard/open_llm_... lists davidkim205/Rhea-72b-v0.5 as the best open model overall. https://leaderboard.tabbyml.com/ lists DeepSeekCoder-6.7B as the best for coding. The takeaway is that Llama 3 looks a lot better than Llama 2, and I expect this trend to continue to 4 and 5, to the point where 5 could overtake GPT4.

Brave API (Bing is good as well). Here's a little gist (Elixir). It's pretty rudimentary so far and needs refining, but works alright enough (result at bottom): https://gist.github.com/cpursley/b4af2ff3b56c912f659bd5300e4... The most useful part is probably the prompt and usage of Phi 3 Mini 128K Instruct for web page summarization and Llama 3 for the final summary (of the summaries). I'm parsing out all but minimal content html but might even remove that to keep context length down.

I'd be careful sending raw audio to public APIs given the sensitive commercial info. A local pipeline with Whisper and Llama 3 is viable now and solves the privacy issue. It also keeps the long-term inference costs much lower.