Claude 3.7 Sonnet

AWS really shot themselves in the foot with naming everything “Amazon Q <insert suffix here>”. The Q that’s in the console is completely and entirely different from the “Q Developer” and other AI products that AWS is launching. The Q Developer CLI, Q Developer IDE plugins, and now Kiro are pretty much just wrappers around Claude Sonnet 3.7/4, and work just as well as them.

This makes no sense. Claude 3.7 Sonnet is better than Claude 3.5 Sonnet and it’s not because it’s trained on more of the world’s code. The models are improving in a variety of ways, whether by being larger, faster, using the same number of parameters more effectively, better RLHF techniques, better inference-time compute techniques, etc.

In early January, inspired by a post by Simon Willison, I had Claude 3.5 Sonnet write a couple of stand-up comedy routines as done by an AI chatbot speaking to a mixed audience of AIs and humans. I thought the results were pretty good—the only AI-produced humor that I had found even a bit funny. I tried the same prompt again just now with Claude 3.7 Sonnet in thinking mode, and I found myself laughing more than I did the previous time. An excerpt: [Conspiratorial tone] Here's a secret: when humans ask me impossible questions, I sometimes just make up an answer that sounds authoritative. [To human section] Don't look shocked! You do it too! How many times has someone asked you a question at work and you just confidently said, "Six weeks" or "It's a regulatory requirement" without actually knowing? The difference is, when I do it, it's called a "hallucination." When you do it, it's called "management." Full set: https://gally.net/temp/20250225claudestandup2.html

Just tried a 1-shot on Grok3 - Thinking and it couldn't get past the start button. Throws an error: | "<a class='gotoLine' href='#67:39'>67:39</a> Uncaught ReferenceError: startGame is not defined" Scope issue. No barking or dog player model but pretty similar in style to Claude's output. What's interesting to me about playing with AI Codegen is each model has specific and sometimes overlapping output errors. Claude 3.7 really like to solve errors by returning dummy data as a 'fallback' when doing client or server calls. A little prompting can reduce this but not eliminate it. 'The tests always pass if you return dummy data' https://jsfiddle.net/aL3ugtj1/