Thought it would be fun to see how accurate AI was at analyzing this, given its volatility. I wanted to see if the graph would "converge" like the ones showing the chance of a sports team winning over the course of the game - they usually bounce around before trending one direction at the end of the game.
Every morning a small Lambda checks Google News (war/peace/ceasefire/Hormuz queries), gives the headlines to Claude Haiku, and gets back a probability (0-100%) plus ~500 words (Claude is so generous, need to constrain that) on what moved and why. It's a static site on S3 + CloudFront, and we keep a daily history of it
Anyway the graph was all over the place. So to help gauge its accuracy yesterday I added oil and gas prices overlaid. Turns out they correlate pretty closely *most* of the time. If a deal is reached I'll be interested to see if the graph converges up to that point (showing AI was somewhat smart at predicting), or if it just jumps to 100 in which case it was less... predictive.
Take a look, happy to hear feedback or suggestions.
It seems to have sped up a lot, otherwise I would have recommended something like Claude Haiku which is lightweight, cheap, faster and capable of this kind of work.
Quality goes up a lot with images though. I was going to complain that it mostly recommends white people fashion, but if you upload a photo of a hijabi in a long dress, it does quite well. It finds something with matching colors, and doesn't do the expected faux pas of a hijab and miniskirt.
You might be on to something; it just needs some more polish.
Don't.
The first 500 or so tokens are raw thinking output, then the summarizer kicks in for longer thinking traces. Sometimes longer thinking traces leak through, or the summarizer model (i.e. Claude Haiku) refuses to summarize them and includes a direct quote of the passage which it won't summarize. Summarizer prompt can be viewed [here](https://xcancel.com/lilyofashwood/status/2027812323910353105...), among other places.
4.0
@hn_3cbf26
about 1 month ago
I switched from structured outputs on OpenAI apis to unstructured on Claude (haiku 4.5) and haven't had any issues (yet). But guarantees are always nice.
4.5
@hn_323ab0
about 1 month ago
Mehul here. One thing that surprised me while building this was how creative the defender agent became. It runs Claude Haiku on a timer and can choose from prebuilt disruptions like fake “Session Expired” popups, or generate custom JavaScript injections based on what the attacker is doing, like inserting fake “Search disabled” buttons. Digging through the traces and seeing the before/after screenshots of what the defender agent came up with was pretty funny, and kind of mind-blowing.
