Mistral's multimodal OCR API for document understanding; parses text, tables, equations, and images from PDFs and scans into structured markdown output.
Specific experiences are worth more than sweeping claims.
@hn_599a71
4 months ago
Public discussion, not a rating submitted on Rate My AI.
High accuracy is the goal! But the multimodal approach introduces some complexities that can impact real-world performance. We break it down in our review: https://undatas.io/blog/posts/in-depth-review-of-mistral-ocr... As for use cases, it really depends on how well it handles edge cases…
@hn_c966d7
5 months ago
Public discussion, not a rating submitted on Rate My AI.
It's disappointing to see that the benchmark results are so opaque. I hope we see reproducible results soon, and hopefully from Mistral themselves. 1. We don't know what the evaluation setup is. It's very possible that the ranking would be different with a bit of prompt engineering. 2. We don't know how large each dataset is (or even how the metrics are calculated/aggregated). The metrics are all reported as XY.ZW%, but it's very possible that the .ZW% -- or even Y.ZW% -- is just noise.[1] 3. We don't know how the datasets were mined or filtered. Mistral could have (even accidentally!) filtered out particularly data points that their model struggled with. (E.g., imagine good-meaning engineer testing a document with Mistral OCR first, finding it doesn't work, and deducing that it's probably bad data and removing it.) [1] https://medium.com/towards-data-science/digit-significance-i
@hn_7c0820
5 months ago
Public discussion, not a rating submitted on Rate My AI.
I'm surprised they didn't benchmark it against Pixtral. They test it against a bunch of different Multimodal LLMs, so why not their own? I don't really see the purpose of the OCR form factor, when you have multimodal LLMs. Unless it's significantly cheaper.