Gemini 2.0 Flash

LLM can take audio as input, e.g. Gemini 2.0 Flash can take audio as input and is very fast and cheap (costs 25 audio tokens for a second, and prices $0.1 for 1 million input tokens). But I get your point that e.g. whisper is also very good and can run fast locally on edge devices. We also have recently phi4-multimodal LLM that is 5.6B params and is good enough to do inference locally on beefy smartphones. Problem is probably those smart speakers have really low CPU/GPU. Also LLM with audio input probably can be better re WER and take your mood/emotion into account when doing inference.