2026-10-11 # Apple study finds LLMs and humans both struggle to grade machine translation Apple researchers report that large language models, tested against human judgment on the MQM and ESA quality-annotation schemes across varied language pairs and domains, show mixed reliability, per machinelearning.apple.com. ## What else the source says - Apple researchers tested LLMs against human judgment for MT quality evaluation - Two annotation schemes compared: MQM and ESA - Evaluation covered multiple language pairs, domains and granularity levels ## Why it matters Apple's finding suggests LLM translation grading still needs human oversight. Source. [machinelearning.apple.com](https://machinelearning.apple.com/research/llm-translation-quality-annotation) ## TRACE - First seen Oct 12, 2026, 3:31 PM PT - Published Oct 12, 2026, 4:34 PM PT, 1.1 h after first seen - Source machinelearning.apple.com (Company source) [SOTO home](https://signalsoftomorrow.com/)