AI models can perform complex tasks such as writing essays and generating art – but they have yet to master some skills that humans carry out with ease, researchers say.
A team from the University of Edinburgh has shown that state-of-the-art AI models are unable to reliably interpret clock-hand positions or correctly answer questions about dates on calendars.
Unlike simply recognizing shapes, understanding analog clocks and calendars requires a combination of spatial awareness, context and basic math – something that remains challenging for AI, the team says.
(Photo by Miguel Á. Padriñán via Pexels)
By Talker
Overcoming this could enable AI systems to power time-sensitive applications like scheduling assistants, autonomous robots and tools for people with visual impairments, researchers say.
The team tested if AI systems that process text and images – known as multimodal large language models (MLLMs) – can answer time-related questions by looking at a picture of a clock or a calendar.
Researchers tested various clock designs, including some with Roman numerals, with and without second hands, and different coloured dials.
Their findings show that AI systems, at best, got clock-hand positions right less than a quarter of the time.
Mistakes were more common when clocks had Roman numerals or stylized clock hands.
Jem Sahagun
By Talker
AI systems also did not perform any better when the second hand was removed, suggesting there are deep-seated issues with hand detection and angle interpretation, the team says.
The researchers asked AI models to answer a range of calendar-based questions, such as identifying holidays and working out past and future dates.
The team found that even the best-performing AI model got date calculations wrong one-fifth of the time.
“Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people.
“These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies.”
Aryo Gema, also of the School of Informatics, said: “AI research today often emphasizes complex reasoning tasks, but ironically, many systems still struggle when it comes to simpler, everyday tasks.
“Our findings suggest it’s high time we addressed these fundamental gaps.
Otherwise, integrating AI into real-world, time-sensitive applications might remain stuck at the eleventh hour.”
