On 18 June, reports emerged that fast food chain McDonald’s will cease operating its AI automated voice ordering service in more than 100 of its drive-through restaurants.
The company’s AI automated voice ordering service, launched in 2021 in partnership with IBM, was said to be only processing around 85% of orders correctly and required human intervention for around one-in-five orders.
Go deeper with GlobalData
Perhaps more concerning were the GenAI enabled voice mishaps documented on video by customers that subsequently went viral online, causing considerable reputational damage to the company.
According to GlobalData thematic intelligence research director Josep Bori, excitement around GenAI use cases is well justified. “However, the technology still has some limitations, such as hallucinations, poor accuracy and a lack of an internal representation of the world, which would allow it to separate truth from falsehood. Therefore, it should be deployed carefully,” he said.
According to Bori, an automated ordering system has a low tolerance for risk because customers’ money and product shipments are involved. “Other business processes would have been more suitable for early adoption of AI,” he adds.
Bori points to other examples of early rollout failures including IBM’s ambitious plans with its Watson AI product to help diagnose cancer and work on genomics. IBM partnered with reputable healthcare research institutions in the US, including the University of North Carolina and the Memorial Sloan Kettering Cancer Centre, among others.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalData“Unfortunately, as with the McDonald’s case, the accuracy of the outputs from these systems was not high enough, particularly given that it was dealing with people’s health, so eventually were discontinued,” said Bori.
In another high-profile example from 2016, well ahead of the current large language model boom, Microsoft’s Tay chatbot became lewd and racist and had to be eventually shut down.
Executives must not rush to implement AI
Indeed, McDonald’s is just the latest in a line of multinationals to experience high-profile teething errors in early AI rollouts, according to Partha Gopalakrishnan, a partner and president at Brane Group, who advises enterprises on AI strategy implementation.
“Executives are rushing – they don’t want to be seen as lagging behind, and investors and boards are hungry for AI transformation due to the promised efficiency gains – but this means that they are less focused on putting in place the necessary foundations and guardrails,” said Gopalakrishnan.
“AI systems need to have the right staff behind them, robust internal training structures, a broader company culture of relearning and upskilling, and reliable AI models – these all take years to build,” he added.
In the case of McDonald’s, faulty AI has been rushed in front of customers, combined with not having the right staff and culture behind it, and the result has been financial and reputational damage, warned Gopalakrishnan.
“However, we shouldn’t get carried away. This is a trip, not a fall, and McDonald’s management will likely roll out AI later down the line, with some lessons under their belt,” said Gopalakrishnan, who cautioned executives to view the case of McDonald’s as a warning simply to “touch on the brakes of their AI transformation projects and ensure the road is properly paved first”.
As part of a digital transformation strategy, McDonald’s has developed a centralised social media strategy for better customer connection, which involves collecting data from social messaging channels and performing social data analytics for customising marketing assessments. The viral video circulation of AI mishaps will have damaged this social media strategy.
GlobalData predicts that the AI market will grow from $103bn in 2023 to $1.04trn by 2030 at a compound annual growth rate of 39%. GenAI, in particular, will provide significant growth opportunities for enterprises.
McDonald’s has embraced this potential with a digital transformation strategy that includes acquisitions such as technology company Apprente in 2019, which developed natural language processing technology to automatically make recommendations based on customers’ purchase history.
The company is integrating its outlets with digital menu boards, digital signage and self-ordering kiosks as well as integrating its mobile app to capture customer data for personalised recommendations.
AI can deliver huge value for an organisation, but premature implementation can do massive damage overnight. “Executives need to balance the drive for innovation with protecting customers from disruptive and malfunctioning tech,” said Gopalakrishnan.
GlobalData’s AI Executive Briefing ranks launching internal processes to review and determine the opportunities and risks of GenAI at the process/task level as critically important, with a focus on tolerance for accuracy, need for factual advice and product substitution potential, well before deployment.
GlobalData senior analyst Beatriz Valle said that the McDonald’s case also demonstrates how AI guardrails are so important, in terms of biases and representation.
“Looking at diverse accents, in this case the bot could not recognise the different accents apparently; it seems we are a long way from applications that are refined enough. AI may struggle to understand accents, dialects or background noise, leading to incorrect orders and frustrated customers,” said Valle.
The McDonald’s example highlights the potential pitfalls of relying on AI without adequate human oversight, proper testing and clear ethical guidelines. “If the AI encounters a situation it can’t handle, there might not be a human readily available to intervene, leading to further complications,” said Valle.
