Britain’s National Cyber Security Centre (NCSC) has cautioned businesses on the dangers of incorporating machine learning (ML) and large language models (LLMs) into their services.
LLMs are algorithms that use ML to summarise, generate and predict new content.
Go deeper with GlobalData
Since ChatGPT’s release in late 2022, the unprecedented popularity of the chatbot has seen businesses integrating LLMs into their products.
In two blog posts on Wednesday, officials stated: “Organisations building services that use LLMs need to be careful, in the same way they would be if they were using a product or code library that was in beta.”
The NCSC identified two potential weak spots in LLMs that could be exploited by attackers: data poisoning attacks and prompt injection attacks.
Prompt injection attacks involve an input designed to cause the model to ‘generate offensive content, reveal confidential information, or trigger unintended consequences in a system.’
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataData poisoning, meanwhile, relies upon the inherent weakness of machine learning: the vast training data the model requires. Data is typically sourced from the internet and can include content that is inaccurate or controversial.
In 2021, Google ethics researchers predicted the risks of data poisoning for LLMs in a paper titled ‘On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?’
The paper found that LLMs are likely to absorb worldviews belonging to dominant groups from their training data. Harmful stereotypes against women and minorities risk being embedded in algorithms that are trained on datasets that do not represent all people.
US President Joe Biden said in April that artificial intelligence (AI) “could be” dangerous and technology companies should ensure that their products are safe
The rising concern around AI was highlighted by an open letter by the Future of Life Institute, which said: “Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable.”
The letter, titled ‘Pause Giant AI Experiments,’ has now received over 33,000 signatures.
