On Monday, IBM showcased its artificial intelligence (AI) technology in a debating contest against one of the most successful competitive debaters in the world. The results were very impressive.
While Project Debater did not win the debate outright, it gave a very strong and coherent performance, showing its ability to formulate an argument, bolster that argument with information and present it in a fluid and articulate way.
IBM acknowledges that there is still a lot of work to do, but Project Debater’s performance represents a significant step forward in the domain of natural language processing (NLP) and AI.
The debate structure
IBM’s Project Debater is designed to compete on equal terms with a human opponent, following established rules for debating competitions. A topic is decided upon and revealed to both participants a few minutes before the contest begins, giving them only a short amount of time to prepare an argument.
In Monday’s debate, the proposal was that preschool education should be subsidised by the state. IBM’s Debater spoke in favour of the proposal and Harish Natarajan (a world debating championship grand finalist in 2016 and 2012 European debating champion) had the job of opposing it.
The format of the contest is fixed. The proposer speaks for four minutes to introduce their argument, and then the opposer has four minutes to introduce theirs. There follows a short pause, after which the proposer has four minutes to rebut the remarks made by its opponent, who then gets four minutes to make their own rebuttal.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataFinally, the proposer is allowed two minutes to summarize, followed by the opponent.
War of the words
IBM’s Debater gave a compelling opening speech, presenting empirical evidence of the benefits of preschool education and making a strong moral argument that society has a duty to help children that are less well-off get a good start to their education.
Crucially, IBM’s AI platform presented its argument as a coherent narrative spoken with a persuasive tone, even adding elements of humour to its segment. Interestingly, it also predicted one of the arguments that would be made by its opponent, insight that requires an added level of sophistication.
Natarajan gave the kind of fluent and coherent opening you’d expect of a champion debater but, when called on to respond, IBM’s AI impressed again by getting a good, if not quite perfect, grip on his argument, demonstrating its ability to not only transpose its opponent’s speech to text but also to draw meaningful inferences from it.
The closing statement from the IBM machine was a real tear-jerker as it argued that, while it had never personally experienced hunger, we all have a responsibility to support the poor in our society.
The performance of IBM’s Debater was far stronger than many in the audience had expected and, while the platform didn’t win the competition (which awards victory to the participant who convinces more people to change their view), it did win a vote to determine which of the two provided more information about the topic.
Project Debater wouldn’t have voted for President Trump
Project Debater calls upon a corpus of knowledge in the form of several billion ‘sentences’, or statements, which it parses to select those which are most relevant to the topic at hand. It then evaluates the set of relevant statements in order to select those that it determines are likely to provide the most persuasive argument.
The fact that it depends on a curated corpus of knowledge is crucial because the views it holds, and the arguments it selects, will be dependent on the views of whoever curated that corpus of knowledge. On this occasion, Debater had evidently been fed a diet of information that you would associate with a respectable New England liberal arts college – no Fox News or Infowars for this AI platform.
One had a strong sense that this incarnation of Debater wouldn’t have voted for President Trump but, had the corpus of knowledge been assembled from different sources, the system’s politics might have been radically different.
This reliance on a curated knowledge base highlights one of the biggest steps that AI needs to take. To be truly cognitive, Project Debater needs to be allowed to search the internet independently for its information. This would, in turn, require it to make judgements about the value, persuasiveness, and veracity of the information it found and would represent a radical extension of the platform’s powers.
This kind of ability remains some way off but the Project Debater team has made huge steps forward in natural language analysis and processing and, while Debater would not make a particularly entertaining dinner guest today, that may not be the case for too much longer.
Read more: Garry Kasparov: We need better humans not less technology
