Abbott’s examples are just hype

Returning to the question of ‘computational invention’, a key assertion made by Abbott is that computers already independently or autonomously create potentially patentable inventions. He offers three examples in support of this claim:

the ‘Creativity Machine’, which was developed in the early 1990’s by computer scientist Stephen Thaler;

the ‘Invention Machine’, a term for systems based on ‘genetic programming’ (GP), coined by Dr John Koza; and

IBM’s Watson.

In all three cases, however, I fear that the claims of ‘creativity’ and ‘inventiveness’ – which invariably originate with the developers and their associates – are more marketing hype than a fair and accurate assessment of technical capability.

The ‘Creativity Machine’

Thaler’s ‘Creativity Machine’ is based on a neural network – the same general type of ML structure that lies at the heart of AlphaGo. As I have already explained, while such systems can ‘learn’, by updating their internal state in response to training data and/or past performance, for a given (trained) state they are entirely deterministic, in the sense that the same inputs will always generate the same output.

Thaler’s idea was to randomly perturb the state of a trained neural network with ‘noise’, to see if this would then result in it doing something unexpected, new and useful. Here is an excerpt from his own description of the resulting invention, for which US patent no. 5,659,666 was granted in 1997:

Whereas the discovery of just how to adjust the noise level within a trained neural network to produce new ideas is a significant scientific finding, a viable patent was not achieved until a critic algorithm was added, whether heuristic, Bayesian, or neural network based, to monitor for the very best notions emerging from the perturbed network. This is the preferred embodiment of the invention called a Creativity Machine, a “dreaming” network, “imagination engine,” or “imagitron” that is monitored by another constantly vigilant algorithm that we appropriately call an ‘alert associative center’.

Numerous exceptional claims for the Creativity Machine are reproduced in Abbott’s paper. These include: that ‘the two artificial neural networks mimic the human brain’s major cognitive circuit: the thalamo-cortical loop’; that ‘like the human brain, the Creativity Machine is capable of generating novel patterns of information rather than simply associating patterns, and it is capable of adapting to new scenarios without additional human input’; and that the Creativity Machine was actually responsible for inventing the subject matter of Thaler’s second patent.

If all of this sounds more like marketing hype than sound academic evaluation, there may be a good reason for that: Thaler, his associates, and/or documents on the web site of Thaler’s company, Imagination Engines Inc., are the direct or ultimate source of every one of these extraordinary claims.

Genetic Programming – the ‘Invention Machine’

Abbott’s second example of existing ‘computational invention’ is a technology known as ‘Genetic Programming’ (GP). As the name suggests, the basic idea of GP is to develop computer programs structured in such a way that they are able to ‘evolve’. As with other ML systems, such programs generally have inputs and outputs corresponding with some specific pre-defined task. A ‘genetic program’ may be represented, for example, as a set of mathematical or other operations organised into a data structure, and which act on the inputs in order to generate the outputs. Modifying the data structure thus changes the program, and its outputs. Various rules (i.e. ‘genetic algorithms’) can be applied to produce ‘generations’ of such programs, where each new generation consists of variations on the previous generation, and to ‘select’ better-performing variations to carry their code forward into further generations.

According to Abbott, the USPTO granted US patent no. 6,847,851 on an invention that was ‘created by the “Invention Machine”— the moniker for a GP-based AI developed by John Koza.’ Abbott goes on to describe Dr Koza as ‘a computer scientist and pioneer in the field of GP’ who ‘claims the Invention Machine has created multiple “patentable new invention[s].”’ I have a number of issues with this description.

Firstly, as far as I have been able to ascertain, there is no single ‘GP-based AI’ that Koza, or anyone else, has developed and called ‘the Invention Machine’. Rather, ‘Invention Machine’ appears to be a generic term used by Koza to describe GP as applied to generating new solutions to technical problems (see Genetic Programming is an Automated Invention Machine on Koza’s web site genetic-programming.com). Koza contends that he is ‘considered the inventor’ of GP, although the Wikipedia article on the topic identifies significant prior work on the subject dating back to the 1950s.

Since 2004, Koza has been running an annual competition called the ‘Human-Competitive Awards’ (or ‘Humies’), in which cash prizes are given for applications of GP which result in solutions to real-world problems that are competitive with the work done by creative and inventive humans. There are a number of alternative criteria for assessing human-competitiveness, including that ‘the result was patented as an invention in the past, is an improvement over a patented invention, or would qualify today as a patentable new invention.’ In 2010, Koza published an article, ‘Human-Competitive Results Produced by Genetic Programming,’ Genetic Programming & Evolvable Machines, Vol. 11, March 2010 (PDF copy available, 400kB), in which he summarised the various successes of entrants in the Humies. This one paper is Abbott’s sole source for his assertion that ‘by 2010, there were at least thirty-one instances in which GP generated a result that duplicated a previously patented invention, infringed a previously issued patent, or created a patentable new invention.’

It is therefore apparent that Koza – a person with a clear interest in promoting the virtues of GP and his own contributions to the field – is the ultimate source of every one of the claims documented by Abbott that GP is an ‘Invention Machine’.

Which is More Creative – Watson or IBM’s Marketing Team?

As Abbott notes, ‘IBM describes Watson as one of a new generation of machines capable of “computational creativity”’ (emphasis added). Abbott also quotes IBM documents which contend that such machines generate ‘ideas the world has never imagined before’ and that Watson ‘generates millions of ideas out of the quintillions of possibilities, and then predicts which ones are [best], applying big data in new ways.’

To misquote Mandy Rice-Davies, ‘well, they would say that, wouldn’t they?’ IBM markets a range of Watson-based products and services to a variety of potential customers – including Australian government agencies such as IP Australia – to provide ‘actionable insights from large amounts of unstructured data via natural language processing and machine learning.’

This is not to say that Watson is not remarkable, however a more sober analysis concludes that Watson is not some ‘super-smart Siri’, but IBM’s branding for ‘a wide range of AI techniques and related applications’ that it is offering to a growing market of organisations looking for ways ‘to apply these techniques, and to tap into the expertise required to do so.’