Beyond the Fence, and the Advent of the Creative Machines

Creative machines have been making their influence felt for some time, but an upcoming stage production challenges preconceived notions of what art is.

Computers being involved in the creation of art is not a new development.

(Consider my use of the word computer throughout this post as roughly equivalent to the cringe-worthy use of the word server in any of your favorite TV shows or movies: it means a bunch of different stuff. I will use it as a stand-in for the often-confounded terms of "technology," "hardware," and "software," (and even "computer") as well as "machine learning system" and "artificial intelligence.")

In any event, for years our electronic sidekicks have gradually moved from assisting us with our creative endeavors to controlling many of these processes themselves. Consider one particular technology-assisted creative undertaking: music recording. From modestly facilitating the editing and processing of raw creative material into musical end products all the way to the automated detection and optimization of certain aspects of the overall process, such as audio input levels, computers have long been involved in this creative process, as they have with many others.

Of course, this is only assisting us with making our art. In fact, the example above is simply providing technological assistance for our own creative processes. But the line between creative assistant and creator is one that becomes blurred rather quickly.

Mandelbrot Set

Philosophical Foundations

The idea of computers outright creating art, while not entirely new, is certainly a more often heard and increasingly less ridiculous notion, given the massive recent advances in computing technologies, both hard and soft.

But what exactly would be considered "computer art?" Is it any art generated by a computer? As a very simple attempt to answer this question, take the Mandelbrot Set, visualizations of which can be "drawn" using algorithms and automated fractal engineering software. Is this art? Can an algorithm be artistic? Do people not use their own algorithms and work within given bounds to produce artistic works? Is this alone enough to be considered art? There are a great number of further philosophical considerations, including this important one: does the categorization of art depend on the end product, or the creation process... or both?

The line between art and imitation is a fine one, for both humans and machines. Humans take artistic influence as input and create new works, so is it any different when a deep neural network takes millions of examples of literary or visual art and then creates something "original" from what it has learned? We could embark on a lengthy philosophical tangent at this point, yet I am certain we would end up with no more definition on the subject than we have now.

"Representing and speedily..." and Other Generative Models

Instead, let's look at a few concrete examples of generative models which produce outcomes which could be considered art.

in 1948, Claude Shannon and his successive generative text models (dubbed "representing and speedily" for the words one of the generated sentences began with) were able to increasingly approximate proper English sentence structure. While not art in the conventional sense, such models lay the foundation for larger scale generation of text, which could be considered art depending on the particular input. Text generation models since then have produced poetry, Shakespeare-like dialog, and convincing click-bait (hey, it's an artform), among other things. Note that these generative models are often flawed in numerous ways, and producing a literary work that could pass as human-written is not an easy feat. However, these models are often quite impressive in their achievements, regardless of your view on whether or not their output is art.

Deep Dream arches

And it's not just the written artforms being transformed by the computer. Turning to recent achievements in the computational visual arts, Google's Inceptionism and Deep Dream have likely helped solidify the current widespread infatuation with deep neural networks and computer vision among a new wave of theorists and practitioners alike. Algorithmic music composition has been undertaken in the recent past by different learning algorithms, including Restricted Boltzmann Machines and Recurrent Neural Networks.

Generative neural networks have become a part of the rich deep learning research landscape. Online searches turn up new models weekly, be they research papers or blog posts. Some of the better potentially art-related generative networks I have come across over the past few months include:
Another project that I have (anecdotally) seen, shared via tweet but which I can find no further information about whatsoever, is a 'Friends' script generative model, which claims to have fed in every script of the TV show Friends and generated new scripts out the other side. This image comes from the Twitter handle Andy Pandy, and nothing further is known of the recurrent neural network it claims to represent. However, just read these few lines and then tell me this isn't the best Friends episode you've never seen.

New Friends episode

Of course, many of these new models are based either entirely or heavily on previous implementations; the results are entertaining nonetheless. And could we not argue that this follows the practice of art; one innovation leads to the exploration and development of subsequent innovations? In this way, art and computers are quite similar.

Next, we examine "Beyond the Fence", a whole musical, conceived and substantially crafted by computer.