Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
298
Do Robots Dream of Digital Sheep?
1 Refik Anadol, StyleGAN generated building study using a data base of designs by ZHA
Neil LeachFIU/Tongji/EGS
1
ABSTRACTAI is playing an increasingly important role in everyday life. But can AI actually design?
This paper takes its point of departure from Philip K Dick’s novel, Do Androids Dream of
Electric Sheep? and refers to Google’s DeepDream software, and other AI techniques such
as GANs, Progressive GANs, CANs and StyleGAN, that can generate increasingly convincing
images, a process often described as ‘dreaming’. It notes that although generative AI does
not possess consciousness, and therefore cannot literally dream, it can still be a powerful
design tool that becomes a prosthetic extension to the human imagination. Although the
use of GANs and other deep learning AI tools is still in its infancy, we are at the dawn of an
exciting – but also potentially terrifying – new era for architectural design. Most impor-
tantly, the paper concludes, the development of AI is also helping us to understand human
intelligence and 'creativity'.
299
The movie Blade Runner (1982), directed by Ridley Scott,
is based on the novel by Philip K Dick, Do Androids Dream
of Electric Sheep? (1) It depicts a dystopian future world
involving ‘replicants’ – bio-engineered robots – manu-
factured by the Tyrrell Corporation to have super human
abilities so that they can survive in the hostile conditions
of off-world colonies. Replicants are therefore potentially
dangerous, and as a safety measure are given a limited
life span of four years. In the movie a group of six repli-
cants return to earth in a bid to extend their lives. Rick
Deckard, played by Harrison Ford, is a ‘blade runner’, a
kind of policeman/bounty hunter, charged with hunting
down and ‘retiring’ – killing – these replicants. The problem,
however, is that replicants – especially the advanced Nexus
7 model – look almost identical to human beings, and can
only be distinguished by using the elaborate ‘Voight-Kampff’
test designed to check whether their emotional responses
and eye reflexes meet the standard of human beings (2).
The movie is thus primarily about the difference between
human beings and replicants, such that replicants become
a mirror in which to understand what it is to be human. As
the late Rutger Hauer, who plays the leader of the repli-
cants, Roy Batty, comments, ‘In many ways, Blade Runner
wasn’t about the replicants. It was about what does it mean
to be human.’ (Roxborough)
Fast forward to 2019 – the year in which Blade Runner is
set – and it is worth reflecting on how prescient the movie
has proved to be. We don’t have replicants infiltrating
society, but we do have AI personal assistants, Siri, Alexa
and Google Assistant, colonizing our everyday lives, and
we do have AI filtering our spam, and performing other
tasks on our cell phones. We don’t have flying cars, but
we do have Maglev trains, drones and self-driving cars.
We don’t have the Tyrrell Corporation, but corporate life
is dominated nonetheless by hi-tech companies, such as
Google, Amazon, Apple and Microsoft. And, as predicted in
Blade Runner, we do talk to our computers, and do have
LED advertising all over our buildings, especially in cities
like Shanghai. Clearly Blade Runner has proved to be highly
prescient.
Although replicants are not necessarily controlled by AI,
they are clearly an artificial life form endowed with some
kind of ‘intelligence’. They therefore make a productive
vehicle by which to introduce the topic of AI, especially in
the context of debates about human intelligence. The title
of the novel, Do Androids Dream of Electric Sheep?, can
also be extended to raise the interesting question as to
whether AI can not only dream, but also design. Of course,
‘dreaming’ is not the same as ‘designing’, in that dreaming
– at least according to some commentators – refers to a
2 Rudger Hauer as Roy Batty in Blade Runner, directed by Ridley Scott, 1982
bottom-up process of unleashing repressed ideas, whereas
designing must also entail some measure of top down
control. (Freud) Nonetheless, it provides a useful starting
point for a discussion about AI and creativity.
CAN AI DREAM?It is generally assumed that computers cannot be creative.
This is the conclusion, at any rate, reached by Japanese
computational architect, Makoto Sei Watanabe, writing
about the potential applications of artificial intelligence (AI)
in the field of design: ‘Machines are better than people at
solving complex problems with many intertwined conditions.
In that realm, people are no match for machines. But people
are the only ones who can create an image that does not yet
exist. Machines do not have dreams.’ (3) (Watanabe, 2017)
The term ‘dream’, however, has been used – albeit meta-
phorically – in connection with a technique, DeepDreaming,
discovered by Alex Mordvintsev of Google Artists and
Machine Intelligence [AMI], while analyzing the operations
at work in the process of recognition using artificial neural
networks. Mordvintsev found that he was able to generate
images by reversing the flow of information in a neural
network.
Artificial neural networks are often used to recognize
images. They are used, for example, to recognize faces on
FaceBook and to classify images on Instagram, and are
often based on a class of deep neural networks, known as
convolutional neural networks [CNNs]. Generally speaking,
a network consists of 10-30 stacked layers of artificial
neurons. As Alex Mordvintsev et al. comment: ‘Each image
is fed into the input layer, which then talks to the next layer,
300
3 50 iterations of DeepDream trained to perceive dogs, Martin Thoma, 2015
until eventually the “output” layer is reached. The network’s
“answer” comes from this final output layer.’ (Mordvintsev,
Olah, Tyka)
With DeepDream, however, the neural network operates in
the opposite direction. Instead of recognizing an image and
assigning it a category, DeepDream starts with a category
and proceeds to generate an image. For example, whereas
a standard neural network can recognize an image of a
bird, and categorize it as a ‘bird’, DeepDream is able to start
with the category ‘bird’, and generate an image of a bird.
Thus instead of operating ‘from image to media’, DeepDream
operates ‘from media to image’.
But how exactly can a neural network typically used for
recognizing images also be used to generate – or ‘synthe-
size’ – images? Importantly, although computational neural
networks are trained to discriminate between images, it
is essential that they have some understanding of those
images in order to distinguish them. This allows them to
work in reverse, and generate images, instead of merely
categorizing them (4).
However, the process of inverting the operation of a neural
net in order to generate images is not so straightforward
(5). Also, it often produces a somewhat ‘trippy’ picture that
appears vaguely surrealistic with a multiplicity of objects
generated in a variety of poses (6).
These generated images are referred to as ‘hallucinations’.
Technically, however, they cannot strictly be hallucina-
tions as such, in that true hallucinations are phantasms or
figments of the imagination, composed of images that do
not actually exist. These ‘hallucinations’, by contrast, are a
form of overly strong perceptual prediction of what is there
already, and are generated by ‘maximizing the activation of
the semantic neurons at the end of a recognition network.’
(8) (Agüera y Arcas)
GENERATIVE ADVERSARIAL NETWORKS (GANS)The next step in the development of generative AI came
with the introduction of Generative Adversarial Networks
(GANs), a technique first proposed by Ian Goodfellow in
2014, which has undergone a process of rapid develop-
ment. (Goodfellow, 2014). GANs represent a significant
step forward in the quest to ‘synthesize’ images, in that the
issue of invariance to ‘pose’ is no longer a problem, and the
images generated have significantly better resolution.
GANs are based on a competition between two neural
networks. There is always a bottom-up generator – or
‘artist’ – (typically a deconvolutional neural network) that
generates images, and a top-down discriminator – or ‘critic’
– (typically a convolutional neural network) that evaluates
those images. In the competition, the generator attempts to
fool the discriminator by producing images so realistic that
the discriminator is unable to distinguish them from a real
data set (9).
Effectively GANs are a way of training a computer to
perform complex tasks through a generative process. ‘[A]
Generator (an artist) generates an image. The Generator
does not know anything about the real images and learns
by interacting with the Discriminator. Discriminator (an art
critic) determines whether an object is “real” and “fake” .
. . The Generator keeps creating new images and refining
its process until the Discriminator can no longer tell the
difference between the generated images and the real
training images.’ (Nayak 2018) The two work in tandem, so
that the ‘artist’ trains the ‘critic’, and the ‘critic’ trains the
‘artist’. Once the ‘artist’ has been trained, the ‘critic’ can be
removed (10). Although GANs have been used primarily
to generate faces, they have also been used for clothing,
shoes, industrial design objects, and even interiors and
exteriors of buildings.
A number of variations of GANs have now been devel-
oped. A recent variation of GANs – Progressive Growing of
GANs – has improved the speed and stability of the training
process, by starting with a very low resolution image and
increasingly the resolution progressively with every layer.
‘The key idea is to grow both the generator and discrimi-
nator progressively: starting from a low resolution, we add
new layers that model increasingly fine details as training
Do Robots Dream of Digital Sleep? Leach
301
4 Edmond de Belamy (2018): A Creative Adversarial Network portrait painting generated by the Paris-based arts collective Obvious
5-6 StyleGAN Faces, generated using www.thispersondoes- notexist.com
64
progresses.’ (Karras, Aila, Laine, Lehtinen) This allows the
whole process to produce some highly convincing faces of
non-existent people with far greater realism than previ-
ously achieved. The results, while still not perfect, show
that true photo realism is not far away (11).
Likewise, a version of GANs has now been developed to
produce Creative Adversarial Networks [CANs]. CANs
have been used in particular to generate art; ‘The system
generates art by looking at art and learning about style;
and becomes creative by increasing the arousal potential
of the generated art by deviating from the learned styles.’
(Elgammal, Liu, Elhoseiny, Mazzone) The intention here is to
open up the range of creative possibilities ‘by maximizing
deviation from established styles and minimizing deviation
from art distribution.’ (Elgammal, Liu, Elhoseiny, Mazzone)
Interestingly, it was found that human beings were unable
to distinguish ‘art’ created by CANs from art created by
artists. As such, it would appear that CANs are capable of
passing the Turing Test.
A further version, StyleGAN, is the latest in this line of
development and the most sophisticated GAN system to
date. StyleGAN is an open-source style based generator
architecture that borrows from style transfer literature to
produce images, and offers remarkable improvements in
terms of resolution and quality. In particular, it overcomes
the problem of ‘entanglement’ whereby any slight tweak or
amendment to one feature would have a knock on effect
on other features, by reducing the correlation between
different features (12). (Horev) As a result, it generates
5
faces so convincing that it is often almost impossible to tell
the difference between the generated artificial faces and
those of real human beings.
Perhaps the most significant improvement, however, is that
images can simply be fed in, and do not need to be tagged
or classified as with previous versions of GAN. This saves
considerable time. As the authors note: ‘The new archi-
tecture leads to an automatically learned, unsupervised
separation of high-level attributes (e.g., pose and identity
when trained on human faces) and stochastic variation in
the generated images (e.g., freckles, hair), and it enables
intuitive, scale-specific control of the synthesis.’ (Karras,
Laine, Aila)
ARCHITECTURAL APPLICATIONS OF GANSOne of the first applications of GANs to an architectural
project was a projection on to the Walt Disney Concert Hall
[WDCH], by media artist, Refik Anadol, in collaboration with
Google AMI and computational sound consultant, Parag
Mital, and others, commissioned to mark the 100th anni-
versary of the LA Philharmonic in October 2018.
‘When we dream, our minds process memories to form
new combinations of images and ideas.’ (Google Arts and
Culture) This, at any rate, is the premise behind the projec-
tion. The intention was to give ‘consciousness’ to the WDCH
in three distinct stages. The first stage of the projection
– ‘Memory’ – involved the ‘training’ of a neural network by
classifying thousands of images and sound recordings of
events from the archives of LA Philharmonic. These were
302
then processed by the network, and the results projected
onto the surface of the WDCH (13). The second stage –
‘Consciousness’ – involved the use of a neural network to
categorize and rearrange the material, by finding similar-
ities within the material (14). The third stage – ‘Dreaming’
– was probably the most spectacular. This was when the full
‘hallucination’ takes place (15).
More recently Anadol has explored the use of StyleGAN as
a means of ‘hallucinating’ an animated movie from which
still images of fictional buildings can be extracted, drawing
upon a data base of thousands of photographs of more
progressive buildings, designed by architects such as
Gehry and Partners, Zaha Hadid Architects, Toyo Ito, Tadao
Ando and so on.
This work raises the interesting question as to whether
StyleGAN might be able to generate designs that are even
more experimental. Indeed, it might equally be possible to
use alternative data, such as natural forms like flora and
fauna – orchids, perhaps – or even rocks and landscapes,
to ‘hallucinate’ novel hybrid or cross-pollinated forms.
Xkool, an AI startup in Shenzhen, China, whose CEO is
Wanyu He, has probably engaged with the most sophisti-
cated application of AI to architectural design to date. The
intention is to streamline the design process and make
it more efficient and creative by using deep learning to
not only search through a vast range of possibilities, but
also automatically generate designs based on the trained
models, and then, evaluate and return the outcomes from
various evaluation models.
Xkool has also used StyleGAN and other AI techniques
to generate images of buildings that do not exist (16). On
the whole these buildings are more traditional that those
generated by Anadol, and are largely modernist in their
aesthetic, reflecting the content of the database of images
used. These images also reveal how much depends on the
quantity of data, in that they draw upon a larger data-
base, and are therefore somewhat clearer and more
detailed than those generated by Anadol. However, they
do not reach the same standard as the faces generated by
StyleGAN, and at present there are still too many tell-tale
glitches giving away the fact that they are not images of
actual buildings.
Although StyleGAN can already generate some star-
tlingly innovative images, there a number of constraints
holding back its development. Firstly, the use of StyleGAN
is constrained by the power of Graphic Processing Units
(GPUs) available. As a result, Anadol, for example, would
not have been unable to generate images so quickly without
the support of extremely powerful GPUs provided by
NVIDIA. Secondly, it should be borne in mind that at present
StyleGAN has so far been restricted largely to the gener-
ation of 2D images. In order to generate truly convincing
‘designs’, it would be necessary to move beyond the domain
of two dimensions and operate in three dimensions (17).
Thirdly, it would be necessary to deepen the level of infor-
mation to incorporate some more highly refined details, and
to engage with performative constraints.
Since GANs are already in use for the purposes of optimi-
zation, some aspects of the introduction of performative
constraints should be fairly straightforward. Moreover, the
introduction of KoolPlan, an AI assist for floor plans and
façade design, which builds upon the logic of StyleGAN, is
Do Robots Dream of Digital Sleep? Leach
7
8
7-8 Refik Anadol, StyleGAN generated building studies using a database of designs by ZHA
303
already helping to increase the refinement of the detailing.
Meanwhile Autodesk is continuing to explore the poten-
tial use of AI into structural optimization through Project
Dreamcatcher, automation in design through Fusion 360,
and even AI based software for risk assessment in the
construction industry. (Rajagopal, 2018)
Indeed, given the logic of Moore’s Law, it is surely safe to
assume that that the process of development might even
accelerate, and that a robust AI based technique for gener-
ating 3D architectural drawings will be available within the
next few years (18).
CAN AI BE CREATIVE?Can AI be creative? In some senses, the answer would
appear to be clearly ‘yes’. Take the example of the highly
novel Move 37 made by AlphaGo in the second game of
its match against Lee Sedol. As Fan Hui comments, ‘When
AlphaGo chose that move, I assumed that it had made a
mistake. I immediately looked to see Lee Sedol’s reaction. At
first, he seemed to smile – as though he too thought it had
made a mistake – but as the minutes rolled by it was clear
that he was starting to realize its brilliance. In fact, after the
match, he said that when he saw this move he finally real-
ised that AlphaGo was creative.’ (Hassabis, Hui, 89)
If, as Richard and Daniel Susskind note, AlphaGo could
come up with such an original move, and it can be viewed
a creative does this not suggest that creativity amounts
to little more than imagining a range of already available
options that have not been thought about before? (Susskind,
Susskind, x) In other words, could we even say that being
‘creative’ is little more than searching for the unexpected,
or merely proposing that which has not been proposed
before? As the Susskinds note: ‘Contrary to widespread
belief, machines are now capable of generating novel
outcomes, entirely beyond the contemplation of their orig-
inal human designers.’ (Susskind, Susskind, xi)
In order to answer the question more comprehensively,
however, we would need to understand the nature of
creativity in more detail. According to Margaret Boden,
‘Creativity can be defined as the ability to generate novel,
and valuable, ideas.’ (Boden, 2009) For Boden ‘valuable’ has
many meanings: ‘interesting, useful, beautiful, simple, richly
complex and so on’. Meanwhile ‘ideas’ could take many
forms from ideas as such, ‘concepts, theories, interpreta-
tions, stories,’ but also artefacts, such as ‘graphic images,
sculptures, houses and jet engines.’ (Boden, 2009)
Boden notes that there is nothing magical about creativity
9
10
9-11 Xkool, StyleGAN generated building images
11
304
even shocking – results, as something might have been
generated that could never have been previously imagined.
Importantly, for Boden, exploratory creativity can lead to
transformational creativity. (Boden, 2009)
For Boden collage would be an example of combinational
creativity. Painting, architecture and music, and artistic
production in general, would be an example of explor-
atory creativity. Boden offers the example of a shape
grammar study of the works of Frank Lloyd Wright, which
suggested many other possible designs that could have
been produced. (Boden, 1999) Finally, science offers some
of the best examples of transformational creativity, where
paradigmatic shifts in knowledge can occur (20).
From this perspective, the use of StyleGANs in architec-
ture would seem to fall into the category of exploratory
H-creativity, in that the material generated is novel from
a global perspective, and depends upon an initial set of
constraints. Although not limited to fixed forms, as in the
case of shape grammars, the operations of StyleGAN
would seem to amount to a blending of pre-existing forms.
Although novel forms can be generated, the search space
of possibilities is constrained by the range of forms in the
data base.
CONCLUSIONAnd so do robots dream of digital sheep? Or – to be more
precise – can AI dream of digital buildings? Is Watanabe
correct to claim that humans are the only ones that can
create an image that does not exist, and that machines do
not have dreams? (21) (Watanabe, 2017)
Let us start by stating the obvious. All faces or buildings
generated by StyleGAN do not actually exist (22). In terms
of dreams, however, what becomes clear is that the word
‘dream’ appears in many of the AI applications and projects
cited above, whether it be ‘DeepDream’, ‘Dreamcatcher’ or
‘WDCH Dreams’. Clearly, in one sense, computers are able
to ‘dream’ not just of sheep but also of buildings through the
use of various forms of GANs. However, the term, ‘dream’ –
along with the terms, ‘memory’ and ‘hallucination’ – should
not be taken literally. They need to appear in inverted
commas. Until computers have consciousness, they cannot
literally dream (23). In this sense, Watanabe is correct (24).
But can AI really be as creative as humans beings? For
Boden, this is a philosophical question unanswerable for
now, as many highly contentious philosophical issues would
need to be resolved. (Boden, 2009) For Boden, however, the
key issue is not whether AI can match human creativity, but
how AI has helped us to understand human intelligence,
(19). Although we have come to think of creativity as a
mysterious process, that often depends on the ‘black box’
of intuition, this is simply because most people do not
understand how it operates. In this sense, it is similar to
magic. For magicians do not perform magic. They simply
conceal the processes that actual happen, so that the
audience comes to attribute them to magic. (Leach, 1999;
Boden, 2009) Nor does creativity need to operate at a high
level, and Boden offers examples of some relatively banal
jokes generated by AI that nonetheless display a low level
of creativity, such as, ‘What do you call a strange market? A
bizarre bazaar.’ (Boden, 2009)
Boden distinguishes between what she calls psycho-
logical creativity (P-creativity) and historical creativity
(H-creativity). P-creativity could be defined as something
novel in terms of the person who created it, whereas
P-creativity refers to something that has been created for
the first time in history. She further divides creativity into
three distinct categories, combinational, exploratory and
transformational.
Combinational creativity ‘produces unfamiliar combina-
tions of familiar ideas, and works by making associations
between ideas that were only indirectly linked.’ (Boden,
2009) Exploratory creativity, meanwhile, is based on a
culturally accepted style of thinking, or ‘conceptual space’,
defined by a set of generative rules. Finally, in transfor-
mational creativity, ‘the space or style is transformed by
altering (or dropping) one or more of its defining dimen-
sions’. (Boden, 2009) This can lead to surprising – or
12-13 Xkool, AI assist-generated drawings
12
13
Do Robots Dream of Digital Sleep? Leach
305
14 Xkool, StyleGAN generated building images
‘Thanks in part to AI, we have already begun to understand
what sort of phenomenon creativity is.’ (Boden, 2009)
Certainly, there are a number of specific questions about
human creativity that we might pose in the light of what
we know about AI. Could the process of classifying objects
to form a data base in machine learning be similar to how
children themselves are taught to identify objects, ‘This is
a cow’; ‘this is a horse’, and so on? Could we even surmise
that the teaching of architectural design is based on a
similar principle, since architectural design is not taught
as an overarching theory, but rather by way of examples:
‘this is a good design by Le Corbusier’; ‘this is a good design
by Rem Koolhaas’, and so on? Could we even claim that the
architectural design process consists of a directed search
of latent space based on a database of such examples, not
dissimilar to how StyleGANs work? Does the reversal in
direction of operation of a neural network, pioneered with
the development of DeepDream, suggest that the process
of interpretation is – in some senses – the opposite of the
process of creativity? If so, might this help to explain why
architectural theorists/critics tend not to be so creative,
while architectural designers tend to be less theoretical?
(25)
Blade Runner, as has been noted, has proved to be extraor-
dinarily prescient in predicting the future. But are there
not also uncanny parallels between Blade Runner and
the operations of AI? Does the victory at chess of repli-
cant, Roy Batty, over Dr Eldon Tyrrell, not foreshadow the
eventual moment when DeepBlue, IBM’s super computer,
beat the then leading chess player Gary Kasparov at chess
in 1997? Does not the way in which replicants are almost
indistinguishable from human beings, echoe how StyleGAN
generated faces are almost indistinguishable from human
faces? And, does not the Voight-Kampff test, used to distin-
guish replicants from human beings, echo the Turing test,
used to establish whether the operations of a computer
might be indistinguishable from those of human beings?
There is, however, a further parallel that deserves to be
mentioned. Just as for Hauer, Blade Runner is primarily
about the difference between human beings and repli-
cants, such that replicants become a mirror in which to
understand what it is to be human, so too AI – and robotic
intelligence in general - can become a mirror in which to
understand what it means to be human. As Hiroshi Ishiguro
notes, ‘The robot is a kind of mirror that reflects humanity
and by creating intelligent robots we can open up new
opportunities to contemplate what it means to be human.’
(Ishiguro, 179)
37 years ago Ridley Scott released a movie, Blade Runner,
set in October-November 2019, that predicted a world
inhabited by artificial life forms almost identical to human
beings. Indeed the fact that two replicants, Rachael and
Deckard, could fall in love, and that Roy Batty could deliver
a soliloquy as eloquent as ‘Tears in the Rain’ implies that
these artificial life forms had developed at least some form
of consciousness. But whether AI will ever achieve full
consciousness and match human creativity remains an
open question.
NOTES1. With this very title, the author raises an important question
about the potential for androids/robots to ‘dream’. Indeed the
theme of dreaming runs throughout both Blade Runner movies.
In the original movie, Blade Runner, Deckard has a dream of a
unicorn, while in the sequel, Blade Runner 2049, Anna Stelline,
whom we find out to be the daughter of Deckard and his lover
306
5. The problem, as Blaise Agüera y Arcas explains, is that the
process cannot literally be inverted, as it is a non-linear
operation, and therefore some ingenuity is required: ‘With a
convolutional net you can’t do this directly exactly, as it is not a
linear operator, and therefore not invertible. But you can ‘cheat’,
by basically saying that I am going to optimize the pixels, so I’ll
start maybe with noise – with pure noise – and run the same
gradient ascent algorithms that you use to learn a neural net
on the pixels until the output corresponds to the embedding
vector that I want.’ [Agüera y Arcas]
6. ‘The problem,’ as Agüera y Arcas notes, ‘is that the convolu-
tional nets are designed to be invariant to pose, which means
that when they are run backwards they do not know what pose
to render things in. This leads to an aggregation of several
poses at once.’ [Agüera y Arcas]
7. As Oliver Sacks notes, ‘Precise definitions of the word “hallu-
cination” still vary considerably, chiefly because it is not always
easy to discern where the boundary lies between hallucina-
tion, misperception and illusion. But generally, hallucinations
are defined as percepts arising in the absence of any external
reality – seeing things or hearing things that are not there.’
[Sacks, ix]
8. It is perhaps not insignificant that according to neuroscientist,
Anil Seth, perception is a form of perceptual prediction, and is
actively generated – ‘hallucinated’ – on the part of the viewer.
Thus perception becomes a form of ‘controlled hallucination’:
Rachel, is a designer of dreams for replicants. Since replicants
are not born and brought up like humans, they need to be given
dreams, just as they need to be given memories.
2. Rachael, who ultimately has an affair with Deckard, is herself a
Nexus 7 model. While previous models were required to answer
20 to 30 questions before they could be identified as a replicant,
it took 100 questions before Rachael could be positively identi-
fied as a replicant.
3. While Watanabe does not use the term ‘design’ here, he does
refer to a process of ‘dreaming’ whose role would seem to be to
imagine that which does not yet exist. For Watanabe, the term
‘designing’ would therefore seem to imply ‘imagining that which
does not yet exist.’
4. It is also possible to produce an image by starting with an
arbitrary image instead of ‘noise’ or a specific embedding, and
allowing the network to analyze and optimize it. It is important,
however, to specify a layer, and ask the network to enhance
what it sees. Whereas lower-level layers simply produce
‘strokes’ and ‘simple ornament-like patterns’, higher–level
layers ‘identify more sophisticated features in images, complex
features and even whole objects’. Higher-level layers are there-
fore more productive, and can potentially work by feedback
loops to enhance the image that they are fed, to unexpectedly
generate a further image. As Mordvintsev notes, ‘We ask the
network: “Whatever you see there, I want more of it!” This
creates a feedback loop: if a cloud looks a little bit like a bird,
the network will make it look more like a bird. This in turn will
make the network recognize the bird even more strongly on
the next pass and so forth, until a highly detailed bird appears,
seemingly out of nowhere.’ [Mordvintsev]
15
15 Refik Anadol with Google AMI, WDCH projection, October 2019, ‘Dream’ section
Do Robots Dream of Digital Sleep? Leach
307
‘If hallucination is a kind of uncontrolled perception, then
perception. . . is also a kind of hallucination, but a controlled
hallucination in which the brain's predictions are being reined
in by sensory information from the world.’ (Seth, 2017). As Anil
Seth notes, instead of passively recognizing an object, the brain
actively partakes in trying to make sense of what it is sensing
through a form of perceptual prediction: ‘Instead of perception
depending largely on signals coming into the brain from the
outside world, it depends as much, if not more, on perceptual
predictions flowing in the opposite direction. We don't just
passively perceive the world, we actively generate it. The world
we experience comes as much, if not more, from the inside
out as from the outside in.’ (Seth) Seth goes on to illustrate this
with an algorithm based on Google’s DeepDream to simulate
the effects of an overly strong perceptual prediction: ‘When
perceptual predictions are too strong, as they are here, the
result looks very much like the kinds of hallucinations people
might report in altered states, or perhaps even in psychosis.’
(Seth) Seth’s comments about altered states is highly relevant,
in that research has shown that altered states can also result
from a combination of top down and bottom up processes, on
which computational hallucinations are based. In fact Seth
uses the ‘hallucinations’ of DeepDream to illustrate how the
brain itself works. Others, such as Agüera y Arcas, go further,
and claim that the brain itself is literally computational. (Agüera
y Arcas, 2016) Whether or not the brain is actually computa-
tional, there are clear parallels between the operations of AI
and the operations of the brain.
9. As Horev notes, ‘The generator input is a random vector
(noise) and therefore its initial output is also noise. Over time, as
it receives feedback from the discriminator, it learns to synthe-
size more “realistic” images. The discriminator also improves
over time by comparing generated samples with real samples,
making it harder for the generator to deceive it.’ (Horev)
10. This echoes the way in which architect and critic work
together in architectural culture. It also echoes the dialectical
relationship between architectural design and architectural
criticism, in that the critic needs to think creatively, while the
designer needs to generate work critically.
11. ‘While the quality of our results is generally high compared
to earlier work on GANs, and the training is stable in large
resolutions, there is a long way to true photorealism. Semantic
sensibility and understanding dataset-dependent constraints,
such as certain objects being straight rather than curved,
leaves a lot to be desired. There is also room for improvement
in the micro-structure of the images. That said, we feel that
convincing realism may now be within reach, especially in
CELEBA-HQ.’ [Karras, Aila, Laine, Lehtinen]
12. As Rani Horev notes, ‘ProGAN generates high-quality images
but, as in most models, its ability to control specific features
of the generated image is very limited. In other words, the
features are entangled and therefore attempting to tweak the
input, even a bit, usually affects multiple features at the same
time. A good analogy for that would be genes, in which changing
a single gene might affect multiple traits.’ (Horev)
13. As Anadol notes: “We take every folder, every image, every
sound recording in the archive, one by one, and let the archive
load itself into a RAM or buffer so the building goes through
its nostalgic folders, finds the first music directors, finds the
first iconic moments in time, and the building plots them for us.”
(Huewe)
14. ‘The second part,’ notes Anadol, ‘is completely organic. The
building starts to look at its own entire history. Suddenly, we
see a bunch of Big Bang moments. Data universes appear in
front of us, as emotional as possible.’ (Huewe)
15. As Anadol comments, ‘It is completely a machine hallucination.
An architectural cultural beacon reconstructs its own skin, and
even reconstructs a memory.’ (Huewe)
16. Wanyu He used to work for OMA, hence the name ‘Xkool’ (ie
ex-Koolhaas). According to the Urban Dictionary, Xkool also
means ‘super cool’. [Urban Dictionary]
17. Matias del Campo, however, is exploring the use of a 2d to
3D style transfer GAN in research work that has yet to be
published.
18. ‘Moore’s Law’ refers to a comment made in 1965 by Gordon
Moore, then CEO of Fairchild Semiconductor, and later to
become the co-founder of Intel, that the number of transistors
on an integrated circuit board would double every year, while
unit costs would fall correspondingly. This leads to exponential
growth. It has since been applied by Ray Kurzweil and others to
all forms of technology. (Kurzweil) The recent development of
ArchiGAN by Stanslav Chaillou - too recent to be included in this
article - is surely a sign of things to come.
19. Do we not therefore need to question our understanding of
human creativity? Is not human creativity, as has been claimed,
something of a ‘myth’? (King, Churchill, Tan, xi)
20. By way of example, Boden offers the redefinition in our
understanding of benzene, which opened up a whole new field
of enquiry for chemists: ‘If all organic molecules are basically
strings of molecules, then benzene can’t be a ring structure.
In suggesting that this is indeed what benzene is, the chemist
Friedrich von Kekule had to transform the constraint string
(open curve) into that of a ring (closed curve).’ (Boden, 2009)
21. The very notion that there might be a form that ‘does not
yet exist’ is itself contentious. According to Kostas Terzidis,
for example, what AI establishes above all else is that every
possible form is already out there, and it is simply a matter
of searching for it. (Terzidis, 2006, 11; 2014, 85) For example,
in the game of Go against Sedol, the possibility of Move 37
had always already existed. It is just that no human beings
had ‘encountered’ it before. According to this logic, the notion
that architects are genius creators constantly ‘dreaming up’
new forms is clearly absurd, in that it would be simply impos-
sible to invent forms that do not yet exist. Arguably, then, we
308
could claim that when we design something novel, we are not
inventing anything, but merely finding out about something that
was there in the first place.
22. Indeed there is a website that automatically generates faces,
‘This person does not exist’: www.thispersondoesnotexist.com
23. One of the main criticisms of AI is that machines do not have
consciousness. As such, although DeepBlue and AlphaGo both
appeared to be good at ‘playing’ chess and Go, neither was
conscious of actually playing the game. Nor indeed does AI have
emotions, thus establishing a further parallel with replicants.
Here we must differentiate between ‘narrow AI’ or ‘weak AI’ –
AI, that is, designed to work on one general task – and ‘general
AI’ or ‘strong AI’ – AI, that is, that can perform any task that a
human can, in some cases involving full consciousness. While
many have speculated about the possibility of general AI, it is
clearly a long way off. Until such time as AI can be endowed
with consciousness, it will never be able to hold a meaningful
conversation, still less dream. As such, attempts to produce
humanoid robots, such as Hansen Electronics’ robot, Sophia,
that not only look human – much like replicants – but also
supposedly ‘think’ like humans, are clearly misguided. For
Rodney Brooks, Sophia is ‘completely bogus and a total sham’
while for Benedict Evans Sophia is merely ‘a tape recorder with
a rubber head on it’. [Sinapayen]
24. But is consciousness so important? What if we were to judge
AI not in terms of human intelligence, but on its own terms, as
a form of alternative, alien intelligence? (Leach, 2019) As such,
the question of consciousness becomes largely irrelevant. In
any case, ultimately the issue is not competition between AI and
human intelligence, but rather a potential synergy between the
two. Instead of pure AI, we should therefore be taking about
‘Intelligence Augmentation’ [IA], whereby AI operates in tandem
with human intelligence, and becomes a prosthetic extension to
the human imagination. (Leach, 2018) As such, with IA there will
always be consciousness on the part of the human operator.
25. This is not to overlook the fact that theorists need a degree of
creativity, and designers need a degree of criticality. In other
words, the processes of interpreting and creating are to some
extent reciprocal.
REFERENCESAgüera y Arcas, Blaise. 2017. Lecture 9, Washington State
University, https://intelligentmachinerycourse.com/recordings/
Agüera y Arcas, Blaise. 2016. How Computers are Learning
to be Creative, TED Talk. https://www.ted.com/talks/
blaise_aguera_y_arcas_how_computers_are_learning_to_be_
creative?language=en
Boden, Margaret. 1999. ‘Creativity and Artificial Intelligence.’
Artificial Intelligence, 103 (I 998) 347-356
Boden, Margaret. 2009. ‘Computer Models of Creativity.’
Association for the Advancement of Artificial Intelligence. 103 (I
998) 347-356
Brooks, Rodney. 27 October 2018. twitter.com/rodneyabrooks/
status/1056259030962716672
Elgammal, Ahmed; Liu, Bingchen; Elhoseiny, Mohamed; Mazzone,
Marian, ‘CAN: Creative Adversarial Networks, Generating "Art" by
Learning About Styles and Deviating from Style Norms,’ 21 June
2017, https://arxiv.org/abs/1706.07068
Del Campo, Matias; Manniger, Sandra; Carlson, Alexa. 2019.
‘Imaginary Plans: The Potential of 2D to 2D Style Transfer in
Planning Processes.’ Acadia
Freud, Sigmund. 2010. The Interpretation of Dreams. Trans.
Strachey, James. New York: Basic Books
Harari, Yuval Noah. 2017. Homo Deus: A Brief History of Tomorrow.
NY: HarperCollins
Hassabis, Demis; Hui, Fan. 2019. ‘AlphaGo: Moving Beyond the
Rules.’ In Wood, Chloe; Livingston, Suzanne; Uchida, Maholo (eds.),
AI: More Than Human. London: Barbican International Enterprises.
Pp. 174-179
Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing;
Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio,
Yoshua (2014). Generative Adversarial Networks. Proceedings of
the International Conference on Neural Information Processing
Systems (NIPS 2014). pp. 2672–2680.
Google Arts and Culture. 2018. ‘WDCH Dreams’ https://artsandcul-
ture.google.com/exhibit/yQIyh25RSGAtLg
Horev, Rani. 2018. ‘Explained: A Style Based Generator
Architecture for GANs – Generating and Tuning Realistic Artificial
Faces.’ Towards Data Science, 30 December 2018. https://
towardsdatascience.com/explained-a-style-based-generator-ar-
chitecture-for-gans-generating-and-tuning-realistic-6cb2be0f431
Huewe, Kyle, ‘Refik Anadol: “WDCH Dreams”, Flaunt, November 20,
2018, www.flaunt.com/content/refik-anadol
Ishiguro, Hiroshi (in conversation with Uchida, Maholo). 2019.
‘A Reflection of Ourselves: Robots and the Journey Towards
Understanding Human Intelligence.’ In Wood, Chloe; Livingston,
Suzanne; Uchida, Maholo (eds.), AI: More Than Human. London:
Barbican International Enterprises. Pp. 174-179
Ito, Joichi. 2017. “Resisting Reduction: A Manifesto.” Journal
Do Robots Dream of Digital Sleep? Leach
309
of Design and Science: MIT Media Lab, November 01, 2017.
Accessed 1 May, 2018. https://jods.mitpress.mit.edu/pub/
resisting-reduction
Jhingran, Anant. 2016. “Obsessing about AI is the
Wrong Way to Think about the Future.” Wired, January
22, 2016. https://www. wired.com/2016/01/
forget-ai-the-human-friendly-future-of- computing-is-already-here
Karras, Tero; Aila, Timo; Laine, Samuli; Lehtinen, Jaakko. 2017.
‘Progressive Growing of GANs for Improved Quality, Stability, and
Variation,’ 27 October 2017, arxiv.org/abs/1710.10196
Karras, Tero, Laine, Samuli. Aila, Timo. 2018. ‘A Style Based
Generator Architecture for Generative Adversarial Networks’, 12
December 2018, https://arxiv.org/pdf/1812.04948.pdf
King, Rochelle; Churchill, Elizabeth; Tan, Caitlin. 2017. Designing
with Data, Sebastopol, CA: O’Reilly
Kurzweil, Ray. 2012. How to Create a Mind: The Secret of Human
Thought Revealed, NY: Penguin
Leach, Neil. 1999. Millennium Culture. London: Ellipsis
Leach, Neil. 2018. ‘We Have Never Been Digital’ in Phillip
Anzalone, Marcella del Signore, Andrew Wit (eds.), ACADIA 2018:
Recalibration: On Imprecision and Infidelity, Proceedings of the
38th Annual Conference of the Association for Computer Aided
Design in Architecture, Universidad Iberoamericana, Mexico.
Leach, Neil. 2019. “The Second Copernican Revolution,” in New
Geographies 11: Extraterrestrial. Edited by Jeffrey S Nesbit and
Guy Trangos. Cambridge, MA and Barcelona: Harvard Graduate
School of Design and Actar Publishers.
Mordvintsev, Alexander; Olah, Christopher, Tyka, Mike. 2015.
‘Inceptionism: Going Deeper into Neural Networks.’
https://web.archive.org/web/20150703064823/http://
googleresearch.blogspot.co.uk/2015/06/inceptionism-go-
ing-deeper-into-neural.html
Nayak, Mishak. 2018. ‘Deep Convolutional Generative Adversarial
Networks (DCGANs). Data Driven Investor. https://medium.com/
datadriveninvestor/deep-convolutional-generative-adversari-
al-networks-dcgans-3176238b5a3d
Rajagopal, Anand. 2018. “The Rise of AI and Machine Learning in
Construction.” https://www.autodesk.com/autodesk-university/
article/Rise-AI-and-Machine-Learning-Construction-2018
Roxborough, Scott. 2018. ‘Rutger Hauer on 'Blade Runner
2049' and Why Films Today "Lack Balls"’ Hollywood Reporter,
February 18 2018, hollywoodreporter.com/heat-vision/
rutger-hauer-blade-runner-2049-why-films-today-lack-
balls-1085827
Sacks, Oliver. 2012. Hallucinations. New York: Knopf/Picador
Seth, Anil. 2017. Your Brain Hallucinates Your Conscious Reality
(TED Talk). https://www.ted.com/talks/anil_seth_how_your_
brain_hallucinates_your_conscious_reality?language=en
Sinapayen, Lana. 2018. ‘Sophia the Robot, More Marketing
Machine than AI Marvel.’ Skynet Today, November 20, 2018. https://
www.skynettoday.com/briefs/sophia
Susskind, Richard; Susskind, Daniel. 2017. The Future of the
Professions: How Technology Will Transform the Work of Human
Experts. Oxford: Oxford UP
Terzidis, Kostas. 2006. Algorithmic Architecture. London: Routledge
Terzidis, Kostas. 2014. Permutation Design. London: Routledge
Urban Dictionary, https://www.urbandictionary.com/define.
php?term=xkool
Watanabe, Makoto Sei. 2017. ‘Can AI Make Designs?’ in Leach, Neil,
Yuan; Philip (eds.), Computational Design. Shanghai: Tongji UP.
[Conference Paper] Cremers, Jan. 2011. “Energy Saving Design
of Membrane Building Envelope." In International Conference
on Textile Composites and Inflatable Structures: Structural
Membranes 2011, edited by E. Oñate, B. Kröplin and K.-U.Bletzinger,
148–155. Barcelona: CIMNE.
IMAGE CREDITSFigure 1: © Refik Andadol, 2019
Figure 2: © Bernard Goldbach, Wiki Commons, 2008
Figure 3: © Martin Thoma, 2015
Figure 4: © Obvious, 2018
Figures 5-6: © Neil Leach, images generated using StyleGAN
website, www.thispersondoesnotexist.com
Figures 7-8: © Refik Andadol, 2019
Figures 9-14: © Wanyu He, 2019
Figure 15: © Refik Andadol, 2018