Mental Work is an exhibition and large-scale participatory neuroscientific thought experiment. Executed in collaboration between the artist Jonathon Keats and neuroengineer José Millán, the project employs brain-machine interfaces (BMI) to explore philosophical questions surrounding the Industrial Revolution, society’s connection to emerging technologies and the future of work.
Potential labourers sign up for a 90-minute session and, upon arrival at the factory, are equipped with a dry, wireless EEG helmet with 24 electrodes. After a short training phase to learn the technique of modulating brain activity, workers are led onto the factory floor where three chromed mechanical sculptures await the mind-machine connection. In order to engage these devices, the worker must follow instructions on a tablet and reproduce the mental activity they practised during the training phase: imagining the muscle movements of opening and closing their hand to activate their motor cortex. Machine-learning algorithms interpret these electrical frequencies and their points of emanation to find brain patterns specific to individual participants.
As the workers navigate their assignment, they experience a progression from machine to machine in terms of both the mechanical intricacy of the sculptures and the design complexity of their respective interfaces – a narrative arc that begins with the awe and unfamiliarity of mind control, proceeds through a deeper understanding of its underlying mechanisms, and finishes with the revelation that no interface is ever fully complete. The first machine is a simple slider-crank mechanism activated by a piston, where a worker sends either a binary or continuous command to engage the sculpture. The second machine features a gearing system, which can be controlled in tandem with a co-worker who manipulates an internal parameter – the partner’s command threshold for successfully activating the device. The final and most complex machine introduces a new type of brain signal and comprises gears, a linkage system and two pistons – each of which is activated by a different brain signal selected by the co-worker.
In this way, Mental Work is designed to explore both the excitement of moving machinery with one’s mind, as well as the current limitations of BMI technology. Unlike most artistic expressions or technological demonstrations that employ passive EEG technology – a field in which it is currently near impossible to distinguish the signal output of a human brain from that of a wet sponge – Mental Work’s brain-machine interface mirrors the parameters used in contemporary neuroscience laboratories where users must consciously engage voluntary modulations of their neurological activity. It is exhausting work.
The tasks are not immediately accessible to everyone, although around 70 per cent of participants have been successful to date. Each of the three machines poses different practical and philosophical challenges. If the worker discovers the right strategy to send a strong enough signal, however, the brain-machine interface loop is completed and the machines glide into motion.
It is simultaneously a humbling and empowering experience. Less than a month before the exhibition’s opening, after five years of working on the project with more than two-dozen collaborators, I activated the first machine – it was an otherworldly sensation. Over the three months that Mental Work was operational at EPFL, my performance fluctuated from day to day, but the strangeness faded away, leaving comfortably intuitive control in its place: I had done this type of apprenticeship before. Whether learning to drive a car, navigating the intricacies of a new language or sending a signal to a machine, the brain’s capacity to adapt and find novel strategies is an infinite well of individual and collective potentiality. Technology is a way to exercise this potential, but if we wilfully ignore that it also holds the danger of unwanted consequences, we are heading blindly into a future that could one day be out of our control.
Jonathon Keats I’ve always been fascinated by the machinery of the Industrial Revolution because you could look at it and essentially understand how it works. Those mechanisms had a physicality to them – they emulated what we do with our muscles, which meant that the Industrial Revolution was beautifully instantiated in its machines. As I began to think about this project, and the cognitive revolution we are currently facing through artificial intelligence and brain-computer interfaces, I immediately thought back to the Industrial Revolution. My idea was that taking the technology of the present and looping it back to that period – into machines which we experience through their motion – might provide a way of examining the electronics and computers that are driving what our future may become.
José Millán The machines of the Industrial Revolution amplified or augmented our manual capabilities. Today we have the opportunity to augment our cognitive capabilities as well. Also, the Industrial Revolution didn’t give autonomy to machines – someone had to operate them – whereas many contemporary machines are completely autonomous, which is where responsibility starts coming into the equation: do we want amplifications of our cognitive or mental processes going on without our intervention? In the case of Mental Work, we’re asking our workers to initiate voluntary, self-generated processes to operate the machines. We’re not exploiting any kind of automatism within the brain, which we could have done by using unconscious processes to drive the machines. Making an effort to engage something costs energy and time. If we exploited less conscious processes our responses would be faster but, as with everything in life, there is no free lunch: you delegate too much, you lose too much. It is our responsibility when designing things for the public to make our choices transparent, because sooner or later you might wonder about the whys and why nots.
Jonathon Exactly – there is always a question of who is doing the delegation. Who is making those decisions? I think that is where an exhibition – undertaken before these technologies are implemented in the real world – can open up territory that is all too infrequently addressed. These far futures can make us more sensitive to near futures and the present, and they can also augment our ability to interrogate, question and navigate present relationships. Look at the degree to which smartphones are reshaping who and what we are. They slipped into our
lives in such a way that we didn’t reflect on what they did and who we became as a result; what we became when we had the total memory and knowledge of the internet – or lack thereof – on our bodies. Coming into Mental Work and having this almost sci-fi experience of operating one of these machines brings you back to your cyborg relationship with your phone.
José We don’t have all the answers. This area of technology will require new ethical regulations because if our cognitive capabilities are suddenly amplified, they will become apparent to everybody. What happens to privacy in that case? Until now, we could keep our mental processes secret, whereas we are now opening those up. I don’t have the solution to that problem, but these are the kind of questions that we want to generate. There are three pillars in Mental Work. One is philosophical, another artistic and the third scientific. We look at this experience as a large-scale experiment that gives us an unprecedented opportunity to recruit thousands of potential subjects under relatively well-controlled conditions. Analysing the data we collect will provide opportunities to improve the design of brain-machine interfaces. For example, I’ve been told that some people are very good at mental work straight away. Why is that? What kind of characteristics do those people have? We also hope that there will be many people who will start with a certain level of brain performance and, as they move from machine to machine, experience a better level of control. What is the process behind that improvement and how can we design better algorithms to follow the mental skills the workers are developing? Conversely, what happens for mental workers who, despite all their efforts, have a limited feeling of control? You need to find the most appropriate algorithm to fit your individual brain, such that you feel that this machinery is not something that comes over you, but something which follows you.
Michael Mitchell One of the things I found most exciting was being able to take a technology from the clinical setting of a laboratory and bring it to the public before it was offered by any of the commercial giants such as Google or Facebook, who might have other designs for it. We can create something abstract and philosophical that allows people to think about what kind of future they would like to see this technology be developed in. I’ve been a Mental Worker several times this week and sometimes you’re good at it and sometimes you’re not, but it does always provoke something. It makes you imagine a seamless brain-machine interaction with the world around you. I’ve gone home, for instance, and imagined turning the lights off or controlling my environment with my mind. If we do that frequently enough through modulating brain activity, we might reach a point where we don’t even realise we’re interacting with the world through this interface. Is that telling us that our relationship with our body isn’t as special as we have imagined it to be? Up until now, we have always thought that our body was the only thing with which we could interact without an intermediary interface – the only thing we can relate to on that very intimate level. So how does the potential of these brain-machine interfaces and the hyperconnected world change that relationship? It raises very profound questions about who we are and how we define ourselves.
Jonathon In the past, we had stories as mechanisms for understanding ourselves – for speculating on who and what we are, and what we might become. Various art forms have developed around that, ranging from plays, to movies, to dance. Here, we’re speaking about something that is a little bit different, because of the fact that it’s a new human-machine hybrid that is being made on the factory floor every time somebody walks in. Rather than standing outside of some possible future, and observing that possible future play out in the form of a movie about another person, you’re becoming that hypothetical being, if only for a short period of time. You’re experiencing it in some internal way that genuinely changes who and what you perceive yourself to be. To me, that is less within the realm of amplification of abilities than it is about what happens when your experience of the world is shifted, modified or undermined. Think about another revolution that happened before the Industrial Revolution: the Copernican Revolution. That was, on one hand, a very humbling moment when we found that the earth was not at the centre of things. On the other hand, we realised that by virtue of this mediocrity of who, what and where we are in the solar system, anything we observe locally has significance throughout the universe. Suddenly, we could derive universal laws based on experiments done in the here and now. That’s extraordinarily empowering. The process of being humbled by the cosmos can also be a process of finding some other aspect of yourself that is ultimately greater than your self-centred individuality: a communal self that is essential to mutual tolerance and collective decision-making in a democratic society. Even if we are only humbled by the Mental Work machines, that may be profound enough. We desperately need humility. The degree of hubris with which we’re operating in the world today is absolutely toxic.
Michael There’s a certain contradiction to some of the advertising that we put out for the exhibition in Lausanne and on the web. We had slogans like “Stop Machine Domination” and “Join the Cognitive Revolution”, while at the same time the exhibit has a very capitalistic element: we want you to come in because we want your data, and your data has value. So you have this communist, Red revolution discourse on one side, and then you come in here and find, “Oh, they cheated and now I’m just going to give my data away.” That contradiction is part of the society we live in. In a data-driven society, technology companies are making products that continually harvest our data, and I think as a whole we’re not really taking into consideration the everyday “proletariat” and how they’re going to be affected by these advances in technology. There is a collective mentality among companies and many people which says that “The more data, any data that can feed AI, the better.” The technocratic class want people to participate as much as possible in the data economy, but we’re not necessarily giving them anything back for it. That is the underlying contradiction that our promotional campaign is trying to evoke.
Jonathon That rhetoric is completely consistent with the way in which most capitalist companies go about marketing themselves. Think about how Facebook purports to make the whole world one big community. The propaganda is usually disingenuous, and often contrary to what is really being achieved. So, by self-consciously mimicking corporate propaganda, there’s a socio-political dimension to Mental Work. On first encounter, it might be categorised as sci-art, in the sense of being a hybrid art-and‑science project. But I think sci-art is problematic as conventionally practised because it’s too strictly academic and often also lopsided. Sci-art all too frequently boils down to artists subserviently illustrating what the scientist is achieving, would like to achieve, or would like others to believe is being achieved. It’s just hyped-up data visualisation.
Michael The fact that these machines leave enough space for us to be able to think about the type of future we want to have with this technology means that they’re a good example of the convergence of art and science. We did an event, for instance, in which an actor read Etienne Bonnet de Condillac’s Traité des sensations, an 18th-century text within the Lockean tradition of empiricism. For empiricists, the world around you can only be described and experienced through your senses. This text describes, in a phenomenological way, the different experiences that you can have through smell, sight and hearing, and how you develop your personality and identity through memory of these. I thought that story had an interesting resonance with what we are doing here. These machines allow the general public to experience another capacity and go beyond the five senses that are tied to the physicality of our world. We can project ourselves into a machine using only our brain activity, which goes beyond anything that an empirical philosopher of the 18th century could have imagined.
Jonathon Which is not to say that our exhibition would have been completely unfamiliar to an 18th-century visitor. One of the inspirations for Mental Work’s machines in terms of their design was the Industrial Revolution. I started with 507 Mechanical Movements, a catalogue published in the 1880s showing many of the machines designed during the Industrial Revolution, from which we chose a few that were modified to operate in a more sculptural way. I based the designs on slider cranks: mechanisms that undertake calculations through their movement and geometry. These were the first mechanical computers and so they connect historically to the computers of the present.
Michael These slider cranks were used to help calculate ballistic trajectories so as to find better ways to eliminate the enemy. After the Industrial Revolution came industrialised warfare and one of the bloodiest centuries in history. We’re now seeing the same dangers surrounding warfare with artificial intelligence.
Jonathon Most technology has at least some military use, historically and in the present, and artificial intelligence is one example. Exploring military history was not really at the front of my mind while developing Mental Work, but it does seem in retrospect that the exhibition can provide a way to reflect on the power inherent in technology and the responsibility that we have in terms of implementing it and determining who and what we leverage it against. That is also a point that needs to be considered as we confront the Cognitive Revolution if we’re to find our way to a better future than the future wrought by the Industrial Revolution. Any technology can either dehumanise us or make us more human by enhancing our abilities and focusing our bodies and minds. The hand axe, for instance, really made us who and what we are today, but there are many ways in which technology can get out of hand and become a controlling factor. Once again, the smartphone comes to mind.
José We are already in a realm where artificial intelligence is taking up more and more space in our daily lives. Now, imagine that we are capable of replicating all our brain and cognitive processes – one day those machines will be here, but will we allow them to become independent? I am not afraid that they will become superhumans, because our brain is only what it is because of the bodies that we have, with all of the body’s attendant limitations. We make a number of decisions in determining what form a machine takes, so what I am putting on the table is that it is up to us what kind of cognitive revolution lies ahead of humankind. During the Industrial Revolution, everything was driven by money – who can build the biggest machine in order to produce more and faster than anybody else – and we now know what that brought: many good things, but also very many bad things. Now, we face very real threats, so we need to design not only with the bright side of the coin in mind, but with the dark side of the moon too.
Jonathon When designing the machines, I was thinking about the long history of the orrery, an instrument which was a model of the universe as it was understood in the 1700s and 1800s. Orreries not only allowed for scientific study, but also for philosophical contemplation of the universe as a clockwork mechanism which had significant religious implications. I wanted to borrow from this tradition of the philosophical instrument, and that purely mental interaction where a machine is there to help augment your thought processes. It seems that an exhibition space can be reflective, but only if the exhibited mechanism is detached from some sort of external purpose or practicalities. The space has to become a self-contained world that operates according to its own internal logic and has its own self-reinforcing aesthetic. That is one of the reasons why we chromed these machines. Having a set of machines that are immaculate allows for a relationship with them that is not industrial – you wouldn’t chrome these machines if you were using them to operate a mill, for instance. By chroming them, the physical work that they can do is undercut by their immaculate finish: this perfection can only exist as long as we are in the space of doing philosophy, as opposed to milling wood. This allows for a relationship that is not industrial and which is ultimately otherworldly. It doesn’t belong to any particular place or time, and as a result it allows for that kind of movement back and forth between the Industrial Revolution and the Cognitive Revolution, as well as an extrapolation forward in time.
José In my lab, we don’t put too much effort into the final product – if you produce things that are too rigid, you are constraining, not helping, research. Having said that, something I am particularly proud of is the fact that we put a lot of attention to detail into how the subjects of the research in our lab – people with physical disabilities – interact with systems that allow them to regain control of parts of their body. When they interact with the final machine, it is seamless. When you’re in front of these Mental Work machines, you feel like you’re reflecting upon them, and they’re conveying back the mental effort you’re putting in. It’s a kind of mirror, which is a powerful image because a basic principle for acquiring brain-control skills is that you need feedback from whatever activity you are generating such that you can learn to moderate it. Our brain is the only organ in our body that doesn’t have any sensor – we can feel all of our organs except the brain – and so this exhibition becomes the perfect sensory space in which to reflect on its activity. As Jonathon has said, these are abstract machines, detached from what they might have been initially designed for. That’s important because whenever we go public with our brain-control devices I worry that there may be false expectations from people who need such a system because of their physical disabilities. Detaching the machines from practicality is a way to say, “Look, this is where brain control technology is at the moment. Don’t be fooled by any news that you hear to the contrary.”
Jonathon The project is designed to problematise as much as it is to resolve. You start out with a case that offers a relatively straightforward relationship – you are mentally turning on and off a machine. Only when you come to subsequent machines do you start to realise the degree to which this is reliant on a whole backend system – a supervisor is capable of manipulating the algorithm for instance, or can switch you back and forth between different modes of actuation. I hope that’s a bit of a traumatic experience, in the sense of recognising all the ways in which interfaces hide so many of the decisions that have been made by others – the lack of actual control. If you are not going to make those decisions for yourself, you at least need to be aware of them. For instance, there is a whole question of how artificial intelligence may be biased in ways that disadvantage women, because women are less represented than men in the development of the technology. So one of the challenges is to bring more women into the technological process, but another is to make it such that the technology is accountable and can be interrogated. Part of what we’re trying to achieve here is a system that is not only capable of being interrogated, but which inherently must be interrogated and which interrogates you in the process. There’s an increasing amount of interest in trying to design possible futures. This practice of speculative design has especially been taken up by consultancies that work with companies, helping businesses position themselves for the near future. In effect, these designers are mercenary visual futurists. What is interesting with Mental Work is that we are putting forth a fiction in the form of a factory that everyone can engage in out of their own interest. It is a total fiction in which all aspects are put together, visually and narratively, to create a seamless internal experience for the audience. What is even more unusual is that this is not happening at the stage of a technology that is about to be available through Amazon, but rather something that is still in the lab. The process of reflection that would ordinarily take place when a speculative designer has been hired by a company is to develop some near-term way in which to make a product more appealing or even more functional. Here we are looking at a much more distant future. We’re exposing the earliest technological instantiations of the science, and even the science itself, to speculative design, and, in so doing, trying to find the best possible way forward.
José People use the term “speculative design”, but let me use another term, “disruptive design”, in the sense of confronting people with the limits of technology. If people come out of this experiment thinking that the technology is already there and that everything works fine, we have failed. We need to make people understand that they have choices around this technology. The word “responsibility” is very central for me. The human must be at the centre of the equation, which doesn’t mean that you need to be constantly aware and conscious of every single thing that is going on. We are not conscious of every single process in our bodies – thank God – because we would be drunk on the weight of information. Our body has evolved to shield us from many low-level processes, but we can take back control if we want. The important thing is that we always have the reins and that’s key to the vision of the Cognitive Revolution.
Jonathon Where we have the ability to project possible futures, we need to act on that knowledge. That is part of human evolution and to not act would be to abdicate something essential in a way that could be highly detrimental to us and even more detrimental to those who come after us. The Cognitive Revolution can be different from the Industrial Revolution by fostering a collective self-discovery of who and what we will and want to become. Entering Mental Work is a way to become more informed as a citizen and more prepared. You position yourself individually and collectively in relation to that future, without feeling that it is inevitable – because it isn’t. Brain-computer interfaces can be tools for reflection because of the feedback loops that are inherent in this kind of mental and physical interaction, but given the plasticity of the brain and the way in which the algorithms involved can adapt, they can also potentially result in change over evolutionary time. That is to say, these interfaces can be a means by which we affect how we evolve as a species.