“This third decade [of the 21st century] is likely to decide our fate. Will we make the digital future better, or will it make us worse? Will it be a place that we can call home?”
Shoshana Zuboff
Professor at Harvard University
Preamble
Technology can be a wonderful tool for humanity, one that has allowed it to overcome many obstacles and afflictions over the centuries. However, we believe that today’s prevailing paradigm of technological development is not entirely aligned with humanity’s interests, and that there is an urgent need for the kind of deep public awareness that leads to a call for hard political measures to regain control. We want to continue to enjoy the advantages technology offers us, without being forced to endure some of the most harmful ramifications it can bring with it, either by design or through lack of regulation.
A substantial part of the digital technology and AI currently being promoted fosters an environment in which human beings are vulnerable, and their freedom and integrity are threatened. They run the risk of having their will conditioned, of not being able to make decisions autonomously and without external interference, of living under a state of surveillance in which intense control is exercised over their lives, or of finding it increasingly difficult to discern the truth. If we allow this model to consolidate and take hold, the world shaped by some of today’s technologies may irreversibly become one in which humanity itself has no place, because the logics at play do not recognise its unique nature and its interests. It could even lead to the very extinction of our species.
Faced with this existential risk to humanity, this MANIFESTO sets out a series of urgent measures to regain control over digital technology and ensure that it remains at the service of human beings, whilst containing, as effectively as possible, the adverse consequences it presents.
A Manifesto
Regaining Control Over Digital Technology
We stand at a historic turning point, where the power of technology is growing exponentially and presenting a challenge unprecedented in the history of our species.
The accelerated technological deployment we are witnessing is taking place with little democratic control. As the short-term benefits of many technological innovations tend to be more easily perceptible than the risks they entail, the hasty adoption of technological applications — by individuals, professionals, businesses, administrations, educational institutions — is a temptation that is difficult to resist in the absence of clear barriers and protocols that evaluate their implications, and ensure that they respect fundamental rights and do not contravene our general interest.
Given this context, we reject the view that we are facing an inexorable technological determinism of which we can only be spectators. On the contrary, we are convinced that we must act to agree on a framework that can be applied to reduce technological control over humans and in which technological development is at the service of humanity and not the other way around.
WHAT WE OBSERVE
A disturbing evolution
Irrespective of the advantages that today's technological development offers humanity in terms of speed and efficiency, we are deeply concerned by three dynamics by which technology is likely to fundamentally affect human beings:
The growing asymmetry between the capabilities of human beings and those of technology means that the latter can be used by individuals or by public or private organisations, to exert control over others, whether for commercial, security, or other purposes, reducing or annihilating the individual's freedom, free will, and judgement.
Our individual and collective dependence on digital infrastructures constitutes a vulnerability that can be exploited for malicious purposes – criminal, terrorist or ideological – paralysing, destroying or damaging elements that are critical to our existence.
Increased autonomy of technology means less human control. Whereas until recently machines were largely directed by humans, their growing complexity and autonomy means that understanding how they work tends to escape even their own creators, and machines are increasingly issuing orders to humans.
These features of technological evolution are already profoundly affecting individuals and society. Among other things, we observe:
-
A profound deterioration in both attention span and mental health among the general population – and particularly young people – accompanied by an increased sense of loneliness and isolation. [1]
-
An alarming unlearning of certain functions inherent to human beings (such as memory, orientation, reasoning, or critical thinking), qualitatively modifying their nature and increasing their dependence on technology. [2]
-
The distortion of key electoral processes through disinformation campaigns fuelled by large digital platforms, encouraging intense social and political polarisation. [3]
-
A significant increase in the control exercised by authoritarian states over their subjects through a sophisticated surveillance network based on the ever more comprehensive collection of personal information processed with the help of artificial intelligence. [4]
-
A tendency, also on the part of democratic states, to accumulate and excessively exploit citizens' personal data and violate their privacy through, for example, the abuse of facial recognition. [5]
-
A high concentration in the technology sector, meaning a very small number of companies have a disproportionate amount of power that allows them to intensify the influence they exert over the lives of citizens, conditioning their behaviour, beliefs, and decisions. These companies also hold a dominant position in relation to other companies, and sometimes to nation states themselves. [6]
-
Cyber-attacks perpetrated against critical infrastructures – such as hospitals or electricity distribution networks – highlighting the vulnerability of a society in which all essential systems are connected, coupled with the fact that security criteria are not sufficiently prioritised in technological design and deployment. [7]
-
An increasingly significant environmental impact resulting from the indiscriminate deployment and intensified use of technology: the harmful extraction of rare-earth elements for the production of devices, an exponential growth in the number of satellites, an increase in digital greenhouse gas emissions, the intensive mining of cryptocurrencies, etc. [8]
These seemingly disparate phenomena are, in fact closely related. They are the flip side of our uncontrolled relationship with technology, the full potential of which is unknown.
1
2
3
WHAT WE WANT TO AVOID
The principles behind our proposed action
In an environment in which human existence in all its facets is increasingly conditioned by technology, we refuse to accept:
-
A world ruled by machines, in which technology reaches a level of autonomy so high that it completely escapes human understanding and can lead us in directions that we have not chosen.
-
That criteria based exclusively on optimisation reduce the perimeter of human activities in such a way that our species is not only marginalised but also deprived of what makes life meaningful.
-
That trading on human will and conscience is permitted, making use of personal data and predictive models that limit or disable a person’s ability to discern what they want, and thereby weaken or annihilate their freedom.
-
That private organisations exclusively own technologies that give them a dominant position which concentrates power, allowing them to control or condition a very substantial part of people's lives, a trend which is difficult to reverse.
-
That it is permitted to discriminate or classify individuals or groups of humans and give or deny them access to certain resources on the grounds of criteria established by machines, based on the exploitation of data that are not directly related to the purpose for which this categorisation is established, leading to arbitrary or opaque results that cannot be challenged or appealed.
-
That the artificial character of a machine cannot be identified, generating confusion between the human and the non-human. That technologies are used which, by design, present artificial human representations as truthful.
-
States using technology to exert excessive and pervasive control over their citizens and in particular abusing facial recognition or technologies that do not guarantee anonymity in public spaces; or asserting the right to trace movements, transactions, or any other aspect of human activity.
-
That systemic technologies are deployed without sufficient security precautions and may therefore create large-scale vulnerability.
-
An increasingly dualistic humanity, in which the choice of one part of the population to merge with the machine would de facto force the rest of humanity to either do the same, or to subject itself to the power of the former.
Ultimately, we believe that we must ensure an environment that respects the unique nature of human beings and minimises the potential for technology to be used to alienate them and undermine their freedom.
WHAT WE PROPOSE
An appeal to the authorities and the public
This manifesto does not in any way intend to oppose technological development per se, and indeed we acknowledge its many advantages, but rather to challenge political authorities and civil society on the price humanity is paying and the risks it assumes if it continues to accept a paradigm of technological deployment without discernment and democratic control, and to encourage urgent action to minimise this cost.
In order to preserve the freedom and dignity of individuals as key values in a context of growing asymmetry between technology and human beings, democracies must profoundly reinvent their repertoire of actions and defend these rights more fiercely. The efforts of some states or regions, both in terms of data protection and in the drafting of charters or declarations on the protection of digital rights (Spanish Charter 2021, European Declaration 2023, or the Ibero- American Charter of March 2023), are commendable, but they must be intensified and implemented on a much larger scale.
We are aware that this is a real challenge for political systems that tend to trust the individual to regulate issues considered private. We are, however, faced with technologies that have the collateral effect of overriding individual freedom and affecting what makes us human. We also understand that economic and geostrategic balances largely depend on technological competitiveness, which discourages any measure that could be perceived as a limitation to technological development. Therefore, this existential challenge demands unprecedented solutions, beyond soft recommendations.
Whatever the level of political action, a paradigm shift will only occur if there is a real mobilisation of citizens that exerts the necessary pressure to put this issue at the top of the political agenda. For the moment, public opinion is more sensitive to the short-term advantages of technological development than to its risks, which are often unknown. There is a need for debate in parliament and civil society to weigh up the risks and benefits of the prevailing technological paradigm and to jointly define the values that should guide and frame technological development.
We propose exploring a broad spectrum of measures to respond to the challenges we face, including:
1. Legally binding regulatory measures, such as:
-
The prohibition of micro-targeted advertising.
-
The constitutional and transnational protection of neuro-rights (prohibition of accessing our neural activity and collecting its data with the possibility of interfering with our mental activity).
-
An international legislation for the prohibition of autonomous lethal weapons.
-
The pursuit of a "technological de-escalation" in areas where technology has a proven negative collective impact, and its deployment poses a substantial risk to the population.
-
A right to disconnection that guarantees access to services – especially public services – in a non-digital capacity.
-
The obligation of tech companies to provide access to data that allows third parties, within the framework of the law, to autonomously analyse how their platforms and algorithms function.
-
The expansion of protocols aimed at validating the ethical appropriateness of algorithms used by companies and governments.
2. Measures linked to industrial production and marketing, including:
-
The reintroduction and revaluing of “off” functionality in the design of technological applications and devices, allowing users to consciously switch them on and off.
-
Certain disruptive technologies being subject to strict regulatory procedures before they reach the market, as is already the case in sectors such as pharmaceuticals and food.
-
Measures to educate, inform and empower consumers, investors, and professionals so that their demands for technology are aligned with their rights and interests.
3. Measures in the field of research and education, such as:
-
A moratorium on the indiscriminate introduction of technology in schools below a certain age.
-
More training for various age groups on programming and the general functioning of algorithms to reduce the asymmetry between technology and people.
-
Systematic actions aimed at parents/guardians to raise awareness of the risks to which young people are exposed.
-
The promotion of in-depth studies on the effect of specific technologies on the individual and society, particularly in terms of cognitive and behavioural issues, well-being, and freedom.
-
Age-appropriate use of devices, in particular smartphones and other connected devices.
Regulating technological innovation is complex, and together with the environmental crisis it constitutes a fundamental challenge to humanity remaining a free species. The issue is too important to allow the debate to become bogged down by sectarian positions. Mobilisation cannot be delayed as long as has been the case with the environmental movement if we want to avoid reaching a point of no return. This is why a consensus is urgently needed to ensure that prompt and sufficiently effective measures are taken to respond to a challenge of this magnitude.
Let us urgently do what is necessary for humanity to take control of its own destiny and ensure its permanence as a free species.
References
Appendix
[1] Among others, psychologist Jean M. Twenge of the University of San Diego has conducted many studies that
have revealed the extent of the decline in mental health among young people during the 2010s. Together with
Jonathan Haidt of New York University, they have established correlations between smartphone use and this
decline.
See for example:
-
Jean M. Twenge, “The Mental Health Crisis Has Hit Millennials. Why It Happened and Why It's Bad For Democracy”, 25/04/2023.
-
“Smartphones and social media are destroying children's mental health”, Financial Times, 10/03/2023.
See for example:
-
Nicholas Carr's works, from The Shallows. What the Internet Did To Our Brains to "Is Google Making Us Stupid?” The Atlantic, 08/2008.
-
Will Conaway, "Technology Is On The Rise, While IQ Is On The Decline", Forbes, 29/04/2020.
-
Adrian F. Ward, Kristen Duke, Ayelet Gneezy, and Maarten W. Bos, "Brain Drain: The Mere Presence of One's Own Smartphone Reduces Available Cognitive Capacity", Journal of the Association for Consumer Research, 04/2017.
-
Kaspersky Lab, "Digital amnesia" means that we don't remember information we think we have at hand, especially on our mobile devices. Eight years ago, half of people could no longer remember their partner's number when in the early 2000s they remembered dozens of numbers by heart. This phenomenon of memory externalisation has since spread to other cognitive faculties, in an increasingly cross-cutting way, now affecting the ability to argue, when constantly resorting to services such as ChatGPT.
-
Investigations by The Observer and The New York Times revealed a few years ago how Cambridge Analyticaprovided Donald Trump's campaign with highly accurate psychometric data on tens of millions of Facebook users, facilitating the delivery of personalised advertising.
More information:
• Kurt Wagner, "Mark Zuckerberg says it's 'crazy' to think fake news stories got Donald Trump elected"., Vox, 11/11/2016.
See for example:
-
Katie Canales & Aaron Mok, "China's 'social credit' system ranks citizens and punishes them with reduced internet speeds and a ban on flying if the Communist Party deems them untrustworthy," Business Insider, 11/28/2023.
-
BBC, "In Your Face: China’s all-seeing state”, 10/12/2017.
[5] From the West, we watch in horror at the deployment of surveillance technologies in China. The omnipresence of smart cameras and biometric controls that foster a society of hyper-control seems dystopian to us. However, we too are giving in to this temptation in our democracies, by means of "temporary experiments". Video surveillance with facial recognition technology opens a Pandora's box that threatens our privacy and anonymity in public spaces. It represents a differential leap towards the systematic tracking of our movements and the control of our lives. Our face has the particularity of being both public (understood as visible, uncovered) and private (because it harnesses a great deal of information about us, especially our identity and our emotions). If AI combined with video surveillance analyses what is hidden behind our face in
the public domain, we lose the main shield that protects our inner selves. If we allow facial recognition to be deployed, not only will we lose the right to move freely without being identified, but also information about us will be accumulated without knowing how it will be used in the future.
More information:
-
Amnesty International called for a ban on the use of facial recognition systems as a form of mass surveillance that amplifies racist policing and threatens the right to demonstrate. Campaign: Your face is being tracked.
[6] See for example:
-
Market capitalisation of the world's largest Internet companies in April 2023 (in billions of US dollars).
-
Almost all of the world's data (99%) travels over submarine cables. 70% of global cable projects are led by Google, Facebook and others, which already manage more than 50% of the world's bandwidth. With 21 submarine cables, 6 of which it owns exclusively, Google is the champion in this field.
-
Bruce Schneier, "When It Comes to Security, We're Back to Feudalism", Wired, 26/11/2012.
[7] See for example:
• "Cyberattack hits major hospital in Spanish city of Barcelona", AP NEWS, 06/03/2023.
[8] See for example:
-
Sarah Griffiths, "Why your internet habits are not as clean as you think", BBC, 06/03/2020.
-
"Bitcoin consumes 'more electricity than Argentina", BBC, 10/02/2021.
The annual electricity consumption generated by bitcoin is equivalent to that of entire countries.
