The limits of (con)science

Mr. de Rachewiltz, the Manhattan Project demonstrated the terrible destructive power that science can unleash with the development of the atomic bomb. How far should research go?  

Michael de Rachewiltz: It really depends on what is happening in the present and what kind of context researchers are confronted with. The limits of research are heavily influenced by the current societal or political situation. Looking at Robert Oppenheimer and the Manhattan Project, which is often considered the starting point for modern ethics of science, the development of the atomic bomb was justified by the existential threat posed by the Nazis. In ethics of science, we talk about internal and external responsibility. The internal refers to the responsibility within the research process itself, including considerations about methodology and adherence to moral and legal guidelines, such as in animal testing or testing with human subjects. The external responsibility of science refers to the possible impacts of research findings on society. This external responsibility can be harder to define, particularly in basic research where the application possibilities of results are often unclear. 

Just as it is in feasibility studies, is it first about seeing what might be possible? 

de Rachewiltz: Exactly. A classic example is nuclear fission: This discovery ultimately led to the atomic bomb. With far-reaching consequences for humanity. For instance, we know that Otto Hahn, who discovered nuclear fission with Lise Meitner and others, felt partially responsible for the deaths in Hiroshima and Nagasaki. He and Meitner both advocated for peaceful uses of nuclear fission and international understanding throughout their lives. Hahn had also conducted research on chemical warfare gases during World War I and trained soldiers in handling them. He was well aware of the potential impacts of scientific discoveries. Responsibility and foresight do not come automatically. Werner Heisenberg, the founder of quantum mechanics and a leading figure in the German uranium project felt, perhaps for reasons of pride, that he should have been the one to invent the atomic bomb – even if it were for the Nazis.  

At the Manhattan Project, about 160,000 people were involved. Many did not know exactly what they were working on. In such a case, where does responsibility begin? 

de Rachewiltz: The Manhattan Project presents a special case. At least the researchers involved clearly understood the goal of basic research: to harness one of the fundamental natural forces and release the previously unimaginable power of the atomic nucleus in the form of nuclear energy. The Manhattan Project is also fascinating because, despite its devastating outcome, it demonstrated what humans are capable of when immense brainpower is combined with existential fear – in this case, the fear that Hitler could have reduced the world to rubble. Equally fascinating is how many small steps were necessary for success. For example, if Enrico Fermi had not succeeded in calculating the critical mass, others would not have been able to continue their work. In this context, each person bears an individual share of responsibility. Much like with climate change, where the responsibility is divided among billions of people, including development and research institutions. 

You approach this debate from a philosophical perspective. What role does philosophy play in the ethics of science?  

de Rachewiltz: In philosophy of science, we are interested in what makes science science: how it functions, whether and why it can produce reliable knowledge. As a subfield of philosophy of science, ethics of science deals with the practice of science. It provides guidance and orientation for moral questions within and outside of research. A classic internal issue would be the misrepresentation of research results. On the other hand, the possible consequences of the CRISPR/Cas method, colloquially known as the ‘gene editors’, would be an external issue. 

And who is responsible for ethics of science now? 

de Rachewiltz: Aside from the commissions at universities or research institutions, hardly anyone is responsible. Although some philosophers try to bring the topic to the public, real discussions often only arise afterward. Dolly, the cloned sheep, is presented to the public – only then is cloning and the ethical concerns about it publicly debated. However, most of the ethical considerations have already been addressed in the fields of medical and bioethics, yet they garnered little interest. We face a similar issue with climate change. Climate ethics is clear, but until the effects are felt personally, not much will change. According to evolutionary psychology, humans – with smoking being the best example – are simply extremely poor at assessing long-term risks that develop gradually. With a bomb, it’s different. This danger is immediately recognizable. Ethics of science thus often has an educational function for the future similar problems, and moral dilemmas. 

You also visit schools to talk about this topic. How do young people react to questions of science and responsibility? 

de Rachewiltz: The events surrounding the Manhattan Project have a high moral and educational value. However, most young people tend to connect the question of scientific responsibility with current issues. For instance, there’s concern about whether they will still be able to find work in the future, especially with the rise of artificial intelligence. Existential fears about the future of humanity or the planet are less commonly discussed. It might be a trivial observation, but science plays a fundamental role in all our lives, including through technology. Also, in particular, young people tend to grapple with the broader question of what contribution science should make to society. 

You mentioned AI. What topics and questions are ethics of science currently grappling with? 

de Rachewiltz: Artificial intelligence is currently one of the most controversial topics being discussed. Apocalyptic scenarios like those we know from science fiction, such as ‘Terminator’, are unlikely, but the use of AI does pose significant risks. These range from distorting public discourse or interfering with political systems to influencing voting behavior. An ethical challenge is the decentralization of AI systems. Regulating large companies is one thing, but when individuals or smaller groups independently develop and use AI, control becomes more difficult. Recently, a mother in the US sued an AI company after her son’s death, claiming a chatbot whispered suicidal thoughts to him. The mathematician Hannah Fry from University College London has already called for a Hippocratic Oath for researchers in mathematics and computer science. However, establishing guidelines in research is challenging due to different ethical standards across the international context. This brings us back to the tension between moral responsibility and the practical pressure that others might do research without such restrictions. Basic research doesn’t always have concrete goals, making it difficult for individual researchers to predict what unintended consequences their work might have. Even in genetic engineering and synthetic biology, we still lack a clear understanding of the potential health effects or risks associated with the commercialization of such technologies. 

Can science set the course for a livable, peaceful future? 

de Rachewiltz: Not all is lost yet. I would like to answer the question with a hopeful ‘yes’. The more I engage with the history of science, the more I am fascinated by what humans can achieve when we set responsible goals and collaborate – not in an anthropocentric sense, but in a holistic one, where we see ourselves as part of the biosphere we all depend on. Science is the greatest adventure of humanity. Ethics of science ultimately acts as an ethical-social guideline for responsibility toward humanity and all life. 

About the Interviewee

Michael de Rachewiltz studied environmental science in North Carolina and philosophy in Innsbruck. He conducts research at the Center for Advanced Studies at Eurac Research in applied ethics, philosophy of science, and the philosophy of mind. In addition, he explores cultural anthropology and transhumanism and was responsible for organizing two futurological conferences focused on energy and the future of humanity in the age of artificial intelligence. Together with historian Josef Prackwieser, he holds workshops and lectures on responsibility and science in secondary schools across South Tyrol.