AI & Scientific Discovery

When I hear the Silicon Valley elite wax poetic about the potential of AI, I am struck by a glaring flaw in their reasoning. They make grandiose claims about what AI will do, from solving the climate crisis and eradicating poverty to curing cancer. These promises ring hollow and I suspect that deep down most people can sense the malodorous bullshit emanating from such statements. The idea that AI alone will be our savior, magically resolving the most complex and deeply rooted challenges facing humanity, is not only unrealistic but also insulting to our collective intelligence.

If we want AI to benefit humanity, we need more than such grand promises. We need leaders who communicate a realistic narrative about the technology's genuine potential to make groundbreaking discoveries. People need concrete examples of AI's ability to do the important—and superhuman work—of navigating vast data spaces, uncovering novel insights, and collaborating with human expertise. Only then will people have a realistic perspective on how AI can transform scientific discovery.

To grasp the vision for AI’s transformative impact on scientific discovery and creativity, we first need a mental model for the concept of combinatorial space or the space of possibilities.

Imagine a wilderness with its high peaks, deep valleys, trees, and deserts. As a human, you search this space by following well-trodden paths. At a human scale it’s impossible to efficiently move across the landscape without using paths taken by those before you. This landscape is also full of places where humans can’t go, where the world is so hostile or foreign that it is essentially unsearchable. We are cut off from its contents.

In science, traditional human search strategies are like wandering through a wilderness with limited visibility, relying on intuition and serendipity. In the pursuit of new insights, we can choose to follow the well-worn paths trodden by others or venture along the trails revealed by low-dimensional data. AI, in contrast, can take in the whole landscape, quickly and effectively exploring the vast range of possible combinations. That's what gives AI the power to reveal hidden paths and uncover new territory.

A combinatorial space represents the set of all possible configurations or solutions to a given problem. In fields like protein folding and material science, the number of potential combinations is staggeringly vast—far beyond the scope of what human researchers can explore unaided. The laws of physics set the limits of what's possible in our universe. Life as we know it has followed a particular path, shaped by its history, but when it comes to materials, there's a huge untapped space of potential waiting to be explored.

With AI, it's as if we can instantaneously scan the entire wilderness or follow a hyper-dimensional trail leading directly to the most promising discoveries. It can identify promising avenues for further exploration and uncover novel structures, patterns, and combinations that might otherwise remain hidden. In this sense, AI acts as a powerful search tool, extending the reach of human cognition well beyond what we can do on our own. In our landscape analogy, AI can cover inhospitable and hostile territory. It could search every branch of every tree and hand off to a human who can find the one leaf that is meaningful.

"Meaningful" is fluid and subjective. Meaning might be found in causal relationships, where we want to understand the web of cause and effect that might explain more of how the world works. It can also emerge from simplicity and clarity, as exemplified by Occam's Razor, which suggests that the simplest explanation is often the most meaningful. Meaning often emerges from anomalies—those puzzling observations that don't quite fit the prevailing paradigm. Meaningful can be something that's interesting, sparks curiosity, has a phenomenological quality as intuition often does. Meaningful in the mind of an expert has an aesthetic quality— a thing of beauty, curiosity, imagination, or awe.

With this metaphor it should be clear that, in science, AI is not a replacement for human creativity and insight. By working with AI systems, researchers can leverage the strengths of both human and machine intelligence, finding the sweet spot where human creativity and computational power intersect. The hyperspace of AI-cognition reveals the complexity of the universe and provides an entirely new information-based system for exploring the space of possibles. To me, this is the story we need to hear—one in which discoveries aren’t fantastical scenarios that ignore the complexity of scientific progress but instead where the collective ability to discover is hugely amplified by machines that have different capabilities.

A better example of the power of AI is to describe how it can simulate and analyze thousands of potential material combinations to identify those with desirable properties, such as high strength or thermal resistance. By predicting how different atoms will bond and behave under various conditions, AI helps scientists pinpoint promising new materials without the need for time-consuming experiments and trials.

The key ideas behind large language models, which are great at predicting the next word in a sequence, can be cleverly tweaked and used for making new scientific discoveries in all sorts of different fields. In this framework, the "tokens" become elements of scientific data—be they chemical compounds, material properties, or genetic sequences.

Scientists can employ this kind of AI model to predict and design new material structures. By training the model on extensive datasets of material properties and their corresponding molecular structures, the AI can learn to suggest new materials that might exhibit specific characteristics, such as conductivity or flexibility. This technology essentially transforms the way materials are discovered, moving from traditional experimental methods to generative innovations.

As AI is integrated into the process of scientific discovery, we have to remember that there is a critical distinction between novelty and creativity. True creativity encompasses more than novelty, a discovery has to also be valuable. Creativity is concerned with impact, which is a concept that is embedded in our complex human social systems.

A new artifact is only valuable when humans—in the context of culture—decide it is valuable. This is "Capital C" creativity, as Mihaly Csikszentmihalyi defined it. And while AI can undoubtedly generate novel solutions and insights, true creativity only happens in relation to the social and cultural context in which discoveries are made. This is the essence of peer review. Is the discovery truly novel? Is it valuable? Does it represent new knowledge based on the process of peer review? Is it recognized as being, not just creative, but Creative?

Here's a simpler way to think about AI: it's not just a tool, but a tool for creativity in all sorts of fields. AI provides a more complex and dynamic environment in which novel ideas can emerge. By exposing scientists to a wider range of possibilities and configurations, AI can help to break down the mental barriers that often constrain human thinking, opening up new avenues for creative exploration.

The use of AI doesn’t have to be just about a landscape of data, it can be put to use in the landscape of domains. Many academics complain of the narrow space of their discipline or of how the distinct language of specialization can put walls up to cross-discipline collaboration. Large language models can help break down these walls by translating between disciplines and enabling young scientists to develop their specialist expertise but to also be fluent in other fields, opening up more possibilities for cross-pollination of ideas, interdisciplinary insights, and for new fields of scholarship to emerge.

AI can also be strategically employed in science to address some of today’s key challenges, including reproducibility, misinformation, and how publication incentives affect the visibility of negative results. AI has the potential to uncover "sleeping beauties"—valuable insights and discoveries that have been overlooked or forgotten in the vast sea of scientific literature. By reducing the communication overhead that often plagues large-scale research efforts, AI can help to create a better system for sharing knowledge and building upon each other's work. AI can amplify the collective intelligence of the scientific community, enabling us to tackle ever more complex and challenging problems whilst holding true to the foundations of the scientific method.

Scientific complexity is ever-expanding. Each new discovery, theory, and measurement technique propels a continuous cycle of advancement. Incomplete theories are supplanted. AI can genuinely provide many new minds for ours in the ever-complexifying world of scientific discovery.