Generative AI has been hailed as a game-changer in fields as diverse as science, entertainment, and finance. Many are touting its achievements while others are concerned about its drawbacks.
In this article, experts and users explore its pros and cons.
A disruptive technology that will transform science
Babak Mostaghaci is a scholarly publishing expert, editor-in-chief of Advanced Intelligent Systems and a deputy editor of Advanced Materials at Wiley-VCH.
Generative AI models, such as ChatGPT, are powerful tools that can create realistic and coherent texts on various topics and domains. They have the potential to significantly disrupt scientific publishing, both positively and negatively.
On the positive side, generative AI models can democratize scientific writing by enabling researchers from different backgrounds and languages to communicate their findings effectively and efficiently. They can also save time and effort for authors by transforming their results into ready-to-publish manuscripts that follow the standards and conventions of their fields. This could accelerate the pace and quality of scientific activities and discoveries.
On the negative side, generative AI models can also pose serious ethical and practical challenges for scientific publishing. They can enable bad actors to submit fraudulent and non-scientific papers that are difficult to detect and verify (such as the case of paper mills). They can also introduce errors and speculations in the generated manuscripts that may mislead or confuse readers if the authors do not pay attention. Moreover, they can raise questions about the credibility and validity of AI-written reviewers’ reports that may affect the peer review process. Additionally, there are still legal and moral dilemmas about the ownership and attribution of AI-generated texts that may conflict with the existing norms and policies of scientific publishing.
In conclusion, generative AI models are a disruptive technology that will transform science and scientific publishing in the upcoming years. They offer great opportunities and challenges for researchers, publishers, editors, reviewers, and readers. They require careful and responsible use and regulation to ensure their benefits outweigh their risks.
No “mind” here
Judith Donath is a writer, designer and artist whose work examines how new technologies transform the social world. She is author of The Social Machine: Designs for Living Online and faculty associate at the Berkman-Klein Center, Harvard University.
The anthropomorphizing language used to talk about generative AI (genAI) — as well as the language used by various instances of it (e.g., ChatGPT, Bing) — implies consciousness, the presence of an entity with thoughts and feelings. This obscures what these systems actually are and creates a potent and dangerously misleading mythology around them.
Microsoft’s Bing, for example, starts its responses with “Sure, I can suggest…” and often ends with a cheery send-off such as, “I hope this helps you plan your party. Enjoy! 😋” or “I hope this helps you find an app that works for you!” The word “I” implies a self, and the bantering tone conjures up a being with personality in the (actual) mind of the (truly sentient) user.
“It’s hallucinating” is the explanation technologists have given when asked why a genAI produced plausible-seeming but erroneous results. This vivid metaphor further confuses a public whose understanding of the capabilities and very nature of these systems is already cloudy, and adds to the impression that they are strange and potentially menacing beings.
Anthropomorphizing makes these systems highly persuasive — they become entities whose opinions we care about, whose approval we seek. Exacerbated because they can be trained to find the most effective and affecting phrase. Framing AI systems as mysterious yet almost supernaturally intelligent beings discourages people from questioning its omniscience.
In reality, these are huge computational systems that have been trained on the Internet’s vast accumulation of articles, images, arguments, fan-fiction, reviews, and rants. From this stew of phrases and statements, they develop predictive models that can synthesize new material based on the innumerable patterns they have ingested.
They have been trained on material created by billions of conscious human beings. This allows them to generate — at rates far faster than any individual human could, drawing from a range of material far beyond the grasp of any individual human — new material that looks and sounds like it was thought up by a human, or indeed a sort of super-human, fluent in all languages and disciplines.
But — and this is the big key thing — there is no “mind” there. No thoughts, feelings, or desires. Yes, the statements they generate may seem as if there is, but that is because they have been trained on material created by us humans, who wrote expressing our thoughts and feelings. If they are asked about the likelihood of an AI apocalypse or about whether your wife really loves you, their answer may seem chillingly prescient but ultimately it is a statistical prediction based on the prompting question (and perhaps additional context).
If tech leaders choose to acknowledge that the use of anthropomorphic language about and by AI systems is harmful, they could be quite effective at fixing it. But will they? This depends on why they use such language to begin with. Do they want to create more buzz and controversy around these new programs? Is it because entity-like chatbots are more persuasive than neutral query systems? Or does it reflect a deep, perhaps subconscious, wish to be the god-like creators of a truly novel sentient being?
Revolutionizing research or a threat to critical thinking?
Sergei V. Kalinin is a Weston Fulton Professor at the University of Tennesse, Knoxville, following a year as principal scientist at Amazon (special projects) and 20 years at Oak Ridge National Laboratory. His interests include active machine learning in electron and probe microscopy, including physics discovery and atomic fabrication by electron beams.
Over the last year, generative models have taken the scientific world by storm. I started to apply machine learning (ML) methods including shallow neural networks and multivariate statistics in microscopy and experimental physics more than 15 years ago. However, even two years ago I was giving lectures about ML tools in science, with the central idea that machine learning changed everything from shopping to transportation to social life — but in the experimental research labs we were still using tools like Origin, Word, Mathematica, and ISI that were created in the late ‘80s. This did not age well!
As the name suggests, generative models can create new data, such as images, music, text, chemical formulae, and protein sequences. For experimentalists and casual coders like me, generative models offer an unprecedented opportunity to translate ideas into working code and test them faster than ever before. This means that I can iterate and experiment with new code and new concepts much more rapidly, cutting development times by a factor of five or more.
Furthermore, generative models allow me to explore what is known about a field on the level of knowledge of an undergraduate program without having to delve into multiple new textbooks, bounce the ideas and guesses off the model, and generally have much more fun.
While generative models cannot extrapolate, hypothesize, or answer counterfactual questions, they already make the work of scientists much faster and more efficient. The key here is to identify the right set of inputs to generate the desired output, and use it as a basis for experimentation, exploration, and discovery. Dynamic language programming if you will.
The most exciting current developments in the field of generative models are their operationalization by combining them with search and reference functions, as the recent paper-qa and ChemCrow projects by Andrew White, Gabe Gomes workflow design, and mathematics engines. These combined workflows synthesize the generative ability of the models with the fact-checking and mathematical rigor, pinning them to the reality. Overall, it looks like we are close to building our own AI assistants that can be deployed to help us rapidly check hypotheses, write reviews, run microscopes, or do grant paperwork.
That said, I think that these models are a “severity 5” event for education. While they are incredibly powerful, they are not a substitute for a deep understanding of the underlying principles and concepts in a given field. To fully realize the potential of generative models, researchers and students alike must continue to develop their critical thinking skills and engage in rigorous intellectual inquiry.
In some sense, the celebrated short story Profession by Isaac Asimov from 1957 has foreseen potential effects. This story talks about a society where knowledge is directly uploaded to the human mind — and for a very few that are not capable of it, life becomes uncertain. But only they can create the new knowledge. Much like in Profession, the large language models can help us access past knowledge but can impede the capability to create new knowledge or recognize the truth from plausibility.
A nuanced approach is needed
Karsten Wenzlaff is a researcher in alternative finance at the University of Hamburg, research associate at the Cambridge Centre for Alternative Finance at the European Centre for Alternative Finance
With ChatGPT, conversational AI has found a very intuitive interface. For the first time, the problem-solving capabilities of large language models are visible to a wide range of users. This feels and is disruptive, and policy-responses are coming: some countries are contemplating regulating the use and training of conversational AI.
However, there is a real threat in overshooting as well — regulating conversational AI needs a nuanced approach. Currently, regulators often lack the tools and capacities to monitor AI tools, especially when AI is used in finance — however, it should be an urgent priority to get regulators empowered to understand the very dynamic developments in AI.
A helpful assistant
Lisa Smith is the editor-in-chief of Nano Select and a deputy editor of Small at Wiley-VCH
Generative AI could represent a major upgrade to repetitive tasks such as basic copywriting and templating or producing suitable, good-quality images to match informative texts. In scientific research, it could assist authors to generate abstracts, cover letters, grant applications, even titles for their manuscripts — everything needed to “sell” their work, to communicate it clearly, and make it more discoverable.
Most scientists are not formally trained in such tasks and tend to learn them in a hit-and-miss way right at the point where the results are critical to their careers. That’s daunting enough even before considering that a great majority are working in their second language.
In this respect, generative AI is no different from any other technology applied to simplify our lives, and research is just one of many jobs which will benefit: having an assistant to draft texts or create simple images frees up time for us to focus on more complex tasks and to develop what we do in new and different directions.
However, the key word here is “assistant”: generative AI should be considered a tool, providing support rather than completing tasks without oversight. Accuracy of the content it provides is a proven concern, and we have a responsibility to ensure that what we claim is true is, in fact, true. Equally concerning is the question of authorship: AI cannot take responsibility for the content it creates, nor can anyone using purely AI-generated content honestly claim to be its author.
There are two points here: the ethics I’ve just touched on of accuracy and authorship, and defining the point of “creation”.
If I want an email template explaining the submission process for a manuscript, that doesn’t need much creativity; I would gladly have AI draft it for me. As a writer, however, I relish the ability to produce my own content. I write for the pleasure of the craft and to connect with my readers, and I think most writers I know would agree.
But not everyone feels that way, and another thing AI greatly simplifies is cheating.
We learn best by applying ourselves to a task, and we improve by repetition. It’s important to make mistakes, discover what doesn’t work, and learn how to problem-solve. Students getting AI to do their homework may be left unable to analyze a situation, form opinions and articulate them without a computer telling them what to say and think. This is an old problem, but AI provides a tantalizingly easy path (which teachers are already exploring ways to exploit).
Those seeking quick recognition can now use generative AI to produce a lot of content in a short time, flooding editorial inboxes with works of questionable accuracy and dubious ethics, and complicating the lives of editors and more genuine content creators.
These issues are not limited to text: artists have played a huge role in defining culture across the world and through the ages. But the irresponsible acquisition of content used to train generative AI, coupled with the question of authorship, has already left many artists uncertain how to proceed when seeing adaptations of their copyrighted artworks touted as wholly new works by others, not to mention how devalued they feel at being told a button-click can replace them.
AI can generate new versions of old content, but cannot replace the important role of creators in a society that values what I will call “the heart behind the art”. Nor should it replace a human mind analyzing the implications of scientific results. In the end, as with any tool, it will be up to the individual to use AI responsibly, and for society to determine an acceptable standard of behaviour.
Exciting times, but regulations are needed
Gerardo Adesso is a professor of mathematical physics and director of research in the Faculty of Science at Nottingham University.
When ChatGPT first launched last year, I had very little previous experience with generative AI. However, like many, I was instantly taken aback by its communicative potential. I am particularly intrigued by prompt engineering; that is, finding the right way to ask questions so that the AI can take on different roles and unlock creative processes beyond what may have been initially envisioned by the developers.
I have experimented with both game-making and also in the domain of scientific discovery. One of the most exciting latest developments is certainly the possibility of integrating GPT-4 with plugins, especially the Wolfram one, which adds a whole new layer of mathematical ability to the AI model.
These days, generative AI is exploding with new functionalities on an almost daily basis. Multi-modality, i.e., the ability to process and generate not only text but also images and sound, is certainly helping make AI tools even more useful and with minimum effort from the human operator.
There are a whole lot of worries of course: first of all, the fact that we do not have complete access to the inner workings of these models, and that by their own nature, they are non-reproducible, meaning that they can give very different answers when prompted with the same question multiple times. Letting an AI take over actions like sending emails, booking reservations, etc., is now possible with the right plugins, but it may easily lead to unintended consequences, which can be quite serious despite the various filters in place — I don’t think it could ever be possible to get fully aligned generative AI, and even that raises some questions, such as the “right” set of ethical values to align to.
A more long-term worry is that soon the internet will be flooded with text and outputs, including “fake news” and alt-images generated by AI tools, which will be used to train new versions of these tools. Therefore, we risk losing the human creative basis that started all this, and AI might get stuck into a self-referential loop without us being able to put a brake to this. Exciting times ahead, but more regulations are needed.
Feature image: DeepMind on Unsplash
