GPT-4 – what you should know and what to expect

OpenAI launched GPT-4 with great timing from a marketing point of view, leveraging the worldwide hype. Unfortunately, the model is a black box.

Yes, there is this one-hundred-page technical report released in PDF to read and to study: it’s pretty interesting from several points of view, in particular the sections related to limits and risks, prompt examples and techniques adopted to mitigate possibly harmful answers by the machine, and so on. 

But the document is quite disappointing if you are looking for transparency: “Given both the competitive landscape and the safety implications of large-scale models like GPT-4”, you can read, “this report contains no further details about the architecture (including model size), hardware, training computer, dataset construction, training method, or similar.”

OpenAI’s choice not to disclose details about its upgrade is a problem widely criticised because it blinds us to how the machine actually works – understanding which we must infer from its predecessors, from what we know about LLMs in general, from reverse engineering that we can apply by testing the machine’s capabilities and limitations, and from a leap of faith in comparisons of the details entered by OpenAI in its report. 

Hallucinations are everywhere

Moreover, even if better than its predecessor – at least according to OpenAI’s statements – the model still suffers from hallucinations, that is, confident but completely factually incorrect outputs. This is a crucial point – GPT-4 is not a search engine; it is not an oracle; it has no general knowledge about the world.

It is still a Large Language Model, with several capabilities, and, as a transformer, it’s trained with a large amount of data to predict the next token (words, or, better, strings with an assigned and thus identified meaning). It can receive larger input and produce larger output compared to GPT-3.5. It has been presented – and actually appears to be – as extremely effective in passing standard tests in various disciplines. It’s a great assistant to brainstorm ideas, summarise, create mental maps, titles, bullet points and so on.

Despite these capabilities, it still can fabricate articles (see this story by The Guardian), it can fabricate scientific abstracts (see Forbes), it can fabricate stories about real people (see The Washington Post), and we must know that. If ChatGPT were human, we’d say it lies and deceives. But we have to abandon the anthropocentric vision to be able to use these machines.

If I ask ChatGPT Plus, “Who founded Slow News in Italy?”, referring to the slow journalism outlet I run, not only is the answer completely wrong, but it is always different. 

This behaviour depends on several reasons, starting from the fact that Slow News is not well-known. 

Evidently, given the small and independent nature of Slow News, the machine has received little text in its training data to get the answer right. Things get very different, for example, if you ask the machine who founded the New York Times. In this case, not only is the answer always correct, but it is also always the same.

Between a small independent reality like Slow News and the most famous newspaper in the world there are, of course, all possible intermediate nuances. And this goes for any topic.

So, if we have a problem with hallucinations, it is the fact that they are inherent in the very essence of these machines.

“Despite its capabilities, GPT-4 has similar limitations to earlier GPT models: it is not fully reliable (e.g., can suffer from ‘hallucinations’), has a limited context window, and does not learn from experience,” states OpenAI’s report, which continues: “Care should be taken when using the outputs of GPT-4, particularly in contexts where reliability is important”.

Moreover, hallucinations could also depend on the prompts: we need to learn how to ask these machines.

So, the first rule for using these machines is always the same: don’t use them to write content from scratch, and don’t publish anything generated without human supervision. “There are several reasons for this,” says ChatGPT itself – and in this case, I can assure you that these are not hallucinations at all because I double-checked the answer:

• Quality control: While AI-generated content can be informative and well-written, there is always a risk of inaccuracies, inconsistencies, or lack of coherence. A human editor can ensure the content meets the desired quality standards.
• Ethical considerations: AI-generated content can sometimes inadvertently perpetuate biases or generate inappropriate content. Human supervision helps to prevent such issues and ensure the content aligns with ethical standards.
• Context and nuance: AI language models can sometimes misinterpret context or struggle to convey the nuances that human writers can easily include. Human review can help ensure that the article is accurate, relevant, and well-structured.
• Legal and copyright concerns: AI-generated content may unintentionally violate copyright or trademark laws. Human supervision can help to detect and resolve any such issues before publication.

I would add a fifth point:

• Verification method: AI can’t substitute humans in the verification process. At least, not yet.

GPT-4 can be a great assistant, and it can help you produce semi-finished products.

The series of journalistic stories about ChatGPT making things up is – generally speaking – missing a crucial point, beautifully explained by Stephen Wolfram, British-American computer scientist, physicist, and founder and CEO of the software company Wolfram Research: “What ChatGPT is always fundamentally trying to do is to produce a ‘reasonable continuation’ of whatever text it’s got so far, where by ‘reasonable’ we mean ‘what one might expect someone to write after seeing what people have written on billions of webpages, etc.’”

Wolfram isn’t offering an oversimplification (conversely, the whole article is probably the most accurate explanation of how these machines work): it is precisely what ChatGPT does, repeating this operation for each word, starting from an enormous mass of contents which allows the device to generate essays, articles, conversations in what we humans call natural language.

Plugin as a possible solution to several hallucinations

Being indignant about GPT-4 hallucinations serves no purpose. As Wolfram writes, “ChatGPT—for all its remarkable prowess in textually generating material ‘like’ what it’s read from the web, etc.—can’t itself be expected to do actual nontrivial computations, or to systematically produce correct (rather than just ‘looks roughly right’) data, etc.”

This ChatGPT’s essence is something that journalists should remember all the time and properly report all the time.

You can react to this fact by arguing that we have to ban the tool, or you can look for solutions.

A possible solution is offered by one of the plugins ChatGPT came with (unfortunately, plugins are still under waitlist).

The most important one from a journalistic perspective is the “Wolfram” plugin. It connects ChatGPT with Wolfram Alpha, an answer engine that Wolfram Research developed. Wolfram Alpha is a “computational knowledge engine: it generates output by doing computations from the Wolfram Knowledgebase, instead of searching the web and returning links”, you can read on the FAQ page. It answers factual queries by computing answers from externally sourced and reliable data.

You can use it in several fields.

For example, you can ask Wolfram Alpha “How far is Tokyo from Rome?” and you’ll get a result like this one. 

The integration solves some ChatGPT’s hallucination problems, also creating several possibilities for a combo between a human knowing the tools and this technology.

Wolfram himself offers a series of remarkable examples of this integration. 

Alien Intelligence

As Nello Cristianini, professor of Artificial Intelligence at the University of Bath, writes in his book “La Scorciatoia,” “An alien intelligence is entirely different from ours, such as that of ant colonies, machines or animals of the deep.”

We should probably abandon the usage of artificial intelligence and focus on “alien intelligence.”

GPT-4 is an alien intelligence because it is a machine that can do a variety of specific things and is different from human intelligence. “In abandoning an anthropocentric view of intelligence,” Cristianini argues, “we should not expect things like sentience or self-awareness […] What we should expect from our intelligent artefacts is rather a continuous pursuit of simple goals, indifferent to any consequence for us, perhaps facilitated by an ability to learn and improve with experience”.

As journalists, not only we should learn how to use these tools, but we also need to learn how to report about them: the paradigm shift from artificial intelligence to alien intelligence could help us in

• avoiding the apocalyptic tones of the longtermists who think of AI that it will dominate us (see, for example, the notorious petition to suspend AI research for six months, signed by, among others, Elon Musk and Yuval Noah Harari;
• allowing  us not to ask ourselves the wrong or useless questions (for example: “do these machines understand?” is a useless question because it is anthropocentric)
• allowing us to avoid the pitfalls of marketing and hype to tell these technologies to the public properly

Finally, if we are to devote ourselves to disseminating petitions, as journalists genuinely interested in people, we should probably focus on other instances. For example, supporting the creation of a large work project on these technologies, 100% non-profit and 100% open like Laion (Large-scale Artificial Intelligence Open Network). “The potential applications of these technologies,” they write in Laion, “are vast, spanning scientific research, education, governance, and small and medium-sized enterprises. To harness their full potential as tools for societal betterment, it is vital to democratise research on and access to them, lest we face severe repercussions for our collective future.”

---