Who controls ChatGpt?
At first glance, ChatGPT's capabilities appear extraordinary. However, if you look closely, all that glitters is not gold. Here's what doesn't work and why it needs to be supported by ethical work. The article by Andrea Vestrucci, artificial intelligence expert, for Stefano Feltri's Appunti blog
Nicola Lattanzi and Andrea Vestrucci, two true experts in artificial intelligence, are working on a very promising book which will soon be released in Italy too. Below is an article written for the Appunti blog (Stefano Feltri).
ChatGPT is a Large Language Model, a software trained on an immense amount of data (texts, web pages, programming pages…) to which grammatical and syntactic rules have been provided to be able to interact with users.
As a type of hybrid AI, ChatGPT consists of a subsymbolic and a symbolic component. The subsymbolic component deals with generating answers on the basis of a calculation of the probability of the correctness of the answer itself, while the symbolic component deals with implementing the syntactic rules in generating answers.
At first glance, ChatGPT's capabilities appear extraordinary. ChatGPT generates responses as if it were a human being, but with a knowledge base far superior to that of a human being.
Within a single chat, ChatGPT responses are connected to previous responses, creating the impression of a true dialogue with an entity that has not only knowledge, but also cognition of a succession of interactions over time.
However, if you look closely, all that glitters is not gold. In a series of experiments aimed at testing the logical-mathematical capabilities of ChatGPT, it was found that ChatGPT3 has problems deducing the correct premise in a syllogism slightly more complex than average, and ChatGPT4 fails to demonstrate that, given a binary relation with some properties, not all elements of a set are connected to each other by the relation.
In both cases, you can tell ChatGPT that the answers are incorrect; ChatGPT will apologize, and formulate another response that will take the error into account. But this new answer will be along the lines of the previous conclusion: ChatGPT will simply change the way you get there.
This problem is related to the semantics of ChatGPT, i.e. the relationship between the terms (of an answer or a formula) and the objects of a given set or "domain".
In the sentence “it's raining”, the semantics is the meaning of this sentence, that is, the relationship between the sentence and the fact that drops are falling from the sky (objects) at the time and place where I am (domain).
The semantics of ChatGPT are derived, statistically, from the immense amount of texts that ChatGPT has been trained on. Given that the texts, taken from the internet, are considered coherent and meaningful, i.e. correct from a syntactic and semantic point of view, they constitute the basis on which ChatGPT builds the meaning of its answers.
This means that if the texts provided to ChatGPT were not semantically correct, then ChatGPT's responses would also not be semantically correct – or rather, they would be even less semantically correct than they already are.
Therefore, ChatGPT has no semantic autonomy: it cannot evaluate whether its response is semantically correct because it has no rule to construct the semantics of its decisions, that is, it has no abstract notion of what meaning is.
This absence of semantic rules means one important thing: that artificial intelligences such as ChatGPT have no understanding of what they say and decide.
They simply repeat what they have learned from the enormous amount of learned texts, modifying and rearranging parts of these theses according to the semantic scope of the user's question. For this, it is common to apply the metaphor of the stochastic parrot: the stochastic parrot does not create anything new, but simply repeats in a probabilistic way. More precisely, the stochastic parrot formulates answers whose semantic correctness is only probable, and this probability is based solely on the quantity of what has been learned.
The statistical parrot
What's the deal with the stochastic parrot? The problem has to do with the distinction between logic and rhetoric. As seen, ChatGPT formulates a response – i.e. a decision – which conforms to syntactic rules but not to semantic rules: consequently, ChatGPT does not have control over the semantics of its responses, i.e. over their meaning. Therefore, ChatGPT's answers appear perfectly correct, but, in some cases, this correctness is only apparent, given that there is no comparison with semantic rules. In the examples mentioned, ChatGPT's answers on syllogism and binary relationship are incorrect. However, these incorrect answers seem perfectly correct because they are very well argued.
Furthermore, precisely because it is a machine, ChatGPT is not affected by the emotional conditioning that we humans have when we formulate an answer that we are not sure is correct: ChatGPT does not hesitate, does not stutter, does not sweat, does not show signs of uncertainty. ChatGPT formulates its incorrect answer just as if it were the only correct answer, argues it, and continues to propose it even after it has been pointed out that some passages are not logically valid.
ChatGPT's answers are absolutely convincing, but not necessarily correct. In even more explicit terms: ChatGPT's answers hide a potential logical incorrectness under excellent rhetoric.
Who controls ChatGPT?
One could argue that this is not a huge problem because it is limited to some detailed answers from ChatGPT, among other things formulated precisely to test its logical capabilities.
However, let's ask ourselves what a standard user who asks a question to ChatGPT without ulterior motives such as, for example, testing his logical abilities would do. How would this user react to such a rhetorically well-packaged response? Would you go and verify the logical correctness of the answer, or would you trust ChatGPT's perfectly plausible decision?
The time factor should also be underlined: ChatGPT is used precisely to obtain a quick response to a complex problem.
A ChatGPT response check would waste this time gain. In short, the rhetoric of ChatGPT's responses instills confidence in the user regarding the correctness of these responses. But this confidence is not at all supported by a true validity of the answers. For some answers, it is a fictitious confidence, and yet difficult to refute.
In short, artificial intelligences such as ChatGPT are able to create answers, decisions, and even narratives, with such rhetorical strength that they make us believe they are correct. Not only that: ChatGPT has access to our expectations, habits, preferences, precisely from our interactions with it.
This means that the machine's excellent rhetoric is capable of modifying our beliefs: making us believe things that do not correspond to the reality of the facts. And this is possible precisely because our beliefs are not based on evidence or confirmation of their validity, but depend exclusively on their compatibility with certain of our expectations, on their narrative impact, and on their rhetorical force.
The logical inconsistency of some automated decisions does not appear at first glance, just as the fragility or inconsistency of some of our beliefs does not appear at first glance: at first glance the power of persuasion based on the harmony of artificial decisions with our beliefs appears, in as equally characterized by this separation between rhetorical persuasion and logical correctness.
Beliefs are central to our decisions and regulate our orientation in the world and in our lives in general. It is therefore clear that the separation between logic and rhetoric, which characterizes artificial decisions, has enormous ethical relevance, because it implicitly influences our decisions, actions, conduct – all things initiated and supported by our beliefs. And I say “implicitly”, because, again, artificial decisions are perfectly convincing: why waste time seeing if they are also logically coherent and correct?
Ethical work is connected to ethical relevance. Once we are aware of this discrepancy between rhetorical persuasion and logical correctness in artificial decisions, we may want to delve deeper and test the artificial decisions we receive; we might recognize them as siren songs, we might want to be even more firmly anchored to the mast of logical correctness, semantic coherence, and relationship with facts and circumstances, to preserve our autonomy. It is an ethical work, because it aims to remedy the ethical problem of artificial decisions that are potentially harmful because they are illusively correct.
But how can we really carry out this ethical work towards artificial narratives? How can we have the resources, skills and time to engage in this operation of confirming the correctness of the decisions formulated by artificial intelligence?
The rest of the book answers this question! The answer includes the direction of my work: the creation of a 100% symbolic AI – that is, 100% transparent and rule-based from both a semantic and syntactic point of view – which acts as an auditor for the decisions of other AIs.
With my students we are applying this idea to the EU AI Act, which is the first and so far only legislation on AI formulated by a parliamentary body.
(Excerpt from Stefano Feltri's Notes blog )
This is a machine translation from Italian language of a post published on Start Magazine at the URL https://www.startmag.it/innovazione/chi-controlla-chatgpt/ on Sat, 25 Nov 2023 06:29:11 +0000.