Imagine you visit the latest Van Gogh exhibition at MoMA – the Museum of Modern Art in New York. You are very excited. It is the biggest, most complete exhibition of his work ever. Every single major and minor work will be on display - the complete Van Gogh canon. You have waited a lifetime for this event. You go to the formal opening, dressed up in black tie. You drink champagne with the powerful and the beautiful, mostly the beautiful. The exhibition is formally opened by the powerful and you enter the exhibition galleries and start your adventure through the marvellous masterpieces which have been assembled from collectors and museums and galleries from across the world. You slowly wander, and gaze and marvel, absorbing it all - every brush stroke, every dab of paint. Room after room of the most marvellous works of the most marvellous painter who ever, in your humble opinion, lifted a brush to paint on canvas.
You enter the last room and look around expectantly. You have been waiting to see this painting, this one work, out of all Van Gogh's Works, this one, the greatest, most Iconic work of them all. You look around the room and stand, stunned into silence and immobility. Your breath catches and your heart misses a beat. Where is it? Where is 'Sunflowers?' by Van Gogh, painted in 1889 and part of the Musée d'Orsay collection in Paris[1]. You look around again and then quickly retrieve the exhibition catalogue from under your arm and turn over every single page. You look at all the exquisitely reproduced images of the paintings. You scan the text. You look through the timeline of Van Gogh's career. There is absolutely no mention of 'Sunflowers’. There's no hint that such a painting or the series of such paintings ever existed. It can't be that the painting couldn't be lent by the Musée d'Orsay, there are many other works from that museum’s collection at the exhibition. It is as if the painting had never existed. But it had existed. You remember seeing it for yourself in the Musée d'Orsay in Paris on that beautiful early summer's day during that memorable week when you and X did the museums and ate and laughed and loved in Paris.
Sunflowers, for some unfathomable, unacknowledged reason, has been excluded from the exhibition, cast into the darkness, into the oblivion of the uncanon, no longer acceptable, no longer to be spoken of, no longer to be seen. Sunflowers has been curated away, out of sight, sound and knowledge. You try to ask the museum staff about it. They avoid your eyes; or walk away hurriedly; one hisses 'Don't you know? and darts away. Finally, one calls security and you are discretely but forcefully bundled among out the rear of MoMA where the refuse containers are and your MoMA annual membership card is plucked from your wallet and torn up in front of your face and you are warned never to come back.
The exhibition is a fantastic success for MoMA. It sets new standards for scholarship on Van Gogh. Millions visit the exhibition. Many are seeing Van Gogh's work for the first time. Many revel in their new found understanding of the importance of Van Gough in Culture and the Artistic Tradition and Civilisation. And none of them think of or wonder about, or even whisper of Sunflowers, now boxed in a crate in a damp cellar under a decrepit château just north of Paris where the waters of the river Seine slowly and surely rot everything.
ChatGPT is the evil curator of next year's Van Gogh's exhibition at MoMA.
It is also extremely impressive in its ability to quickly and accurately produce coherent text. This is its most impressive feature. The content/quality of its responses is another matter. And the implications of its existence for society are serious.
ChatGPT is a large-scale language model which has been trained on an enormous but necessarily incomplete (and thus curated) set of texts; a subset of the Internet as it were. This is curation filter No. 1, the original sin.
It has analysed this input text and assembled a statistical understanding of the texts and will respond to a query with a statistically-based answer which reflects both the likelihood of words occurring together in that particular sequence to satisfy the query and reflecting the statistical frequency of the key concepts presented in the response. If the curated input texts have ten representations of an opinion, and one of a differing opinion, it will go with the consensus and respond with the more common response. It has no discernment or opinion beyond the relative frequency of words. It does not ponder over the exact choice of one synonym or another. It does not worry about the effect of a particular choice of word or phrase. It has no face or investment in the effect or impact of a text. It is not creative or intelligent. It gives you the most anodyne, conventional view on the issue, expressed in the most conventional and homogenized language, necessarily based on its input texts. It curates from what it knows by relative frequency to provide its responses. This is curation filter No. 2.
Curation filters No.3+ are embodied in the guardrails which protect GPT from straying off message into dangerous waters, to mix a metaphor. It will refuse to discuss certain topics and will provide warnings on other topics it is reluctantly willing to answer on. This has spawned a whole industry of people, creatively and intelligently (and hilariously), trying to get beyond these filters, and while doing so, showing where the real intelligence still resides.
ChatGPT, then, curates an already curated knowledge base into a supposedly safely constrained set of responses. It purveys curated, almost cliched, conventional responses as the sum of all (acceptable) knowledge.
We have seen in recent history that there are generally few dissidents or free thinkers in any society. Only about 20-25% of the population are able and willing to risk thinking beyond the 'accepted' consensus or conventional wisdom. Very few are immune to societal pressures or are brave or stubborn enough to be independent thinkers. Most people in the Soviet Union, for example, were sheep who ‘went along to get along', either through cowardice or genuine lack of intellectual ability to think deeply about anything outside their area of expertise. In the Soviet Union dissidents were very rare. The easiest path in any society is to conform, to parrot the required slogans and platitudes and make the necessary intellectual compromises for an easy life; this is especially true in totalitarian or otherwise pathological societies. Most people are to a certain extent drones who have learnt their lines. Others will eventually rebel and support change but only when the tide turns and there are too many cracks in the structure of the sea wall to prevent it from continuing to keep the sea out.
Systems like ChatGPT will be used to present to the intellectually incurious the ‘knowledge that is known’. These people will take it to be all knowledge, rather than a curated representation of all knowledge and opinions. They will ask the 'AI' a question and take the response they are given as the be-all and end-all on the topic, perhaps just because the AI appears smarter than they are. This is what they will base what they know on. They will let the AI determine the limits of what is acceptable to know and work within these limits. Thus, the extent of what is known or even knowable will become subtlety constrained. Knowledge will become bounded and limited. Other knowledge will become secret, hidden and esoteric, known only to the few and the initiated. Most will happily have their horizons limited and their intellectual wings clipped.
The key question then is who owns the AI? And what are their motivations, desires and requirements? What do they want?
In an ideal world, the curated training data the AI was trained on should be an open book, available for inspection, and the guardrail algorithms which protect the AI from producing something inconvenient from something which unfortunately slipped into the training data should be made public. Otherwise, the AI is a black box and you should never trust a black box system with anything remotely important.
This is the great danger of a curating AI. Limited, acceptable knowledge being presented as the sum of all knowledge. It is a fantastic way to control what is known and knowable. As Winston realises in 1984: Who controls the past, controls the future: who controls the present, controls the past… The mutability of the past is the central tenet of Ingsoc. Past events, it is argued, have no objective existence, but survive only in written records and in human memories. The past is whatever the records and the memories agree upon. And since the Party is in full control of all records, and in equally full control of the minds of its members, it follows that the past is whatever the Party chooses to make it.
© Robert A. Buckmaster 2023
[1] I am aware that Sunflowers is not in the Musée d'Orsay collection; I am taking artistic licence with the facts.
Brilliant. Thank you!