Automation as a Cue of Factitiousness
In this post, I will tell a small anecdote that happened to me this week while I was trying, together with a few friends, to (re)start exercising, and show how it illustrates a possibly rapid change in our ability to trust the information circulating in today's digital information environments.
AI and the colonization of digital environments by automated media
To pull the threads of this anecdote together, a few preliminaries are needed.
The first is that the development of conversational AI and other automatic content generation tools, and above all the very rapid spread of their use by actors with varying degrees of good faith, raises a whole series of questions and fears about the proper functioning of information environments.
One of these fears is that the web, and digital information environments more generally, are gradually filling up with slop, that is, false or low-quality information generated automatically by AI tools. It follows that having to navigate every day a digital environment saturated with slop could aggravate the information disorders already identified: disinformation, malinformation, misinformation, etc.
A second dimension of this fear is that, in a digital information environment where content is increasingly produced by what my colleagues Jerry Jacques and Antonin Descampe call "automated media", it will become more and more difficult to assess both the reliability of sources and the factitiousness of the content we are confronted with.
With the generalization of automated media, the very notions of author and source are transformed. The growing automation of digital media content production could therefore not only intensify information disorders but, above all (and this is, I think, what my anecdote from this week will show), reshape our relationship to information deemed reliable or of good quality.
To navigate a digital environment filled with "slop", we must (or at least should) constantly ask ourselves the following questions:
- Was the press or scientific journal article I am reading really published in the journal I am interested in, or is it a fake copy meant to deceive me?
- Does the email I received really come from my boss, or is it a very realistic phishing attempt imitating a message from my boss?
- Is the sender of this Instagram message a real person, or a bot?
- Is the answer to the question I asked Grok or Gemini actually valid, or does it rest on inaccurate information, or even on an ideologically skewed view of reality?
- Was the gardening blog I am reading (yes, at 40 you read gardening blogs...) written by an expert on the subject, or was it generated in two minutes by an AI that says whatever comes to mind?
In short, the ongoing colonization of the digital environment by a new type of media, automated media, could well destabilize that environment's functioning as an information environment, that is, as a collective space in which we form our relationship to the world by acquiring knowledge and information that we deem reliable.
Reception and trust in information environments
The second preliminary is that the effect of a new medium and its content on society and individuals is never direct. If this effect is measurable, it is measured largely by taking into account how the people who use the medium to communicate receive its content.
If we are interested in the effect of the ongoing colonization of digital environments by automated media, it is therefore relevant to ask how the reception practices they may provoke could change.
Faced with media production, we are never passive; we are in an active interpretive stance. We try to make sense of the messages we receive and of the content we read or watch, and it is in this continuous interpretive activity (often very rapid, even unconscious) that the meaning of the content is ultimately constituted, and with it its possible "effect" on our perceptions, attitudes or activities.
We can hypothesize that in a functioning information environment it is relatively easy (whatever criteria the participating person uses) to assess the quality and reliability of both a source and a piece of information. In any case, it must be possible to know whether or not we can trust the source of the information.
In 2025, our information environment consists of a variety of sources, both physical or analog and digital: friends, parents, colleagues, teachers, books, traditional news media (press, TV, radio), blogs, social media influencers, scientific journals, professional newsletters, local newspapers, and so on.
These sources are relatively stable over time, and we have learned either that we can trust them or that they are not reliable. This operation is mainly personal and the result of an interpretive act. For example, one person may judge the RTBF news (the Belgian public broadcaster) to be extremely reliable and another may see it as a collection of lies; depending on this initial evaluation, its content will not be interpreted in the same way and will not have the same "effect" on us. This activity also has an important social dimension, however: it is, for instance, generally accepted in Belgium that the RTBF is a quality source that can be relied upon.
Once this trust has been acquired and stabilized, there is no need to re-evaluate it at each new interaction with the source. This stability of media trust is probably one of the conditions of possibility of a functioning information environment. And the existence of a functioning and relatively stable information environment is probably one of the conditions of possibility of life in democratic societies in the twenty-first century.
From this point of view, the digitization of information environments, caused by the development of the web in the 1990s and above all of Web 2.0 from 2000 to 2020, had significant consequences. New types of media sources appeared, such as encyclopedias written by non-professionals (Wikipedia), online newspapers, blogs, social media accounts, and so on. Almost all of these digital sources have in common that they are produced by non-professionals with very diverse skills, so their reliability could no longer be automatically inferred from the specialized status of their author (journalists, scientists, experts). The evaluation of source reliability thus became more central, and so, probably, did the collective processes by which trust is attributed to certain sources judged more reliable than others.
What is interesting is that we have gradually learned collectively, and it took almost 30 years, whether and how we could (or could not) trust these new sources and how they could contribute to our information mix. This process took time, is still not over, and gives rise to many debates.
With these preliminaries in place, I can come to my aborted attempt to exercise and show why it is a good example of a possible ongoing destabilization of the trust structures of digital information environments, and how this destabilization can cause very concrete problems in our everyday lives.
My (non-)participation in a trail run and a debate among friends about the factitiousness of an email
A few weeks ago, I signed up with three friends for a trail run scheduled for the end of May in a town near Brussels. Nothing out of the ordinary. The event was being organized for the first time by an experienced company, in a very professional manner. A flawless website in French and Dutch powered by Odoo, impeccable social media accounts (Facebook, Instagram and TikTok) with slick promotional videos, and so on.
Last week, surprisingly, all four of us received an email announcing that, for lack of sufficient registrations, the event was going to be cancelled and that we would therefore be contacted about refunding the registration fees. Great disappointment on our part, but above all, great surprise.
Reactions to this email within our group were very varied. For two of us, there was no doubt: this email was spam, a phishing attempt to harvest our bank details. The event was going ahead and we should keep training for it. The third, on the contrary, saw it as an authentic email and thus a simple cancellation notice, and the fourth could not decide but leaned more toward spam than toward an authentic email. So there was no way for us to agree.
The elements used to justify the claim that this email could not be considered reliable were the following:
- The sender address was of the type "info@45698122.xxx.com". No doubt: the sender had nothing to do with the race organizer. The domain of the address was not that of the event's website, and the address was not the same as the one used in the previous emails we had received. Above all, the series of digits in the address was a sure sign that the sending was automated. A robot must have taken part in this mailing in one way or another.
- The email contained the following sentence: "We will get back to you shortly regarding the terms of the refund of your registration." A sign of a possible scam designed to collect our bank details.
- Finally, after checking, none of the event's other communication channels (website, Facebook, Instagram or TikTok accounts) mentioned its cancellation.
Conversely, for the third of us, there was no doubt:
- The email was formatted with very professional Mailchimp-type software, which was consistent with the other communication tools used by the organizers.
- It was written in perfect French and Dutch, without errors and very concisely, with no trace of approximate machine translation.
- Above all, it asked for no direct action on our part. No link to click that would take us outside the event's known media, or that would ask us to enter our bank details somewhere.
- Moreover, the email had clearly been sent only to people registered for the event. If it were spam, it would have meant that the hackers had access to the organizers' database, which is not impossible but still quite unlikely, and above all a great deal of energy expended by hackers for such a small event. In short, no reason to doubt its authenticity.
In the end, after five days of debate among ourselves, and tired of waiting for parallel information to appear on the event's website or social media (it never did: either the event was cancelled and they should have published the information, or their database had been corrupted and they should have informed us), we found a contact email address to write to, and a person at the organizing company did confirm that the event was cancelled. End of the suspense.
The trigger for doubt in our case was the identification of an automated process behind the sending of the email (the email address made of numbers being a cue of automation, and therefore of possible factitiousness, which we have learned to recognize). What intrigued me most was the extremely rapid and stable character of this evaluation for two of us (and afterwards for a third person outside our group whom we asked for an opinion). There was no doubt, it was spam (even though our mailboxes' spam filters had not flagged it as such). The fact that the verification work also failed to yield a result (the information was not available redundantly across the company's other media, which is yet another characteristic of digital communication that we have learned to recognize over time) was clear confirmation of factitiousness. Despite a critical effort of investigation, we could not remove the doubt that had assailed us.
I think that a year or two ago, before the acceleration in the development of automated media and their colonization of digital environments, we would not have had the almost automatic reflex of doubting the reliability of this email.
This little episode illustrates that this rapid transformation of digital information environments may have as a consequence a rapid change in our relationship to information and in our ability to distinguish reliable from unreliable digital sources. It is possible that these new automated media, more than their predecessors, complicate the activity of content reception and therefore destabilize, perhaps severely, the functioning of our information spaces by complicating the interpretive work they impose on us.
To the automation of media, there might thus correspond an automation of doubt in the process of receiving media content, and this automation of doubt could be a problem in itself.
Fortunately, in the end, it was not this small communicational event that prevented us from getting back into sport and getting our heads out of the digital environments in which we are immersed for a large part of our days.
#GenAI #automated_media #trust
Updated on May 21, 2025.