For years now a set of disinformation tools and tactics have spread cascades of falsehood across the Internet. This problem could take a turn for the worse with the development of machine learning models, an emerging technology powered by Artificial Intelligence (AI) that hostile actors could use to support false narratives.
Consider this: a hostile actor creates a false headline, builds a story around it, and uses AI to design an image that perfectly supports the erroneous narrative. This is what fully synthetic content, such as hyperrealistic images created through text prompts and powered by AI, enables. Also known as text-to-image-generation, this impressive technology, with fascinating potential in its legitimate uses, could have daunting effects on our democratic public discourse.
Aware of this looming threat, Democracy Reporting International has interviewed a series of leading global experts in the field of AI, disinformation and text-to-image generation. We tried to understand the nature of this intricate technology, assess how prepared we are for its unrolling and explore possible solutions to the disinformation problem it will create. We have compiled all the insight from the interviews into a report titled “What a Pixel Can Tell: Text-to-Image Generation and its Disinformation Potential”.
Report Launch event
Join us on 29 September for our launch event “The deeper the fake, the more dangerous: The disinformation potential of text-to-image generation”, where our researchers will present the findings of the report and our guest experts will offer their views on synthetic content.
How will text-to-image generation threaten our public discourse?
Text-to-image generation will take some time and effort to become the method of choice for political disinformation, but we must be ready for its impact.
text prompt “a billboard with an image of a pink strawberry” (Source: OpenAI).
These are the specific risks it brings:
- It allows for the creation of misleading images of politicians and other public figures;
- It reinforces sexualised and racial stereotypes because AI works on models that already include such biases or stereotyping;
- Text prompt-generated images can be combined with other photo editing and manipulation tools to increase their believability;
- Authoritarian state actors with the resources, infrastructures and vast amounts of data necessary can train models in a shorter span of time; and
- Automated content production could “flood the zone”, where a significant increase in synthetic content could overwhelm the capabilities of forensic detection
Are we prepared?
We are not well prepared for this threat, according to the experts consulted in the report.
Society is not prepared because we are lacking sufficient digital literacy among adults and in our school systems.
Digital forensic experts put forward some image authentication techniques, but these require that AI service providers agree to incorporate invisible watermarks or metadata in their outputs.
Platforms are also not prepared. It is unclear to what extent platforms have the technical capacity to effectively identify and prevent misleading synthetic imagery based on text from going viral. Despite the adoption of some policies on media manipulation and synthetic media, the effectiveness of those policies is unclear. Whatever the case, no platform has yet tackled the issue of merging multiple tools or addressing the risk posed by AI-generated images based on text prompts, as images alone, without fake text as supporting evidence, would simply fall under the already existing guidelines and standards for manipulated media.
How can we respond?
Increase protection mechanisms
- Implement a binding and standardised “AI responsibility” code of conduct for AI service providers, thus going beyond self-regulation;
- Enforce “product safety” standards for code-hosting platforms;
- AI companies should work with filters to reduce the risk of biased models;
- Introduce model evaluation around potential harms of text-to-image generation models; and
- Establish a cooperative relationship between regulators and the AI industry to create minimum standards jointly
Encourage platform transparency
- Tech platforms should apply stricter platform community standards for the combination of synthetic media; and
- Platforms need to be more transparent about policy-enforcement data
Go beyond technical solutions
- Introduce media literacy programmes that apply innovative sensitisation formats (i.e., gamification elements);
- Shift the focus from debunking disinformation to pre-bunking; and
- Establish an exchange forum for effective collaboration between researchers, civil society organisations (CSOs) and tech platforms.
Don’t forget to sign up for our newsletter to receive the latest work and other updates from DRI’s Digital Democracy team! Pick the 'Digital Democracy' topic when you sign in.
