The rise of AI has sparked countless debates about its potential for good and its risks for misuse. While the future remains unwritten, one application stands out today as undeniably impactful: AI generated ALT text.
ALT text serves a dual purpose: it ensures screen readers can describe images to those who can’t see them and provides search engines with contextual cues to improve discoverability. But how well can AI craft these descriptions?
I decided to put AI to the test after discovering the AI Services Plugin for WordPress. This powerful framework allows developers to integrate AI models like OpenAI (ChatGPT), Anthropic (Claude), and Google (Gemini) into their WordPress projects. If you’ve ever wanted to explore adding AI capabilities to your own plugins, this is a fantastic starting point.
Inspired by its possibilities, I decided to take the plunge and build something of my own. Enter the AI SEO Tools Plugin, a project still in its early stages but already sparking my creativity. Using the AI Services Plugin as a foundation, I began experimenting with AI-powered features, blending my interest in generative AI with my love for building software.
Generative AI has been the “don’t miss out on” technology of the year, and diving into this project was both exciting and a little unsettling. Seeing what AI can do—especially in areas like computer vision—sparked joy and curiosity while raising questions about its broader impact. Naturally, I couldn’t wait to share the results of my tests and get others thinking about what’s possible with AI in WordPress development.
My test image was a photo from the local sandcastle contest. I thought it would be a tricky image for AI to write accurate ALT text for. The results were surprising.
I tested all of the OpenAI Models which supported the image input. While they were all surprisingly descriptive, none of the models mentioned the scroll or the carrot nose. One of the models provided an inaccurate description.
AI Prompt for Generating ALT Text
Create a brief description of what the following image shows, suitable as alternative text for screen readers.
OpenAI Model ALT Text Results
gpt-4o
A detailed sand sculpture featuring a large snowman wearing a beanie and scarf, surrounded by several gnomes with bushy beards and pointed hats. The sculpture includes a sign that reads, “Gnome for the Holidays.” The background shows grassy dunes and a cloudy sky.
gpt-4o-2024-08-06
A detailed sand sculpture depicting a large snowman wearing a beanie and scarf, surrounded by several small gnomes with beards and pointed hats. They are positioned around a large flat section of sand with the words “Gnome for the Holidays” etched into it. The scene is set against a backdrop of sand dunes with grassy vegetation.
gpt-4o-mini-2024-07-18
A sandy sculpture featuring a snowman wearing a knit hat and scarf, sitting next to three gnomes. The gnomes are depicted with long hats and beards, and they hold various items. A sand sign in front reads “Gnome for the Holidays.” The background includes grassy dunes under an overcast sky.
gpt-4o-mini
A sand sculpture depicting a whimsical holiday scene featuring a large snowman with a hat and scarf, holding a small gnome. Surrounding the snowman are several smaller gnomes, each with distinctive pointed hats. In the foreground, there is a sandy plaque that reads “Gnome for the Holidays.” The setting includes sandy textures and patches of grass in the background, indicating a beach environment.
However, Computer Vision can Make Funny Mistakes…. and Miss out on Key Details
The inaccuracy is highlighted in red below. It made me LOL! 🤣
gpt-4-turbo
A detailed sand sculpture depicting a large snowman wearing a hat and scarf, silently hushing with its finger to its mouth, surrounded by small gnomes wearing pointed hats. The phrase “Gnome for the Holidays” is carved into the sand in front of them. The scene is set on a beach with grass and flora in the background.
gpt-4-turbo-2024-04-09
Sand sculpture featuring a snowman with a hat, scarf, and a finger to its lips, surrounded by smaller figures wearing pointed hats, with the text “Gnome for the Holidays” carved in the front. The sculpture is located on a sandy area with vegetation in the background.
gpt-4o-2024-05-13
A detailed sand sculpture on a beach depicts a large snowman wearing a beanie and scarf, surrounded by four smaller gnomes with tall hats and long beards. One gnome is being hugged by the snowman. Beside them, a flat sand surface bears the carved words “Gnome for the Holidays.” In the background, there are grassy dunes under an overcast sky.
AI Image Generation – Reverse Engineering ALT Text
Curious to see how well AI could “reverse engineer” its own descriptions, I decided to have a little fun. Using one of the generated ALT texts as input, I asked OpenAI to create a photorealistic image based on the description.
Other Notes
I tried out Gemini AI for ALT text generation, but it was consistently too verbose. I frequently received responses with multiple options or preambles.
I briefly tested Anthropic’s Claude AI. My initial impression is that it works well. I don’t know yet if it’s more accurate or better in any way. My colleagues have sung the praises of Claude AI so I am eager to work with that AI Service more. However, a shootout of Claude AI models would require writing a bit more code. I see more AI experimenting and AI pondering in my future… Stay tuned.