The proliferation of immersive models has become a crucial part of development in providing virtual experiences, creating interactive online models, generating realistic sentences and paragraphs by observing the images, etc. This massive tech revolution has compelled researchers to develop AI to an extent where the models can create images through captions. That’s right, just when you thought about what AIs can not do!
OpenAI’s GPT-3 Model Generates Images from Captions
The researchers at Allen Institute for Artificial Intelligence created an AI technology with an interface that integrates “text in, text out” services, authorizing users to generate texts with logical prompts. The application programming interface (API) outlined by the tech experts can deliver sensible sentences, paragraphs, even write poems, whatever the user demands. The AI model will perform the given task(s) accurately only when the user feeds the right codes with specific commands and tasks.
Like a newborn who takes a few years to understand languages and start speaking, OpenAI’s GPT-3 needs to be told the same set of commands at least a million times to get it right. It is alarmingly surprising how the process to train the AI model is almost similar to teaching a human being how to perform particular tasks. GPT-3 can generate short stories, lyrics, poems, articles, etc., with the right set of prompts, images, and codes. With the assistance of a ‘masking’ technique, the AI model can fill in the blanks. For instance, “An __ a day keeps the doctor away.” The text-based model, GPT-3, can potentially autocomplete sentences/paragraphs; however, it may make grammatical errors.
While this AI model baffled people with its predictive power, the experts saw the immense potential in advancing the same technology, take it up a notch, i.e., contextualizing images.
Generate Images from Captions: Image Masking
Before we understand the wizardry of how an AI model can generate images from texts, one needs to comprehend the underlying technology of predictive texts and images. With the help of Google’s BERT, AI models can now interpret texts and generate coherent sentences, slowly getting closer to understand the English language.
The wonders of image masking allow the AI model to understand all the words and the image to fill in the gaps with current or close second words. The understanding behind this technology is that AI can use closely related text and visual references to generate articulate sentences and paragraphs. To imitate human minds, rather on an experimental basis, if AIs have the conceptual understanding of the real world, researchers at Allen Institute for Artificial Intelligence (AI2) wanted to level up GPT-3, where the AI model can potentially create apt images based on textual references.
Visual Language Model: X-LXMERT
The curious minds at AI2 wondered if it is possible to generate images just with a string of words? And voila! Here we have, X-LXMERT, an AI model designed to deliver pictures with written captions. At first, the model generated meaningless and pixelated photographs. An extension of LXMERT, X-LXMERT, after refinements and training, the model can essentially generate pictures, answer questions, and write captions resulting from multi-modal transformers.
However, with repeated actions, different approaches at image masking, and after a million trials, the model could generate images close to the meaning of captions. Pictures weren’t exactly realistic, although the results were satisfactory; it proved how AI is getting closer to understand the human world (it’s equally scary, we don’t want robots to colonize the world after all).
In a report by MIT Technology Review, researcher Jiasen Lu says that the missing piece of the puzzle has been image generation. With this notable advancement, AI models can interpret and represent the world with more accuracy.
Afterthoughts
Developments in the field of AI will continue with an array of new opportunities. With the ever-changing landscape in the digital revolution, all organizations across various sectors must maximize the power of technology to their advantage. With the help of increased understanding of the online world and prominent technological advancement, there is a bright future for research of virtual environments.
In this era of ‘virtual data,’ data-driven technologies will lead to transformation; a big contributor to the growth of AI worldwide is the willingness of people to adapt to such technologies, which eventually paves the path for a more general-purpose AI. The AI ecosystem will thrive on the right mix of technology, creativity, and vernacular. While it does so, you get to generate images from captions alone!
Dear Reader,
First, thank you for your precious time reading the stories (without paywalls) I publish on Startups to Enterprises covering the EU, China, the US, and India. Second, I request you to contribute financially (any amount) to help me sustain this as an independent digital business news media.
If I receive a request for a sponsored post, I ensure I see merit that is meaningful for erudite and informed readers like you. In the bargain, I lose out on sponsorships wherein I need funds to sustain this effort. Your contribution helps me stay afloat.
Please note that your contribution is treated as revenue generated and not a donation; hence, there are no 80G or other donation certificates. In fact, as I am eligible to pay for the revenue generated, I will pay taxes on the same.
You deserve to know that I abide by journalistic ethics and practices to ensure I tell the stories as is, unbiased. You can follow us on Facebook, Linkedin, and Twitter, bookmark us on Google News, and finally, PayPal us here.
Founding Editor
Linda Ashok