It can also draw and combine multiple objects and offer different points of view, including recesses and objects. Unlike previous text-to-image programs, it even contains details that are not mentioned in the description, but that are necessary for a realistic image. With the description “a painting of a fox sitting in a field during the winter”, the agent was able to determine that a shadow was needed.
“Unlike a 3D rendering engine, the input of which must be unambiguously and fully specified, DALL · E is often able to ‘fill in the blanks’ if the caption implies that the image must contain a certain detail. which is not explicitly stated, “according to the OpenAI team.
OpenAI also utilizes a capability called ‘zero-shot reasoning’. It enables an agent to generate a response from a description and direction without any additional training, and is used for translation and other tasks. This time, the researchers applied it to the visual domain to perform both image-to-image and text-to-image translation. In one example, it was able to generate an image of a cat from a sketch, with the indication “exactly the same cat at the top as the sketch at the bottom.”
The system has numerous other talents, such as understanding how phones and other objects change over time, grasping geographic facts and landmarks, and creating images in photographic, illustrative, and even illustrations.
For now, DALL-E is pretty limited. Sometimes it delivers what you expect from the description, and other times you get some weird or crazy images. As with other AI systems, even the researchers themselves do not understand exactly how it produces certain images due to the black box nature of the system.
If developed further, DALL-E still has great potential to disrupt fields like photography and illustrations, with all the good and bad things. “In the future, we plan to analyze how models such as DALL · E relate to social issues such as economic impact on certain work processes and occupations, the possibility of bias in model outputs and the longer-term ethical challenges that this technology implies,” the team written. To play with DALL-E yourself, go to OpenAI’s blog.