Software recognizes and describes images in words
Last Updated: Thursday, November 2, 2006 | 4:39 PM ET
CBC News
U.S. researchers have developed a computer system that can automatically recognize the content of a photograph and describe it in English.
The system by Penn State University professors uses a vocabulary of up to 332 words to annotate a photo with subject-relevant descriptors or keywords. For example, an image of a polo match could be described by the system as "sport," "people," "horse," "polo," the researchers said.
The technology makes it possible to automatically tag images with keywords, rather than having a person manually label the photos. The system can tag online collections of images as they are uploaded.
Image search engines currently rely on text tags to help index and sort images, so those that don't have descriptions are effectively invisible to search requests.
The Automatic Linguistic Indexing of Pictures Real-Time (ALIPR) system developed by Penn State associate professors James Wang and Jia Li solves the problem by analyzing the images and comparing them against a database. The computer then suggests 15 possible tags for the image.
"By inputting tens of thousands of images, we have trained computers to recognize certain objects and concepts and automatically annotate those new or unseen images," Wang said in a statement. "More than half the time, the computer's first tag out of the top 15 tags is correct."
The analysis takes about 1.4 seconds per image and in 98 per cent of tests suggests at least one correct tag in the top 15.
The university has applied for a patent on the invention.
