Software recognizes and describes images in words
Last Updated: Thursday, November 2, 2006 | 4:39 PM ET
CBC News
Related
External Links
(Note: CBC does not endorse and is not responsible for the content of external sites - links will open in new window)
U.S. researchers have developed a computer system that can automatically recognize the content of a photograph and describe it in English.
The system by Penn State University professors uses a vocabulary of up to 332 words to annotate a photo with subject-relevant descriptors or keywords. For example, an image of a polo match could be described by the system as "sport," "people," "horse," "polo," the researchers said.
The technology makes it possible to automatically tag images with keywords, rather than having a person manually label the photos. The system can tag online collections of images as they are uploaded.
Image search engines currently rely on text tags to help index and sort images, so those that don't have descriptions are effectively invisible to search requests.
The Automatic Linguistic Indexing of Pictures Real-Time (ALIPR) system developed by Penn State associate professors James Wang and Jia Li solves the problem by analyzing the images and comparing them against a database. The computer then suggests 15 possible tags for the image.
"By inputting tens of thousands of images, we have trained computers to recognize certain objects and concepts and automatically annotate those new or unseen images," Wang said in a statement. "More than half the time, the computer's first tag out of the top 15 tags is correct."
The analysis takes about 1.4 seconds per image and in 98 per cent of tests suggests at least one correct tag in the top 15.
The university has applied for a patent on the invention.
Share Tools
Top News Headlines
- Quebec man charged with killing mother, 2 nieces
- A 35-year-old man has been charged with three counts of first-degree murder in connection with the deaths of his mother and two young nieces in Quebec's Eastern Townships. more »
- Manitoba trailer fire kills 4
- Four people are dead after an early-morning fire quickly engulfed a residential trailer in Selkirk, Man. more »
- Harper's China visit ends with panda pact

- Prime Minister Stephen Harper wrapped up a visit to China aimed seeking new investments by officially announcing that Beijing will loan two of the country's prized giant pandas to Canadian zoos. more »
- Attawapiskat sites not ready for modular homes
- The first two of 22 modular homes promised by the federal government to Attawapiskat are on their way to the remote northern Ontario community, but the minister handling the Aboriginal Affairs portfolio is expressing concern over the "readiness" of the lots. more »
Latest Technology & Science News Headlines
- Ancient Antarctic lake may harbour microbial life
- If scientists find microbes in a frigid lake 3.2 kilometres beneath the thick ice of Antarctica, it will illustrate once again that somehow life finds a way to survive in the strangest and harshest places, and it will offer hope that life exists beyond Earth. more »
- B.C. killer whale habitat protection ruled a legal duty
- The federal minister of fisheries has no discretion when it comes to protecting the critical habitat of B.C.'s southern resident killer whales, the Federal Court of Appeal has ruled. more »
- Game developer seeks $400K, makes $1M in a day
- Videogame studio Double Fine went on the website Kickstarter to raise $400K US in a month to develop a new game. They reached that target in a matter of hours. more »
- McGill asbestos study review criticized
- A group of anti-asbestos activists and scientists are criticizing McGill University's plans for an internal review of a major asbestos research study that has been called into question. more »
Bob McDonald's Blog
Glacier Discovery Walk: Will the visitor centre enhance the view? Feb. 10, 2012 3:17 PM Environment minister Peter Kent has announced the construction of a new Glacier Discovery Walk and visitor centre on the Icefields Parkway in Jasper National Park. It raises the issue of how to balance commercial development in our National Parks against the preservation of the last refuges of wilderness.
Quirks & Quarks
- February 11: Inside the Mind of a Neandertal Feb. 10, 2012 4:01 PM Can we get inside the mind of a species that's been dead for 30,000 years? A new book, How to Think Like a Neanderthal, suggests we can. The authors reconstruct a creature like us in many ways, but with important differences.
Latest Features
- Quebec man charged with killing mother, 2 nieces
- Harper's China visit ends with panda pact
- Weed Man's sales tactics draw fire from consumer ministry
- Attawapiskat sites not ready for modular homes
- Manitoba trailer fire kills 4
- Emailed rave rape pictures earn teen probation
- RCMP shooting suspect hoped to surrender before arrest
- Bus rolls near Redwater, Alberta, injuring dozens
- Crane drops section of Port Mann bridge into B.C. river

