New translator app makes sense of foreign-language food menus

Researchers have created an application that enables cell phones and other portable devices to translate foreign-language food menus for English speakers and could be used for people who must follow restricted diets for medical reasons.

"You type in the menu listing and the application translates it automatically without talking to a server," said Mireille "Mimi" Boutin, an associate professor in Purdue University's School of Electrical and Computer Engineering. "It only takes a fraction of a second, you don't need connection to the Internet and it won't empty your battery."

Before entering a foreign country, the user would download a region- and language-specific configuration and database. From then on, the system can operate without a network connection.

"The problem with menus is that even if you know the language you may still have to ask questions to clarify what a dish contains," Boutin said. "For example, in German, "Schinken" means ham, but it can be raw ham or cooked ham. If you are going to eat the ham, you might want to know which."

The user types the desired dish into a prompt field in the graphical user interface. The text is translated, and the best possible translations are then listed, along with other information, including pictures and ingredients. The user can then browse the multimedia database to obtain more information about the dish or the ingredients. When appropriate, information and questions for the waiter are suggested.

"With Albert Parra Pozo, a graduate student who is fluent in Spanish, we were able to develop and implement this system on the iPod Touch for English speakers traveling in Spain," said Boutin, who specializes in signal and image processing. "Our tests indicate that our system yields a correct translation more often than general-purpose translation engines. Moreover, it does so almost instantaneously. The memory requirements of the application, including the database of pictures, are also well within the limits of the device."

The researchers used a method called n-gram consolidation that improves the translation accuracy and dramatically reduces the size of the database while increasing search speed.

"As a result, our application can be used in a real-time, network-independent environment and produce highly accurate results," Boutin said. "We have a prototype in Spanish and have demonstrated the translation can be done in real time within a fraction of a second.

Findings were detailed in a paper presented in July at the IEEE International Conference on Multimedia and Expo in Barcelona, Spain. The paper was written by Parra, graduate student Andrew W. Haddad, Boutin and Edward J. Delp, the Charles William Harrison Distinguished Professor of Electrical and Computer Engineering.

"We believe the approach we are taking in this project is extendable to other types of mobile devices, including iPads and also the Android mobile phones," Delp said. "The results have been extremely good."