The Rapid-Rich Object Search (ROSE) Lab, a collaboration between the Nanyang Technological University and Peking University, is partnering with Tencent, one of Asia’s largest Internet companies, to develop mobile search technology. WeChat (or Weixin in China) is Tencent’s mobile messaging service with 400 million users worldwide. The WeChat Smart Platformprovides APIs for the development of applications leveraging on image recognition and speech recognition technology.

This Challenge aims to boost the development of mobile search applications in Singapore. Participating teams are expected to develop apps based on the WeChat Smart Platform.

Tencent’s WeChat team collaborated with the ROSE Lab to develop the Image Recognition module of the WeChat Smart Platform. Mobile app developers can use the new technology, available free online, to create new apps on Tencent’s popular WeChat platform. The new image recognition and search software allows for faster processing of information compared to any similar software currently available. Mobile developers could develop a WeChat app using the software’s photo search function, enabling users to retrieve information on almost anything they point their camera at.

To commemorate the initial success of its collaboration with Tencent, NTU is planning to host Singapore’s first competition to develop the most creative and innovative apps for WeChat, using the new Smart Platform software development kit.

Specially organised for the Singapore market, the competition is expected to be held in the second half of this year. This will be Tencent’s first competition based on the WeChat Smart Platform in South-East Asia.

Currently, when a photo is taken on a mobile phone, the entire photo is sent to the server to be analysed and to search for matching photos and related information on the database. This consumes a lot of data bandwidth and server resources, worsened by multiple independent developers, each with their own dataset. For a system with a few hundred million users, the system infrastructure will be overloaded and the search will be slowed down.

In this newly designed database system developed by ROSE and Tencent, the algorithm in the WeChat image recognition software will analyse and extract relevant bits of information (descriptors) from the photo taken.

Compact descriptors are used to search the server database for similar images and related information. By letting the mobile device handle the task of extracting the feature descriptors, the load on the server is reduced and data bandwidth usage is kept to a minimum, thus increasing the speed of the search. Rigorous testing by Tencent showed that the jointly-developed image retrieval techniques are better in search performance and data retrieval, compared to other state-of-the-art approaches.