Introduction

The Rapid-Rich Object Search (ROSE) Lab, a collaboration between the Nanyang Technological University and Peking University, is partnering with Tencent, one of Asia’s largest Internet companies, to develop mobile search technology. WeChat (or Weixin in China) is Tencent’s mobile messaging service with 400 million users worldwide. The WeChat Smart Platformprovides APIs for the development of applications leveraging on image recognition and speech recognition technology.

This Challenge aims to boost the development of mobile search applications in Singapore. Participating teams are expected to develop apps based on the WeChat Smart Platform.

Tencent’s WeChat team collaborated with the ROSE Lab to develop the Image Recognition module of the WeChat Smart Platform. Mobile app developers can use the new technology, available free online, to create new apps on Tencent’s popular WeChat platform. The new image recognition and search software allows for faster processing of information compared to any similar software currently available. Mobile developers could develop a WeChat app using the software’s photo search function, enabling users to retrieve information on almost anything they point their camera at.

To commemorate the initial success of its collaboration with Tencent, NTU is planning to host Singapore’s first competition to develop the most creative and innovative apps for WeChat, using the new Smart Platform software development kit.

Specially organised for the Singapore market, the competition is expected to be held in the second half of this year. This will be Tencent’s first competition based on the WeChat Smart Platform in South-East Asia.

The Technology: WeChat Smart Platform

Currently, when a photo is taken on a mobile phone, the entire photo is sent to the server to be analysed and to search for matching photos and related information on the database. This consumes a lot of data bandwidth and server resources, worsened by multiple independent developers, each with their own dataset. For a system with a few hundred million users, the system infrastructure will be overloaded and the search will be slowed down.

In this newly designed database system developed by ROSE and Tencent, the algorithm in the WeChat image recognition software will analyse and extract relevant bits of information (descriptors) from the photo taken.

Compact descriptors are used to search the server database for similar images and related information. By letting the mobile device handle the task of extracting the feature descriptors, the load on the server is reduced and data bandwidth usage is kept to a minimum, thus increasing the speed of the search. Rigorous testing by Tencent showed that the jointly-developed image retrieval techniques are better in search performance and data retrieval, compared to other state-of-the-art approaches.

 

Challenge Co-Organisers

Rapid-Rich Object Search (ROSE) Lab, NTU

With the proliferation of mobile internet devices, the ROSE Lab was conceived to address the growing need to expand search beyond text’s limited capability in describing real-world objects. The Lab aims to build visual search technology on media cloud platforms to support real-time applications with scalability to user demand.

The ROSE Lab was jointly set up between the Nanyang Technological University (NTU), Singapore, and Peking University (PKU), China, drawing upon each university’s strength in media, computer vision, and cloud computing technologies. The lab is physically located at the Research Techno Plaza at the NTU. The lab is creating an innovation platform for media applications, and develops visual object search tools and services by working with industry players such as Tencent, one of the largest Internet companies in Asia. In addition, ROSE is working with SMEs to license and jointly develop technology.

The ROSE Lab’s research focuses on three areas:

  1. Large-scale structured database of media objects – To facilitate effective mobile search, it is important to collect, analyse, and manage a large-scale database of the images and other information for real-life objects for machine learning and testing. These datasets are structured into vertical application domains (eg. mCommerce & Digital Advertising, Tourism, Lifestyle & Hobbies).
  2. Next generation object search – To enable fast and rich mobile object search with constrained network and computational resources, compact and innovative feature coding, scalable indexing and search algorithms are necessary. Exploiting real-time location- and situation-based data analytics will allow users interact better with their surroundings.
  3. Media cloud platform – This platform will support the enormous demand in storage, transmission and computation for mobile object search. This platform will serve as a test-bed for experimentation.

Tencent

Founded in November, 1998, Tencent has grown into one of China's largest and most used Internet service portal. Since its establishment over the last decade, Tencent has maintained steady growth under its user-oriented operating strategies. In 2004, Tencent Holdings Limited (SEHK 700) went public on the main board of the Hong Kong Stock Exchange.

It is Tencent's mission to enhance the quality of human life through Internet services. Presently, Tencent is providing value-added Internet, mobile and telecom services and online advertising under the strategic goal of providing users with "one-stop online lifestyle services". Tencent’s leading Internet platforms in China – QQ (QQ Instant Messenger), WeChat, QQ.com, QQ Games, Qzone, 3g.QQ.com, SoSo, PaiPai and Tenpay – have brought together China's largest Internet community, to meet the various needs of Internet users including communication, information, entertainment, e-commerce and others. As of Dec 31, 2012, the active QQ users accounts for QQ IM amounted to 798.2 million while its peak concurrent users reached 176.4 million.

WeChat(also known as Weixin in China) is the world’s fastest-growing mobile social application, according to market research firm GlobalWebIndex. As of March 2014, the combined number of monthly active users of WeChat stands at 396 million. The platform brings together messaging, social communication and games all within on easy-to-use app. Users can choose to send free text and multimedia messages, video calls or share photos on their closed Moments social network. Others features include mobile games and convenient friend adding services. Continuously evolving around users' needs, Weixin/WeChat also offers enterprises a platform to create original consumer experiences.

Supporting Partners

NTUitive Pte Ltd

Nanyang Technological University – NTUitive Pte Ltd (pronounced as intuitive), is Nanyang Technological University’s innovation and enterprise company.

At NTUitive, we enable new technologies and ideas to realise their potential by bridging the gap between research and business. We work with researchers in the labs as well as businessmen in the industry to bring new technology in the form of new products, processes, or services to market.

We work with NTU researchers to help identify and protect the intellectual property arising from their research. We also work with them to find market applications for their research by working with industry.

For those who are more entrepreneurial, we help them to create businesses to take their technologies to market. We are an established startup incubator with the infrastructure, access to early stage funding and mentoring support with a dedicated team of incubation managers.

NTUitive is also a one-stop point of contact for industry players looking for technologies to help their businesses grow. We help connect the technologies and expertise at NTU with the needs or businesses.

StarHub i3 (Incubation, Investment, Innovation)

StarHub is Singapore's fully-integrated info-communications company, offering a full range of information, communications and entertainment services for both consumer and corporate markets.

StarHub i3 is formed to lead the company to becoming a top player in technological innovations and digital initiatives. Its role is to discover and deliver new engines of growth outside of the traditional info-communications business for StarHub. This is achieved through forging new partnerships, forming regional think tanks and incubating new ideas from innovative entrepreneurs. These initiatives are underpinned by StarHub’s big data platform – SmartHub, a National Research Foundation-supported project aimed at creating new classes of applications and solutions, in collaboration with higher learning institutions, start-up companies, and large and multinational organisations.