Computer sketches set to make online shopping much easier

Computer sketches set to make online shopping much easier
Demonstration of SketchX computer program in action. Credit: Queen Mary, University of London

A computer program that recognises sketches pioneered by scientists from Queen Mary University of London (QMUL) could help consumers shop more efficiently.

The sketches of a pair of shoes or piece of furniture, for example, are drawn directly by hand on a touchscreen and recognised using a sophisticated image retrieval system, where the top 10 retrieval accuracy is close to 100 per cent on some object categories so that it always displays the desired product on the first page.

"What's great about our system is that the user doesn't have to be an artist for the sketch to be accurate, yet is able to retrieve images in a more precise manner than text," said co-developer Dr Yi-Zhe Song, Director of the SketchX Research Lab, from QMUL's School of Electronic Engineering and Computer Science.

He continued: "With the proliferation of touchscreens, sketching had become a much easier to do and in some ways is actually preferable to text-based or photo searches."

Fine-grained sketch-based image retrieval (SBIR) overcomes problems with using words to describe visual objects in words, especially when dealing with precise details, and with using photos, which can restrict the search far too narrowly.

Co-developer Dr Timothy Hospedales and Director of the Applied Machine Learning Lab also from QMUL's School of Electronic Engineering and Computer Science, said: "For the first time users are presented with a system that can perform fine-grained retrieval of images - this leap forward unlocks the commercial adaptation of image search for companies.

"Current image search capabilities provided by online retailers only perform category-level searches so for customers it is still a tedious 'browsing' exercise to find your ideal item. Now you can simply sketch your mental image of your ideal product and go straight to it.

The system is designed to emulate the human brain's processing through arrays of simulated neurons. It was trained to match sketches to photos based on about 30,000 sketch-photo comparisons, learning how to interpret salient details of photos and how people try to depict them in hand-drawing.

The research team has been working with Queen Mary Innovation for the past year, and have already identified commercial interest from multiple major retailers as well as potential investors.

The research was accepted at International Conference on Computer Vision and Pattern Recognition (CVPR) and will be presented at the end of June.

Citation: Computer sketches set to make online shopping much easier (2016, June 24) retrieved 28 March 2024 from https://phys.org/news/2016-06-online-easier.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

New computer program first to recognise sketches more accurately than a human

94 shares

Feedback to editors