Live Object

LiveObject demonstrates how a computer can learn to recognise objects like pens, glasses, mugs etc. in an image supplied by a camera. Many new and exciting applications would be possible if computers could do this well, so scientific research into object recognition is very important. We could make a car that could drive itself, recognise the items in your shopping basket at the checkout or build a home robot that can do the cleaning for you and find your keys! Object recognition technology could also be used to provide a guide for the blind, for example, by giving a spoken description of the room.

 

How does it work?

LiveObject is able to recognise one object at a time, the instant it is placed in front of the camera.  Rather than being given rules to recognise each object, it learns from examples. To learn about mobile phones for example, we showed the system about twenty different mobiles phones, by placing them in front of the camera, shown from many different angles. Because LiveObject can learn very quickly, it can adapt each time it gets things wrong, so as to be correct next time.  In this way, it ‘gains experience’ as it proceeds and improves in accuracy over time. LiveObject can also learn about new types of objects as it goes along. At the moment, it knows about 15 different types of objects.

 

What aspects of the image does it use?

For a given set of example images, LiveObject searches through tens of thousands of image features (such as colour, texture, shape and edge information) to automatically find a small set of features that discriminate between example images of different classes.  For example, if the images were only of snooker balls then the system would work out that it only needs to use colour to distinguish between them.  For the 15 objects it knows about, the system has learned to use just a few hundred image features.  These features can be computed very quickly, which is why the system works so fast.

 

Where is this research going next?

Currently, our system can recognise only one object at a time.  There are other object recognition methods that we have developed for recognising multiple objects, but they take much longer to run (between 10 and30 seconds per image).  We are interested in speeding up these methods so that we can recognise many objects simultaneously in real time.  Ultimately, we would also like a machine to be able to recognise hundreds or even thousands of types of object in a fraction of a second – just like we can!

 

To read more about our object recognition research, please take a look at other object recognition projects taking place at Microsoft Research Cambridge.