Remarkable new method lets AI learn with almost no data

Researchers from the University of Waterloo have just developed a technique that lets them train AI on almost no data.

Published in a paper, the researchers have shown how their technique works which they have dubbed the “less than one” shot learning. Typically any Machine Learning algorithm takes huge amounts of data to ‘learn’ different classes. A class represents an object of the real world e.g. a Car. These algorithms also require different images of the same object as well. This allows it to be robust and identify the object from any angle, direction, or in any lighting conditions.

So how does their algorithm work on very little data? Researchers usually use the MNIST dataset to test new algorithms. It consists of 60,000 thousand images of digits from 0 to 9.

The researchers first used a technique called distilling to reduce this dataset from 60,000 images to only 10 images. They reduced the images such that there is an overlap of digits. For example, if we have an image of the digit 9, they also added features of similar digits like 7 or 0.

After they get 10 images, they gave soft labels to these images. What this means is that for the above-mentioned example, they told the AI that the image is 70% the digit 9, 15% the digit 7, and 15% the digit 0. This allows the AI to set thresholds, both upper and lower, for what an image might be.

So where is the limit for this algorithm? Simply put, there might not even be one. The model is built on the inference ability of normal humans. Like humans can identify something just from its description, the AI should as well. If you tell a kid that a unicorn is something in between a rhino and a horse and show him a picture of a unicorn. He will most likely point out that the image is a unicorn without ever seeing one before. This algorithm tends to do the same.

One limitation however maybe that you need to make a custom set of images that still requires a lot of images to produce.

Image Source: Ditech, Vizard, Tongzhou et al, Medium