Real-time image recognition demonstration on a smartphone using TensorFlow.js

There are many examples of image recognition (especially classification) by AI, but the "Google AI Experiments" introduced here seems to be further evolved. It was announced at TensorFlow Dev Summit 2019. Do you already know? This Summit's  38 lecture videos are available on youtube:

https://www.youtube.com/watch?v=GRMvCeIKvps

I tried the demo called "Teachable Machine" among them. As shown in Figure 1, the smartphone classifies images into three types. The Javascript version of Tensorflow runs under the web browser. A local Neural Network in the smartphone performs image training and evaluation without sending the image to the server. Of course, as long as this demo is used, no coding is required at all.

Here, we prepare three types of images of a hat, i.e. "back", "side", and "front", as shown in Figure 2, about 10 images each. Is it all right with such a small number of images? You may think that "transfer learning" is working there. In other words, this Teachable Machine should be using a Neural Network, which has been trained with a huge amount of images for a certain purpose.

Let's train this as shown in Figure 3. That is, press the corresponding button (train button) at the bottom to label each image. Each time, the image is trained in real time. For each of the three types, 10 images should be trained. After that, if you take a picture of the hat with either the back, side, or front, the orientation of the hat will be immediately determined. Actually, as you can see, everything was accurately identifiable! 100% confidence!

Next, try shooting with the hat in an ambiguous direction, not exactly on the back, side, nor front. As human beings feel, the result seems that Neural Network can not judge with certainty. It was a judgment that "maybe a side (59% confidence)". This is another good point.

● The university's graduation research is becoming increasingly severe. Even if you work hard to code and create an AI application, it seems that it is difficult to go beyond applications that can easily be made with no coding like this example. But on the other hand, the Teachable Machine demo app is just a demo. Kindly, the source list is also published. It's great if you can use this to solve your own problems. The level of graduation study goes up by this.

● This trial was carried out with smartphone Pixel3a (Android OS 9). Of course, it is also possible to use a PC or Mac, but the point is that you can train and evaluate in real time on your smartphone. Note that this demo will not work on iPhone and iPad.