2019年4月15日月曜日

Using MIT App Inventor AWS AI Services Extension

Modern AI image recognition allows us to recognize not only objects, but also the accompanying text (strings). Here we will try it using Amazon AWS Rekognition. Using the APIs provided there, we can create a variety of applications on our PC, but for more convenient use, the applications on our smartphone will be preferable.

Recently, I focused on the AWS AI Services Extension developed by Ceyhun Özgün. This is an Extension that allows MIT App Inventor applications to easily take advantage of such recognition features. References [1] and [2] provide descriptions and demo programs. I've created a simple smartphone application like Fig.1 below, using this useful information as a reference.


(Please click to enlarge)

Let me illustrate the recognition results in this diagram. Fig.1 (a) is an image of the US $2 bill. Immediately, “Money, Person, Dollar” was recognized. In addition, the text in the image was also recognized. In particular, the number “J045... 5A”, which is the id of the banknote, is also recognized precisely. Fig.1 (b), on the other hand, is an image of a notebook and books. It is recognized as “Book”, and the text of the titles of the books (although handwriting is also included) is almost accurate. I think it's great. These smartphone apps are fun to use wherever you take them!

The design part of this app is shown in Fig.2. Here, I used AppyBuilder. You can choose to take a picture now or use an image that you already have in your library. After that, recognition is performed automatically and the results of recognition of objects and text are displayed.


Next Fig.3 is the complete blocks of this app. Thanks to the Extension mentioned above, you will find that it has become a very concise program. In order to use this Extension, you need to set up an AWS account. In particular, be aware that you will give your AWS Services credentials on the first run. Details of this are also described in the literature [1]. Thanks to Ceyhun Özgün!


References
[1] https://hackernoon.com/your-kid-can-code-a-fruit-detector-with-this-mit-app-inventor-aws-ai-services-extension-231665c8bcdc
[2] https://github.com/ceyhunozgun/awsAIServicesAppInventorExtension/wiki

0 件のコメント:

コメントを投稿