This model uses the Wio Terminal to recognize hand gestures: rock, paper and scissors using built-in Light sensor.
It is quite hard to accomplish using rule-based programming because gestures are not performed the same way all the time. Trying to solve this problem using traditional Programming requires hundred of different rules for each mode of action. Even if we consider an idealized situation, we still miss a lot of motions such as speed, angle, and direction. A slight change in these factors requires a new set of rules to be defined but machine learning can handle these variations very easily. A well-known possibility is to use camera sensors combined with machine learning to recognize gestures. Using a light sensor is equivalent to using only a one-pixel point of the camera to recognize gestures which is a completely different level of challenge. Let's meet the challenge with this project.
In the figure below, Wio Terminal displays the current results of gesture recognition in real-time. One of rock, scissors, and paper.
Hardware requirements: Wio Terminal Connection method:
In this lesson, we are going to train and deploy a simple neural network for classifying rock-paper-scissors gestures with just a single light sensor.
The working principle of this project is quite trivial. Different gestures being moved above the light sensor will block certain amount of light for certain periods of time. For example, for rock, we will have high values at first (nothing above sensor) but lower values when "rock" passes above the sensor and then high values again. For paper, we will have high-low-high-low-high-low-high-low values when each of the fingers in "paper" passes above the sensor.
- Paper
- Rock
- Scissors
There is a high variance in speed and amplitude of the values from the sensor which makes a great case for using machine learning model instead of a hand-crafted algorithm for the task.
- Creating and Selecting Models
- Data Acquisition
- Training and Deployment
- Programming
Click on "Create and select model", click on " Gesture Recognition (Built-in Light Sensor)", as shown in steps 1 and 2 below.
Enter a NAME according to the requirements.
Click Ok and it will automatically jump to the Data Acquisition interface.
When the Wio Terminal is connected, in the Codecraft surface, click
as shown in the figure. This action will upload the default data acquisition program.
Usually it takes 10 seconds to upload. Once the program is uploaded, "Upload Successful" window will appear on the screen shown below. This is shown in the image below.
Click "Roger" to close the upload success pop-up window above and return to the programming screen.
In the upper right hyperlink, you will find a step-by-step introduction to data acquisition. Follow the instructions to collect data.
Attention:
- Wio Terminal button location.
- Animated gif has been accelerated, the actual action can slightly slow down.
- Please notice the red tips.
- Point the curser over Description Texts for more detailed content.
Wio Terminal will be displayed during the collection process. Start and end collecting data according to the Wio Terminal screen.
Indicates the data is being collected
Indicates the Data collection is complete
Now, the data acquisition is finished.
Click on "Training & Deployment"
Select the suitable neural network size: one of small, medium and large Set parameters, number of training cycles (positive integer), learning rate (number from 0 to 1) and minimum confidence rating(number from 0 to 1). The interface provides default parameter values. In this case we are using MEDIUM. It will take quite a long time.
Click "Start training".
When you click "Start training", the interface displays "Loading...".
The duration of "Loading.." varies depending on the size of the selected neural network (small, medium and large) and the number of training cycles. Larger is the network size and number of training cycles, the longer it takes.
You can also infer the waiting time by observing the "Log". In the figure below, "Epoch: 68/500" indicates the total number of training rounds is 500 while 68 rounds have been trained.
In the "Model Training Report" window, you can observe training results including the accuracy, loss and performance of the model. If the training results are not satisfactory, you can go back to the first step of training the model, select another size of the neural network or adjust the parameters and train it until you get a model with satisfactory results.
In the "Model Training Report" window, click 
Once the deployment is completed, Click "Ok" to jump to the Programming window
In the "Programming" interface, click on "Use Model" to use the deployed model.
Try to use your model by writing the following programme.
Click the "Upload" button.
The first upload time is usually long and it increases with the complexity of the model. The upload time for smaller models is about 4 minutes or longer(depending on the performance of your machine).
Make a gesture "scissors" to see whether Wio Terminal's screen show "scissors". Try other gestures, and see whether the Wio Terminal recognizes your gestures.
Congratulations! You have completed your TinyML model, I believe you are familiar with your input(dataset and labels) and know your output. A good score of "Accuracy" leads you to draw a conclusion that your model is working perfectly but it is not enough for you to evaluate the performance of your model. So, let's take a look at the output.
- ACCURACY
- LOSS
- Confusion matrix
- F1 SCORE
Accuracy is a method of measuring the performance of a classification model. It is typically expressed in percentage value. Accuracy is the count of predictions where the predicted value is equal to the true value. It is binary (true/false) for a particular sample. Accuracy is often graphed and monitored during the training phase though the value is often associated with the overall or final model accuracy. Accuracy is easier to interpret as compared to the loss.
Machine learning model accuracy is the measurement used to determine which model is best at identifying relationships and patterns between variables in a dataset based on the input, or training, data. It is typically expressed in percentage value. Accuracy is the count of predictions where the predicted value is equal to the true value and is simple to calculate. You can check the accuracy of your model by simply dividing the number of correct predictions by the total number of predictions.
A loss function determines the probability or uncertainty of a prediction based on how much prediction varies from the true value. This gives us a more nuanced view of how well the model is performing.
Unlike accuracy, loss is not a percentage. It is a summation of the errors occurred for each sample in training or validation sets. Loss is often used in the training process to find the "best" parameter values for the model (e.g. weights in a neural network). During the training process, the goal is to minimize this value.
Most of the time we observe that accuracy increases with the decrease in loss but this is not always the case. Accuracy and loss have different definitions and they measure different parameters. They often appear to be inversely proportional but there is no mathematical relationship between these two metrics.
Accuracy and loss appear to be inversely proportional, as we can often observe that accuracy increases with the decrease in loss, but in reality, there is no mathematical relationship between the two. Accuracy and loss have different definitions and they measure different parameters. when the loss decreases the accuracy also decreases.
The F1-score is a measure of a model's accuracy on a dataset.
- Theroy:
- Light sensor
- TinyML practice
- ML theory (Output/the training performance)
- Accuracy: A method of measuring the performance of a model.
- Loss: A loss function, also known as a cost function determines the probability or uncertainty of a prediction based on how much prediction varies from the true value.
- Confusion Matrix: A matrix that lays out the performance of a learning algorithm
- F1 Score:
