Each input needs to be multiplied by its corresponding weight:
-| Phrase | -Phrase | -Input \boldsymbol{\times} Weight | -
|---|---|---|
| 12 | -0.5 | -6 | -
| 4 | -–1 | -–4 | -
Each input needs to be multiplied by its corresponding weight:
+| Phrase | +Phrase | +Input \boldsymbol{\times} Weight | +
|---|---|---|
| 12 | +0.5 | +6 | +
| 4 | +–1 | +–4 | +
The weighted inputs are then added together:
Here’s the code to generate a guess:
// Create the perceptron. let perceptron = new Perceptron(3); -// The input is three values: x, y, and the bias. +// The input is three values: x, y, and the bias. let inputs = [50, -12, 1]; // The answer! let guess = perceptron.feedForward(inputs);@@ -521,17 +529,17 @@ function draw() {
Now imagine you’re classifying plants according to soil acidity (x-axis) and temperature (y-axis). Some plants might thrive in acidic soils but only within a narrow temperature range, while other plants prefer less acidic soils but tolerate a broader range of temperatures. A more complex relationship exists between the two variables, so a straight line can’t be drawn to separate the two categories of plants, acidophilic and alkaliphilic (see Figure 10.10, right). A lone perceptron can’t handle this type of nonlinearly separable problem. (Caveat here: I’m making up these scenarios. If you happen to be a botanist, please let me know if I’m anywhere close to reality.)
+One of the simplest examples of a nonlinearly separable problem is XOR (exclusive or). This is a logical operator, similar to the more familiar AND and OR. For A AND B to be true, both A and B must be true. With OR, either A or B (or both) can be true. These are both linearly separable problems. The truth tables in Figure 10.11 show their solution space. Each true or false value in the table shows the output for a particular combination of true or false inputs. See how you can draw a straight line to separate the true outputs from the false ones?
One of the simplest examples of a nonlinearly separable problem is XOR (exclusive or). This is a logical operator, similar to the more familiar AND and OR. For A AND B to be true, both A and B must be true. With OR, either A or B (or both) can be true. These are both linearly separable problems. The truth tables in Figure 10.11 show their solution space. Each true or false value in the table shows the output for a particular combination of true or false inputs. See how you can draw a straight line to separate the true outputs from the false ones?
The XOR operator is the equivalent of (OR) AND (NOT AND). In other words, A XOR B evaluates to true only if one of the inputs is true. If both inputs are false or both are true, the output is false. To illustrate, let’s say you’re having pizza for dinner. You love pineapple on pizza, and you love mushrooms on pizza, but put them together—yech! And plain pizza, that’s no good either!
+The XOR truth table in Figure 10.12 isn’t linearly separable. Try to draw a straight line to separate the true outputs from the false ones—you can’t!
The XOR truth table in Figure 10.12 isn’t linearly separable. Try to draw a straight line to separate the true outputs from the false ones—you can’t!
The fact that a perceptron can’t even solve something as simple as XOR may seem extremely limiting. But what if I made a network out of two perceptrons? If one perceptron can solve the linearly separable OR and one perceptron can solve the linearly separate NOT AND, then two perceptrons combined can solve the nonlinearly separable XOR.
When you combine multiple perceptrons, you get a multilayered perceptron, a network of many neurons (see Figure 10.13). Some are input neurons and receive the initial inputs, some are part of what’s called a hidden layer (as they’re connected to neither the inputs nor the outputs of the network directly), and then there are the output neurons, from which the results are read.
Now for the heart of the machine learning process: actually training the model. Here’s the code:
// The train() method initiates the training process.
classifier.train(finishedTraining);
-
// A callback function for when the training is complete
function finishedTraining() {
console.log("Training complete!");
@@ -833,7 +840,7 @@ classifier.train(options, finishedTraining);
You may have noticed that I never did this. For simplicity, I’ve instead used the entire dataset for training. After all, my dataset has only eight records; it’s much too small to divide three sets! With a large dataset, this three-way split would be more appropriate.
Using such a small dataset risks the model overfitting the data, however: the model becomes so tuned to the specific peculiarities of the training data that it’s much less effective when working with new, unseen data. The main reason to use a validation set is to monitor the model during the training process. As training progresses, if the model’s accuracy improves on the training data but deteriorates on the validation data, it’s a strong indicator that overfitting might be occurring. (The testing set is reserved strictly for the final evaluation, one more chance after training is complete to gauge the model’s performance.)
-For more realistic scenarios, ml5.js provides a way to split up the data, as well as automatic features for employing validation data. If you’re inclined to go further, you can explore the full set of neural network examples on the ml5.js website.
+For more realistic scenarios, ml5.js provides a way to split up the data, as well as automatic features for employing validation data. If you’re inclined to go further, you can explore the full set of neural network examples on the ml5.js website.
After the evaluation step, there’s typically an iterative process of adjusting hyperparameters and going through training again to achieve the best performance from the model. While ml5.js offers capabilities for parameter tuning (which you can learn about in the library’s reference), it isn’t really geared toward making low-level, fine-grained adjustments to a model. Using TensorFlow.js directly might be your best bet if you want to explore this step in more detail, since it offers a broader suite of tools and allows for lower-level control over the training process.
In this case, tuning the parameters isn’t strictly necessary. The graph in the Visor shows a loss all the way down at 0.1, which is plenty accurate for my purposes. I’m happy to move on.