noc-book-2/content/10_nn.html

<section data-type="chapter">
<h1 id="chapter-10-neural-networks">Chapter 10. Neural Networks</h1>
<blockquote data-type="epigraph">
  <p>“The human brain has 100 billion neurons, each neuron connected to 10 thousand other neurons. Sitting on your shoulders is the most complicated object in the known universe.”</p>
  <p>— Michio Kaku</p>
</blockquote>
<p>I began with inanimate objects living in a world of forces, and gave them desires, autonomy, and the ability to take action according to a system of rules. Next, I allowed those objects, now called creatures, to live in a population and evolve over time. Now I’d like to ask: What is each creature’s decision-making process? How can it adjust its choices by learning over time? Can a computational entity process its environment and generate a decision?</p>
<p>The human brain can be described as a biological neural network—an interconnected web of neurons transmitting elaborate patterns of electrical signals. Dendrites receive input signals and, based on those inputs, fire an output signal via an axon. Or something like that. How the human brain actually works is an elaborate and complex mystery, one that I certainly am not going to attempt to tackle in rigorous detail in this chapter.</p>
<figure>
  <img src="images/10_nn/10_nn_1.png" alt="Figure 10. An illustration of a neuron with dendrites and an axon connected to another neuron.">
  <figcaption>Figure 10. An illustration of a neuron with dendrites and an axon connected to another neuron.</figcaption>
</figure>
<p>The good news is that developing engaging animated systems with code does not require scientific rigor or accuracy, as you've learned throughout this book. You can simply be inspired by the idea of brain function.</p>
<p>In this chapter, I'll begin with a conceptual overview of the properties and features of neural networks and build the simplest possible example of one (a network that consists of a single neuron). I’ll then introduce you to more complex neural networks using the ml5.js library. Finally, I'll cover “neuroevolution”, a technique that combines genetic algorithms with neural networks to create a “Brain” object that can be inserted into the <code>Vehicle</code> class and used to calculate steering.</p>
<h2 id="artificial-neural-networks-introduction-and-application">Artificial Neural Networks: Introduction and Application</h2>
<p>Computer scientists have long been inspired by the human brain. In 1943, Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, developed the first conceptual model of an artificial neural network. In their paper, "A logical calculus of the ideas immanent in nervous activity,” they describe the concept of a neuron, a single cell living in a network of cells that receives inputs, processes those inputs, and generates an output.</p>
<p>Their work, and the work of many scientists and researchers that followed, was not meant to accurately describe how the biological brain works. Rather, an <em>artificial</em> neural network (hereafter referred to as a “neural network”) was designed as a computational model based on the brain to solve certain kinds of problems.</p>
<p>It’s probably pretty obvious to you that there are problems that are incredibly simple for a computer to solve, but difficult for you. Take the square root of 964,324, for example. A quick line of code produces the value 982, a number your computer computed in less than a millisecond. There are, on the other hand, problems that are incredibly simple for you or me to solve, but not so easy for a computer. Show any toddler a picture of a kitten or puppy and they’ll be able to tell you very quickly which one is which. Say “hello” and shake my hand one morning and you should be able to pick me out of a crowd of people the next day. But need a machine to perform one of these tasks? Scientists have already spent entire careers researching and implementing complex solutions.</p>
<p>The most prevalent use of neural networks in computing today involves these “easy-for-a-human, difficult-for-a-machine” tasks known as pattern recognition. These encompass a wide variety of problem areas, where the aim is to detect, interpret, and classify data. This includes everything from identifying objects in images, recognizing spoken words, understanding and generating human-like text, and even more complex tasks such as predicting your next favorite song or movie, teaching a machine to win at complex games, and detecting unusual cyber activities.</p>
<figure class="half-width-right">
  <img src="images/10_nn/10_nn_2.png" alt="Figure 10.2">
  <figcaption>Figure 10.2</figcaption>
</figure>
<p>One of the key elements of a neural network is its ability to learn. A neural network is not just a complex system, but a complex <strong>adaptive</strong> system, meaning it can change its internal structure based on the information flowing through it. Typically, this is achieved through the adjusting of weights. In the diagram above, each line represents a connection between two neurons and indicates the pathway for the flow of information. Each connection has a <strong>weight</strong>, a number that controls the signal between the two neurons. If the network generates a “good” output (which I'll define later), there is no need to adjust the weights. However, if the network generates a “poor” output—an error, so to speak—then the system adapts, altering the weights in order to improve subsequent results.</p>
<p>There are several strategies for learning, and I'll examine two of them in this chapter.</p>
<ul>
  <li><strong>Supervised Learning</strong> —Essentially, a strategy that involves a teacher that is smarter than the network itself. For example, let’s take the facial recognition example. The teacher shows the network a bunch of faces, and the teacher already knows the name associated with each face. The network makes its guesses, then the teacher provides the network with the answers. The network can then compare its answers to the known “correct” ones and make adjustments according to its errors. Our first neural network in the next section will follow this model.</li>
  <li><strong>Unsupervised Learning</strong> —Required when there isn’t an example data set with known answers. Imagine searching for a hidden pattern in a data set. An application of this is clustering, i.e. dividing a set of elements into groups according to some unknown pattern. I won’t be showing at any examples of unsupervised learning in this chapter, as this strategy is less relevant for the examples in this book.</li>
  <li><strong>Reinforcement Learning</strong> —A strategy built on observation. Think of a little mouse running through a maze. If it turns left, it gets a piece of cheese; if it turns right, it receives a little shock. (Don’t worry, this is just a pretend mouse.) Presumably, the mouse will learn over time to turn left. Its neural network makes a decision with an outcome (turn left or right) and observes its environment (yum or ouch). If the observation is negative, the network can adjust its weights in order to make a different decision the next time. Reinforcement learning is common in robotics. At time <code>t</code>, the robot performs a task and observes the results. Did it crash into a wall or fall off a table? Or is it unharmed? I'll showcase how reinforcement learning works in the context of our simulated steering vehicles.</li>
</ul>
<p>Reinforcement learning comes in many variants and styles. In this chapter, while I will lay the groundwork of neural networks using supervised learning, my primary focus will be a technique related to reinforcement learning known as <em>neuroevolution</em>. This method builds upon the code from chapter 9 and "evolves" the weights (and in some cases, the structure itself) of a neural network over generations of "trial and error" learning. It is especially effective in environments where the learning rules are not precisely defined or the task is complex with numerous potential solutions. And yes, it can indeed be applied to simulated steering vehicles!</p>
<p>A neural network itself is a “connectionist” computational system. The computational systems I have been writing in this book are procedural; a program starts at the first line of code, executes it, and goes on to the next, following instructions in a linear fashion. A true neural network does not follow a linear path. Rather, information is processed collectively, in parallel throughout a network of nodes (the nodes, in this case, being neurons).</p>
<p>Here I am showing yet another example of a complex system, much like the ones seen throughout this book. Remember how the individual boids in a flocking system, following only three rules—separation, alignment, cohesion, created complex behaviors? The individual elements of a neural network network are equally simple to understand. They read an input, a number, process it, and generate an output, another number. A network of many neurons, however, can exhibit incredibly rich and intelligent behaviors, echoing the complex dynamics seen in a flock of boids.</p>
<p>This ability of a neural network to learn, to make adjustments to its structure over time, is what makes it so useful in the field of artificial intelligence. Here are some standard uses of neural networks in software today.</p>
<ul>
  <li><strong>Pattern Recognition</strong> — As I’ve discussed, this is one of the most common applications, with examples that range from facial recognition and optical character recognition to more complex tasks like gesture recognition.</li>
  <li><strong>Time Series Prediction and Anomaly Detection</strong> — Neural networks are utilized both in forecasting, such as predicting stock market trends or weather patterns, and in recognizing anomalies, which can be applied to areas like cyberattack detection and fraud prevention.</li>
  <li><strong>Natural Language Processing (or “NLP” for short)</strong> — One of the biggest developments in recent years has been the use of neural networks for processing and understanding human language. They are used in various tasks including machine translation, sentiment analysis, text summarization, and are the underlying technology behind many digital assistants and chat bots.</li>
  <li><strong>Signal Processing and Soft Sensors</strong> — Neural networks play a crucial role in devices like cochlear implants and hearing aids by filtering noise and amplifying essential sounds. They're also involved in 'soft sensor' scenarios, where they process data from multiple sources to give a comprehensive analysis of the environment.</li>
  <li><strong>Control and Adaptive Decision-Making Systems</strong> — These applications range from autonomous systems like self-driving cars and drones, to adaptive decision-making used in game playing, pricing models, and recommendation systems on media platforms.</li>
  <li><strong>Generative Models</strong> — The rise of novel neural network architectures has made it possible to generate new content. They are used for synthesizing images, enhancing image resolution, style transfer between images, and even generating music and video.</li>
</ul>
<p>This is by no means a comprehensive list of applications of neural networks. But hopefully it gives you an overall sense of the features and possibilities. Today, leveraging machine learning in creative coding and interactive media is not only feasible, but increasingly common. Two libraries that you may want to consider exploring further for working with neural networks are tensorflow.js and ml5.js. TensorFlow.js<strong> </strong>is an open-source library that lets you define, train, and run machine learning models in JavaScript. It's part of the TensorFlow ecosystem, which is maintained and developed by by Google. ml5.js is a library built on top of tensorflow.js designed specifically for use with p5.js. It’s goal is to be beginner friendly and make machine learning approachable for a braod audience of artists, creative coders, and students.</p>
<p>One of the more common things to do with tensorflow.js and ml5.js is to use something known as a “pre-trained model.” A “model” in machine learning is a specific setup of neurons and connections and a “pre-trained” model is one that has already been trained on a dataset for a particular task. It can be used “as is” or as a starting point for additional learning (commonly referred to as “transfer learning”).</p>
<p>Examples of popular pretrained models are ones that can classify images, identify body poses, recognize facial landmarks or hand positions, or even analyze the sentiment expressed in a text. Covering the full gamit of possibilities in this rapidly expanding and evolving space probably merits an entire additional book, maybe a series of books. And by the time that book was printed it would probably be out of date.</p>
<p>So instead, for me, as I embark on this last hurrah in the nature of code, I’ll stick to just two things. First, I’ll look at how to build the simplest of all neural networks from scratch using only p5.js. The goal is to gain an understanding of how the concepts of neural networks and machine learning are implemented in code. Second, I’ll explore one library, specifically ml5.js, which offers the ability to create more sophisticated neural network models and use them to drive simulated vehicles.</p>
<h2 id="the-perceptron">The Perceptron</h2>
<p>Invented in 1957 by Frank Rosenblatt at the Cornell Aeronautical Laboratory, a perceptron is the simplest neural network possible: a computational model of a single neuron. A perceptron consists of one or more inputs, a processor, and a single output.</p>
<figure>
  <img src="images/10_nn/10_nn_3.png" alt="Figure 10.3: The perceptron ">
  <figcaption>Figure 10.3: The perceptron </figcaption>
</figure>
<p>A perceptron follows the “feed-forward” model, meaning inputs are sent into the neuron, are processed, and result in an output. In the diagram above, this means the network (one neuron) reads from left to right: inputs come in, output goes out.</p>
<p>Let’s follow each of these steps in more detail.</p>
<p><span class="highlight">Step 1: Receive inputs.</span></p>
<p>Say I have a perceptron with two inputs—let’s call them <em>x0</em> and <em>x1</em>.</p>
<table>
  <thead>
    <tr>
      <th>Input</th>
      <th>Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>x0</td>
      <td>12</td>
    </tr>
    <tr>
      <td>x1</td>
      <td>4</td>
    </tr>
  </tbody>
</table>
<p><span class="highlight">Step 2: Weight inputs.</span></p>
<p>Each input sent into the neuron must first be weighted, meaning it is multiplied by some value, often a number between -1 and 1. When creating a perceptron, the inputs are typically assigned random weights. Let’s give the example inputs the following weights:</p>
<table>
  <thead>
    <tr>
      <th>Weight</th>
      <th>Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>w0</td>
      <td>0.5</td>
    </tr>
    <tr>
      <td>w1</td>
      <td>-1</td>
    </tr>
  </tbody>
</table>
<p>The next step is each input and multiply it by its weight.</p>
<table>
  <thead>
    <tr>
      <th>Weight</th>
      <th>Input</th>
      <th>Weight <span data-type="equation">\times</span> Input</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>12</td>
      <td>0.5</td>
      <td>6</td>
    </tr>
    <tr>
      <td>4</td>
      <td>-1</td>
      <td>-4</td>
    </tr>
  </tbody>
</table>
<p><span class="highlight">Step 3: Sum inputs.</span></p>
<p>The weighted inputs are then summed.</p>
<p><span data-type="equation">6 + -4 = 2</span></p>
<p><span class="highlight">Step 4: Generate output.</span></p>
<p>The output of a perceptron is produced by passing the sum through an activation function. Think about a “binary” output, one that is only “off” or “on” like an LED. In this case, the activation function determines whether the perceptron should "fire" or not. If it fires, the light turns on; otherwise, it remains off.</p>
<p>Activation functions can get a little bit hairy. If you start reading about activation functions in artificial intelligence textbooks, you may find yourself reaching for a calculus textbook. However, with your new friend the simple perceptron, there’s an easy option which demonstrates the concept. Let’s make the activation function the sign of the sum. In other words, if the sum is a positive number, the output is 1; if it is negative, the output is -1.</p>
<p><span data-type="equation">\text{sign}(2) = +1</span></p>
<p>Let’s review and condense these steps and translate them into code.</p>
<p><strong>The Perceptron Algorithm:</strong></p>
<ol>
  <li>For every input, multiply that input by its weight.</li>
  <li>Sum all of the weighted inputs.</li>
  <li>Compute the output of the perceptron based on that sum passed through an activation function (the sign of the sum).</li>
</ol>
<p>I can start writing this algorithm in code using two arrays of values, one for the inputs and the weights.</p>
<pre class="codesplit" data-code-language="javascript">let inputs  = [12 , 4];
let weights = [0.5, -1];</pre>
<p>Step #1 "for every input" implies a loop that multiplies each input by its corresponding weight. To obtain the sum, the results can be added up in that same loop.</p>
<pre class="codesplit" data-code-language="javascript">// Steps 1 and 2: Add up all the weighted inputs.
let sum = 0;
for (let i = 0; i &#x3C; inputs.length; i++) {
  sum += inputs[i] * weights[i];
}</pre>
<p>With the sum, I can then compute the output.</p>
<pre class="codesplit" data-code-language="javascript">// Step 3: Passing the sum through an activation function
let output = activate(sum);

// The activation function
function activate(sum) {
  //{!5} Return a 1 if positive, -1 if negative.
  if (sum > 0) {
    return 1;
  } else {
    return -1;
  }
}</pre>
<h2 id="simple-pattern-recognition-using-a-perceptron">Simple Pattern Recognition Using a Perceptron</h2>
<p>Now that I have explained the computational process of a perceptron, let's take a look at an example of one in action. As I mentioned earlier, neural networks are commonly used for pattern recognition applications, such as facial recognition. Even simple perceptrons can demonstrate the fundamentals of classification. Let’s demonstrate with the following scenario.</p>
<figure class="half-width-right">
  <img src="images/10_nn/10_nn_4.png" alt="Figure 10.4">
  <figcaption>Figure 10.4</figcaption>
</figure>
<p>Consider a line in two-dimensional space. Points in that space can be classified as living on either one side of the line or the other. While this is a somewhat silly example (since there is clearly no need for a neural network; on which side a point lies can be determined with some simple algebra), it shows how a perceptron can be trained to recognize points on one side versus another.</p>
<p>Let’s say a perceptron has 2 inputs: <span data-type="equation">x,y</span> coordinates of a point). When using a sign activation function, the output will either be -1 or 1. The input data are classified according to the sign of the output, the weighted sum of inputs. In the above diagram, you can see how each point is either below the line (-1) or above (+1).</p>
<p>The perceptron itself can be diagrammed as follows. In machine learning <span data-type="equation">x</span>’s are typically the notation for inputs and <span data-type="equation">y</span> is typically the notation for an output. To keep this convention I’ll note in the diagram the inputs as <span data-type="equation">x_0</span> and <span data-type="equation">x_1</span>. <span data-type="equation">x_0</span> will correspond to the x cooordinate and <span data-type="equation">x_1</span> to the y. I name the output simply “<span data-type="equation">\text{output}</span>”.</p>
<figure>
  <img src="images/10_nn/10_nn_5.png" alt="Figure 10.5 Two inputs (x_0 and x_1), a weight for each input (\text{weight}_0 and \text{weight}_1) as well as a processing neuron that generates the output.">
  <figcaption>Figure 10.5 Two inputs (<span data-type="equation">x_0</span> and <span data-type="equation">x_1</span>), a weight for each input (<span data-type="equation">\text{weight}_0</span> and <span data-type="equation">\text{weight}_1</span>) as well as a processing neuron that generates the output.</figcaption>
</figure>
<p>There is a pretty significant problem in Figure 10.5, however. Let’s consider the point <span data-type="equation">(0,0)</span>. What if I send this point into the perceptron as its input: <span data-type="equation">x_0 = 0</span> and <span data-type="equation">x_1=1</span>? What will the sum of its weighted inputs be? No matter what the weights are, the sum will always be 0! But this can’t be right—after all, the point <span data-type="equation">(0,0)</span> could certainly be above or below various lines in this two-dimensional world.</p>
<p>To avoid this dilemma, the perceptron requires a third input, typically referred to as a <strong>bias</strong> input. A bias input always has the value of 1 and is also weighted. Here is the perceptron with the addition of the bias:</p>
<figure>
  <img src="images/10_nn/10_nn_6.png" alt="Figure 10.6: Diagram of a perceptron with the added “bias” input.">
  <figcaption>Figure 10.6: Diagram of a perceptron with the added “bias” input.</figcaption>
</figure>
<p>Let’s go back to the point <span data-type="equation">(0,0)</span>.</p>
<table>
  <thead>
    <tr>
      <th>input value</th>
      <th>weight</th>
      <th>result</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>0</td>
      <td><span data-type="equation">w_0</span></td>
      <td>0</td>
    </tr>
    <tr>
      <td>0</td>
      <td><span data-type="equation">w_1</span></td>
      <td>0</td>
    </tr>
    <tr>
      <td>1</td>
      <td><span data-type="equation">w_\text{bias}</span></td>
      <td><span data-type="equation">w_\text{bias}</span></td>
    </tr>
  </tbody>
</table>
<p>The output is then the sum of the above three results: <span data-type="equation">0 + 0 + w_\text{bias}</span>. Therefore, the bias, by itself, answers the question of where <span data-type="equation">(0,0)</span> is in relation to the line. If the bias's weight is positive, then <span data-type="equation">(0,0)</span> is above the line; if negative, it is below. Its weight <strong>biases</strong> the perceptron's understanding of the line's position relative to <span data-type="equation">(0,0)</span>!</p>
<h2 id="coding-the-perceptron">Coding the Perceptron</h2>
<p>I am now ready to assemble the code for a <code>Perceptron</code> class. The perceptron only needs to track the input weights, which I can store using an array.</p>
<pre class="codesplit" data-code-language="javascript">class Perceptron {
  constructor() {
    this.weights = [];
  }</pre>
<p>The constructor could receive an argument indicating the number of inputs (in this case three: <span data-type="equation">x_0</span>, <span data-type="equation">x_1</span>, and a bias) and size the array accordingly.</p>
<pre class="codesplit" data-code-language="javascript">  // The argument "n" determines the number of inputs (including the bias)
  constructor(n) {
    this.weights = [];
    for (let i = 0; i &#x3C; n; i++) {
      //{!1} The weights are picked randomly to start.
      this.weights[i] = random(-1, 1);
    }
  }</pre>
<p>A perceptron’s job is to receive inputs and produce an output. These requirements can be packaged together in a <code>feedForward()</code> function. In this example, the perceptron's inputs are an array (which should be the same length as the array of weights), and the output is an a number, <span data-type="equation">+1</span> or <span data-type="equation">-1</span>, depending on the sign as returned by the activation function.</p>
<pre class="codesplit" data-code-language="javascript">  feedForward(inputs) {
    let sum = 0;
    for (let i = 0; i &#x3C; this.weights.length; i++) {
      sum += inputs[i] * this.weights[i];
    }
    //{!1} Result is the sign of the sum, -1 or +1.
    // Here the perceptron is making a guess.
    // Is it on one side of the line or the other?
    return this.activate(sum);
  }</pre>
<p>I’ll note that the name of the function "feed forward" in this context comes from a commonly used term in neural networks to describe the process data passing through the network. This name relates to the way the data <em>feeds</em> directly <em>forward</em> through the network, read from left to right in a neural network diagram.</p>
<p>Presumably, I could now create a <code>Perceptron</code> object and ask it to make a guess for any given point.</p>
<figure>
  <img src="images/10_nn/10_nn_7.png" alt="Figure 10.7">
  <figcaption>Figure 10.7</figcaption>
</figure>
<pre class="codesplit" data-code-language="javascript">// Create the Perceptron.
let perceptron = new Perceptron(3);
// The input is 3 values: x, y, and bias.
let inputs = [50, -12, 1];
// The answer!
let guess = perceptron.feedForward(inputs);</pre>
<p>Did the perceptron get it right? At this point, the perceptron has no better than a 50/50 chance of arriving at the right answer. Remember, when I created it, I gave each weight a random value. A neural network is not a magic tool that can guess things correctly on its own. I need to teach it how to do so!</p>
<p>To train a neural network to answer correctly, I will use the method of <em>supervised learning</em>, which I described in section 10.1. In this method, the network is provided with inputs for which there is a known answer. This enables the network to determine if it has made a correct guess. If it is incorrect, the network can learn from its mistake and adjust its weights. The process is as follows:</p>
<ol>
  <li>Provide the perceptron with inputs for which there is a known answer.</li>
  <li>Ask the perceptron to guess an answer.</li>
  <li>Compute the error. (Did it get the answer right or wrong?)</li>
  <li>Adjust all the weights according to the error.</li>
  <li>Return to Step 1 and repeat!</li>
</ol>
<p>Steps 1 through 4 can be packaged into a function. Before I can write the entire function, however, I need to examine Steps 3 and 4 in more detail. How do I define the perceptron’s error? And how should I adjust the weights according to this error?</p>
<p>The perceptron’s error can be defined as the difference between the desired answer and its guess.</p>
<div data-type="equation">\text{error} = \text{desired output} - \text{guess output}</div>
<p>Does the above formula look familiar to you? Maybe you are thinking what I’m thinking? What was that formula for a steering force again?</p>
<div data-type="equation">\text{steering} = \text{desired velocity} - \text{current velocity}</div>
<p>This is also a calculation of an error! The current velocity serves as a guess, and the error (the steering force) indicates how to adjust the velocity in the correct direction. In a moment, you will see how adjusting a vehicle's velocity to follow a target is similar to adjusting the weights of a neural network to arrive at the correct answer.</p>
<p>In the case of the perceptron, the output has only two possible values: <span data-type="equation">+1</span> or <span data-type="equation">-1</span>. This means there are only three possible errors.</p>
<p>If the perceptron guesses the correct answer, then the guess equals e the desired output and the error is 0. If the correct answer is -1 and it guessed +1, then the error is -2. If the correct answer is +1 and it guessed -1, then the error is +2.</p>
<table>
  <thead>
    <tr>
      <th>Desired</th>
      <th>Guess</th>
      <th>Error</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><span data-type="equation">-1</span></td>
      <td><span data-type="equation">-1</span></td>
      <td><span data-type="equation">0</span></td>
    </tr>
    <tr>
      <td><span data-type="equation">-1</span></td>
      <td><span data-type="equation">+1</span></td>
      <td><span data-type="equation">-2</span></td>
    </tr>
    <tr>
      <td><span data-type="equation">+1</span></td>
      <td><span data-type="equation">-1</span></td>
      <td><span data-type="equation">+2</span></td>
    </tr>
    <tr>
      <td><span data-type="equation">+1</span></td>
      <td><span data-type="equation">+1</span></td>
      <td><span data-type="equation">0</span></td>
    </tr>
  </tbody>
</table>
<p>The error is the determining factor in how the perceptron’s weights should be adjusted. For any given weight, what I am looking to calculate is the change in weight, often called <span data-type="equation">\Delta\text{weight}</span> (or “delta” weight, delta being the Greek letter <span data-type="equation">\Delta</span>).</p>
<div data-type="equation">\text{new weight} = \text{weight} + \Delta\text{weight}</div>
<p><span data-type="equation">\Delta\text{weight}</span> is calculated as the error multiplied by the input.</p>
<div data-type="equation">\Delta\text{weight} = \text{error} \times \text{input}</div>
<p>Therefore:</p>
<div data-type="equation">\text{new weight} = \text{weight} + \text{error} \times \text{input}</div>
<p>To understand why this works, I will again return to steering. A steering force is essentially an error in velocity. By applying a steering force as an acceleration (or <span data-type="equation">\Delta\text{velocity}</span>), then the velocity is adjusted to move in the correct direction. This is what I want to do with the neural network’s weights. I want to adjust them in the right direction, as defined by the error.</p>
<p>With steering, however, I had an additional variable that controlled the vehicle’s ability to steer: the <em>maximum force</em>. A high maximum force allowed the vehicle to accelerate and turn quickly, while a lower force resulted in a slower velocity adjustment. The neural network will use a similar strategy with a variable called the "learning constant."</p>
<div data-type="equation">\text{new weight} = \text{weight} + (\text{error} \times \text{input}) \times \text{learning constant}</div>
<p>Note that a high learning constant causes the weight to change more drastically. This may help the perceptron arrive at a solution more quickly, but it also increases the risk of overshooting the optimal weights. A small learning constant, however, will adjust the weights slowly and require more training time, but allow the network to make small adjustments that could improve overall accuracy.</p>
<p>Assuming the addition of a <code>this.learningConstant</code> property to the <code>Perceptron</code>class, , I can now write a training function for the perceptron following the above steps.</p>
<pre class="codesplit" data-code-language="javascript">// Step 1: Provide the inputs and known answer.
// These are passed in as arguments to train().
train(inputs, desired) {

  // Step 2: Guess according to those inputs.
  let guess = this.feedforward(inputs);

  // Step 3: Compute the error (difference between desired and guess).
  let error = desired - guess;

  //{!3} Step 4: Adjust all the weights according to the error and learning constant.
  for (let i = 0; i &#x3C; this.weights.length; i++) {
    this.weights[i] += error * inputs[i] * this.learningConstant;
  }
}</pre>
<p>Here’s the <code>Perceptron</code> class as a whole.</p>
<pre class="codesplit" data-code-language="javascript">class Perceptron {
  constructor(n) {
    //{!2} The Perceptron stores its weights and learning constants.
    this.weights = [];
    this.learningConstant = 0.01;
    //{!3} Weights start off random.
    for (let i = 0; i &#x3C; n; i++) {
      this.weights[i] = random(-1,1);
    }
  }

  //{!7} Return an output based on inputs.
  feedforward(inputs) {
    let sum = 0;
    for (let i = 0; i &#x3C; this.weights.length; i++) {
      sum += inputs[i] * this.weights[i];
    }
    return this.activate(sum);
  }

  // Output is a +1 or -1.
  activate(sum) {
    if (sum > 0) {
      return 1;
    } else {
      return -1;
    }
  }

  //{!7} Train the network against known data.
  train(inputs, desired) {
    let guess = this.feedforward(inputs);
    let error = desired - guess;
    for (let i = 0; i &#x3C; this.weights.length; i++) {
      this.weights[i] += error * inputs[i] * this.learningConstant;
    }
  }
}</pre>
<p>To train the perceptron, I need a set of inputs with a known answer. Now the question becomes, how do I pick a point and know whether it is above or below a line? Let’s start with the formula for a line, where <span data-type="equation">y</span> is calculated as a function of <span data-type="equation">x</span>:</p>
<div data-type="equation">y = f(x)</div>
<p>In generic terms, a line can be described as:</p>
<div data-type="equation">y = ax + b</div>
<p>Here’s a specific example:</p>
<div data-type="equation">y = 2x + 1</div>
<p>I can then write a function with this in mind.</p>
<pre class="codesplit" data-code-language="javascript">// A function to calculate y based on x along a line
f(x) {
  return 2 * x + 1;
}</pre>
<p>So, if I make up a point:</p>
<pre class="codesplit" data-code-language="javascript">let x = random(width);
let y = random(height);</pre>
<p>How do I know if this point is above or below the line? The line function <span data-type="equation">f(x)</span> returns <span data-type="equation">y</span> value on the line for that <span data-type="equation">x</span> position. Let’s call that <span data-type="equation">y_\text{line}</span>.</p>
<pre class="codesplit" data-code-language="javascript">// The y position on the line
let yline = f(x);</pre>
<p>If the <span data-type="equation">y</span> value I am examining is above the line, it will be less than <span data-type="equation">y_\text{line}</span>.</p>
<figure>
  <img src="images/10_nn/10_nn_8.png" alt="Figure 10.8: If y is less than y_\text{line} then it is above the line. Note this is only true for a p5.js canvas where the y axis points down in the positive direction.">
  <figcaption>Figure 10.8: If <span data-type="equation">y</span> is less than <span data-type="equation">y_\text{line}</span> then it is above the line. Note this is only true for a p5.js canvas where the y axis points down in the positive direction.</figcaption>
</figure>
<pre class="codesplit" data-code-language="javascript">// Start with the value of +1
let desired = 1;
if (y &#x3C; yline) {
  //{!1} The answer is -1 if y is above the line.
  desired = -1;
}</pre>
<p>I can then make an inputs array to go with the <code>desired</code> output.</p>
<pre class="codesplit" data-code-language="javascript">// Don't forget to include the bias!
let trainingInputs = [x, y, 1];</pre>
<p>Assuming that I have a <code>perceptron</code> variable, I can train it by providing the inputs along with the desired answer.</p>
<pre class="codesplit" data-code-language="javascript">perceptron.train(trainingInputs, desired);</pre>
<p>Now, it’s important to remember that this is just a demonstration. Remember the Shakespeare-typing monkeys? I asked the genetic algorithm to solve for “to be or not to be”—an answer I already knew. I did this to make sure the genetic algorithm worked properly. The same reasoning applies to this example. I don’t need a perceptron to tell me whether a point is above or below a line; I can do that with simple math. By using an example that I can easily solve without a perceptron, I can both demonstrate the algorithm of the perceptron and verify that it is working properly.</p>
<p>Let’s look the perceptron trained with with an array of many points.</p>
<div data-type="example">
  <h3 id="example-101-the-perceptron">Example 10.1: The Perceptron</h3>
  <figure>
    <div data-type="embed" data-p5-editor="https://editor.p5js.org/natureofcode/sketches/sMozIaMCW" data-example-path="examples/10_nn/10_1_perceptron_with_normalization"></div>
    <figcaption></figcaption>
  </figure>
</div>
<pre class="codesplit" data-code-language="javascript">// The Perceptron
let perceptron;
//{!1} 2,000 training points
let training = [];
// A counter to track training points one by one
let count = 0;

//{!3} The formula for a line
function f(x) {
  return 2 * x + 1;
}

function setup() {
  createCanvas(640, 240);
  
  // Perceptron has 3 inputs (including bias) and learning rate of 0.01
  perceptron = new Perceptron(3, 0.01);

  //{!1} Make 1,000 training points.
  for (let i = 0; i &#x3C; 2000; i++) {
    let x = random(-width / 2,width / 2);
    let y = random(-height / 2,height / 2);
    //{!2} Is the correct answer 1 or -1?
    let desired = 1;
    if (y &#x3C; f(x)) {
      desired = -1;
    }
    training[i] = {
      input: [x, y, 1],
      output: desired
    };  
  }
}


function draw() {
  background(255);
  translate(width/0wiu2, height/2);

  ptron.train(training[count].inputs, training[count].answer);
  //{!1} For animation, we are training one point at a time.
  count = (count + 1) % training.length;

  for (let i = 0; i &#x3C; count; i++) {
    stroke(0);
    let guess = ptron.feedforward(training[i].inputs);
    //{!2} Show the classification—no fill for -1, black for +1.
    if (guess > 0) noFill();
    else           fill(0);
    ellipse(training[i].inputs[0], training[i].inputs[1], 8, 8);
  }
}</pre>
<p>Section on Normalizing Here?</p>
<div data-type="exercise">
  <h3 id="exercise-101">Exercise 10.1</h3>
  <p>Instead of using the supervised learning model above, can you train the neural network to find the right weights by using a genetic algorithm?</p>
</div>
<div data-type="exercise">
  <h3 id="exercise-102">Exercise 10.2</h3>
  <p>Visualize the perceptron itself. Draw the inputs, the processing node, and the output.</p>
</div>
<h2 id="its-a-network-remember">It’s a “Network,” Remember?</h2>
<p>Yes, a perceptron can have multiple inputs, but it is still a lonely neuron. The power of neural networks comes in the networking itself. Perceptrons are, sadly, incredibly limited in their abilities. If you read an AI textbook, it will say that a perceptron can only solve <strong>linearly separable</strong> problems. What’s a linearly separable problem? Let’s take a look at the first example, which determined whether points were on one side of a line or the other.</p>
<figure>
  <img src="images/10_nn/10_nn_9.png" alt="Figure 10.11">
  <figcaption>Figure 10.11</figcaption>
</figure>
<p>On the left of Figure 10.11, is an example of classic linearly separable data. Graph all of the possibilities; if you can classify the data with a straight line, then it is linearly separable. On the right, however, is non-linearly separable data. You can’t draw a straight line to separate the black dots from the gray ones.</p>
<p>One of the simplest examples of a non-linearly separable problem is <em>XOR</em>, or “exclusive or.” I’m guessing, as someone who works with coding and p5.js, you are familiar with a logical <span data-type="equation">\text{AND}</span>. For <span data-type="equation">A \text{ AND } B</span> to be true, both <span data-type="equation">A</span> and <span data-type="equation">B</span> must be true. With <span data-type="equation">\text{OR}|</span>, either <span data-type="equation">A</span> or <span data-type="equation">B</span> can be true for <span data-type="equation">A \text{ OR } B</span> to evaluate as true. These are both linearly separable problems. Let’s look at the solution space, a “truth table.”</p>
<figure>
  <img src="images/10_nn/10_nn_10.png" alt="Figure 10.12">
  <figcaption>Figure 10.12</figcaption>
</figure>
<p>See how you can draw a line to separate the true outputs from the false ones?</p>
<p><span data-type="equation">\text{XOR}</span> (”exclusive” or) is the equivalent <span data-type="equation">\text{OR}</span> and <span data-type="equation">\text{NOT AND}</span>. In other words, <span data-type="equation">A \text{ XOR } B</span> only evaluates to true if one of them is true. If both are false or both are true, then we get false. Take a look at the following truth table.</p>
<figure>
  <img src="images/10_nn/10_nn_11.png" alt="Figure 10.13">
  <figcaption>Figure 10.13</figcaption>
</figure>
<p>This is not linearly separable. Try to draw a straight line to separate the true outputs from the false ones—you can’t!</p>
<p>So perceptrons can’t even solve something as simple as <span data-type="equation">\text{XOR}</span>. But what if we made a network out of two perceptrons? If one perceptron can solve <span data-type="equation">\text{OR}</span> and one perceptron can solve <span data-type="equation">\text{NOT AND}</span>, then two perceptrons combined can solve <span data-type="equation">\text{XOR}</span>.</p>
<figure>
  <img src="images/10_nn/10_nn_12.png" alt="Figure 10.14">
  <figcaption>Figure 10.14</figcaption>
</figure>
<p>The above diagram is known as a <em>multi-layered perceptron</em>, a network of many neurons. Some are input neurons and receive the inputs, some are part of what’s called a “hidden” layer (as they are connected to neither the inputs nor the outputs of the network directly), and then there are the output neurons, from which the results are read.</p>
<p>Training these networks is more complex. With the simple perceptron, you could easily evaluate how to change the weights according to the error. But here there are so many different connections, each in a different layer of the network. How does one know how much each neuron or connection contributed to the overall error of the network?</p>
<p>The solution to optimizing the weights of a multi-layered network is known as <strong>backpropagation</strong>. In this process, the output of the network is generated in the same manner as a perceptron. The inputs multiplied by the weights are summed and fed forward through the network. The difference here is that they pass through additional layers of neurons before reaching the output. Training the network (i.e. adjusting the weights) also involves taking the error (desired result - guess). The error, however, must be fed backwards through the network. The final error ultimately adjusts the weights of all the connections.</p>
<p>Backpropagation is beyond the scope of this book and involves a variety of different activation functions (one class example is the “sigmoid” function) as well as some calculus. If you are interested in continuing down this road and learning more about how backpropagation works, you can find <a href="https://github.com/CodingTrain/Toy-Neural-Network-JS">my “toy neural network” project at github.com/CodingTrain</a> with links to accompanying video tutorials. They go through all the steps of solving <span data-type="equation">\text{XOR}</span> using a multi-layered feed forward network with backpropagation. For this chapter, however, I’d like to get some help and phone a friend.</p>
<h2 id="machine-learning-with-ml5js">Machine Learning with ml5.js</h2>
<p>That friend is ml5.js. Inspired by the philosophy of p5.js, ml5.js is a JavaScript library that aims to make machine learning accessible to a wide range of artists, creative coders, and students. It is built on top of TensorFlow.js, Google's open-source library that runs machine learning models directly in the browser without the need to install or configure complex environments. TensorFlow.js's low-level operations and highly technical API, however, can be intimidating to beginners. That's where ml5.js comes in, providing a friendly entry point for those who are new to machine learning and neural networks.</p>
<p>Before I get to my goal of adding a "neural network" brain to a steering agent and tying ml5.js back into the story of the book, I would like to demonstrate step-by-step how to train a neural network model with "supervised learning." There are several key terms and concepts important to cover, namely “classification”, “regression”, “inputs”, and “outputs”. By walking through the full process of a supervised learning scenario, I hope to define these terms, explore other foundational concepts, introduce the syntax of the ml5.js library, and provide the tools to train your first machine learning model with your own data.</p>
<h3 id="classification-and-regression">Classification and Regression</h3>
<p>The majority of machine learning tasks fall into one of two categories: classification and regression. Classification is probably the easier of the two to understand at the start. It involves predicting a “label” (or “category” or “class”) for a piece of data. For example, an image classifier might try to guess if a photo is of a cat or a dog and assign the corresponding label.</p>
<p><strong><em>[FIGURE OF CAT OR DOG OR BIRD OR MONKEY OR ILLUSTRATIONS ASSIGNED A LABEL?]</em></strong></p>
<p>This doesn’t happen by magic, however. The model must first be shown many examples of dogs and cats with the correct labels in order to properly configure the weights of all the connections. This is the supervised learning training process.</p>
<p>The classic “Hello, World” demonstration of machine learning and supervised learning is known as “MNIST”. MNIST, short for “Modified National Institute of Standards and Technology,” is a dataset that was collected and processed by Yann LeCun and Corinna Cortes (AT&#x26;T Labs) and Christopher J.C. Burges (Microsoft Research). It is widely used for training and testing in the field of machine learning and consists of 70,000 handwritten digits from 0 to 9, with each one being a 28x28 pixel grayscale image.</p>
<p><strong><em>[FIGURE FOR MNIST?]</em></strong></p>
<p>While I won't be building a complete MNIST model with ml5.js (you could if you wanted to!), it serves as a canonical example of a training dataset for image classification: 70,000 images each assigned one of 10 possible labels. This idea of a “label” is fundamental to classification, where the output of a model involves a fixed number of discrete options. There are only 10 possible digits that the model can guess, no more and no less. After the data is used to train the model, the goal is to classify new images and assign the appropriate label.</p>
<p>Regression, on the other hand, is a machine learning task where the prediction is a continuous value, typically a floating point number. A regression problem can involve multiple outputs, but when beginning it’s often simpler to think of it as just one. Consider a machine learning model that predicts the daily electricity usage of a house based on any number of factors like number of occupants, size of house, temperature outside. Here, rather than a goal of the neural network picking from a discrete set of options, it makes more sense for the neural network to guess a number. Will the house use 30.5 kilowatt-hours of energy that day? 48.7 kWh? 100.2 kWh? The output is therefore a continuous value that the model attempts to predict.</p>
<p><strong><em>[FIGURE ILLUSTRATING REGRESSION?]</em></strong></p>
<h3 id="inputs-and-outputs">Inputs and Outputs</h3>
<p>Once the task has been determined, the next step is to finalize the configuration of inputs and outputs of the neural network. In the case of MNIST, each image is a collection of 28x28 grayscale pixels and each pixel can be represented as a single value (ranging from 0-255). The total pixels is <span data-type="equation">28 \times 28 = 784</span>. The grayscale value of each pixel is an input to the neural network.</p>
<figure>
  <img src="images/10_nn/10_nn_13.jpg" alt="Place holder figure (just show the inputs first?, borrowed from https://ml4a.github.io/ml4a/looking_inside_neural_nets/">
  <figcaption>Place holder figure (just show the inputs first?, borrowed from <a href="https://ml4a.github.io/ml4a/looking_inside_neural_nets/">https://ml4a.github.io/ml4a/looking_inside_neural_nets/</a></figcaption>
</figure>
<p>Since there are 10 possible digits 0-9, the output of the neural network is a prediction of one of 10 labels.</p>
<p><strong><em>[FIGURE NOW ADDS THE OUTPUTS IN]</em></strong></p>
<p>Let’s consider the regression scenario of predicting the electricity usage of a house. Let’s assume you have a table with the following data:</p>
<table>
  <tbody>
    <tr>
      <td><strong>Occupants</strong></td>
      <td><strong>Size (m²)</strong></td>
      <td><strong>Temperature Outside (°C)</strong></td>
      <td><strong>Electricity Usage (kWh)</strong></td>
    </tr>
    <tr>
      <td>4</td>
      <td>150</td>
      <td>24</td>
      <td>25.3</td>
    </tr>
    <tr>
      <td>2</td>
      <td>100</td>
      <td>25.5</td>
      <td>16.2</td>
    </tr>
    <tr>
      <td>1</td>
      <td>70</td>
      <td>26.5</td>
      <td>12.1</td>
    </tr>
    <tr>
      <td>4</td>
      <td>120</td>
      <td>23</td>
      <td>22.1</td>
    </tr>
    <tr>
      <td>2</td>
      <td>90</td>
      <td>21.5</td>
      <td>15.2</td>
    </tr>
    <tr>
      <td>5</td>
      <td>180</td>
      <td>20</td>
      <td>24.4</td>
    </tr>
    <tr>
      <td>1</td>
      <td>60</td>
      <td>18.5</td>
      <td>11.7</td>
    </tr>
  </tbody>
</table>
<p>Here in this table, the inputs to the neural network are the first three columns (occupants, size, temperature). The fourth column on the right is what the neural network is expected to guess, or the output.</p>
<p><strong><em>[FIGURE SHOWING 3 inputs + 1 output]</em></strong></p>
<h3 id="setting-up-the-neural-network-with-ml5js">Setting up the Neural Network with ml5.js</h3>
<p>In a typical machine learning scenario, the next step after establishing the inputs and outputs is to configure the architecture of the neural network. This involves specifying the number of hidden layers between the inputs and outputs, the number of neurons in each layer, which activation functions to use, and more! While all of this is possible with ml5.js, it will make its best guess and design a model for you based on the task and data.</p>
<p>As demonstrated with Matter.js and toxiclibs.js in chapter 6, you can import the ml5.js library into your <strong>index.html</strong> file.</p>
<pre class="codesplit" data-code-language="javascript">&#x3C;script src="https://unpkg.com/ml5@latest/dist/ml5.min.js">&#x3C;/script></pre>
<p>The ml5.js library is a collection of machine learning models that can be accessed using the syntax <code>ml5.functionName()</code>. For example, to use a pre-trained model that detects hands, you can use <code>ml5.handpose()</code>. For classifying images, you can use <code>ml5.imageClassifier()</code>. While I encourage exploring all that ml5.js has to offer (I will reference some of these pre-trained models in upcoming exercise ideas), for this chapter, I will focus on only one function in ml5.js: <code>ml5.neuralNetwork()</code>, which creates an empty neural network for you to train.</p>
<p>To create a neural network, you must first create a JavaScript object that will configure the model. While there are many properties that you can set, most of them are optional, as the network will use default values. Let’s begin by specifying the "task" that you intend the model to perform: "regression" or "classification.”</p>
<pre class="codesplit" data-code-language="javascript">let options = { task: "classification" }
let classifier = ml5.neuralNetwork(options);</pre>
<p>This, however, gives ml5.js very little to go on in terms of designing the network architecture. Adding the inputs and outputs will complete the rest of the puzzle for it. In the case of MNIST, there are 784 inputs (grayscale pixel colors) and 10 possible output labels (digits “0” through “9”). This can be configured in ml5.js with a single integer for the number of inputs and an array of strings for the list of output labels.</p>
<pre class="codesplit" data-code-language="javascript">let options = {
  inputs: 784,
  outputs: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"],
  task: "classification",
};
let digitClassifier = ml5.neuralNetwork(options);</pre>
<p>The electricity regression scenario involved 3 input values (occupants, size, temperature) and 1 output value (usage in kWh).</p>
<pre class="codesplit" data-code-language="javascript">let options = {
  inputs: 3,
  outputs: 1,
  task: "regression",
};
let energyPredictor = ml5.neuralNetwork(options);</pre>
<p>While the MNIST and energy predictor scenarios are useful starting points for understanding how machine learning works, it's important to note that they are simplified versions of what you might encounter in a “real-world” machine learning application. Depending on the problem, there could be significantly higher levels of complexity both in terms of the network architecture and the scale and preparation of data. Instead of a neatly packaged dataset like MNIST, you might be dealing with enormous amounts of messy data. This data might need to be processed and refined before it can be effectively used. You can think of it like organizing, washing, and chopping ingredients before you can start cooking with them.</p>
<p>The “lifecycle” of a machine learning model is typically broken down into seven steps.</p>
<ol>
  <li><strong>Data Collection</strong>: Data forms the foundation of any machine learning task. This stage might involve running experiments, manually inputting values, sourcing public data, or a myriad of other methods.</li>
  <li><strong>Data Preparation</strong>: Raw data often isn't in a format suitable for machine learning algorithms. It might also have duplicate or missing values, or contain outliers that skew the data. Such inconsistencies may need to be manually adjusted. Additionally, neural networks work best with “normalized” data. While this term might remind you of normalizing vectors, it's important to understand that it carries a slightly different meaning in the context of data preparation. A “normalized” vector’s length is set to a fixed value, usually 1, with the direction intact. However, data normalized for machine learning involves adjusting the values so that they fit within a specific range, generally between 0 and 1 or -1 and 1. Another key part of preparing data is separating it into distinct sets: training, validation, and testing. The training data is used to teach the model (Step 5). On the other hand, the validation and testing data (the distinction is subtle, more on this later) are set aside and reserved for evaluating the model's performance (Step 6).</li>
  <li><strong>Choosing a Model:</strong> This step involves designing the architecture of the neural network. Different models are more suitable for certain types of data and outputs.</li>
  <li><strong>Training</strong>: This step involves feeding the "training" data through the model, allowing the model to adjust the weights of the neural network based on its errors. This process is known as “optimization” where the model tunes the weights to <em>optimize</em> for the least amount of errors.</li>
  <li><strong>Evaluation</strong>: Remember that “testing” data that was saved for in step 3? Since that data wasn’t used in training, it provides a means to evaluate how well the model performs on new, unseen data.</li>
  <li><strong>Parameter Tuning:</strong> The training process is influenced by a set of parameters (often called “hyperparameters”), such as the "learning rate," which dictates how much the model should adjust its weights based on errors in prediction. By fine-tuning these parameters and revisiting steps 5 (Training), 4 (Choosing a Model), or even 3 (Data Preparation), you can often improve the model's performance.</li>
  <li><strong>Deployment: </strong>Once the model is trained and its performance is evaluated satisfactorily, it’s time to actually use the model out in the real world with new data!</li>
</ol>
<h2 id="building-a-gesture-classifier">Building a Gesture Classifier</h2>
<p>I’d like to now follow the 7 steps outlined with an example problem well suited for p5.js and build all the code for each step using ml5.js. However, even though 7 is a truly excellent number, I think I missed a critical step. Let’s call it step 0.</p>
<ol>
  <li><strong>Identify the Problem</strong>: This initial step involves defining the problem that needs solving. What is the objective? What are you trying to accomplish or predict with your machine learning model?</li>
</ol>
<p>After all, how are you supposed to collect your data without knowing what you are even trying to do? Are you predicting a number? A category? A sequence? Is it a binary choice, or are there multiple options? These considerations about your inputs (the data fed into the model) and outputs (the predictions) are critical for every other step of the machine learning journey.</p>
<p><strong><em>[POSSIBLE ILLUSTRATION OF A SINGLE MOUSE SWIPE AS A GESTURE: basically can the paragraph below be made into a drawing?]</em></strong></p>
<p>Let’s take a crack at step 0 for an example problem of training your first machine learning model with ml5.js and p5.js. Imagine for a moment, you’re working on an interactive application that responds to a gesture, maybe that gesture is ultimately meant to be classified via body tracking, but you want to start with something much simpler—one single stroke of the mouse. Each gesture could be recorded as a vector (extending from the start to the end points of a mouse movement) and the model’s task could be to predict one of four options: “up”, “down”, “left”, or “right.” Perfect! I’ve now got the objective and boiled it down into inputs and outputs!</p>
<h3 id="data-collection-and-preparation">Data Collection and Preparation</h3>
<p>Next, I’ve got steps 1 and 2: data collection and preparation. Here, I’d like to take the approach of ordering a machine learning “meal-kit,” where the ingredients (data) comes pre-portioned and prepared. This way, I’ll get straight to the cooking itself, the process of training the model. After all, this is really just an appetizer for what will be the ultimate meal later in this chapter when I get to applying neural networks to steering agents.</p>
<p>For this step, I’ll hard-code that data itself and manually keep it normalized within a range of -1 and 1. Here it is directly written into the code, rather than loaded from a separate file. It is organized into an array of objects, pairing the <span data-type="equation">x,y</span> components of a vector with a string label.</p>
<pre class="codesplit" data-code-language="javascript">let data = [
  { x: 0.99, y: 0.02, label: "right" },
  { x: 0.76, y: -0.1, label: "right" },
  { x: -1.0, y: 0.12, label: "left" },
  { x: -0.9, y: -0.1, label: "left" },
  { x: 0.02, y: 0.98, label: "down" },
  { x: -0.2, y: 0.75, label: "down" },
  { x: 0.01, y: -0.9, label: "up" },
  { x: -0.1, y: -0.8, label: "up" },
];</pre>
<p>In truth, it would likely be better to collect example data by asking users to perform specific gestures and recording their inputs, or by creating synthetic data that represents the idealized versions of the gestures I want the model to recognize. In either case, the key is to collect a diverse set of examples that adequately represent the variations in how the gestures might be performed. But let’s see how it goes with just a few servings of data.</p>
<div data-type="exercise">
  <h3 id="exercise-103">Exercise 10.3</h3>
  <p>
    Create a p5.js sketch that collects gesture data from users and saves it to a JSON file. You can use <code>mousePressed()</code> and <code>mouseReleased()</code> to mark the start and end of each gesture and <code>saveJSON()</code> to download the data into a file.
    <em>JSON (JavaScript Object Notation) and CSV (Comma-Separated Values) are two popular formats for storing and loading data. JSON stores data in key-value pairs and follows the same exact format as JavaScript objects. CSV is a file format that stores “tabular” data (like a spreadsheet). There are numerous other data formats you could use depending on your needs what programming environment you are working with.</em>
  </p>
</div>
<p>I’ll also note that, much like some of the genetic algorithm demonstrations in chapter 9, I am selecting a problem here that has a known solution and could have been solved more easily and efficiently without a neural network. The direction of a vector can be classified with the <code>heading2D()</code> function and a series of if statements! However, by using this seemingly trivial scenario, I hope to explain the process of training a machine learning model in an understandable and friendly way. Additionally, it will make it easy to check if the code is working as expected! When I’m done I’ll provide some ideas about how to expand the classifier to a scenario where <code>if</code> statements would not apply.</p>
<h3 id="choosing-a-model">Choosing a Model</h3>
<p>This is where I am going to let ml5.js do the heavy lifting for me. To create the model with ml5.js, all I need to do is specify the task, the inputs, and the outputs!</p>
<pre class="codesplit" data-code-language="javascript">let options = {
  task: "classification",
  inputs: 2,
  outputs: ["up", "down", "left", "right"],
  debug: true
};
let classifier = ml5.neuralNetwork(options);</pre>
<p>That's it! I'm done! Thanks to ml5.js, I can bypass a host of complexities related to the manual configuration and setup of the model. This includes decisions about the network architecture, such as how many layers and neurons per layer to have, the kind of activation functions to use, and the setup of algorithms for training the network. Keep in mind that the default model architecture selected by ml5.js may not be perfect for all cases. I encourage you to read the ml5.js reference for additional details on how to customize the model.</p>
<p>I’ll also point out that ml5.js is able to infer the inputs and outputs from the data itself, so those properties are not entirely necessary to include here in the <code>options</code> object. However, for the sake of clarity (and since I’ll need to specify those for later examples), I’m including them here.</p>
<p>The <code>debug</code> property, when set to <code>true</code>, enables a visual interface for the training process. It’s a helpful tool for spotting potential issues during training and for getting a better understanding of what's happening behind the scenes.</p>
<h3 id="training">Training</h3>
<p>Now that I have the data and a neural network initialized in the <code>classifier</code> variable, I’m ready to train the model! The thing is, I’m not really done with the data. In the “Data Collection and Preparation” section, I organized the data neatly into an array of objects, representing the <span data-type="equation">x,y</span> components of a vector paired with a string label. This format, while typical, isn't directly consumable by ml5.js for training. I need to specify which elements of the data are the inputs and which are the outputs for training the model. I could have initially organized the data into a format that ml5.js recognizes, but I'm including this extra step because it's more likely to be what happens when using a "real" dataset that has been collected or sourced elsewhere.</p>
<p>The ml5.js library offers a fair amount of flexibility in the kinds of formats it will accept, I will choose to use arrays—one for the <code>inputs</code> and one for the <code>outputs</code>.</p>
<pre class="codesplit" data-code-language="javascript">for (let i = 0; i &#x3C; data.length; i++) {
  let item = data[i];
  // An array of 2 numbers for the inputs
  let inputs = [item.x, item.y];
  // A single string "label" for the output
  let outputs = [item.label];
  //{!1} Add the training data to the classifier
  classifier.addData(inputs, outputs);
}</pre>
<p>A term you will often hear when talking about data in machine learning is “shape.” What is the “shape” of your data?</p>
<p>The "shape" of data in machine learning describes its dimensions and structure. It indicates how the data is organized in terms of rows, columns, and potentially even deeper, into additional dimensions. In the context of machine learning, understanding the shape of your data is crucial because it determines how the model should be structured.</p>
<p>Here, the input data's shape is a one-dimensional array containing 2 numbers (representing x and y). The output data, similarly, is an array but just contains a single string label. While this is a very small and simple example, it nicely mirrors many real-world scenarios where input features are numerically represented in an array, and outputs are string labels.</p>
<p>Oh dear, another term to unpack—features! In machine learning, the individual pieces of information used to make predictions are often called <strong>features</strong>. The term “feature” is chosen because it underscores the idea of distinct characteristics of the data are that most salient for the prediction. This will come into focus more clearly in future examples in this chapter.</p>
<p>After passing the data into the <code>classifier</code>, ml5.js provides a helper function to normalize it.</p>
<pre class="codesplit" data-code-language="javascript">// Normalize the data
classifier.normalizeData();</pre>
<p>As I’ve mentioned, normalizing data (adjusting the scale to a standard range) is a critical step in the machine learning process. However, if you recall during the data collection process, the hand-coded data was written with values that already range between -1 and 1. So, while calling <code>normalizeData()</code> here is likely redundant, it's important to demonstrate. Normalizing your data as part of the pre-processing step will absolutely work, but the auto-normalization feature of ml5.js is a quite convenient alternative.</p>
<p>Ok, this subsection is called training. So now it’s time to train! Here’s the code:</p>
<pre class="codesplit" data-code-language="javascript">// The "train" method initiates the training process
classifier.train(finishedTraining);

// A callback function for when the training is complete
function finishedTraining() {
  console.log("Training complete!");
}</pre>
<p>Yes, that’s it! After all, the hard work as already been completed! The data was collected, prepared, and fed into the model. However, if I were to run the above code and then test the model, the results would probably be inadequate. Here is where it’s important to introduce another key term in machine learning: <strong>epoch.</strong> The <code>train()</code> method tells the neural network to start the learning process. But how long should it train for? You can think of an epoch as one round of practice, one cycle of using the entire dataset to update the weights of the neural network. Generally speaking, the longer you train, the better the network will perform, but at a certain point there are diminishing returns. The number of epochs can be set by passing in an options object into <code>train()</code>.</p>
<pre class="codesplit" data-code-language="javascript">
//{!1} Setting the number of epochs for training
let options = { epochs: 25 };
classifier.train(options, finishedTraining);</pre>
<p>There are other "hyperparameters" that you can set in the <code>options</code> variable (learning rate is one again!), but I'm going to stick with the defaults. You can read more about customization options in the ml5.js reference.</p>
<p>The second argument, <code>finishedTraining()</code>, is optional, but it's good to include because it's a callback that runs when the training process is complete. This is useful for knowing when you can proceed to the next steps in your code. There is even another optional callback, which I usually name <code>whileTraining()</code>, that is triggered after each epoch. However, for my purposes, knowing when the training is done is plenty!</p>
<div data-type="note">
  <h3 id="callbacks">Callbacks</h3>
  <p>If you've worked with p5.js, you're already familiar with the concept of a callback even if you don't know it by that name. Think of the <code>mousePressed()</code> function. You define what should happen inside it, and p5.js takes care of <em>calling </em>it at the right moment, when the mouse is pressed.</p>
  <p>A callback function in JavaScript operates on a similar principle. It's a function that you provide as an argument to another function, intending for it to be “called back” at a later time. They are needed for “asynchronous” operations, where you want your code to continue along with animating or doing other things while waiting for another task to finish. A classic example of this in p5.js is loading data into a sketch with <code>loadJSON()</code>.</p>
  <p>In JavaScript, there's also a more recent approach for handling asynchronous operations known as "Promises." With Promises, you can use keywords like <code>async</code> and <code>await</code> to make your asynchronous code look more like traditional synchronous code. While ml5.js also supports this style, I’ll stick to using callbacks to stay aligned with p5.js style.</p>
</div>
<h3 id="evaluation">Evaluation</h3>
<p>If <code>debug</code> is set to true in the initial call to <code>ml5.neuralNetwork()</code>, once <code>train()</code> is called, a visual interface appears covering most of the p5.js page and canvas.</p>
<figure>
  <img src="images/10_nn/10_nn_14.png" alt="">
  <figcaption></figcaption>
</figure>
<p>This panel, called "Visor," represents the evaluation step, as shown in Figure X.X. The Visor is a part of TensorFlow.js and includes a graph that provides real-time feedback on the progress of the training. Let’s take a moment to focus on the "loss" plotted on the y-axis against the number of epochs along the x-axis.</p>
<p>So, what exactly is this "loss"? Loss is a measure of how far off the model's predictions are from the “correct” outputs provided by the training data. It quantifies the model’s total error. When training begins, it's common for the loss to be high because the model has yet to learn anything. As the model trains through more epochs, it should, ideally, get better at its predictions, and the loss should decrease. If the graph goes down as the epochs increase, this is a good sign!</p>
<p>Running the training for 200 epochs might strike you as a bit excessive. In a real-world scenario with more extensive data, I would probably use fewer epochs. However, because the dataset here is so tiny, the higher number of epochs helps the model get enough "practice" with the data. Remember, this is a "toy" example, aiming to make the concepts clear rather than to produce a sophisticated machine learning model.</p>
<p>Below the graph, you will find a "model summary" table that provides details on the lower-level TensorFlow.js model architecture created behind the scenes. The summary includes layer names, neuron counts per layer, and a "parameters" count, which is the total number of weights, one for each connection between two neurons.</p>
<p>Now, before moving on, I’d like to refer back to the data preparation step. There I mentioned the idea of splitting the data between “training,” “validation,” and “testing.”</p>
<ol>
  <li><strong><em>training</em></strong>: primary dataset used to train the model</li>
  <li><strong><em>validation</em></strong>: subset of data used to check the model during training</li>
  <li><strong><em>testing</em></strong>: additional untouched data never considered during the training process to determine its final performance.</li>
</ol>
<p>With ml5.js, while it’s possible to incorporate all three categories of data. However, I’m simplifying things here and focusing only on the training dataset. After all, my dataset only has 8 records, it’s much too small to divide three different sets! Using such a small dataset risks the model “overfitting” the data. Overfitting is a term that describes when a machine learning model has learned the training data <em>too well</em>. In this case, it’s become so “tuned” to the specific peculiarities of the training data, that is is much less effective when working with new, unseen data. The best way to combat overfitting, is to use validation data during the training process! If it performs well on the training data but poorly on the validation data, it's a strong indicator that overfitting might be occurring.</p>
<p>ml5.js provides some automatic features to employ validation data, if you are inclined to go further, you can explore the full set of neural network examples at <a href="http://ml5js.org/">ml5js.org</a>.</p>
<h3 id="parameter-tuning">Parameter Tuning</h3>
<p>After the evaluation step, there is typically an iterative process of adjusting "hyperparameters" to achieve the best performance from the model. The ml5.js library is designed to provide a higher-level, user-friendly interface to machine learning. So while it does offer some capabilities for parameter tuning (which you can explore in the reference), it is not as geared towards low-level, fine-grained adjustments as some other frameworks might be. Using TensorFlow.js directly might be your best bet since it offers a broader suite of tools and allows for lower-level control over the training process. For this demonstration—seeing a loss all the way down to 0.1 on the evaluation graph—I am satisfied with the result and happy to move onto deployment!</p>
<h3 id="deployment">Deployment</h3>
<p>This is it, all that hard work has paid off! Now it’s time to deploy the model. This typically involves integrating it into a separate application to make predictions or decisions based on new, unseen data. For this, ml5.js offers the convenience of a <code>save()</code> and <code>load()</code> function. After all, there’s no reason to re-train a model every single time you use it! You can download the model to a file in one sketch and then load it for use in a completely different one. However, for simplicity, I’m going to demonstrate deploying and utilizing the model in the same sketch where it was trained.</p>
<p>Once the training process is complete, the resulting model is saved in the <code>classifier</code> variable and is, in essence, deployed. You can detect the completion of the training process using the <code>finishedTraining()</code> callback and use a boolean variable or other logic to initiate the prediction stage of the code. For this example, I’ll include a global variable <code>status</code>to track the training process and ultimately display the predicted label on the canvas.</p>
<pre class="codesplit" data-code-language="javascript">// When the sketch starts, it will show a status of "training"
let status = "training";

function draw() {
  background(255);
  textAlign(CENTER, CENTER);
  textSize(64);
  text(status, width / 2, height / 2);
}

// This is the callback for when training is complete, and the message changes to "ready"
function finishedTraining() {
  status = "ready";
}</pre>
<p>Once the model is trained, the <code>classify()</code> method can be called to send new data into the model for prediction. The format of the data sent to <code>classify()</code> should match the format of the data used in training, in this case two floating point numbers, representing the <code>x</code> and <code>y</code> components of a direction vector.</p>
<pre class="codesplit" data-code-language="javascript">// Manually creating a vector
let direction = createVector(1, 0);
// Converting the x and y components into an input array
let inputs = [direction.x, direction.y];
// Asking the model to classify the inputs
classifier.classify(inputs, gotResults);</pre>
<p>The second argument of the <code>classify()</code> function is a callback. Although it would be more convenient to receive the results immediately and move on to the next line of code, the results are returned later through a separate callback event (just as with model loading and training).</p>
<pre class="codesplit" data-code-language="javascript">function gotResults(results) {
  console.log(results);
}</pre>
<p>The model’s prediction arrives in the argument to the callback, which I’m calling <code>results</code> in the code. Inside, you’ll find an array of the labels, sorted by “confidence.” Confidence refers to the probability assigned by the model to each label, representing how sure it is of that particular prediction. It ranges from 0 to 1, with values closer to 1 indicating higher confidence and values near 0 suggesting lower confidence.</p>
<pre class="codesplit" data-code-language="json">[
  {
    "label": "right",
    "confidence": 0.9669702649116516
  },
  {
    "label": "up",
    "confidence": 0.01878807507455349
  },
  {
    "label": "down",
    "confidence": 0.013948931358754635
  },
  {
    "label": "left",
    "confidence": 0.00029277068097144365
  }
]</pre>
<p>In the example output here, the model is highly confident (approximately 96.7%) that the correct label is "right," while it has minimal confidence in the "left" label, 0.03%. The confidence values are normalized and add up to 100%.</p>
<div data-type="example">
  <h3 id="example-102-gesture-classifier">Example 10.2: Gesture Classifier</h3>
  <figure>
    <div data-type="embed" data-p5-editor="https://editor.p5js.org/natureofcode/sketches/SbfSv_GhM" data-example-path="examples/10_nn/gesture_classifier"></div>
    <figcaption></figcaption>
  </figure>
</div>
<pre class="codesplit" data-code-language="javascript">
// Storing the start of a gesture when the mouse is pressed
function mousePressed() {
  start = createVector(mouseX, mouseY);
}

// Updating the end of a gesture as the mouse is dragged
function mouseDragged() {
  end = createVector(mouseX, mouseY);
}

// The gesture is complete when the mouse is released
function mouseReleased() {
  // Calculate and normalize a direction vector
  let dir = p5.Vector.sub(end, start);
  dir.normalize();
  // Convert to an inputs array and classify
  let inputs = [dir.x, dir.y];
  classifier.classify(inputs, gotResults);
}

// Store the resulting label in the status variable for showing in the canvas
function gotResults(error, results) {
  status = results[0].label;
}</pre>
<p>Since the array is sorted by confidence, if I just want to use a single label as the prediction, I can access the first element of the array with <code>results[0].label</code> as in the <code>gotResults()</code> function in Example 10.2.</p>
<div data-type="note">
  <h3 id="exercise-104">Exercise 10.4</h3>
  <p>Divide Example 10.2 into three different sketches, one for collecting data, one for training, and one for deployment. Using the <code>ml5.neuralNetwork</code> functions <code>save()</code> and <code>load()</code> for saving and loading the model to and from a file.</p>
</div>
<div data-type="note">
  <h3 id="exercise-105">Exercise 10.5</h3>
  <p>Expand the gesture recognition to classify a sequence of vectors, capturing more accurately the path of a longer mouse movement. Remember your input data must have a consistent shape! So you’ll have to decide on how many vectors to use to represent a gesture and store no more and no less for each data point. While this approach can work, other machine learning models (such as Recurrent Neural Networks) are specifically designed to handle sequential data and might offer more flexibility and potential accuracy.</p>
</div>
<h2 id="what-is-neat-neuroevolution-augmented-topologies">What is NEAT? “neuroevolution augmented topologies”</h2>
<p>flappy bird scenario (classification) vs. steering force (regression)?</p>
<p>features?</p>
<h2 id="neuroevolution-steering">NeuroEvolution Steering</h2>
<p>obstacle avoidance example?</p>
<h2 id="other-possibilities">Other possibilities?</h2>
<p></p>
<div data-type="project">
  <h3 id="the-ecosystem-project-9">The Ecosystem Project</h3>
  <p>Step 10 Exercise:</p>
  <p>Try incorporating the concept of a “brain” into your creatures.</p>
  <ul>
    <li>Use reinforcement learning in the creatures’ decision-making process.</li>
    <li>Create a creature that features a visualization of its brain as part of its design (even if the brain itself is not functional).</li>
    <li>Can the ecosystem as a whole emulate the brain? Can elements of the environment be neurons and the creatures act as inputs and outputs?</li>
  </ul>
</div>
<h3 id="the-end">The end</h3>
<p>If you’re still reading, thank you! You’ve reached the end of the book. But for as much material as this book contains, we’ve barely scratched the surface of the world we inhabit and of techniques for simulating it. It’s my intention for this book to live as an ongoing project, and I hope to continue adding new tutorials and examples to the book’s website as well as expand and update the printed material. Your feedback is truly appreciated, so please get in touch via email at <code>(daniel@shiffman.net)</code> or by contributing to the GitHub repository, in keeping with the open-source spirit of the project. Share your work. Keep in touch. Let’s be two with nature.</p>
</section>