Miroirs/noc-book-2
1
0
Fork 0
mirror of https://github.com/nature-of-code/noc-book-2 synced 2024-11-17 07:49:05 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Notion - Update docs

Browse source
This commit is contained in:
shiffman 2023-08-13 18:42:46 +00:00 committed by GitHub
parent 389b210edd
commit bf4f48c280
9 changed files with 487 additions and 34 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

274
content/10_nn.html
View file

@ -7,8 +7,8 @@
<p>I began with inanimate objects living in a world of forces, and gave them desires, autonomy, and the ability to take action according to a system of rules. Next, I allowed those objects, now called creatures, to live in a population and evolve over time. Now I’d like to ask: What is each creature’s decision-making process? How can it adjust its choices by learning over time? Can a computational entity process its environment and generate a decision?</p>
<p>The human brain can be described as a biological neural network—an interconnected web of neurons transmitting elaborate patterns of electrical signals. Dendrites receive input signals and, based on those inputs, fire an output signal via an axon. Or something like that. How the human brain actually works is an elaborate and complex mystery, one that I certainly am not going to attempt to tackle in rigorous detail in this chapter.</p>
<figure>
<img src="images/10_nn/10_nn_1.png" alt="Figure 10. An illustration of a neuron with dendrites and an axon connected to another neuron.">
<figcaption>Figure 10. An illustration of a neuron with dendrites and an axon connected to another neuron.</figcaption>
<img src="images/10_nn/10_nn_1.png" alt="Figure 10.1 An illustration of a neuron with dendrites and an axon connected to another neuron.">
<figcaption>Figure 10.1 An illustration of a neuron with dendrites and an axon connected to another neuron.</figcaption>
</figure>
<p>The good news is that developing engaging animated systems with code does not require scientific rigor or accuracy, as you've learned throughout this book. You can simply be inspired by the idea of brain function.</p>
<p>In this chapter, I'll begin with a conceptual overview of the properties and features of neural networks and build the simplest possible example of one (a network that consists of a single neuron). I’ll then introduce you to more complex neural networks using the ml5.js library. Finally, I'll cover “neuroevolution”, a technique that combines genetic algorithms with neural networks to create a “Brain” object that can be inserted into the <code>Vehicle</code> class and used to calculate steering.</p>
@ -454,26 +454,26 @@ function draw() {
<h2 id="its-a-network-remember">It’s a “Network,” Remember?</h2>
<p>Yes, a perceptron can have multiple inputs, but it is still a lonely neuron. The power of neural networks comes in the networking itself. Perceptrons are, sadly, incredibly limited in their abilities. If you read an AI textbook, it will say that a perceptron can only solve <strong>linearly separable</strong> problems. What’s a linearly separable problem? Let’s take a look at the first example, which determined whether points were on one side of a line or the other.</p>
<figure>
<img src="images/10_nn/10_nn_9.png" alt="Figure 10.11">
<figcaption>Figure 10.11</figcaption>
<img src="images/10_nn/10_nn_9.png" alt="Figure 10.9">
<figcaption>Figure 10.9</figcaption>
</figure>
<p>On the left of Figure 10.11, is an example of classic linearly separable data. Graph all of the possibilities; if you can classify the data with a straight line, then it is linearly separable. On the right, however, is non-linearly separable data. You can’t draw a straight line to separate the black dots from the gray ones.</p>
<p>One of the simplest examples of a non-linearly separable problem is <em>XOR</em>, or “exclusive or.” I’m guessing, as someone who works with coding and p5.js, you are familiar with a logical <span data-type="equation">\text{AND}</span>. For <span data-type="equation">A \text{ AND } B</span> to be true, both <span data-type="equation">A</span> and <span data-type="equation">B</span> must be true. With <span data-type="equation">\text{OR}|</span>, either <span data-type="equation">A</span> or <span data-type="equation">B</span> can be true for <span data-type="equation">A \text{ OR } B</span> to evaluate as true. These are both linearly separable problems. Let’s look at the solution space, a “truth table.”</p>
<figure>
<img src="images/10_nn/10_nn_10.png" alt="Figure 10.12">
<figcaption>Figure 10.12</figcaption>
<img src="images/10_nn/10_nn_10.png" alt="Figure 10.10">
<figcaption>Figure 10.10</figcaption>
</figure>
<p>See how you can draw a line to separate the true outputs from the false ones?</p>
<p><span data-type="equation">\text{XOR}</span> (”exclusive” or) is the equivalent <span data-type="equation">\text{OR}</span> and <span data-type="equation">\text{NOT AND}</span>. In other words, <span data-type="equation">A \text{ XOR } B</span> only evaluates to true if one of them is true. If both are false or both are true, then we get false. Take a look at the following truth table.</p>
<figure>
<img src="images/10_nn/10_nn_11.png" alt="Figure 10.13">
<figcaption>Figure 10.13</figcaption>
<img src="images/10_nn/10_nn_11.png" alt="Figure 10.11">
<figcaption>Figure 10.11</figcaption>
</figure>
<p>This is not linearly separable. Try to draw a straight line to separate the true outputs from the false ones—you can’t!</p>
<p>So perceptrons can’t even solve something as simple as <span data-type="equation">\text{XOR}</span>. But what if we made a network out of two perceptrons? If one perceptron can solve <span data-type="equation">\text{OR}</span> and one perceptron can solve <span data-type="equation">\text{NOT AND}</span>, then two perceptrons combined can solve <span data-type="equation">\text{XOR}</span>.</p>
<figure>
<img src="images/10_nn/10_nn_12.png" alt="Figure 10.14">
<figcaption>Figure 10.14</figcaption>
<img src="images/10_nn/10_nn_12.png" alt="Figure 10.12">
<figcaption>Figure 10.12</figcaption>
</figure>
<p>The above diagram is known as a <em>multi-layered perceptron</em>, a network of many neurons. Some are input neurons and receive the inputs, some are part of what’s called a “hidden” layer (as they are connected to neither the inputs nor the outputs of the network directly), and then there are the output neurons, from which the results are read.</p>
<p>Training these networks is more complex. With the simple perceptron, you could easily evaluate how to change the weights according to the error. But here there are so many different connections, each in a different layer of the network. How does one know how much each neuron or connection contributed to the overall error of the network?</p>
@ -484,21 +484,21 @@ function draw() {
<p>Before I get to my goal of adding a "neural network" brain to a steering agent and tying ml5.js back into the story of the book, I would like to demonstrate step-by-step how to train a neural network model with "supervised learning." There are several key terms and concepts important to cover, namely “classification”, “regression”, “inputs”, and “outputs”. By walking through the full process of a supervised learning scenario, I hope to define these terms, explore other foundational concepts, introduce the syntax of the ml5.js library, and provide the tools to train your first machine learning model with your own data.</p>
<h3 id="classification-and-regression">Classification and Regression</h3>
<p>The majority of machine learning tasks fall into one of two categories: classification and regression. Classification is probably the easier of the two to understand at the start. It involves predicting a “label” (or “category” or “class”) for a piece of data. For example, an image classifier might try to guess if a photo is of a cat or a dog and assign the corresponding label.</p>
<p><strong><em>[FIGURE OF CAT OR DOG OR BIRD OR MONKEY OR ILLUSTRATIONS ASSIGNED A LABEL?]</em></strong></p>
<p><strong><em>[FIGURE 10.13 OF CAT OR DOG OR BIRD OR MONKEY OR ILLUSTRATIONS ASSIGNED A LABEL?]</em></strong></p>
<p>This doesn’t happen by magic, however. The model must first be shown many examples of dogs and cats with the correct labels in order to properly configure the weights of all the connections. This is the supervised learning training process.</p>
<p>The classic “Hello, World” demonstration of machine learning and supervised learning is known as “MNIST”. MNIST, short for “Modified National Institute of Standards and Technology,” is a dataset that was collected and processed by Yann LeCun and Corinna Cortes (AT&#x26;T Labs) and Christopher J.C. Burges (Microsoft Research). It is widely used for training and testing in the field of machine learning and consists of 70,000 handwritten digits from 0 to 9, with each one being a 28x28 pixel grayscale image.</p>
<p><strong><em>[FIGURE FOR MNIST?]</em></strong></p>
<p><strong><em>[FIGURE 10.14 FOR MNIST?]</em></strong></p>
<p>While I won't be building a complete MNIST model with ml5.js (you could if you wanted to!), it serves as a canonical example of a training dataset for image classification: 70,000 images each assigned one of 10 possible labels. This idea of a “label” is fundamental to classification, where the output of a model involves a fixed number of discrete options. There are only 10 possible digits that the model can guess, no more and no less. After the data is used to train the model, the goal is to classify new images and assign the appropriate label.</p>
<p>Regression, on the other hand, is a machine learning task where the prediction is a continuous value, typically a floating point number. A regression problem can involve multiple outputs, but when beginning it’s often simpler to think of it as just one. Consider a machine learning model that predicts the daily electricity usage of a house based on any number of factors like number of occupants, size of house, temperature outside. Here, rather than a goal of the neural network picking from a discrete set of options, it makes more sense for the neural network to guess a number. Will the house use 30.5 kilowatt-hours of energy that day? 48.7 kWh? 100.2 kWh? The output is therefore a continuous value that the model attempts to predict.</p>
<p><strong><em>[FIGURE ILLUSTRATING REGRESSION?]</em></strong></p>
<p><strong><em>[FIGURE 10.15 ILLUSTRATING REGRESSION?]</em></strong></p>
<h3 id="inputs-and-outputs">Inputs and Outputs</h3>
<p>Once the task has been determined, the next step is to finalize the configuration of inputs and outputs of the neural network. In the case of MNIST, each image is a collection of 28x28 grayscale pixels and each pixel can be represented as a single value (ranging from 0-255). The total pixels is <span data-type="equation">28 \times 28 = 784</span>. The grayscale value of each pixel is an input to the neural network.</p>
<figure>
<img src="images/10_nn/10_nn_13.jpg" alt="Place holder figure (just show the inputs first?, borrowed from https://ml4a.github.io/ml4a/looking_inside_neural_nets/">
<figcaption>Place holder figure (just show the inputs first?, borrowed from <a href="https://ml4a.github.io/ml4a/looking_inside_neural_nets/">https://ml4a.github.io/ml4a/looking_inside_neural_nets/</a></figcaption>
<img src="images/10_nn/10_nn_13.jpg" alt="Figure 10.16 Place holder figure (just show the inputs first?, borrowed from https://ml4a.github.io/ml4a/looking_inside_neural_nets/">
<figcaption>Figure 10.16 Place holder figure (just show the inputs first?, borrowed from <a href="https://ml4a.github.io/ml4a/looking_inside_neural_nets/">https://ml4a.github.io/ml4a/looking_inside_neural_nets/</a></figcaption>
</figure>
<p>Since there are 10 possible digits 0-9, the output of the neural network is a prediction of one of 10 labels.</p>
<p><strong><em>[FIGURE NOW ADDS THE OUTPUTS IN]</em></strong></p>
<p><strong><em>[FIGURE 10.17 NOW ADDS THE OUTPUTS IN]</em></strong></p>
<p>Let’s consider the regression scenario of predicting the electricity usage of a house. Let’s assume you have a table with the following data:</p>
<table>
<tbody>
@ -553,7 +553,7 @@ function draw() {
</tbody>
</table>
<p>Here in this table, the inputs to the neural network are the first three columns (occupants, size, temperature). The fourth column on the right is what the neural network is expected to guess, or the output.</p>
<p><strong><em>[FIGURE SHOWING 3 inputs + 1 output]</em></strong></p>
<p><strong><em>[FIGURE 10.18 SHOWING 3 inputs + 1 output]</em></strong></p>
<h3 id="setting-up-the-neural-network-with-ml5js">Setting up the Neural Network with ml5.js</h3>
<p>In a typical machine learning scenario, the next step after establishing the inputs and outputs is to configure the architecture of the neural network. This involves specifying the number of hidden layers between the inputs and outputs, the number of neurons in each layer, which activation functions to use, and more! While all of this is possible with ml5.js, it will make its best guess and design a model for you based on the task and data.</p>
<p>As demonstrated with Matter.js and toxiclibs.js in chapter 6, you can import the ml5.js library into your <strong>index.html</strong> file.</p>
@ -673,8 +673,8 @@ classifier.train(options, finishedTraining);</pre>
<h3 id="evaluation">Evaluation</h3>
<p>If <code>debug</code> is set to true in the initial call to <code>ml5.neuralNetwork()</code>, once <code>train()</code> is called, a visual interface appears covering most of the p5.js page and canvas.</p>
<figure>
<img src="images/10_nn/10_nn_14.png" alt="">
<figcaption></figcaption>
<img src="images/10_nn/10_nn_14.png" alt="Figure 10.19">
<figcaption>Figure 10.19</figcaption>
</figure>
<p>This panel, called "Visor," represents the evaluation step, as shown in Figure X.X. The Visor is a part of TensorFlow.js and includes a graph that provides real-time feedback on the progress of the training. Let’s take a moment to focus on the "loss" plotted on the y-axis against the number of epochs along the x-axis.</p>
<p>So, what exactly is this "loss"? Loss is a measure of how far off the model's predictions are from the “correct” outputs provided by the training data. It quantifies the model’s total error. When training begins, it's common for the loss to be high because the model has yet to learn anything. As the model trains through more epochs, it should, ideally, get better at its predictions, and the loss should decrease. If the graph goes down as the epochs increase, this is a good sign!</p>
@ -787,11 +787,11 @@ function gotResults(error, results) {
<p>There is so much more to working with data, machine learning, ml5.js, and beyond. I’ve only scratched the surface. As I close out this book, my goal is to tie the foundational machine learning concepts I’ve covered back into animated, interactive p5.js sketches that simulate physics and complex systems. Let’s see if I can bring as many concepts from the entire book back together for one last hurrah!</p>
<p>Towards the start of this chapter, I referenced an approach to incorporating machine learning into a simulated environment called “reinforcement learning.” Imagine embedding a neural network into any of the example objects (walker, mover, particle, vehicle) and calculating a force or some other action. The neural network could receive inputs related to the environment (such as distance to an obstacle) and produce a decision that requires a choice from a set of discrete options (e.g., move “left” or “right”) or a set of continuous values (e.g., magnitude and direction of a steering force). This is starting to sound familiar: it’s a neural network that receives inputs and performs classification or regression!</p>
<p>Here is where things take a turn, however. To better illustrate the concept, let’s start with a hopefully easy to understand and possibly familiar scenario, the game “Flappy Bird.” The game is deceptively simple. You control a small bird that continually moves horizontally across the screen. With each tap or click, the bird flaps its wings and rises upward. The challenge? A series of vertical pipes spaced apart at irregular intervals emerges from the right. The pipes have gaps, and your primary objective is to navigate the bird safely through these gaps. If you hit one, it’s game over. As you progress, the game’s speed increases, and the more pipes you navigate, the higher your score.</p>
<p><strong><em>[ILLUSTRATION OF FLAPPY BIRD]</em></strong></p>
<p><strong><em>[Figure 10.20 ILLUSTRATION OF FLAPPY BIRD]</em></strong></p>
<p>Suppose you wanted to automate the gameplay, and instead of a human tapping, a neural network will make the decision as to whether to “flap” or not. Could machine learning work here? Skipping over the “data” steps for a moment, let’s think about “choosing a model.” What are the inputs and outputs of the neural network?</p>
<p>Let’s being with the inputs. This is a quite the intriguing question because there isn’t a definitive answer! In a scenario where you want to see if you could train an automated neural network player without any knowledge of the game itself, it might make the most sense to have the inputs be all the pixels of the game screen. Maybe you don’t want to put your thumb on the scale in terms of what aspects of the game are important so this approach attempts to feed <em>everything</em> about the game into the model.</p>
<p>For me, I understand the flappy bird game quite well, I believe I can identify the important points data points needed to make a decision. I can bypass all the pixels and boil the essence of the game down into the important <strong>features </strong>that define the game. Remember the discussion about features in the context of the gesture classifier? It applies here as well. These features are not arbitrary aspects of the game; they represent the distinct characteristics of Flappy Bird that are most salient for the neural network's decisions.</p>
<p><strong><em>[ILLUSTRATION OF FLAPPY BIRD WITH FEATURES MARKED]</em></strong></p>
<p><strong><em>[FIGURE 10.21 OF FLAPPY BIRD WITH FEATURES MARKED]</em></strong></p>
<ol>
<li><span data-type="equation">y</span> position of the bird</li>
<li><span data-type="equation">y</span> velocity of the bird.</li>
@ -805,7 +805,7 @@ function gotResults(error, results) {
<li>flap</li>
<li>don’t flap</li>
</ol>
<p><strong><em>[DIAGRAM OF THE NEURAL NETWORK AS ML5 MIGHT ARCHITECT IT]</em></strong></p>
<p><strong><em>[FIGURE 10.22 OF THE NEURAL NETWORK AS ML5 MIGHT ARCHITECT IT]</em></strong></p>
<p>This gives me the information needed to choose the model and I can let ml5.js build it.</p>
<pre class="codesplit" data-code-language="javascript">let options = {
inputs: 5,
@ -839,7 +839,7 @@ let birdBrain = ml5.neuralNetwork(options);</pre>
// The bird flaps its wings
flap() {
this.y += this.flapForce;
this.velocity += this.flapForce;
}
update() {
@ -847,7 +847,7 @@ let birdBrain = ml5.neuralNetwork(options);</pre>
this.velocity += this.gravity;
this.y += this.velocity;
// Dampen velocity
this.velocity *= 0.9;
this.velocity *= 0.95;
// Handle the "floor"
if (this.y > height) {
@ -860,7 +860,7 @@ let birdBrain = ml5.neuralNetwork(options);</pre>
<pre class="codesplit" data-code-language="javascript">class Pipe {
constructor() {
// The size of the opening between the two parts of the pipe
this.spacing = 75;
this.spacing = 100;
// A random height for the top of the pipe
this.top = random(height - this.spacing);
// The starting position of the bottom pipe (based on the top)
@ -942,7 +942,229 @@ function draw() {
<h3 id="exercise-107">Exercise 10.7</h3>
<p>Implement a scoring system that awards points for successfully navigating through each set of pipes. Feel free to add your own visual design elements for the bird, pipes, and environment!</p>
</div>
<h2 id="evolving-the-bird">Evolving the Bird</h2>
<h2 id="neuroevolution-flappy-bird">NeuroEvolution Flappy Bird</h2>
<p>The game, as it currently stands, is controlled by mouse clicks. The first step to implementing neuro-evolution is to give each bird a brain so that it can intelligently decide whether or not to flap its wings.</p>
<h3 id="the-bird-brain">The Bird Brain</h3>
<p>In the previous section on reinforcement learning, I established a list of input features that comprise the bird's decision-making process. I’m going to use that same list with one simplification. Since the size of the opening between the pipes will remain constant, there’s no need to include both the <span data-type="equation">y</span> positions of the top and bottom and can just use one.</p>
<ol>
<li><span data-type="equation">y</span> position of the bird.</li>
<li><span data-type="equation">y</span> velocity of the bird.</li>
<li><span data-type="equation">y</span> position of the next pipe’s top (or the bottom!) opening.</li>
<li><span data-type="equation">x</span> distance to the next pipes.</li>
</ol>
<p>The outputs have just two options: to flap or not to flap! With the inputs and outputs set, I can add a <code>brain</code> property to the bird’s constructor with the appropriate configuration. Just to demonstrate a different style here, I’ll skip including a separate <code>options</code> variable and pass the properties as an object literal directly into the <code>ml5.neuralNetwork()</code> function. Note the addition of a <code>neuroEvolution</code> property set to <code>true</code>. This is necessary to enable some of the features I’ll be using later in the code.</p>
<pre class="codesplit" data-code-language="javascript"> constructor() {
this.brain = ml5.neuralNetwork({
// A bird's brain receives 5 inputs and classifies them into one of two labels
inputs: 4,
outputs: ["flap", "no flap"],
task: "classification",
// A new property necessary to enable neuro evolution functionality
neuroEvolution: true
});
}</pre>
<p>Next, I’ll add a new method called <code>think()</code> to the <code>Bird</code> class where I will calculate all of the necessary inputs for the bird. The first two are easy, as they are simply the <code>y</code> and <code>velocity</code> properties of the bird itself. However, for inputs 3 through 5, I need to determine which pipe is the “next” pipe.</p>
<p>At first glance, it might seem that the next pipe is always the first one in the array, since the pipes are added one at a time to the end of the array. However, once a pipe passes the bird, it is no longer relevant. I need to find the first pipe in the array whose right edge (x-position plus width) is greater than the bird’s x position.</p>
<pre class="codesplit" data-code-language="javascript"> think(pipes) {
let nextPipe = null;
for (let pipe of pipes) {
//{!4} The next pipe is the one who hasn't passed the bird yet.
if (pipe.x + pipe.w > this.x) {
nextPipe = pipe;
break;
}
}</pre>
<p>Once I have the next pipe, I can create the four inputs:</p>
<pre class="codesplit" data-code-language="javascript"> let inputs = [
// y-position of bird
this.y,
// y-velocity of bird
this.velocity,
// top opening of next pipe
nextPipe.top,
//{!1} distance from next pipe to this pipe
nextPipe.x - this.x,
];</pre>
<p>However, I have forgotten a critical step! The range of all input values is determined by the dimensions of the canvas. The neural network, however, expects values in a standardized range, such as 0 to 1. One simple method to normalize these values is to divide the inputs related to vertical properties by<code>height</code>, and those related to horizontal ones by <code>width</code>.</p>
<pre class="codesplit" data-code-language="javascript"> let inputs = [
//{!5} All of the inputs are now normalized by width and height
this.y / height,
this.velocity / height,
nextPipe.top / height,
(nextPipe.x - this.x) / width,
];</pre>
<p>With the inputs in hand, I’m ready to pass them to the neural network’s <code>classify()</code> method. There is, however, one small problem. Remember, <code>classify()</code> is asynchronous! This means I need implement a callback inside the <code>Bird</code> class to process the decision! Unfortunately, doing so adds a level of complexity to the code here which is entirely unnecessary. Asynchronous callbacks with machine learning functions in ml5.js are typically necessary due to the time required to process a large amount of data in a model. Without a callback, the code might have to wait a long time and if it’s in the context of a p5.js animation, it could severely impact the smoothness of any animation. The neural network here, however, only has four floating point inputs and two output labels! It’s tiny and can run so fast there’s no reason to implement this asynchronously.</p>
<p>For completeness, I will include a version of this example on the book’s website that implements this example with asynchronous callbacks. For the discussion here, however, I’m going to use a feature of ml5.js which allows me to take a shortcut. The method <code>classifySync()</code> is identical to <code>classify()</code>, but runs synchronously meaning the code stops and waits for the results before moving on. You should be very careful about using this version of the function as it can cause problems in other contexts, but it will work quite well here. Here is the end of the <code>think()</code> method with <code>classifySync()</code>.</p>
<pre class="codesplit" data-code-language="javascript"> let results = this.brain.classifySync(inputs);
if (results[0].label == "flap") {
this.flap();
}
}</pre>
<p>The neural network's prediction is in the same format as before and the decision can be made by checking the first element of the <code>results</code> array. If the output label is <code>"flap"</code>, then call <code>flap()</code>.</p>
<p>Now is where the real challenge begins: teaching the bird to navigate the game and flap its wings at the opportune moments! Recalling the discussion of genetic algorithms from Chapter 9, there are three key principles that underpin Darwinian evolution: <strong>Variation</strong>, <strong>Selection</strong>, and <strong>Heredity</strong>. Let’s go through each of these principles, implementing all the steps of the genetic algorithm itself.</p>
<h3 id="variation-a-flock-of-flappy-birds">Variation: A Flock of Flappy Birds</h3>
<p>A single bird with a randomly initialized neural network isn’t likely to have any success at all. That lone bird is most likely to jump incessantly and fly way offscreen or sit perched at the bottom of the canvas awaiting collision after collision with the pipes. This erratic and nonsensical behavior serves as a reminder: a randomly initialized neural network lacks any knowledge or experience! The bird is essentially making wild guesses for its actions and success is going to be very rare.</p>
<p>This is where the first key principle of genetic algorithms comes in: variation! The hope is that by introducing as many different neural network configurations as possible, a few might perform slightly better than the rest. And so the very first step is to add an array of many birds.</p>
<pre class="codesplit" data-code-language="javascript">// Population size
let populationSize = 200;
// Array of birds
let birds = [];
function setup() {
//{!3} Create the bird population
for (let i = 0; i &#x3C; populationSize; i++) {
birds[i] = new Bird();
}
//{!1} Run the computations on the "cpu" for better performance
ml5.tf.setBackend("cpu");
}
function draw() {
for (let bird of birds) {
//{!1} This is the new method for the bird to make a decision to flap or not
bird.think(pipes);
bird.update();
bird.show();
}
}</pre>
<p>You might notice a peculiar line of code that's crept into setup: <code>ml5.tf.setBackend("cpu")</code>. When running neural networks, a lot of the heavy computational lifting is often offloaded to the GPU. This is the default behavior, and especially critical for larger pre-trained models included as part of ml5.js.</p>
<div data-type="note">
<h3 id="gpu-vs-cpu">GPU vs. CPU</h3>
<ul>
<li><strong>GPU (Graphics Processing Unit)</strong>: Originally designed for rendering graphics, GPUs are adept at handling a massive number of operations in parallel. This makes them excellent for the kind of math operations and computations that machine learning models frequently perform.</li>
<li><strong>CPU (Central Processing Unit)</strong>: Often considered the "brain" or general-purpose heart of a computer, a CPU handles a wider variety of tasks than the specialized GPU.</li>
</ul>
</div>
<p>But there's a catch! Transferring data to and from the GPU introduces some overhead. In most cases, the gains from the GPU's parallel processing offset this overhead. However, for such a tiny model like the one here, copying data to the GPU and back slows things down more than it helps.</p>
<p>This is where <code>ml5.tf.setBackend("cpu")</code> comes in. By specifying <code>"cpu"</code>, the neural network computations will instead run the “Central Processing Unit” —the general-purpose heart of your computer— which handles the operations more efficiently for a population of many tiny bird brains.</p>
<h3 id="selection-flappy-bird-fitness">Selection: Flappy Bird Fitness</h3>
<p>Once I’ve got a diverse population of birds, each with their own neural network, the next step in the genetic algorithm is selection. Which birds should pass on their genes (in this case, neural network weights) to the next generation? In the world of Flappy Bird, the measure of success is the ability to stay alive the longest avoiding the pipes. This is the bird's "fitness." A bird that dodges many pipes is considered more "fit" than one that crashes into the first one it encounters.</p>
<p>To track the bird’s fitness, I am going to add two properties to the <code>Bird</code> class: <code>fitness</code> and <code>alive</code>.</p>
<pre class="codesplit" data-code-language="javascript"> constructor() {
// The bird's fitness
this.fitness = 0;
// Keeping track if the bird is alive or not
this.alive = true;
}</pre>
<p>The fitness will be a numeric value that increases by 1 every cycle through <code>draw()</code>, as long as the bird remains alive. The birds that survive longer will have a higher fitness.</p>
<pre class="codesplit" data-code-language="javascript"> update() {
// Incrementing the fitness each time through update
this.fitness++;
}</pre>
<p>The <code>alive</code> property is a <code>boolean</code> flag that is initially set to true. However, when a bird collides with a pipe, it is set to <code>false</code>. Only birds that are still alive are updated and drawn to the canvas.</p>
<pre class="codesplit" data-code-language="javascript">function draw() {
// There are now an array of birds!
for (let bird of birds) {
// Only operate on the birds that are still alive
if (bird.alive) {
bird.think(pipes);
// Update and show the bird
bird.update();
bird.show();
//{!4} Has the bird hit a pipe? If so, it's no longer alive.
for (let pipe of pipes) {
if (pipe.collides(bird)) {
bird.alive = false;
}
}
}
}
}</pre>
<p>In Chapter 9, I demonstrated two techniques for running an evolutionary simulation.The first involved a population living for a fixed amount of time each generation. The same approach would likely work here as well, but I want to allow the birds to accumulate the highest fitness possible. The second technique, demonstrated with the "bloops" example, involved eliminating the fitness score entirely and setting a random probability for cloning alive birds. However, this approach could become messy and risks overpopulation or all the birds dying out completely . Instead, I propose combining elements of both approaches. I will allow a generation to continue as long as at least one bird is still alive. When all the birds have died, I will select parents for the reproduction step and start anew.</p>
<p>Let’s begin by writing a function to check if all the birds have died.</p>
<pre class="codesplit" data-code-language="javascript">function allBirdsDead() {
for (let bird of birds) {
//{!3} If a single bird is alive, they are not all dead!
if (bird.alive) {
return false;
}
}
//{!1} If the loop completes without finding a living bird, they are all dead
return true;
}</pre>
<p>When all the birds have died, then it’s time for selection! In Chapter 9, I demonstrated several different techniques for giving a fair shot to all members of a population, but increasing the changes of selection for those with higher fitness scores. I’ll use the same exact <code>weightedSelection()</code> function here.</p>
<pre class="codesplit" data-code-language="javascript">function weightedSelection() {
let index = 0;
let start = random(1);
while (start > 0) {
start = start - birds[index].fitness;
index++;
}
index--;
//{!1} Instead of returning the entire Bird object, just the brain is returned
return birds[index].brain;
}</pre>
<p>However, for this algorithm to function properly, I need to first normalize the fitness values of the birds so that they collectively sum to 1. This way, each bird's fitness is equal to its probability of being selected.</p>
<pre class="codesplit" data-code-language="javascript">function normalizeFitness() {
// Sum the total fitness of all birds
let sum = 0;
for (let bird of birds) {
sum += bird.fitness;
}
//{!3} Divide each bird's fitness by the sum
for (let bird of birds) {
bird.fitness = bird.fitness / sum;
}
}</pre>
<h3 id="heredity-baby-birds">Heredity: Baby Birds</h3>
<p>There’s only one step left in the genetic algorithm—reproduction. In Chapter 9, I explored in great detail the two step process for generating a “child” element: crossover and mutation. Crossover is where the third key principle of “heredity” arrives. After selecting the DNA of two parents, they are combined to form the child’s DNA. At first glance, the idea of inventing an algorithm for crossover of two neural networks might seem daunting. Yet, it’s actually quite straightforward. Think of the individual “genes” of a bird’s brain to be the weights within the network. Mixing two such brains boils down to creating a new neural network, where each weight is chosen by a virtual coin flip—picking a value from the first or second parent.</p>
<pre class="codesplit" data-code-language="javascript">// Picking two parents and creating a child with crossover
let parentA = weightedSelection();
let parentB = weightedSelection();
let child = parentA.crossover(parentB);</pre>
<p>As you can see, today is my lucky day, as ml5.js includes a <code>crossover()</code> that manages the algorithm for mixing the two neural networks. I can happily move onto the mutation step.</p>
<pre class="codesplit" data-code-language="javascript">// Mutating the child
child.mutate(0.01);</pre>
<p>The ml5.js library provides a <code>mutate()</code> method that accepts a "mutation rate" as its primary argument. This rate determines how often a weight will be altered. For example, a rate of 0.01 indicates a 1% chance that any given weight will mutate. During mutation, ml5.js adjusts the weight slightly by adding a small random number to it, rather than selecting a completely new random value. This behavior mimics real-world genetic mutations, which typically introduce minor changes rather than entirely new traits. Although this default approach works for many cases, ml5.js offers more control over the process by allowing the use of a "custom" function as an optional second argument to <code>mutate()</code>.</p>
<p>These crossover and mutation steps are repeated for the size of the population to create an entire new generation of birds. This is accomplished by populating an empty local array <code>nextBirds</code> with the new birds. Once the population is full, the global <code>birds</code> array is then updated to this fresh generation.</p>
<pre class="codesplit" data-code-language="javascript">function reproduction() {
let nextBirds = [];
for (let i = 0; i &#x3C; populationSize; i++) {
let parentA = weightedSelection();
let parentB = weightedSelection();
let child = parentA.crossover(parentB);
child.mutate(0.01);
nextBirds[i] = new Bird(child);
}
birds = nextBirds;
}</pre>
<p>If you look closely at the <code>reproduction()</code> function, however, you may noticed that I’ve slipped in another new feature of the <code>Bird</code> class, specifically the constructor. When I first introduced the idea of a bird “brain,” each new <code>Bird</code> object was created with a brand new brain—essentially a fresh neural network courtesy of ml5.js. However, I now want the new birds to “inherit” a child brain that was generated through the process of crossover and mutation.</p>
<p>To make this possible, I’ll subtly change the <code>Bird</code> constructor to look for an “optional” argument name, of course, <code>brain</code>.</p>
<pre class="codesplit" data-code-language="javascript"> constructor(brain) {
//{!1} Check if a brain was passed in
if (brain) {
this.brain = brain;
//{!1} If not, proceed as usual
} else {
this.brain = ml5.neuralNetwork({
inputs: 4,
outputs: ["flap", "no flap"],
task: "classification",
neuroEvolution: true,
});
}
}</pre>
<p>Here’s the magic, if no <code>brain</code> is provided when a new bird is created, the <code>brain</code> argument remains <code>undefined</code>. In JavaScript, <code>undefined</code> is treated as <code>false</code> and so the code moves on to the <code>else</code> and calls <code>ml5.neuralNetwork()</code>. On the other hand, if I I do pass in an existing neural network, <code>brain</code> will evaluate to <code>true</code> and is assigned directly to <code>this.brain</code>. This elegant trick allows the constructor to handle different scenarios, a testament to JavaScript’s flexible nature.</p>
<p>And with that, the example is complete. All that is left to do is call <code>normalizeFitness()</code> and <code>reproduction()</code> in <code>draw()</code> at the end of each generation when all the birds have died out.</p>
<div data-type="example">
<h3 id="example-104-flappy-bird-neuroevolution">Example 10.4: Flappy Bird NeuroEvolution</h3>
<figure>
<div data-type="embed" data-p5-editor="https://editor.p5js.org/natureofcode/sketches/PEUKc5dpZ" data-example-path="examples/10_nn/flappy_bird_neuro_evolution"></div>
<figcaption></figcaption>
</figure>
</div>
<pre class="codesplit" data-code-language="javascript">function draw() {
//{inline} all the rest of draw
if (allBirdsDead()) {
normalizeFitness();
reproduction();
}
}</pre>
<p>Example 10.4 also adjusts the behavior of birds so that they die when they leave the canvas, either by crashing into the ground or soaring too high above the top.</p>
<p><strong>EXERCISE: SPEED UP TIME, ANNOTATE PROCESS, ETC.</strong></p>
<p><strong>EXERCISE: SAVE AND LOAD BIRD</strong></p>
<h2 id="evolving-a-steering-forces">Evolving a Steering Forces</h2>
<p></p>
<p>The end?</p>

4
content/examples/10_nn/flappy_bird/bird.js
View file

@ -12,7 +12,7 @@ class Bird {
// The bird flaps its wings
flap() {
this.y += this.flapForce;
this.velocity += this.flapForce;
}
update() {
@ -20,7 +20,7 @@ class Bird {
this.velocity += this.gravity;
this.y += this.velocity;
// Dampen velocity
this.velocity *= 0.9;
this.velocity *= 0.95;
// Handle the "floor"
if (this.y > height) {

2
content/examples/10_nn/flappy_bird/pipe.js
View file

@ -1,6 +1,6 @@
class Pipe {
constructor() {
this.spacing = 75;
this.spacing = 100;
this.top = random(height - this.spacing);
this.bottom = this.top + this.spacing;
this.x = width;

8
content/examples/10_nn/flappy_bird/sketch.js
View file

@ -26,13 +26,11 @@ function draw() {
bird.update();
bird.show();
if (frameCount % 75 == 0) {
if (frameCount % 100 == 0) {
pipes.push(new Pipe());
}
}
function keyPressed() {
if (key == " ") {
bird.flap();
}
function mousePressed() {
bird.flap();
}

80
content/examples/10_nn/flappy_bird_neuro_evolution/bird.js Normal file
View file

@ -0,0 +1,80 @@
class Bird {
constructor(brain) {
// A bird's brain receives 5 inputs and classifies them into one of two labels
if (brain) {
this.brain = brain;
} else {
this.brain = ml5.neuralNetwork({
inputs: 4,
outputs: ["flap", "no flap"],
task: "classification",
// change to "neuroEvolution" for next ml5.js release
noTraining: true
// neuroEvolution: true,
});
}
// The bird's position (x will be constant)
this.x = 50;
this.y = 120;
// Velocity and forces are scalar since the bird only moves along the y-axis
this.velocity = 0;
this.gravity = 0.5;
this.flapForce = -10;
// Adding a fitness
this.fitness = 0;
this.alive = true;
}
think(pipes) {
let nextPipe = null;
for (let pipe of pipes) {
if (pipe.x + pipe.w > this.x) {
nextPipe = pipe;
break;
}
}
let inputs = [
this.y / height,
this.velocity / height,
nextPipe.top / height,
(nextPipe.x - this.x) / width,
];
let results = this.brain.classifySync(inputs);
if (results[0].label == "flap") {
this.flap();
}
}
// The bird flaps its wings
flap() {
this.velocity += this.flapForce;
}
update() {
// Add gravity
this.velocity += this.gravity;
this.y += this.velocity;
// Dampen velocity
this.velocity *= 0.95;
// Handle the "floor"
if (this.y > height || this.y < 0) {
this.alive = false;
}
this.fitness++;
}
show() {
strokeWeight(2);
stroke(0);
fill(127, 200);
circle(this.x, this.y, 16);
}
}

16
content/examples/10_nn/flappy_bird_neuro_evolution/index.html Normal file
View file

@ -0,0 +1,16 @@
<!DOCTYPE html>
<html lang="en">
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.7.0/p5.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.7.0/addons/p5.sound.min.js"></script>
<script src="https://unpkg.com/ml5@latest/dist/ml5.min.js"></script>
<link rel="stylesheet" type="text/css" href="style.css" />
<meta charset="utf-8" />
</head>
<body>
<main></main>
<script src="bird.js"></script>
<script src="pipe.js"></script>
<script src="sketch.js"></script>
</body>
</html>

34
content/examples/10_nn/flappy_bird_neuro_evolution/pipe.js Normal file
View file

@ -0,0 +1,34 @@
class Pipe {
constructor() {
this.spacing = 100;
this.top = random(height - this.spacing);
this.bottom = this.top + this.spacing;
this.x = width;
this.w = 20;
this.speed = 2;
}
collides(bird) {
// Is the bird within the vertical range of the top or bottom pipe?
let verticalCollision = bird.y < this.top || bird.y > this.bottom;
// Is the bird within the horizontal range of the pipes?
let horizontalCollision = bird.x > this.x && bird.x < this.x + this.w;
// If it's both a vertical and horizontal hit, it's a hit!
return verticalCollision && horizontalCollision;
}
show() {
fill(0);
noStroke();
rect(this.x, 0, this.w, this.top);
rect(this.x, this.bottom, this.w, height - this.bottom);
}
update() {
this.x -= this.speed;
}
offscreen() {
return this.x < -this.w;
}
}

96
content/examples/10_nn/flappy_bird_neuro_evolution/sketch.js Normal file
View file

@ -0,0 +1,96 @@
let birds = [];
let pipes = [];
function setup() {
createCanvas(640, 240);
for (let i = 0; i < 200; i++) {
birds[i] = new Bird();
}
pipes.push(new Pipe());
ml5.tf.setBackend("cpu");
}
function draw() {
background(255);
for (let i = pipes.length - 1; i >= 0; i--) {
pipes[i].update();
pipes[i].show();
if (pipes[i].offscreen()) {
pipes.splice(i, 1);
}
}
for (let bird of birds) {
if (bird.alive) {
for (let pipe of pipes) {
if (pipe.collides(bird)) {
bird.alive = false;
}
}
bird.think(pipes);
bird.update();
bird.show();
}
}
if (frameCount % 100 == 0) {
pipes.push(new Pipe());
}
if (allBirdsDead()) {
normalizeFitness();
reproduction();
}
}
function allBirdsDead() {
for (let bird of birds) {
if (bird.alive) {
return false;
}
}
return true;
}
function reproduction() {
let nextBirds = [];
for (let i = 0; i < birds.length; i++) {
let parentA = weightedSelection();
let parentB = weightedSelection();
let child = parentA.crossover(parentB);
child.mutate(0.01);
nextBirds[i] = new Bird(child);
}
birds = nextBirds;
}
// Normalize all fitness values
function normalizeFitness() {
let sum = 0;
for (let bird of birds) {
sum += bird.fitness;
}
for (let bird of birds) {
bird.fitness = bird.fitness / sum;
}
}
function weightedSelection() {
// Start with the first element
let index = 0;
// Pick a starting point
let start = random(1);
// At the finish line?
while (start > 0) {
// Move a distance according to fitness
start = start - birds[index].fitness;
// Next element
index++;
}
// Undo moving to the next element since the finish has been reached
index--;
//{!1} Instead of returning the entire Bird object, just the brain is returned
return birds[index].brain;
}

7
content/examples/10_nn/flappy_bird_neuro_evolution/style.css Normal file
View file

@ -0,0 +1,7 @@
html, body {
margin: 0;
padding: 0;
}
canvas {
display: block;
}