Merge pull request #833 from nature-of-code/notion-update-docs

Chapter 10

content/10_nn.html

@@ -37,7 +37,13 @@
   <li><strong>Pattern recognition:</strong> Neural networks are well suited to problems when the aim is to detect, interpret, and classify features or patterns within a dataset. This includes everything from identifying objects (like faces) in images, to optical character recognition, to more complex tasks like gesture recognition.</li>
   <li><strong>Time-series prediction and anomaly detection: </strong>Neural networks are utilized both in forecasting, such as predicting stock market trends or weather patterns, and in recognizing anomalies, which can be applied to areas like cyberattack detection and fraud prevention.</li>
   <li><strong>Natural language processing (NLP):</strong> One of the biggest developments in recent years has been the use of neural networks for processing and understanding human language. They’re used in various tasks including machine translation, sentiment analysis, and text summarization, and are the underlying technology behind many digital assistants and chatbots.</li>
-  <li><strong>Signal processing and soft sensors:</strong> Neural networks play a crucial role in devices like cochlear implants and hearing aids by filtering noise and amplifying essential sounds. They’re also involved in <em>soft sensors</em>, software systems that process data from multiple sources to give a comprehensive analysis of the environment.</li>
+</ul>
+<div class="avoid-break">
+  <ul>
+    <li><strong>Signal processing and soft sensors:</strong> Neural networks play a crucial role in devices like cochlear implants and hearing aids by filtering noise and amplifying essential sounds. They’re also involved in <em>soft sensors</em>, software systems that process data from multiple sources to give a comprehensive analysis of the environment.</li>
+  </ul>
+</div>
+<ul>
   <li><strong>Control and adaptive decision-making systems: </strong>These applications range from autonomous vehicles like self-driving cars and drones to adaptive decision-making used in game playing, pricing models, and recommendation systems on media platforms.</li>
   <li><strong>Generative models:</strong> The rise of novel neural network architectures has made it possible to generate new content. These systems can synthesize images, enhance image resolution, transfer style between images, and even generate music and video.</li>
 </ul>
@@ -114,28 +120,30 @@
     </tr>
   </tbody>
 </table>
-<p>Each input needs to be multiplied by its corresponding weight:</p>
-<table>
-  <thead>
-    <tr>
-      <th style="width:100px">Phrase</th>
-      <th style="width:100px">Phrase</th>
-      <th>Input <span data-type="equation">\boldsymbol{\times}</span> Weight</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td>12</td>
-      <td>0.5</td>
-      <td>6</td>
-    </tr>
-    <tr>
-      <td>4</td>
-      <td>–1</td>
-      <td>–4</td>
-    </tr>
-  </tbody>
-</table>
+<div class="avoid-break">
+  <p>Each input needs to be multiplied by its corresponding weight:</p>
+  <table>
+    <thead>
+      <tr>
+        <th style="width:100px">Phrase</th>
+        <th style="width:100px">Phrase</th>
+        <th>Input <span data-type="equation">\boldsymbol{\times}</span> Weight</th>
+      </tr>
+    </thead>
+    <tbody>
+      <tr>
+        <td>12</td>
+        <td>0.5</td>
+        <td>6</td>
+      </tr>
+      <tr>
+        <td>4</td>
+        <td>–1</td>
+        <td>–4</td>
+      </tr>
+    </tbody>
+  </table>
+</div>
 <h4 id="step-2-sum-the-inputs">Step 2: Sum the Inputs</h4>
 <p>The weighted inputs are then added together:</p>
 <div data-type="equation">6 + -4 = 2</div>
@@ -262,7 +270,7 @@ function activate(sum) {
 <p>Here’s the code to generate a guess:</p>
 <pre class="codesplit" data-code-language="javascript">// Create the perceptron.
 let perceptron = new Perceptron(3);
-// The input is three values: x, y, and the bias.
+// The input is three values: <em>x</em>, <em>y</em>, and the bias.
 let inputs = [50, -12, 1];
 // The answer!
 let guess = perceptron.feedForward(inputs);</pre>
@@ -521,17 +529,17 @@ function draw() {
   <figcaption>Figure 10.10: Data points that are linearly separable (left) and data points that are nonlinearly separable, as a curve is required to separate the points (right)</figcaption>
 </figure>
 <p>Now imagine you’re classifying plants according to soil acidity (x-axis) and temperature (y-axis). Some plants might thrive in acidic soils but only within a narrow temperature range, while other plants prefer less acidic soils but tolerate a broader range of temperatures. A more complex relationship exists between the two variables, so a straight line can’t be drawn to separate the two categories of plants, <em>acidophilic</em> and <em>alkaliphilic</em> (see Figure 10.10, right). A lone perceptron can’t handle this type of <strong>nonlinearly separable</strong> problem. (Caveat here: I’m making up these scenarios. If you happen to be a botanist, please let me know if I’m anywhere close to reality.)</p>
+<p>One of the simplest examples of a nonlinearly separable problem is XOR (exclusive or). This is a logical operator, similar to the more familiar AND and OR. For <em>A</em> AND <em>B </em>to be true, both <em>A</em> and <em>B</em> must be true. With OR, either <em>A</em> or <em>B</em> (or both) can be true. These are both linearly separable problems. The truth tables in Figure 10.11 show their solution space. Each true or false value in the table shows the output for a particular combination of true or false inputs. See how you can draw a straight line to separate the true outputs from the false ones?</p>
 <figure>
   <img src="images/10_nn/10_nn_12.png" alt="Figure 10.11: Truth tables for the AND and OR logical operators. The true and false outputs can be separated by a line.">
   <figcaption>Figure 10.11: Truth tables for the AND and OR logical operators. The true and false outputs can be separated by a line.</figcaption>
 </figure>
-<p>One of the simplest examples of a nonlinearly separable problem is XOR (exclusive or). This is a logical operator, similar to the more familiar AND and OR. For <em>A</em> AND <em>B </em>to be true, both <em>A</em> and <em>B</em> must be true. With OR, either <em>A</em> or <em>B</em> (or both) can be true. These are both linearly separable problems. The truth tables in Figure 10.11 show their solution space. Each true or false value in the table shows the output for a particular combination of true or false inputs. See how you can draw a straight line to separate the true outputs from the false ones?</p>
 <p>The XOR operator is the equivalent of (OR) AND (NOT AND). In other words, <em>A</em> XOR <em>B </em>evaluates to true only if one of the inputs is true. If both inputs are false or both are true, the output is false. To illustrate, let’s say you’re having pizza for dinner. You love pineapple on pizza, and you love mushrooms on pizza, but put them together—yech! And plain pizza, that’s no good either!</p>
+<p>The XOR truth table in Figure 10.12 isn’t linearly separable. Try to draw a straight line to separate the true outputs from the false ones—you can’t!</p>
 <figure>
   <img src="images/10_nn/10_nn_13.png" alt="Figure 10.12: The truth tables for whether you want to eat the pizza (left) and XOR (right). Note how the true and false outputs can’t be separated by a single line.">
   <figcaption>Figure 10.12: The truth tables for whether you want to eat the pizza (left) and XOR (right). Note how the true and false outputs can’t be separated by a single line.</figcaption>
 </figure>
-<p>The XOR truth table in Figure 10.12 isn’t linearly separable. Try to draw a straight line to separate the true outputs from the false ones—you can’t!</p>
 <p>The fact that a perceptron can’t even solve something as simple as XOR may seem extremely limiting. But what if I made a network out of two perceptrons? If one perceptron can solve the linearly separable OR and one perceptron can solve the linearly separate NOT AND, then two perceptrons combined can solve the nonlinearly separable XOR.</p>
 <p>When you combine multiple perceptrons, you get a <strong>multilayered perceptron</strong>, a network of many neurons (see Figure 10.13). Some are input neurons and receive the initial inputs, some are part of what’s called a <strong>hidden layer</strong> (as they’re connected to neither the inputs nor the outputs of the network directly), and then there are the output neurons, from which the results are read.</p>
 <figure>
@@ -798,7 +806,6 @@ classifier.normalizeData();</pre>
 <p>Now for the heart of the machine learning process: actually training the model. Here’s the code:</p>
 <pre class="codesplit" data-code-language="javascript">// The <code>train()</code> method initiates the training process.
 classifier.train(finishedTraining);
-
 // A callback function for when the training is complete
 function finishedTraining() {
   console.log("Training complete!");
@@ -833,7 +840,7 @@ classifier.train(options, finishedTraining);</pre>
 </ul>
 <p>You may have noticed that I never did this. For simplicity, I’ve instead used the entire dataset for training. After all, my dataset has only eight records; it’s much too small to divide three sets! With a large dataset, this three-way split would be more appropriate.</p>
 <p>Using such a small dataset risks the model <strong>overfitting</strong> the data, however: the model becomes so tuned to the specific peculiarities of the training data that it’s much less effective when working with new, unseen data. The main reason to use a validation set is to monitor the model during the training process. As training progresses, if the model’s accuracy improves on the training data but deteriorates on the validation data, it’s a strong indicator that overfitting might be occurring. (The testing set is reserved strictly for the final evaluation, one more chance after training is complete to gauge the model’s performance.)</p>
-<p>For more realistic scenarios, ml5.js provides a way to split up the data, as well as automatic features for employing validation data. If you’re inclined to go further, <a href="http://ml5js.org/">you can explore the full set of neural network examples on the ml5.js website</a>.</p>
+<p>For more realistic scenarios, ml5.js provides a way to split up the data, as well as automatic features for employing validation data. If you’re inclined to go further, <a href="https://ml5js.org/">you can explore the full set of neural network examples on the ml5.js website</a>.</p>
 <h3 id="tuning-the-parameters">Tuning the Parameters</h3>
 <p>After the evaluation step, there’s typically an iterative process of adjusting hyperparameters and going through training again to achieve the best performance from the model. While ml5.js offers capabilities for parameter tuning (which you can learn about in the library’s reference), it isn’t really geared toward making low-level, fine-grained adjustments to a model. Using TensorFlow.js directly might be your best bet if you want to explore this step in more detail, since it offers a broader suite of tools and allows for lower-level control over the training process.</p>
 <p>In this case, tuning the parameters isn’t strictly necessary. The graph in the Visor shows a loss all the way down at 0.1, which is plenty accurate for my purposes. I’m happy to move on.</p>