more corrections to serve post

docs/src/blog/a-site-server-with-live-reload.org

@@ -1,26 +1,25 @@
 ---
 title: A site server with live reload
-date: 2024-03-05
+date: 2024-03-06
 layout: post
 lang: en
 tags: [golang, project]
-draft: true
 ---
 #+OPTIONS: toc:nil num:1
 #+LANGUAGE: en
 
-The core of my static site generator is the ~build~ command: take some input files, process them ---render templates, convert other markup formats into HTML, minify--- and write the output for serving to the web. This is where I started for ~jorge~, not only because it was core functionality but because I needed to see the org-mode output as early as possible, to learn if I could expect this project to ultimately replace my Jekyll setup.
+The core of my static site generator is the ~build~ command: take some input files, process them ---render templates, convert other markup formats into HTML, minify--- and output a ready-to-serve website. This is where I started for ~jorge~, not only because it was core functionality but because I needed to see the org-mode output as early as possible, to learn if I could expect this project to ultimately replace my Jekyll setup.
 
-I technically had a working static site generator as soon as the ~build~ command was done, but for it to be minimally useful I needed to be able to preview a site while working on it: a ~serve~ command. It could be as simple as running a local file server of the ~build~ target directory, but ideally it would also watch for changes in the source files and live-reload the browser tabs looking at them.
+I technically had a static site generator as soon as the ~build~ command was working, but for it to be minimally useful I needed to be able to preview a site while working on it: a ~serve~ command. It could be as simple as running a local file server of the ~build~ target directory, but ideally it would also watch for changes in the source files and live-reload the browser tabs looking at them.
 
-I was aiming for more than just the basics here because ~serve~ was the only non-trivial command of the project: the one with the most Go learning potential ---and the most fun. For similar reasons, I wanted to tackle it early on: since it wasn't immediately obvious how I would implement it, it was here where unknown-unknowns and blockers were most likely to come up.
+I was aiming for more than just the basics here because ~serve~ was the only non-trivial command of this project: the one with the most Go learning potential ---and the most fun. For similar reasons, I wanted to tackle it early on: since it wasn't immediately obvious how I would implement it, it was here where unknown-unknowns and blockers were most likely to come up.
 Once ~build~ and ~serve~ were out of the way, I'd be almost done with the project,  only nice-to-have features and UX improvements remaining.
 
-The beauty of the ~serve~ command was that I could start with a naive implementation and iterate towards the ideal one, keeping a usable command every step of the way. Below is a summary of that process.
+The beauty of the ~serve~ command was that I could start with a naive implementation and iterate towards the ideal one, keeping a usable command at every step of the way. Below is a summary of that process.
 
 *** A basic file server
 
-The simplest ~serve~ implementation consisted of building the site once and serving the target directory on a local file server. The standard [[https://pkg.go.dev/net/http#FileServer][~net/http~]] package had what I needed:
+The simplest ~serve~ implementation consisted of building the site once and serving the target directory locally. The standard ~net/http~ package has a [[https://pkg.go.dev/net/http#FileServer][file server]] for that:
 
 #+begin_src go
 import (
@@ -45,7 +44,7 @@ func Serve(config config.Config) error {
 }
 #+end_src
 
-I only had to make a minor change (based on [[https://stackoverflow.com/a/57281956/993769][this]] StackOverflow answer) for the server to allow omitting the ~.html~ suffix from URLs so, for instance, ~target/blog/hello.html~ was served at ~/blog/hello~:
+I only had to make a minor change to the code above (based on [[https://stackoverflow.com/a/57281956/993769][this]] StackOverflow answer), to omit the ~.html~ extension from URLs such that, for instance, ~target/blog/hello.html~ would be served at ~/blog/hello~:
 
 #+begin_src go
 type HTMLFileSystem struct {
@@ -65,7 +64,7 @@ func (htmlFS HTMLFileSystem) Open(name string) (http.File, error) {
 }
 #+end_src
 
-The server setup thus changed to:
+This ~HTMLFileSystem~ wrapped around the standard ~http.Dir~ one I was handing to the file server:
 
 #+begin_src diff
 -	fs := http.FileServer(http.Dir(config.TargetDir))
@@ -77,9 +76,9 @@ The server setup thus changed to:
 #+end_src
 
 *** Watching for changes
-As a next step, I needed the command to watch the project source directory and trigger new builds whenever a file changed. I found the [[https://github.com/fsnotify/fsnotify][fsnotify]] library for this exact purpose; the fact that both Hugo and gojekyll listed as a dependency suggested that it was the reasonable choice for the job.
+As a next step, I needed the command to watch the project source directory and trigger new builds whenever a file changed. I found the [[https://github.com/fsnotify/fsnotify][fsnotify]] library for this exact purpose; the fact that both Hugo and gojekyll listed as a dependency suggested that it was a reasonable choice for the job.
 
-Following [[https://github.com/fsnotify/fsnotify/blob/c94b93b0602779989a9af8c023505e99055c8fe5/README.md#usage][an example]] from the fsnotify documentation, I created a watcher and a goroutine that triggered a ~site.Build~ call every time a file change event was received:
+Following [[https://github.com/fsnotify/fsnotify/blob/c94b93b0602779989a9af8c023505e99055c8fe5/README.md#usage][an example]] from the fsnotify documentation, I created a watcher and a goroutine that triggered a ~site.Build~ call every time a file-change event was received:
 
 #+begin_src go
 func runWatcher(config *config.Config) {
@@ -117,9 +116,9 @@ func watchProjectFiles(watcher *fsnotify.Watcher, config *config.Config) {
 *** Build optimizations
 At this point I had a useful file server, always responding with the most recent version of the site. But the responsiveness of the ~serve~ command wasn't ideal: it processed the entire website for every small edit I made on a source file. I wanted to attempt some performance improvements here, but without introducing much complexity: rather than supporting incremental or conditional builds ---which would have required tracking state and dependencies between files---, I wanted to keep building the entire site on every change, only faster.
 
-The first cheap optimization was obvious from looking at the command output: most of the work was copying static assets (e.g. images, static CSS files, etc.). So I changed the ~site.Build~ implementation to optionally create links instead of copying files.
+The first cheap optimization was obvious from looking at the command output: most of the work was copying static assets (e.g. images, static CSS files, etc.). So I changed the ~site.Build~ implementation to optionally create links instead of copying the files over to the target.
 
-The next thing I wanted to try was to process source files concurrently. The logic for creating target directories and rendering files was handled by an internal ~site~ method:
+The next thing I wanted to try was to process source files concurrently. The bulk of the work was done by an internal ~site~ method:
 
 #+begin_src go
 type site struct {
@@ -147,7 +146,7 @@ func (site *site) build() error {
 }
 #+end_src
 
-This ~site.build~ method walks the source file tree, recreating it at the target. For non-directory files, it calls another method, ~site.buildFile~, to do the actual processing (rendering templates, converting markdown and org-mode syntax to HTML, and writing the results to the target files). I wanted ~site.buildFile~ to run in a worker pool; I found the facilities I needed in a couple of [[https://gobyexample.com/][Go by Example]] entries:
+This ~site.build~ method walks the source file tree, recreating it at the target. For non-directory files, it calls another method, ~site.buildFile~, to do the actual processing (rendering templates, converting markdown and org-mode syntax to HTML, and writing the results to the target files). I wanted multiple ~site.buildFile~ calls to run in parallel; I found the facilities I needed ([[https://gobyexample.com/worker-pools][worker pools]] and [[https://gobyexample.com/waitgroups][wait groups]]) in a couple of /Go by Example/ entries:
 
 #+begin_src go
 // Runs a pool of workers to build files.
@@ -170,9 +169,9 @@ func spawnBuildWorkers(site *site) (*sync.WaitGroup, chan string) {
 }
 #+end_src
 
-The function above creates a buffered channel to send source file paths, and a worker pool that reads from it. Each worker registers itself on a ~WaitGroup~ that can be used by callers to block until all workers finish their work.
+The function above creates a buffered channel to send source file paths and a worker pool that reads from it. Each worker registers itself on a ~WaitGroup~ that can be used by callers to block until all work is done.
 
-Then I just needed to adapt the ~build~ function to spawn the workers and send them  file paths through the channel, instead of processing them inline:
+Now I just needed to adapt the ~build~ function to spawn the workers and send them  paths through the channel, instead of processing the files inline:
 
 #+begin_src diff
 func (site *site) build() error {
@@ -203,15 +202,17 @@ func (site *site) build() error {
 }
 #+end_src
 
-the ~close(files)~ call informs the workers that no more work will be sent, and ~wg.Wait()~ blocks execution until all pending work is done.
+the ~close(files)~ call informs the workers that no more work will be sent, and ~wg.Wait()~ blocks until all of them finish executing.
 
-I was very satisfied to see a sequential piece of code turned into a concurrent one with minimal structural changes, without affecting callers of the function that contained it. In other languages, a similar operation would have required me to add ~async~ and ~await~ statements all over the place.
+I was very satisfied to see a sequential piece of code turned into a concurrent one with minimal structural changes, without affecting callers of the function that contained it. In other languages, a similar operation would have required me to add ~async~ and ~await~ statements all over the place[fn:2].
+
+This couple of optimizations resulted in a good enough user experience, so I didn't need to attempt more complex ones.
 
 *** Live reload
 
-Without having looked into their code, I presumed that the live-reloading tools I had used in the past (~jekyll serve~, [[https://github.com/shime/livedown/][livedown]]) worked by running WebSocket servers and injecting some JavaScript in the HTML files they served. I wanted to see if I could get away with implementing live reloading for ~jorge serve~ with [[https://en.wikipedia.org/wiki/Server-sent_events][Server-sent events]] instead, a slightly simpler alternative to WebSockets that didn't require a dedicated server.
+Without having looked into their code, I presumed that the live-reloading tools I had used in the past (~jekyll serve~, [[https://github.com/shime/livedown/][livedown]]) worked by running WebSocket servers and injecting some JavaScript in the HTML files they served. I wanted to see if I could get away with implementing live reloading for ~jorge serve~ with [[https://en.wikipedia.org/wiki/Server-sent_events][Server-sent events]], a slightly simpler alternative to WebSockets that didn't require a dedicated server.
 
-Some googling revealed the boilerplate I needed to send events from my Go http server:
+Some googling [[https://medium.com/@rian.eka.cahya/server-sent-event-sse-with-go-10592d9c2aa1][yielded]] the boilerplate code to send events from my Go http server:
 
 #+begin_src go
 func ServerEventsHandler (res http.ResponseWriter, req *http.Request) {
@@ -241,7 +242,7 @@ func ServerEventsHandler (res http.ResponseWriter, req *http.Request) {
 #+end_src
 
 
-In this test setup, clients connected to the ~/_events/~ endpoint would receive an event with the ~"rebuild"~ message every 5 seconds. After some trial-and-error, I arrived to the corresponding JavaScript:
+In this test setup, clients connected to the ~/_events/~ endpoint would receive a  ~"rebuild"~ message every 5 seconds. After a few attempts to get error-handling right, I arrived to the corresponding JavaScript:
 
 #+begin_src html
 <script type="text/javascript">
@@ -275,9 +276,9 @@ newSSE();
 </script>
   #+end_src
 
-Clients would establish an [[https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events][EventSource]] connection through the ~/_events/~ endpoint, and reload the window whenever a server-sent event arrived. I updated  ~site.buildFile~ to inject this ~script~ tag in the header of every HTML file written to the target directory.
+Clients would establish an [[https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events][EventSource]] connection through the ~/_events/~ endpoint and reload the window whenever a server-sent event arrived. I updated  ~site.buildFile~ to inject this ~script~ tag in the header of every HTML file written to the target directory.
 
-With the code above I had everything in place to send and receive events, and reload the browser accordingly. I just needed to update the http handler to only send events in response to site rebuilds triggered by source file changes. I couldn't just use a channel to connect the handler with the fsnotify watcher, since there could be multiple clients connected at a time (multiple tabs browsing the site) and each needed to receive the reload event; a single-channel message would be consumed by a single client. I needed some method to broadcast rebuild events; I introduced an ~EventBroker~[fn:1] struct for that purpose, with this interface:
+With the code above I had everything in place to send and receive events, and reload the browser accordingly. I just needed to update the http handler to only send those events in response to site rebuilds triggered by source file changes. I couldn't just use a channel to connect the handler with the fsnotify watcher, since there could be multiple clients connected at a time (multiple tabs browsing the site) and each needed to receive the reload event ---a single-channel message would be consumed by a single client. I needed some method to broadcast rebuild events; I introduced an ~EventBroker~[fn:1] struct for this purpose:
 
 #+begin_src go
 // The event broker mediates between the file watcher
@@ -302,7 +303,7 @@ func (broker *EventBroker) publish(event string)
 
 See [[https://github.com/facundoolano/jorge/blob/567db560f511b11492b85cf4f72b51599e8e3a3d/commands/serve.go#L175-L238][here]] for the full ~EventBroker~ implementation.
 
-The http handler now needed to subscribe every connected client to receive rebuild events through the broker:
+The http handler now needed to subscribe every connected client to the broker:
 
 #+begin_src diff
 -func ServerEventsHandler (res http.ResponseWriter, req *http.Request) {
@@ -335,10 +336,9 @@ The watcher, in turn, had to publish an event after every rebuild:
 
 #+begin_src diff
 -func runWatcher(config *config.Config) {
-+func runWatcher(config *config.Config) *EventBroker {
++func runWatcher(config *config.Config, broker *EventBroker) {
 	watcher, _ := fsnotify.NewWatcher()
 	defer watchProjectFiles(watcher, config)
-+	broker := newEventBroker()
 
 	go func() {
 		for event := range watcher.Events {
@@ -351,10 +351,31 @@ The watcher, in turn, had to publish an event after every rebuild:
 +			broker.publish("rebuild")
 		}
 	}()
-+	return broker
 }
 #+end_src
 
+The command function connected the pieces:
+
+#+begin_src go
+func Serve(config config.Config) error {
+	// load and build the project
+	if err := site.Build(config); err != nil {
+		return err
+	}
+
+	broker := newEventBroker()
+	runWatcher(config, broker)
+
+	// mount the target dir on a local file server
+	fs := http.FileServer(http.Dir(config.TargetDir))
+	http.Handle("/", fs)
+	// handle client requests to listen to server-sent events
+	http.Handle("/_events/", makeServerEventsHandler(broker))
+
+	fmt.Println("server listening at http://localhost:4001/")
+	return http.ListenAndServe(":4001", nil)
+}
+#+end_src
 
 *** Handling event bursts
 
@@ -394,8 +415,10 @@ func runWatcher(config *config.Config) *EventBroker {
 The initial build is triggered immediately on setup (~time.AfterFunc(0, ...)~) but subsequent rebuilds are delayed 100 milliseconds (~rebuildAfter.Reset(100 * time.Millisecond)~), canceling previous pending ones.
 
 -----
-That's approximately the current implementation of the ~jorge serve~ command, which I used to write this post. You can see the full code [[https://github.com/facundoolano/jorge/blob/28b2d32406c7f4e4f6c3084d521f0123435637c8/commands/serve.go][here]].
+That's (approximately) the current implementation of the ~jorge serve~ command, which I used to write this post. You can see the full code [[https://github.com/facundoolano/jorge/blob/28b2d32406c7f4e4f6c3084d521f0123435637c8/commands/serve.go][here]].
 
 ** Notes
 
 [fn:1] I'm not sure if "broker" is a proper name in this context, since there's a single event type and it's sent to all subscribers. "Broadcaster" is probably more accurate, but it also sounds worse.
+
+[fn:2] Related discussion: [[https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/][What Color is Your Function?]]