diff --git a/.gitignore b/.gitignore index f938ff3..8ed22e9 100755 --- a/.gitignore +++ b/.gitignore @@ -8,6 +8,8 @@ src/toi.rb src/misc/a.out src/misc/cairo2.c src/misc/cairo2.pdf +src/misc/color_square.c +src/misc/color_square.png src/tfv/a.out src/tfe/a.out src/tfe/hello.txt diff --git a/Rakefile b/Rakefile index 3e6ef30..12c1f9a 100644 --- a/Rakefile +++ b/Rakefile @@ -35,7 +35,7 @@ task md: ["Readme.md"] file "Readme.md" => mdfilenames do buf = [ "# Gtk4 Tutorial for beginners\n", "\n" ] - src2md "src/abstract.src.md", "gfm/abstract.md", -1 + src2md "src/abstract.src.md", "gfm/abstract.md" File.open("gfm/abstract.md") do |file| file.readlines.each do |line| buf << line @@ -53,7 +53,7 @@ end 0.upto(srcfiles.size - 1) do |i| file "gfm/#{srcfiles[i].to_md}" => (srcfiles[i].c_files << srcfiles[i].path) do - src2md srcfiles[i].path, "gfm/#{srcfiles[i].to_md}", -1 + src2md srcfiles[i].path, "gfm/#{srcfiles[i].to_md}" if srcfiles.size == 1 nav = "Up: [Readme.md](../Readme.md)\n" elsif i == 0 @@ -74,7 +74,7 @@ task html: ["html/index.html"] file "html/index.html" => htmlfilenames+["html/tfetextview_doc.html"] do buf = [ "# Gtk4 Tutorial for beginners\n", "\n" ] - src2md "src/abstract.src.md", "html/abstract.md", -1 + src2md "src/abstract.src.md", "html/abstract.md" File.open("html/abstract.md") do |file| file.readlines.each do |line| buf << line @@ -105,7 +105,7 @@ end html_md = "html/#{srcfiles[i].to_md}" html_html = "html/#{srcfiles[i].to_html}" file html_html => (srcfiles[i].c_files << srcfiles[i].path) do - src2md srcfiles[i].path, html_md, -1 + src2md srcfiles[i].path, html_md if srcfiles.size == 1 nav = "Up: [index.html](index.html)\n" elsif i == 0 @@ -129,8 +129,8 @@ end end task pdf: "latex" do - sh "cd latex; pdflatex main.tex" - sh "cd latex; pdflatex main.tex" + sh "cd latex; lualatex main.tex" + sh "cd latex; lualatex main.tex" sh "mv latex/main.pdf latex/gtk4_tutorial.pdf" end @@ -141,19 +141,19 @@ file "latex/main.tex" => ["latex/abstract.tex", "latex/tfetextview_doc.tex"] + t end file "latex/abstract.tex" => "src/abstract.src.md" do - src2md "src/abstract.src.md", "latex/abstract.md", 86 - sh "pandoc -o latex/abstract.tex latex/abstract.md" + src2md "src/abstract.src.md", "latex/abstract.md" + sh "pandoc --listings -o latex/abstract.tex latex/abstract.md" File.delete("latex/abstract.md") end file "latex/tfetextview_doc.tex" => "src/tfetextview/tfetextview_doc.md" do - sh "pandoc -o latex/tfetextview_doc.tex src/tfetextview/tfetextview_doc.md" + sh "pandoc --listings -o latex/tfetextview_doc.tex src/tfetextview/tfetextview_doc.md" end 0.upto(srcfiles.size - 1) do |i| file "latex/#{srcfiles[i].to_tex}" => (srcfiles[i].c_files << srcfiles[i].path) do - src2md srcfiles[i].path, "latex/#{srcfiles[i].to_md}", 86 - sh "pandoc -o latex/#{srcfiles[i].to_tex} latex/#{srcfiles[i].to_md}" + src2md srcfiles[i].path, "latex/#{srcfiles[i].to_md}" + sh "pandoc --listings -o latex/#{srcfiles[i].to_tex} latex/#{srcfiles[i].to_md}" File.delete("latex/#{srcfiles[i].to_md}") end end diff --git a/Readme.md b/Readme.md index fc56811..6c823f5 100644 --- a/Readme.md +++ b/Readme.md @@ -36,3 +36,4 @@ You can read it without download. 1. [Template XML](gfm/sec20.md) 1. [GtkDrawingArea and Cairo](gfm/sec21.md) 1. [Combine GtkDrawingArea and TfeTextView](gfm/sec22.md) +1. [Tiny turtle graphics interpreter](gfm/sec23.md) diff --git a/doc/Readme_for_developers.md b/doc/Readme_for_developers.md index 8e8bdb8..bb4e06a 100644 --- a/doc/Readme_for_developers.md +++ b/doc/Readme_for_developers.md @@ -70,17 +70,33 @@ lib_src2md.rb @@@ ~~~ -The inserted text becomes fence code block. +The inserted text is converted to fence code block. +If the target markdown is GFM, then an info string follows the beginning fence. +The following example shows the command includes a C source file "sample.c". + $ cat src/sample.c + int + main (int argc, char **argv) { + ... ... + } + $cat src/sample.src.md + ... ... + @@@include -N + sample.c + @@@ + ... ... + $ ruby src2md.rb src/sample.src.md gfm/sample.md + $ cat gfm/sample.md + ... ... ~~~C int main (int argc, char **argv) { ... ... } ~~~ + ... ... -The string ("C" in the example above) follows the first fence is called info string. -Info string is usually a language like C, ruby, xml and so on. +Info strings are usually language like C, ruby, xml and so on. This string is decided with the filename extension. A line number is inserted at the top of each line in the code block. @@ -91,7 +107,62 @@ Options: - "-n": Inserts a line number at the top of each line (default). - "-N": No line number is inserted. -This command have two advantages. + $cat src/sample.src.md + ... ... + @@@include + sample.c + @@@ + ... ... + $ ruby src2md.rb src/sample.src.md gfm/sample.md + $ cat gfm/sample.md + ... ... + ~~~C + 1 int + 2 main (int argc, char **argv) { + ... ... + 14 } + ~~~ + ... ... + +If the target markdown is an intermediate file to html, then another type of info string follows the beginning fence. +If @@@include command doesn't have -N option, then the generated markdown is: + + ~~~{.C .numberLines} + int + main (int argc, char **argv) { + ... ... + } + ~~~ +The info string `.C` specifies C language. +The info string `.numberLines` is a class of the pandoc markdown. +If the class is given, pandoc generates CSS to insert line numbers to the source code in the html file. +That's why the fence code block in the markdown doesn't have line numbers, which is different from gfm markdown. +If "-N" option is given, then the info string is `{.C}` only. + +If the target markdown is an intermediate file to latex, then an info string follows the beginning fence. + + ~~~{.C .numberLines} + int + main (int argc, char **argv) { + ... ... + } + ~~~ + +Rake uses pandoc with --listings option when it converts markdown to latex. +The generated latex file uses listings package to list source files instead of verbatim environment. +The markdwon above is converted to the following latex source file. + + \begin{lstlisting}[language=C, numbers=left] + int + main (int argc, char **argv) { + ... ... + } + \end{lstlisting} + +Listing package can color or emphasize keywords, strings, comments and directives. +But it doesn't analyze the syntax or token of the language, so the kind of emphasis target is limited. + +@@@include command have two advantages. 1. Less typing. 2. You don't need to modify your src.md file, even if the C source file is modified. @@ -167,6 +238,46 @@ It is based on the state diagram below. ![state diagram](../image/state_diagram.png) +## mktbl.rb script + +The script "mktbl.rb" is in `src` directory. +This script makes a table easy to read. +For example, a text file "sample.md" has a table like this: + +~~~ +Price list + +@@@table +|item|price| +|:-:|:-:| +|mouse|$10| +|PC|$500| +@@@ +~~~ + +Run the script. + +~~~ +$ ruby mktbl.rb sample.md +~~~ + +Then, the file is changed to: + +~~~ +Price list + +|item |price| +|:---:|:---:| +|mouse| $10 | +| PC |$500 | +~~~ + +The script makes a backup file "sample.md.bak". + +The task of the script seems easy, but the program is not so simple. +The script "mktbl.rb" uses a library script "lib/lib\_mktbl.rb" +This script is independent from "src2md.rb". + ## Directory structure There are six directories under `gtk4_tutorial` directory. diff --git a/gfm/sec12.md b/gfm/sec12.md index 1fbb348..339fcc8 100644 --- a/gfm/sec12.md +++ b/gfm/sec12.md @@ -4,7 +4,7 @@ Up: [Readme.md](../Readme.md), Prev: [Section 11](sec11.md), Next: [Section 13] In this section I will explain functions in TfeTextView object. -### tfe.h and tfetextview.h +## tfe.h and tfetextview.h `tfe.h` is a top header file and it includes `gtk.h` and all the header files. C source files `tfeapplication.c` and `tfenotebook.c` include `tfe.h` at the beginning. diff --git a/gfm/sec19.md b/gfm/sec19.md index 1fc2ce9..bdfaa7a 100644 --- a/gfm/sec19.md +++ b/gfm/sec19.md @@ -1111,7 +1111,7 @@ After compilation, you can test your application like this: $ GSETTINGS_SCHEMA_DIR=_build:$GSETTINGS_SCHEMA_DIR _build/tfe ~~~ -### GSettings object and g_setting_bind +### GSettings object and g\_settings\_bind Write gsettings related codes to `tfeapplication.c'. diff --git a/gfm/sec23.md b/gfm/sec23.md new file mode 100644 index 0000000..8bcaa4d --- /dev/null +++ b/gfm/sec23.md @@ -0,0 +1,380 @@ +Up: [Readme.md](../Readme.md), Prev: [Section 22](sec22.md) + +# Tiny turtle graphics interpreter + +A program `turtle` is an example with the combination of TfeTextView and GtkDrawingArea objects. +It is a very small interpreter but it provides a way to draw fractal curves. +The following diagram is a Koch curve, which is a famous example of fractal curves. + +![Kocho curve](../src/turtle/image/turtle_koch.png) + +This program uses flex and bison. +Flex is a lexical analyzer. +Bison is a parser generator. +These two programs are similar to lex and yacc which are proprietary software developed in Bell Laboratry. +However, flex and bison are open source software. +I will write about how to use those software, but they are not topics about gtk. +So, readers can skip that part of this sections. + +## How to use turtle + +The documentation of turtle is [here](../src/turtle/turtle_doc.md). +I'll show you a simple example. + +~~~ +fc (1,0,0) # Foreground color is red, rgb = (1,0,0). +pd # Pen down. +fd 100 # Go forward by 100 pixels. +tr 90 # Turn right by 90 degrees. +fd 100 +tr 90 +fd 100 +tr 90 +fd 100 +tr 90 +~~~ + +1. Compile and install `turtle` (See the documentation above). +Then, run `turtle`. +2. Type the program above in the editor (left part of the window). +3. Click on the `Run` button, then a red square appears on the right part of the window. +The side of the square is 100 pixels long. + +In the same way, you can draw other curves. +The documentation above shows some fractal curves such as tree, snow and square-koch. +The source code in turtle language is located at [src/turtle/example](../src/turtle/example) directory. +You can read these files by clicking on the `Open` button. + +## Combination of TfeTextView and GtkDrawingArea objects + +Turtle uses TfeTextView and GtkDrawingArea. +It is similar to `color` program in the previous section. + +The body of the interpreter is written with flex and bison. +The codes are not thread safe. +So the handler of "clicked" signal of the `Run` button prevents from reentering. + +~~~C + 1 void + 2 run_cb (GtkWidget *btnr) { + 3 GtkTextBuffer *tb = gtk_text_view_get_buffer (GTK_TEXT_VIEW (tv)); + 4 GtkTextIter start_iter; + 5 GtkTextIter end_iter; + 6 char *contents; + 7 int stat; + 8 static gboolean busy = FALSE; + 9 +10 /* yyparse() and run() are NOT thread safe. */ +11 /* The variable busy avoids reentry. */ +12 if (busy) +13 return; +14 busy = TRUE; +15 gtk_text_buffer_get_bounds (tb, &start_iter, &end_iter); +16 contents = gtk_text_buffer_get_text (tb, &start_iter, &end_iter, FALSE); +17 if (surface) { +18 init_flex (contents); +19 init_parse (); +20 stat = yyparse (); +21 #ifdef debug +22 g_print ("yyparse returned %d.\n", stat); +23 #endif +24 if (stat == 0) /* No error */ { +25 run (); +26 } +27 finalize_flex (); +28 } +29 gtk_widget_queue_draw (GTK_WIDGET (da)); +30 busy = FALSE; +31 } +32 +33 static void +34 resize_cb (GtkDrawingArea *drawing_area, int width, int height, gpointer user_data) { +35 if (surface) +36 cairo_surface_destroy (surface); +37 surface = cairo_image_surface_create (CAIRO_FORMAT_ARGB32, width, height); +38 } +~~~ + +- 8-13: The static value `busy` holds a status of the interpreter. +If it is `TRUE`, the interpreter is running and it is not possible to call the interpreter again because it's not a re-entrant program. +If it is `FALSE`, it is safe to call the interpreter. +- 14: Now it is about to call the interpreter so let `busy` variable to be TRUE. +- 15-16: Gets the contents of GtkTextBuffer. +- 17: The varable `surface` is a static variable. +It points to a `cairo_surface_t` object. +It is generated when the GtkDrawingArea object is realized and the each time it is resized. +Therefore, `surface` isn't NULL usually. +But if it is NULL, the interpreter won't be called. +- 18-19: Initializes lexcal analyzer, then parser. +- 20: Calls parser. +Parser analyze the program codes syntactically and generate a tree structured data. +- 24-26: If the parser succesfully parsed, it calls `run` which is the interpreter. +`Run` executes the tree-structured program. +- 27: finalize the lexical analyzer. +- 29: Add the drawing area object to the queue to draw. +- 30: The interpreter program has finished so `busy` is now FALSE. +- 33-38: A handler of "resized" signal. +It generates or regenerates a surface object. + +Other part of `turtleapplication.c` is almost same as the codes of `colorapplication.c` in the previous section. +The codes of `turtleapplication.c` is in the [turtle directory](../src/turtle). + +## What does the interpreter do? + +Suppose that the turtle runs with the following program. +The following is the description how turtle recognizes the program and does the work. + +~~~ +distance = 100 +fd distance*2 +~~~ + +- Generally, a program consists of tokens. +Tokens are "distance", "=", "100", "fd", "*" and "2" in the above example.. +- The interpreter `turtle` calls `yylex` to read a token in the source file. +The `yylex` returns a code which is called "token kind" and sets a global variable `yylval` with a value, which is called a semantic value. +The type of `yylval` is union and `yylval.ID` is string and `yylval.NUM` is double. +There are seven tokens in the program so `yylex` is called seven times. + +| |token kind|yylval.ID|yylval.NUM| +|:-:|:--------:|:-------:|:--------:| +| 1 | ID |distance | | +| 2 | = | | | +| 3 | NUM | | 100 | +| 4 | FD | | | +| 5 | ID |distance | | +| 6 | * | | | +| 7 | NUM | | 2 | + +`yylex` returns a token kind every time, but it doesn't set `yylval.ID` or `yylval.NUM` every time. +It is because keywords (`FD`) and symbols (`=` and `*`) don't have any semantic values. +The function `yylex` is called lexical analyzer or scanner. +- `turtle` makes a tree structured data. +This part of `turtle` is called parser. + +![turtle parser tree](../src/turtle_parser_tree_png) + +- `turtle` analyzes the tree and executes it. +This part of `turtle` is called runtime routine. +The tree consists of line segments and rectangles between line segments. +The rectangles are called nodes. +For example, N\_PROGRAM, N\_ASSIGN, N\_FD and N\_MUL are nodes. + 1. Goes down from N\_PROGRAM to N\_ASSIGN. + 2. At N_ASSIGN node, `turtle` checks if the first child is ID, that means the left side is a variable. +If it's ID, then `turtle` looks for the variable in the variable table. +If it doesn't exist, it registers the ID (`distance`) to the table. +Then go back to the N\_ASSIGN node. + 3. `turtle` calculates the second child. +In this case its a number 100. +Saves 100 to the variable table at the `distance` record. + 4. `turtle` goes back to N\_PROGRAM then go to the next node N\_FD. +It has only one child. +Goes down to the child N\_MUL. + 5. The first child is ID (distance). +Looks for the varable distance in the variable table and gets the value 100. +The second child is a number 2. +Multiplies 100 by 2 and gets 200. +Then `turtle` goes back to N_FD. + 6. Now `turtle` knows the distance is 200. +It moves the cursor forward by 200. + 8. There are no node follows. +Runtime routine returns to the main routine. +-`turtle` draws GtkDrawingArea then stops. + +Actually most programs are more complicated than the example above. +So, `turtle` does much more work to interpret programs. +However, basically it works by the same way above. +Interpritation consists of three parts. + +- Lexical analysis +- Syntax Parsing and tree generation +- Execution by the tree + +## Compilation flow + +The source files are: + +- flex source file ~> turtle.lex +- bison source file => turtle.y +- C header file => turtle.h, turtle_lex.h +- C source file => turtleapplication.c +- other files => turtle.ui, turtle.gresources.xml, meson.build + +The compilation process is a bit complicated. + +1. glib-compile-resources compiles `turtle.ui` to `resources.c` according to `turtle.gresource.xml`. +It also generates `resources.h`. +2. bison compiles `turtle.y` to `turtle\_parser.c` and generates `turtle\_parser.h` +3. flex compiles `turtle.lex` to `turtle\_lex.c`. +4. gcc compiles `application.c`, `resources.c`, `turtle\_parser.c` and `turtle\_lex.c` with `turtle.h`, `resources.h` and `turtle\_parser.h`. +It generates an executable file `turtle`. + +![compile process](../image/turtle_compile_process.png) + +Meson controls the process and the instruction is described in meson.build. + +~~~meson + 1 project('turtle', 'c') + 2 + 3 compiler = meson.get_compiler('c') + 4 mathdep = compiler.find_library('m', required : true) + 5 + 6 gtkdep = dependency('gtk4') + 7 + 8 gnome=import('gnome') + 9 resources = gnome.compile_resources('resources','turtle.gresource.xml') +10 +11 flex = find_program('flex') +12 bison = find_program('bison') +13 turtleparser = custom_target('turtleparser', input: 'turtle.y', output: ['turtle_parser.c', 'turtle_parser.h'], command: [bison, '-d', '-o', 'turtle_parser.c', '@INPUT@']) +14 turtlelexer = custom_target('turtlelexer', input: 'turtle.lex', output: 'turtle_lex.c', command: [flex, '-o', '@OUTPUT@', '@INPUT@']) +15 +16 sourcefiles=files('turtleapplication.c', '../tfetextview/tfetextview.c') +17 +18 executable('turtle', sourcefiles, resources, turtleparser, turtlelexer, turtleparser[1], dependencies: [mathdep, gtkdep], export_dynamic: true, install: true) +19 +~~~ + +- 3: Gets C compiler. +It is usually `gcc` in linux. +- 4: Gets math library. +This program uses trigonometry functions. +They are defined in the math library, but the library is optional. +So, it is necessary to include the library by `#include ` and also link the library with the linker. +- 6: Gets gtk4 library. +- 8: Gets gnome module. +Module is a system provided by meson. +See [Meson build system website](https://mesonbuild.com/Gnome-module.html#gnome-module) for further information. +- 9: Compiles ui file to C source file according to the XML file `turtle.gresource.xml`. +- 11: Gets flex. +- 12: Gets bison. +- 13: Compiles `turtle.y` to `turtle_parser.c` and `turtle_parser.h` by bison. +The function `custom_target`creates a custom top level target. +See [Meson build system website](https://mesonbuild.com/Reference-manual.html#custom_target) for further information. +- 14: Compiles `turtle.lex` to `turtle_lex.c` by flex. +- 16: Specifies C surce files. +- 18: Compiles C source files including generated files by glib^compile-resources, bison and flex. +The argument `turtleparser[1]` refers to `tirtle_parser.h` which is the secound output in the line 13. + +## Turtle.lex + +This subsection and the following subsection are description about flex and bison. +If you want to focus on gtk4, you can skip these subsections. + +### What does flex do? + +Flex creates lexical analizer from flex source file. +Flex source file is a text file and its syntactic rule will be explained later. +Generated lexical analyzer is a C source file. +It is also called scanner. +It reads a text file, which is a source file of a program language, and gets variable names, numbers and symbols. +Suppose here is a text file. + +~~~ +fc (1,0,0) # Foreground color is red, rgb = (1,0,0). +pd # Pen down. +distance = 100 +angle = 90 +fd distance # Go forward by distance (100) pixels. +tr angle # Turn right by angle (90) degrees. +~~~ + +The content of the textfile is separated into `fc`, `(`, `1`, `,` and so on. +The words `fc`, `pd`, `distance` and `100` are called token. +The characters `(`, `<` and `=` are called symbol. +( Sometimes those symbols called token, too.) + +A scanner has `yylex` function and `yytext` variable. +The function returns a type of token and set `yytext` to contain the name of the token. +Each time `yylexc` is called, it returns the followings. + + +Turtle.lex isn't so big program. + +~~~lex + 1 %top{ + 2 #include + 3 #include "turtle.h" + 4 + 5 static int nline = 1; + 6 static int ncolumn = 1; + 7 static void get_location (char *text); + 8 + 9 /* Dinamically allocated memories are added to the single list. They will be freed in the finalize function. */ +10 extern GSList *list; +11 } +12 +13 %option noyywrap +14 +15 DIGIT [0-9] +16 REAL_NUMBER (0|[1-9][0-9]*)(\.[0-9]+)? +17 IDENTIFIER [a-zA-Z][a-zA-Z0-9]* +18 %% +19 /* rules */ +20 #.*\n nline++; ncolumn = 1; /* comment */ +21 #.* ; /* comment in the last line */ +22 [ ] ncolumn++; +23 \t ncolumn += 8; /* assume that tab is 8 spaces. */ +24 \n nline++; ncolumn = 1; +25 /* reserved keywords */ +26 pu get_location (yytext); return PU; /* pen up */ +27 pd get_location (yytext); return PD; /* pen down */ +28 pw get_location (yytext); return PW; /* pen width = line width */ +29 fd get_location (yytext); return FD; /* forward */ +30 tr get_location (yytext); return TR; /* turn right */ +31 bc get_location (yytext); return BC; /* background color */ +32 fc get_location (yytext); return FC; /* foreground color */ +33 dp get_location (yytext); return DP; /* define procedure */ +34 if get_location (yytext); return IF; /* if statement */ +35 rt get_location (yytext); return RT; /* return statement */ +36 rs get_location (yytext); return RS; /* reset the status */ +37 /* constant */ +38 {REAL_NUMBER} get_location (yytext); yylval.NUM = atof (yytext); return NUM; +39 /* identifier */ +40 {IDENTIFIER} { get_location (yytext); yylval.ID = g_strdup(yytext); +41 list = g_slist_prepend (list, yylval.ID); +42 return ID; +43 } +44 "=" get_location (yytext); return '='; +45 ">" get_location (yytext); return '>'; +46 "<" get_location (yytext); return '<'; +47 "+" get_location (yytext); return '+'; +48 "-" get_location (yytext); return '-'; +49 "*" get_location (yytext); return '*'; +50 "/" get_location (yytext); return '/'; +51 "(" get_location (yytext); return '('; +52 ")" get_location (yytext); return ')'; +53 "{" get_location (yytext); return '{'; +54 "}" get_location (yytext); return '}'; +55 "," get_location (yytext); return ','; +56 . ncolumn++; return YYUNDEF; +57 %% +58 +59 static void +60 get_location (char *text) { +61 yylloc.first_line = yylloc.last_line = nline; +62 yylloc.first_column = ncolumn; +63 yylloc.last_column = (ncolumn += strlen(text)) - 1; +64 } +65 +66 static YY_BUFFER_STATE state; +67 +68 void +69 init_flex (const char *text) { +70 state = yy_scan_string (text); +71 } +72 +73 void +74 finalize_flex (void) { +75 yy_delete_buffer (state); +76 } +77 +~~~ + + + + +## Turtle.y source file compiled by bison + + +Up: [Readme.md](../Readme.md), Prev: [Section 22](sec22.md) diff --git a/image/color_square.png b/image/color_square.png new file mode 100644 index 0000000..0a87c82 Binary files /dev/null and b/image/color_square.png differ diff --git a/image/color_square.xcf b/image/color_square.xcf new file mode 100644 index 0000000..7098273 Binary files /dev/null and b/image/color_square.xcf differ diff --git a/image/turtle_compile_process.png b/image/turtle_compile_process.png new file mode 100644 index 0000000..015ead7 Binary files /dev/null and b/image/turtle_compile_process.png differ diff --git a/image/turtle_compile_process.xcf b/image/turtle_compile_process.xcf new file mode 100644 index 0000000..fd6f538 Binary files /dev/null and b/image/turtle_compile_process.xcf differ diff --git a/image/turtle_parser_tree.png b/image/turtle_parser_tree.png new file mode 100644 index 0000000..503e119 Binary files /dev/null and b/image/turtle_parser_tree.png differ diff --git a/image/turtle_parser_tree.xcf b/image/turtle_parser_tree.xcf new file mode 100644 index 0000000..3265879 Binary files /dev/null and b/image/turtle_parser_tree.xcf differ diff --git a/lib/lib_add_head_tail_html.rb b/lib/lib_add_head_tail_html.rb index b4c0eb3..d2a27ac 100644 --- a/lib/lib_add_head_tail_html.rb +++ b/lib/lib_add_head_tail_html.rb @@ -3,27 +3,57 @@ def add_head_tail_html html_file -head=<<'EOS' - - - - - - +sample_md = <<'EOS' +--- +title: 'Gtk4 tutorial for beginners' +--- - +# sample header - gtk4 tutorial - - +Main contents begin here. + +~~~{.C .numberLines} +int main(int argc, char **argv) { +} +~~~ + +|English|Japanese| +|:-----:|:------:| +|potato|jagaimo| +|carrot|ninjin| +|onion|tamanegi| EOS + File.write "sample.md", sample_md + if (! system("pandoc", "-s", "-o", "sample.html", "sample.md")) + raise ("add_head_tail_html: pandoc retuns error status #{$?}.\n") + end + sample_html = File.read("sample.html") + sample_html.gsub!(//,'') + + head = [] + sample_html.each_line do |l| + if l =~ /<\/head>/ + break + elsif l != "\n" + head << l + end + end + i = head.find_index { |line| line =~ /<\/style>/} + raise "No tag in sample.html which is generated by pandoc." unless i.instance_of?(Integer) + head.insert(i, " body {width: 1080px; margin: 0 auto; font-size: large;}\n") + head.insert(i, " h2 {padding: 10px; background-color: #d0f0d0; }\n") + head.insert(i, " div.sourceCode { margin: 10px; padding: 16px 10px 8px 10px; border: 2px solid silver; background-color: ghostwhite; overflow-x:scroll}\n") + head.insert(i, " pre:not(.sourceCode) { margin: 10px; padding: 16px 10px 8px 10px; border: 2px solid silver; background-color: ghostwhite; overflow-x:scroll}\n") + head.insert(i, " table {margin-left: auto; margin-right: auto; border-collapse: collapse; border: 1px solid;}\n") + head.insert(i, " th {padding: 2px 6px; border: 1px solid; background-color: ghostwhite;}\n") + head.insert(i, " td {padding: 2px 6px; border: 1px solid;}\n") + head.insert(i, " img {display: block; margin-left: auto; margin-right: auto;}\n") + head.insert(i, " figcaption {text-align: center;}\n") + head << "\n" + head << "\n" + head = head.join + tail=<<'EOS' @@ -31,4 +61,5 @@ EOS body = File.read(html_file) File.write(html_file, head+body+tail) -end +end + diff --git a/lib/lib_gen_main_tex.rb b/lib/lib_gen_main_tex.rb index f9e1272..aceb99a 100644 --- a/lib/lib_gen_main_tex.rb +++ b/lib/lib_gen_main_tex.rb @@ -7,7 +7,7 @@ def gen_main_tex directory, texfilenames, appendixfilenames # ------ Generate helper.tex ------ # Get preamble from a latex file generated by pandoc. - # 1. Generate sample latex file by `pandoc -s -o sample.tex sample.md` + # 1. Generate sample latex file by `pandoc -s --listings -o sample.tex sample.md` # 2. Extract the preamble of sample.tex. # 3. Add geometry package. @@ -20,10 +20,17 @@ line1 int main(int argc, char **argv) { } ~~~ + +|English|Japanese| +|:-----:|:------:| +|potato|jagaimo| +|carrot|ninjin| +|onion|tamanegi| + EOS File.write "sample.md", sample_md - if (! system("pandoc", "-s", "-o", "sample.tex", "sample.md")) + if (! system("pandoc", "-s", "--listings", "-o", "sample.tex", "sample.md")) raise ("pandoc retuns error status #{$?}.\n") end sample_tex = File.read("sample.tex") @@ -43,6 +50,20 @@ EOS end preamble << "\\usepackage[margin=2.4cm]{geometry}\n" preamble << "\\usepackage{graphicx}\n" + preamble << "\\lstdefinelanguage[]{turtle}{\n" + preamble << " keywords={pu, pd, pw, fd, tr, bc, fc, if, rt, rs, dp},\n" + preamble << " comment=[l]\\#\n" + preamble << "}\n" + preamble << "[keywords, comments]\n" + preamble << "\\lstset {\n" + preamble << " extendedchars=true,\n" + preamble << " basicstyle=\\small\\ttfamily,\n" + preamble << " keywordstyle=\\color{red},\n" + preamble << " commentstyle=\\color{gray},\n" + preamble << " stringstyle=\\color{blue},\n" + preamble << " breaklines=true,\n" + preamble << " breakatwhitespace=true\n" + preamble << "}\n" File.write("#{directory}/helper.tex",preamble.join) File.delete("sample.md") File.delete("sample.tex") diff --git a/lib/lib_mktbl.rb b/lib/lib_mktbl.rb new file mode 100644 index 0000000..d1a95f4 --- /dev/null +++ b/lib/lib_mktbl.rb @@ -0,0 +1,262 @@ +# mktbl.rb -- make table easy to read + +def usage + print "ruby mktbl.rb file\n" + exit +end + +def get_sep line + unless line.instance_of?(String) && line =~ /^\|(:?-+:?\|)+$/ + raise "Ilegal parameter" + end + line = line.chomp.sub(/^\|/,"") + a = [] + while line.length >= 1 + if line =~ /^(:?-+:?)\|(.*)$/ + pat = $1 + line = $2 + if pat[0] == ":" && pat[-1] == ":" + a << :c + elsif pat[0] == ":" + a << :l + elsif pat[-1] == ":" + a << :r + else #default + a << :l + end + else + raise "Unexpected error. line is \"#{line}\"" + end + end + a +end + +def line2cells line + unless line.instance_of?(String) + raise "Argument must be String" + end + unless line.length < 1000 + raise "String too long" + end + a = [] + line = line.chomp.sub(/^\s*\|/,"").sub(/\s*$/, "") + i = 0 + while i < line.length + cell = "" + while i < line.length && (c = line[i]) != "|" + if c != "\\" && c != "|" + cell << c + i += 1 + elsif i+1 < line.length && line[i+1] == "|" + cell << "\\|" + i += 2 + else + cell << "\\" + i += 1 + end + end + unless i < line.length + raise "String format error in #{line}." + end + cell = cell.sub(/^\s*/,"").sub(/\s*$/, "") + a << cell + i += 1 + end + return a +end + +def get_tbl buf + t = [] + buf.each do |line| + t << line2cells(line) + end + n = t[0].size + t.each do |a| + raise "Each row must have the same number of columns." unless a.size == n + end + t +end + +def get_width tbl, sep + n = sep.size + tbl.each do |a| + raise "Each row must have the same number of columns." unless a.size == n + end + width = [] + 0.upto(n-1) do |i| + max = 0 + tbl.each do |a| + max = max < a[i].length ? a[i].length : max + end + if sep[i] == :l || sep[i] == :r + max = max < 2 ? 2 : max + elsif sep[i] == :c + max = max < 3 ? 3 : max + else + raise "Separater format error." + end + width << max + end + width +end + +def mk_sep_l sep, width + raise "Size of sep and width is different." unless sep.size == width.size + n = sep.size + line = "|" + 0.upto(n-1) do |i| + if sep[i] == :l + line << ":"+"-"*(width[i]-1)+ "|" + elsif sep[i] == :r + line << "-"*(width[i]-1) + ":|" + elsif sep[i] == :c + line << ":" + "-"*(width[i]-2) + ":|" + else + raise "Separater format error." + end + end + line +end + +def mk_tbl_l tbl, sep, width + n = width.size + raise "mk_tbl_l: Each row must have the same number of columns." unless sep.size == n + tbl.each do |a| + raise "mk_tbl_l: Each row must have the same number of columns." unless a.size == n + 0.upto(n-1) do |i| + raise "mk_tbl_l: table data [#{i}] has wider width than expected (#{width[i]})." if a[i].size > width[i] + end + end + tbl_l = [] + tbl.each do |row| + line = "|" + 0.upto(n-1) do |i| + space = width[i]-row[i].length + if sep[i] == :l + line << row[i] + " "*space + "|" + elsif sep[i] == :r + line << " "*space + row[i] + "|" + elsif sep[i] == :c + left = space / 2 + right = space - left + line << " "*left + row[i] + " "*right + "|" + else + raise "mk_tbl_l: Separater format error." + end + end + tbl_l << line + end + tbl_l +end + +def mktbl buf + raise "The data not a table." unless buf.size >= 3 # table must have header, separator, body + sep = get_sep buf[1] # Sep is an array contains ALIGNMENT (LEFT, CENTER or RIGHT) information of each column. + buf.delete_at 1 + tbl = get_tbl buf # Tbl is a two dimensional array of the table. Each element is a cell of the table. + width = get_width tbl, sep # Width is an array. Each element is the maximum width of the cells in the column corresponds to the element. + sep_l = mk_sep_l sep, width # Make a text of the separator. Each column in the text has a width given by the second argument. + tbl_l = mk_tbl_l tbl, sep, width # Make an array of texts. Each column in the text has a width given by the second argument. + tbl_l.insert(1, sep_l) + tbl_l +end + +# ---- test ---- +def test_sep + s = "|:-:|:---|---:|-|\n" + sep = get_sep s + raise "test_sep: Error." unless sep == [:c, :l, :r, :l] +end + +def test_tbl + t = "| a | bc|def | gh i | \\|kl|\n" + tbl = line2cells t + raise "test_tbl: Error." unless tbl == ["a", "bc", "def", "gh i", "\\|kl"] +end + +def test_get_tbl + buf = [ "||book|author|price|\n", + "| 1 | Calculus | T. Takagi | \\3,600 |\n", + "|2|Algebra|S. Lang|$50|\n" + ] + tbl = get_tbl buf + tbl_expected = [["", "book", "author", "price"], + ["1", "Calculus", "T. Takagi", "\\3,600"], + ["2", "Algebra", "S. Lang", "$50"] + ] + raise "test_get_tbl: Error" unless tbl == tbl_expected +end + +def test_get_width + tbl = [["", "book", "author", "price"], + ["1", "Calculus", "T. Takagi", "\\3,600"], + ["2", "Algebra", "S. Lang", "$50"] + ] + sep = [:c, :c, :c, :c] + width = get_width tbl, sep + width_expected = [3, 8, 9, 6] + raise "test_get_width: Error" unless width == width_expected +end + +def test_mk_sep_l + sep = [:l, :c, :r, :c] + width = [3, 8, 9, 6] + sep_l = mk_sep_l sep, width + sep_l_expected = "|:--|:------:|--------:|:----:|" + raise "test_mk_sep_l: Error" unless sep_l == sep_l_expected +end + +def test_mk_tbl_l + sep = [:l, :c, :r, :c] + width = [3, 8, 9, 6] + tbl = [["", "book", "author", "price"], + ["1", "Calculus", "T. Takagi", "\\3,600"], + ["2", "Algebra", "S. Lang", "$50"] + ] + tbl_l = mk_tbl_l tbl, sep, width + tbl_l_expected = ["| | book | author|price |", + "|1 |Calculus|T. Takagi|\\3,600|", + "|2 |Algebra | S. Lang| $50 |" + ] + raise "test_mk_tbl_l: Error" unless tbl_l == tbl_l_expected +end + +def test_mktbl + buf = <<'EOS'.split("\n") +| |token kind | yylval.ID | yylval.NUM | +|:-:|:-|-:|:-:| +|1|ID|distance| | +|2|=||| +|3|NUM||100| +|4|FD||| +|5|ID|distance|| +|6|*||| +|7|NUM||2| +EOS + + tbl_l = mktbl buf + + tbl_l_expected = <<'EOS'.split("\n") +| |token kind|yylval.ID|yylval.NUM| +|:-:|:---------|--------:|:--------:| +| 1 |ID | distance| | +| 2 |= | | | +| 3 |NUM | | 100 | +| 4 |FD | | | +| 5 |ID | distance| | +| 6 |* | | | +| 7 |NUM | | 2 | +EOS + raise "test_mktbl: Error" unless tbl_l == tbl_l_expected +end + +def test + test_sep + test_tbl + test_get_tbl + test_get_width + test_mk_sep_l + test_mk_tbl_l + test_mktbl +end + diff --git a/lib/lib_src2md.rb b/lib/lib_src2md.rb index d91f93e..bd78ad7 100644 --- a/lib/lib_src2md.rb +++ b/lib/lib_src2md.rb @@ -1,8 +1,8 @@ # lib_src2md.rb require 'pathname' -# The method 'src2md' convert .src.md file into .md file. -# The output .md file is fit for the final format, which is one of markdown, html and latex. +# The method 'src2md' converts .src.md file into .md file. +# The outputed .md file is fit for the final format, which is one of markdown, html and latex. # - Links to relative URL are removed for latex. Otherwise, it remains. # See "Hyperref and relative link" below for further explanation. # - Width and height for images are removed for markdown and html. it remains for latex. @@ -40,114 +40,52 @@ require 'pathname' # If the target is full URL, which means absolute URL begins with "http", no problem happens. -# This script just remove the links if its target is relative URL. +# This script just remove the links if its target is relative URL if the target is latex. # If you want to revive the link with relative URL, refer the description above. -# ---- Folding verbatim lines ---- -# When C sourcefiles or subshell output are included, the lines are folded to fit in 'width'. -# Width must be positive integer. -# Otherwise the lines are not folded. - # This script uses "fenced code blocks" for verbatim lines. # It is available in GFM and pandoc's markdown but not in original markdown. # Two characters backtick (`) and tilde (~) are possible for fences. # This script uses tilde because info string cannot contain any backticks for the backtick code fence. # Info string follows opening fence and it is usually a language name. + +# GFM has fence code block as follows. # ~~~C # int main (int argc, char **argv) { # ........ # ~~~ # Then the contents are highlighted based on C language syntax. -# This script find the language by the suffix of the file name. +# This script finds the language by the suffix of the file name. # .c => C, .h => C, .rb => ruby, Rakefile, => ruby, .xml => xml, .ui => xml, .y => bison, .lex => lex, .build => meson, .md => markdown # Makefile => makefile -def src2md srcmd, md, width +# Pandoc's markdown is a bit different. +# ~~~{.C .numberLines} +# int main (int argc, char **argv) { +# ........ +# ~~~ +# Then the contents are highlighted based on C language syntax and line numbers are added. +# Pandoc supports C, ruby, xml, bison, lex, markdown and makefile languages, but doesn't meson. +# +# After a markdown file is converted to a latex file, listings package is used by lualatex to convert it to a pdf file. +# Listings package supports only C, ruby, xml and make. +# Bison, lex, markdown and meson aren't supported. + +def src2md srcmd, md # parameters: # srcmd: .src.md file's path. source # md: .md file's path. destination -# width: maximum width of lines in fence code block src_buf = IO.readlines srcmd src_dir = File.dirname srcmd md_dir = File.dirname md type = File.basename md_dir # type of the target. gfm, html or latex +# phase 1 +# @@@if - @@@elif - @@@else - @@@end md_buf = [] - include_flag = "" - shell_flag = false if_stat = 0 src_buf.each do |line| - if include_flag == "-N" || include_flag == "-n" - if line == "@@@\n" - include_flag = false - elsif line =~ /^ *(\S*) *(.*)$/ - c_file = $1 - c_functions = $2.strip.split(" ") - if c_file =~ /^\// # absolute path - c_file_buf = File.readlines(c_file) - else #relative path - c_file_buf = File.readlines(src_dir+"/"+c_file) - end - if c_functions.empty? # no functions are specified - tmp_buf = c_file_buf - else - tmp_buf = [] - spc = false - c_functions.each do |c_function| - from = c_file_buf.find_index { |line| line =~ /^#{c_function} *\(/ } - if ! from - warn "ERROR in #{srcmd}: Didn't find #{c_function} in #{c_file}." - break - end - to = from - while to < c_file_buf.size do - if c_file_buf[to] == "}\n" - break - end - to += 1 - end - n = from-1 - if spc - tmp_buf << "\n" - else - spc = true - end - while n <= to do - tmp_buf << c_file_buf[n] - n += 1 - end - end - end - md_buf << "~~~#{lang(c_file)}\n" - ln_width = tmp_buf.size.to_s.length - n = 1 - tmp_buf.each do |l| - if include_flag == "-n" - l = sprintf("%#{ln_width}d %s", n, l) - end - fold(l, width).each_line do |l2| - md_buf << l2 - end - n += 1 - end - md_buf << "~~~\n" - end - elsif shell_flag - if line == "@@@\n" - shell_flag = false - else - md_buf << "~~~\n" - fold("$ #{line}", width).each_line do |l2| - md_buf << l2 - end - `cd #{src_dir}; #{line.chomp}`.each_line do |l| - fold(l, width).each_line do |l2| - md_buf << l2 - end - end - md_buf << "~~~\n" - end - elsif line =~ /^@@@if *(\w+)/ && if_stat == 0 + if line =~ /^@@@if *(\w+)/ && if_stat == 0 if_stat = type == $1 ? 1 : -1 elsif line =~ /^@@@elif *(\w+)/ if if_stat == 1 @@ -176,21 +114,133 @@ def src2md srcmd, md, width elsif line =~ /^@@@end/ if_stat = 0 elsif if_stat >= 0 - if line == "@@@include\n" || line =~ /^@@@include *-n/ - include_flag = "-n" - elsif line =~ /^@@@include *-N/ - include_flag = "-N" - elsif line == "@@@shell\n" - shell_flag = true - else - line = change_rel_link(line, src_dir, md_dir) - if type == "latex" # remove relative link - line.gsub!(/(^|[^!])\[([^\]]*)\]\((?~http)\)/,"\\1\\2") - else # type == "gfm" or "html", then remove size option from link to image files. - line.gsub!(/(!\[[^\]]*\]\([^\)]*\)) *{width *= *\d*(|\.\d*)cm *height *= *\d*(|\.\d*)cm}/,"\\1") + md_buf << line + end + end + +# phase 2 +# @@@include and @@@shell + src_buf = md_buf + md_buf = [] + include_flag = "" + shell_flag = false + src_buf.each do |line| + if include_flag == "-N" || include_flag == "-n" + if line == "@@@\n" + include_flag = "" + elsif line =~ /^\s*(\S*)\s*(.*)$/ + c_file = $1 + c_functions = $2.strip.split(" ") + if c_file =~ /^\// # absolute path + c_file_buf = File.readlines(c_file) + else #relative path + c_file_buf = File.readlines(src_dir+"/"+c_file) end - md_buf << line + if c_functions.empty? # no functions are specified + tmp_buf = c_file_buf + else + tmp_buf = [] + spc = false + c_functions.each do |c_function| + from = c_file_buf.find_index { |line| line =~ /^#{c_function} *\(/ } + if ! from + warn "lib_src2md: ERROR in #{srcmd}: Didn't find #{c_function} in #{c_file}." + break + end + to = from + while to < c_file_buf.size do + if c_file_buf[to] == "}\n" + break + end + to += 1 + end + if to >= c_file_buf.size + warn "lib_src2md: ERROR in #{srcmd}: function #{c_function} didn't end in #{c_file}." + break + end + n = from-1 + if spc + tmp_buf << "\n" + else + spc = true + end + while n <= to do + tmp_buf << c_file_buf[n] + n += 1 + end + end + end + if type == "gfm" + md_buf << "~~~#{lang(c_file, "gfm")}\n" + elsif type == "html" + language = lang(c_file, "pandoc") + if include_flag == "-n" + if language != "" + md_buf << "~~~{.#{language} .numberLines}\n" + else + md_buf << "~~~{.numberLines}\n" + end + else + if lang(c_file, "pandoc") != "" + md_buf << "~~~{.#{language}}\n" + else + md_buf << "~~~\n" + end + end + elsif type =="latex" + language = lang(c_file, "pandoc") + if include_flag == "-n" + if language == "C" || language == "ruby" || language == "xml" || language == "makefile" + md_buf << "~~~{.#{language} .numberLines}\n" + else + md_buf << "~~~{.numberLines}\n" + end + else + if language == "C" || language == "ruby" || language == "xml" || language == "makefile" + md_buf << "~~~{.#{language}}\n" + else + md_buf << "~~~\n" + end + end + else # This can't happen. + md_buf << "~~~" + end + ln_width = tmp_buf.size.to_s.length + n = 1 + tmp_buf.each do |l| + if type == "gfm" && include_flag == "-n" + l = sprintf("%#{ln_width}d %s", n, l) + end + md_buf << l + n += 1 + end + md_buf << "~~~\n" end + elsif shell_flag + if line == "@@@\n" + md_buf << "~~~\n" + shell_flag = false + else + md_buf << "$ #{line}" + `cd #{src_dir}; #{line.chomp}`.each_line do |l| + md_buf << l + end + end + elsif line == "@@@include\n" || line =~ /^@@@include *-n/ + include_flag = "-n" + elsif line =~ /^@@@include *-N/ + include_flag = "-N" + elsif line == "@@@shell\n" + md_buf << "~~~\n" + shell_flag = true + else + line = change_rel_link(line, src_dir, md_dir) + if type == "latex" # remove relative link + line.gsub!(/(^|[^!])\[([^\]]*)\]\((?~http)\)/,"\\1\\2") + else # type == "gfm" or "html", then remove size option from link to image files. + line.gsub!(/(!\[[^\]]*\]\([^\)]*\)) *{width *= *\d*(|\.\d*)cm *height *= *\d*(|\.\d*)cm}/,"\\1") + end + md_buf << line end end IO.write(md,md_buf.join) @@ -217,17 +267,7 @@ def change_rel_link line, org_dir, new_dir left + right end -def fold line, width - if line.instance_of?(String) && width.instance_of?(Integer) && width > 0 - n = (line.chomp.length - 1) / width - n.downto(1) do |i| - line.insert width*i, "\n" - end - end - line -end - -def lang file +def lang file, type_of_md tbl = {".c" => "C", ".h" => "C", ".rb" => "ruby", ".xml" => "xml", ".ui" => "xml", ".y" => "bison", ".lex" => "lex", ".build" => "meson", ".md" => "markdown" } name = File.basename file @@ -239,7 +279,8 @@ def lang file suffix = File.extname name tbl.each do |key, val| if suffix == key - return val + return val if type_of_md == "gfm" + return val if type_of_md == "pandoc" && val != "meson" end end end diff --git a/sample.html b/sample.html new file mode 100644 index 0000000..09e1c81 --- /dev/null +++ b/sample.html @@ -0,0 +1,110 @@ + + + + + + + Gtk4 tutorial for beginners + + + +
+

Gtk4 tutorial for beginners

+
+

sample header

+

Main contents begin here.

+
int main(int argc, char **argv) {
+}
+ + + + + + + + + + + + + + + + + + + + + +
EnglishJapanese
potatojagaimo
carrotninjin
oniontamanegi
+ + diff --git a/sample.md b/sample.md new file mode 100644 index 0000000..13d35e4 --- /dev/null +++ b/sample.md @@ -0,0 +1,18 @@ +--- +title: 'Gtk4 tutorial for beginners' +--- + +# sample header + +Main contents begin here. + +~~~{.C .numberLines} +int main(int argc, char **argv) { +} +~~~ + +|English|Japanese| +|:-----:|:------:| +|potato|jagaimo| +|carrot|ninjin| +|onion|tamanegi| diff --git a/src/mktbl.rb b/src/mktbl.rb new file mode 100644 index 0000000..4003272 --- /dev/null +++ b/src/mktbl.rb @@ -0,0 +1,32 @@ +require_relative '../lib/lib_mktbl.rb' + +file = ARGV[0] +old = File.readlines file +in_stat = false +new = [] +changed = false +tmp = [] +old.each do |line| + if in_stat + if line == "@@@\n" + in_stat = false + new += mktbl tmp + else + tmp << line + end + elsif line == "@@@table\n" + changed = true + in_stat = true + tmp = [] + else + new << line + end +end +new.each do |line| + if line[-1] != "\n" + line.sub!(/\z/,"\n") + end +end +exit unless changed +File.write file+".bak", old.join +File.write file, new.join diff --git a/src/sec12.src.md b/src/sec12.src.md index c94a463..d2faa98 100644 --- a/src/sec12.src.md +++ b/src/sec12.src.md @@ -2,7 +2,7 @@ In this section I will explain functions in TfeTextView object. -### tfe.h and tfetextview.h +## tfe.h and tfetextview.h `tfe.h` is a top header file and it includes `gtk.h` and all the header files. C source files `tfeapplication.c` and `tfenotebook.c` include `tfe.h` at the beginning. diff --git a/src/sec14.src.md b/src/sec14.src.md index c11efeb..b3d355e 100644 --- a/src/sec14.src.md +++ b/src/sec14.src.md @@ -60,7 +60,11 @@ It implies that CSS can also be applied to GTK windowing system. The syntax of CSS is as follows. +@@@if gfm ~~~css +@@@else +~~~ +@@@end selector { color: yellow; padding-top: 10px; ...} ~~~ @@ -68,7 +72,11 @@ Every widget has CSS node. For example GtkTextView has `textview` node. If you want to set style to GtkTextView, substitute "textview" for the selector. +@@@if gfm ~~~css +@@@else +~~~ +@@@end textview {color: yellow; ...} ~~~ @@ -77,7 +85,11 @@ Refer [GTK4 API reference](https://gnome.pages.gitlab.gnome.org/gtk/gtk/theming. In line 30, the CSS is a string. +@@@if gfm ~~~css +@@@else +~~~ +@@@end textview {padding: 10px; font-family: monospace; font-size: 12pt;} ~~~ diff --git a/src/sec19.src.md b/src/sec19.src.md index dc63ea9..ae7180b 100644 --- a/src/sec19.src.md +++ b/src/sec19.src.md @@ -863,7 +863,11 @@ Meson provides `gnome.compile_schemas` method to compile XML file in the build d This is used to test the application. Write the following to the `meson.build` file. +@@@if gfm ~~~meson +@@@else +~~~ +@@@end gnome.compile_schemas(build_by_default: true, depend_files: 'com.github.ToshioCP.tfe.gschema.xml') ~~~ @@ -880,7 +884,7 @@ After compilation, you can test your application like this: $ GSETTINGS_SCHEMA_DIR=_build:$GSETTINGS_SCHEMA_DIR _build/tfe ~~~ -### GSettings object and g_setting_bind +### GSettings object and g\_settings\_bind Write gsettings related codes to `tfeapplication.c'. @@ -928,7 +932,11 @@ $ meson --prefix=$HOME/local _build Modify `meson.build` abd add install option and set it true in executable function. +@@@if gfm ~~~meson +@@@else +~~~ +@@@end executable('tfe', sourcefiles, resources, dependencies: gtkdep, export_dynamic: true, install: true) ~~~ @@ -942,7 +950,11 @@ However, you need to do one more thing. Copy your XML file to `$HOME/local/share/glib-2.0/schemas/`, which is specified in `GSETTINGS_SCHEMA_DIR` environment variable, and run `glib-compile-schemas` on that directory. +@@@if gfm ~~~meson +@@@else +~~~ +@@@end schema_dir = get_option('prefix') / get_option('datadir') / 'glib-2.0/schemas/' install_data('com.github.ToshioCP.tfe.gschema.xml', install_dir: schema_dir) ~~~ @@ -956,7 +968,11 @@ So, `$HOME/local/share/glib-2.0/schemas` is assigned to the varable `schema_dir` Meson can runs a post compile script. +@@@if gfm ~~~meson +@@@else +~~~ +@@@end meson.add_install_script('glib-compile-schemas', schema_dir) ~~~ diff --git a/src/sec23.src.md b/src/sec23.src.md new file mode 100644 index 0000000..9cc3b9c --- /dev/null +++ b/src/sec23.src.md @@ -0,0 +1,869 @@ +# Tiny turtle graphics interpreter + +A program `turtle` is an example with the combination of TfeTextView and GtkDrawingArea objects. +It is a very small interpreter but it provides a way to draw fractal curves. +The following diagram is a Koch curve, which is a famous example of fractal curves. + +![Koch curve](turtle/image/turtle_koch.png){width=8cm height=5.11cm} + +This program uses flex and bison. +Flex is a lexical analyzer. +Bison is a parser generator. +These two programs are similar to lex and yacc which are proprietary software developed in Bell Laboratry. +However, flex and bison are open source software. +I will write about how to use those software, but they are not topics about gtk. +So, readers can skip that part of this sections. + +## How to use turtle + +The documentation of turtle is [here](turtle/turtle_doc.md). +I'll show you a simple example. + +~~~ +fc (1,0,0) # Foreground color is red, rgb = (1,0,0). +pd # Pen down. +fd 100 # Go forward by 100 pixels. +tr 90 # Turn right by 90 degrees. +fd 100 +tr 90 +fd 100 +tr 90 +fd 100 +tr 90 +~~~ + +1. Compile and install `turtle` (See the documentation above). +Then, run `turtle`. +2. Type the program above in the editor (left part of the window). +3. Click on the `Run` button, then a red square appears on the right part of the window. +The side of the square is 100 pixels long. + +In the same way, you can draw other curves. +The documentation above shows some fractal curves such as tree, snow and square-koch. +The source code in turtle language is located at [src/turtle/example](turtle/example) directory. +You can read these files by clicking on the `Open` button. + +## Combination of TfeTextView and GtkDrawingArea objects + +Turtle uses TfeTextView and GtkDrawingArea. +It is similar to `color` program in the previous section. + +The body of the interpreter is written with flex and bison. +The codes are not thread safe. +So the handler of "clicked" signal of the `Run` button prevents from reentering. + +@@@include +turtle/turtleapplication.c run_cb resize_cb +@@@ + +- 8-13: The static value `busy` holds a status of the interpreter. +If it is `TRUE`, the interpreter is running and it is not possible to call the interpreter again because it's not a re-entrant program. +If it is `FALSE`, it is safe to call the interpreter. +- 14: Now it is about to call the interpreter so let `busy` variable to be TRUE. +- 15-16: Gets the contents of GtkTextBuffer. +- 17: The varable `surface` is a static variable. +It points to a `cairo_surface_t` object. +It is generated when the GtkDrawingArea object is realized and the each time it is resized. +Therefore, `surface` isn't NULL usually. +But if it is NULL, the interpreter won't be called. +- 18-19: Initializes lexcal analyzer, then parser. +- 20: Calls parser. +Parser analyze the program codes syntactically and generate a tree structured data. +- 24-26: If the parser succesfully parsed, it calls `run` which is the interpreter. +`Run` executes the tree-structured program. +- 27: finalize the lexical analyzer. +- 29: Add the drawing area object to the queue to draw. +- 30: The interpreter program has finished so `busy` is now FALSE. +- 33-38: A handler of "resized" signal. +It generates or regenerates a surface object. + +Other part of `turtleapplication.c` is almost same as the codes of `colorapplication.c` in the previous section. +The codes of `turtleapplication.c` is in the [turtle directory](turtle). + +## What does the interpreter do? + +Suppose that the turtle runs with the following program. + +~~~ +distance = 100 +fd distance*2 +~~~ + +The turtle recognizes the program above and works as follows. + +- Generally, a program consists of tokens. +Tokens are "distance", "=", "100", "fd", "*" and "2" in the above example.. +- The interpreter `turtle` calls `yylex` to read a token in the source file. +The `yylex` returns a code which is called "token kind" and sets a global variable `yylval` with a value, which is called a semantic value. +The type of `yylval` is union and `yylval.ID` is string and `yylval.NUM` is double. +There are seven tokens in the program so `yylex` is called seven times. + +| |token kind|yylval.ID|yylval.NUM| +|:-:|:--------:|:-------:|:--------:| +| 1 | ID |distance | | +| 2 | = | | | +| 3 | NUM | | 100 | +| 4 | FD | | | +| 5 | ID |distance | | +| 6 | * | | | +| 7 | NUM | | 2 | + +- `yylex` returns a token kind every time, but it doesn't set `yylval.ID` or `yylval.NUM` every time. +It is because keywords (`FD`) and symbols (`=` and `*`) don't have any semantic values. +The function `yylex` is called lexical analyzer or scanner. +- `turtle` makes a tree structured data. +This part of `turtle` is called parser. + +![turtle parser tree](../image/turtle_parser_tree.png){width=12cm height=5.34cm} + +- `turtle` analyzes the tree and executes it. +This part of `turtle` is called runtime routine. +The tree consists of line segments and rectangles between line segments. +The rectangles are called nodes. +For example, N\_PROGRAM, N\_ASSIGN, N\_FD and N\_MUL are nodes. + 1. Goes down from N\_PROGRAM to N\_ASSIGN. + 2. N_ASSIGN node has two children, ID and NUM. +This node comes from "distance = 100" which is "ID = NUM" syntactically. +First, `turtle` checks if the first child is ID. +If it's ID, then `turtle` looks for the variable in the variable table. +If it doesn't exist, it registers the ID (`distance`) to the table. +Then go back to the N\_ASSIGN node. + 3. `turtle` calculates the second child. +In this case its a number 100. +Saves 100 to the variable table at the `distance` record. + 4. `turtle` goes back to N\_PROGRAM then go to the next node N\_FD. +It has only one child. +Goes down to the child N\_MUL. + 5. The first child is ID (distance). +Searches the variable table for the varable `distance` and gets the value 100. +The second child is a number 2. +Multiplies 100 by 2 and gets 200. +Then `turtle` goes back to N_FD. + 6. Now `turtle` knows the distance is 200. +It moves the cursor forward by 200. + 8. There are no node follows. +Runtime routine returns to the main routine. +-`turtle` draws a segment on GtkDrawingArea then stops. + +Actually most programs are more complicated than the example above. +So, `turtle` does much more work to interpret programs. +However, basically it works by the same way above. +Interpritation consists of three parts. + +- Lexical analysis +- Syntax Parsing and tree generation +- Interprit the tree and execute commands. + +## Compilation flow + +The source files are: + +- flex source file ~> turtle.lex +- bison source file => turtle.y +- C header file => turtle.h, turtle_lex.h +- C source file => turtleapplication.c +- other files => turtle.ui, turtle.gresources.xml, meson.build + +The compilation process is a bit complicated. + +1. glib-compile-resources compiles `turtle.ui` to `resources.c` according to `turtle.gresource.xml`. +It also generates `resources.h`. +2. bison compiles `turtle.y` to `turtle_parser.c` and generates `turtle_parser.h` +3. flex compiles `turtle.lex` to `turtle_lex.c`. +4. gcc compiles `application.c`, `resources.c`, `turtle_parser.c` and `turtle_lex.c` with `turtle.h`, `resources.h` and `turtle_parser.h`. +It generates an executable file `turtle`. + +![compile process](../image/turtle_compile_process.png){width=12cm height=9cm} + +Meson controls the process and the instruction is described in meson.build. + +@@@include +turtle/meson.build +@@@ + +- 3: Gets C compiler. +It is usually `gcc` in linux. +- 4: Gets math library. +This program uses trigonometric functions. +They are defined in the math library, but the library is optional. +So, it is necessary to include the library by `#include ` and also link the library with the linker. +- 6: Gets gtk4 library. +- 8: Gets gnome module. +Module is a system provided by meson. +See [Meson build system website](https://mesonbuild.com/Gnome-module.html#gnome-module) for further information. +- 9: Compiles ui file to C source file according to the XML file `turtle.gresource.xml`. +- 11: Gets flex. +- 12: Gets bison. +- 13: Compiles `turtle.y` to `turtle_parser.c` and `turtle_parser.h` by bison. +The function `custom_target`creates a custom top level target. +See [Meson build system website](https://mesonbuild.com/Reference-manual.html#custom_target) for further information. +- 14: Compiles `turtle.lex` to `turtle_lex.c` by flex. +- 16: Specifies C surce files. +- 18: Compiles C source files including generated files by glib-compile-resources, bison and flex. +The argument `turtleparser[1]` refers to `tirtle_parser.h` which is the secound output in the line 13. + +## Turtle.lex + +### What does flex do? + +Flex creates lexical analizer from flex source file. +Flex source file is a text file and its syntactic rule will be explained later. +Generated lexical analyzer is a C source file. +It is also called scanner. +It reads a text file, which is a source file of a program language, and gets variable names, numbers and symbols. +Suppose here is a turtle source file. + +~~~ +fc (1,0,0) # Foreground color is red, rgb = (1,0,0). +pd # Pen down. +distance = 100 +angle = 90 +fd distance # Go forward by distance (100) pixels. +tr angle # Turn right by angle (90) degrees. +~~~ + +The content of the text file is separated into `fc`, `(`, `1`, `,` and so on. +The words `fc`, `pd`, `distance` and `100` are called tokens. +The characters `(`, `<` and `=` are called symbols. +( Sometimes those symbols called token, too.) + +Flex reads `turtle.lex` and generates a scanner. +The file `turtle.lex` specifies tokens, symbols and the behavior which corresponds to each token or symbol. +Turtle.lex isn't a big program. + +@@@include +turtle/turtle.lex +@@@ + +The file consists of three sections which are separated by "%%" (line 19 and 58). +They are definitions, rules and user code sections. + +### Definitions section + +First, look at the definitions section. + +- 1-12: Lines between "%top{" and "}" are C source codes. +They will be copied to the top of the generated C source file. +- 2-3: The function `strlen`, in line 64, is defined in `string.h` +The function `atof`, in line 39, is defined in `stdlib.h`. +- 6-8: The current input position is pointed by `nline` and `ncolumn`. +The function `get_location` (line 60-65) sets `yylloc`to point the start and end point of `yytext` in the buffer. +This function is declared here so that it can be called before the function is defined. +- 11: GSlist is used to keep allocated memories. +- 14: This option (`%option noyywrap`) must be specified when you have only single source file to the scanner. Reffur to "9 The Generated Scanner" in the flex documentation in your distribution for further information. +(The documentation is not on the internet.) +- 16-17: `REAL_NUMBER` and `IDENTIFIER` are names. +A name begins with a letter or an underscore followed by zero or more letters, digits, underscores (`_`) or dashes (`-`). +They are followed by regular expressions which are the definition of them. +They will be used in rules section and will expand to the definition. +You can leave out such definitions here and use regular expressions in rules section directly. + +### Rules section + +This section is the most important part. +Rules consist of patterns and actions. +For example, line 37 is a rule. + +- `{REAL_NUMBER}` is a pattern +- `get_location (yytext); yylval.NUM = atof (yytext); return NUM;` is an action. + +`{REAL_NUMBER}` is defined in the 16th line, so it expands to `(0|[1-9][0-9]*)(\.[0-9]+)?`. +This regular expression matches numbers like `0`, `12` and `1.5`. +If the input is a number, it matches the pattern in line 37. +Then the matched text is assigned to `yytext` and corresponding action is executed. +A function `get_location` changes the location variables. +It assigns `atof( yytext)`, which is double sized number converted from `yytext`, to `yylval.NUM` and return `NUM`. +`NUM` is an integer defined by `turtle.y`. + +The scanner generated by flex and C compiler has `yylex` function. +If `yylex` is called and the input is "123.4", then it works as follows. + +1. A string "123.4" matches `{REAL_NUMBER}`. +2. Update the location variable `ncolumn` and `yylloc`. +3. `atof` converts the string "123.4" to double sized floating point number 123.4. +4. It is assigned to `yylval.NUM`. +5. `yylex` returns `NUM` to the caller. + +Then the caller knows the input is `NUM` (number), and its value is 123.4. + +- 19-55: Rules section. +- 20: Comment begins `#` followed by any characters except newline. +No action happens. +- 21: White space just increases a varable `ncolumn` by one. +- 22: Tab is assumed to be equal to eight spaces. +- 23: New line increases a variable `nline` by one and resets `ncolumn`. +- 25-35: Keywords just updates the location variables `ncolumn` and `yylloc`, and return the codes of the keywords. +- 37: Real number constant. +- 38: Identifier is defined in line 17. +It begins alphabet followed by zero or more alphabet or digit. +The location variables are updated and the name of the identifier is assigned to `yylval.ID`. +The memory of the name is allocated by the function `g_strdup`. +The memory is registerd to the list (GSlist type list). +The memory will be freed after the runtime routine finishes. +Returns `ID`. +- 43-54: Symbols just update the location variable and return the codes. +The code is the same as the symbol itself. +- 55: If the input doesn't match above patterns, then it is error. +Returns `YYUNDEF`. + +### User code section + +This section is just copied to C source file. + +- 58-63: A function `get_location`. +The location of the input is recorded to `nline` and `ncolumn`. +These two variables are for the scanner. +A variable `yylloc` is shared by the scanner and the parser. +It is a C structure and has four members, `first_line`, `first_column`, `last_line` and `last_column`. +They point the start and end of the current input text. +- 65: `YY_BUFFER_STATE` is a type of the pointer points the input buffer. +- 67-70: `init_flex` is called by `run_cb` signal handler, which is called when `Run` button is clicked on. +`run_cb` has one argument which is the copy of the content of GtkTextBuffer. +`yy_scan_string` sets the input buffer to read from the text. +- 72-75: `finalize_flex` is called after runtime routine finishes. +It deletes the input buffer. + +## Turtle.y + +Turtle.y has more than 800 lines so it is difficult to explain all the source code. +So I will explain the key points and leave out other less important parts. + +### What does bison do? + +Bison creates C source file of a parser from bison source file. +Bison source file is a text file. +A parser analyzes a program source code according to its grammar. +Suppose here is a turtle source file. + +~~~ +fc (1,0,0) # Foreground color is red, rgb = (1,0,0). +pd # Pen down. +distance = 100 +angle = 90 +fd distance # Go forward by distance (100) pixels. +tr angle # Turn right by angle (90) degrees. +~~~ + +The parser calls `yylex` to get a token. +The token consists of its type (token kind) and value (semantic value). +So, the parser gets items in the following table whenever it calls `yylex`. + +| |token kind|yylval.ID|yylval.NUM| +|:-:|:--------:|:-------:|:--------:| +| 1 | FC | | | +| 2 | ( | | | +| 3 | NUM | | 1.0 | +| 4 | , | | | +| 5 | NUM | | 0.0 | +| 6 | , | | | +| 7 | NUM | | 0.0 | +| 8 | ) | | | +| 9 | PD | | | +|10 | ID |distance | | +|11 | = | | | +|12 | NUM | | 100.0 | +|13 | ID | angle | | +|14 | = | | | +|15 | NUM | | 90.0 | +|16 | FD | | | +|17 | ID |distance | | +|18 | TR | | | +|19 | ID | angle | | + +Bison source code specifies the grammar rules of turtle language. +For example, `fc (1,0,0)` is called primary procedure. +A procedure is like a void type function in C source code. +It doesn't return any values. +Programmers can define their own procedures. +On the other hand, `fc` is a built-in procedure. +Such procedures are called primary procedures. +It is described in Bison source code like: + +~~~ +primary_procedure: FC '(' expression ',' expression ',' expression ')'; +expression: ID | NUM; +~~~ + +This means: + +- Primary procedure is FC followed by '(', expression, ',', expression, ',', expression and ')'. +- expression is ID or NUM. + +The description above is called BNF (Backus-Naur form). +More precisely, it is similar to BNF. + +The first line is: + +~~~ +FC '(' NUM ',' NUM ',' NUM ')'; +~~~ + +You can find this is a primary_procedure easily. +The parser of the turtle language analyzes the turtle source code in the same way. + +The grammar of turtle is described in the [document](turtle/turtle_doc.md). +The following is an extract from the document. + +~~~ +program: + statement +| program statement +; + +statement: + primary_procedure +| procedure_definition +; + +primary_procedure: + PU +| PD +| PW expression +| FD expression +| TR expression +| BC '(' expression ',' expression ',' expression ')' +| FC '(' expression ',' expression ',' expression ')' +| ID '=' expression +| IF '(' expression ')' '{' primary_procedure_list '}' +| RT +| RS +| ID '(' ')' +| ID '(' argument_list ')' +; + +procedure_definition: + DP ID '(' ')' '{' primary_procedure_list '}' +| DP ID '(' parameter_list ')' '{' primary_procedure_list '}' +; + +parameter_list: + ID +| parameter_list ',' ID +; + +argument_list: + expression +| argument_list ',' expression +; + +primary_procedure_list: + primary_procedure +| primary_procedure_list primary_procedure +; + +expression: + expression '=' expression +| expression '>' expression +| expression '<' expression +| expression '+' expression +| expression '-' expression +| expression '*' expression +| expression '/' expression +| '-' expression %prec UMINUS +| '(' expression ')' +| ID +| NUM +; +~~~ + +The grammar rule defines `program` first. + +- program is a statement or a program followed by a statement. + +The definition is recursive. + +- `statement` is program. +- `statement statement` is `program statemet`. +Therefore, it is program. +- `statement statement statement` is `program statemet`. +Therefore, it is program. + +You can find that a list of statements is program like this. + +`program` and `statement` aren't tokens. +They don't appear in the input. +They are called non terminal symbols. +On the other hand, tokens are called terminal symbols. +The word "token" used here has wide meaning, it includes tokens and symbols which appear in the input. +Non terminal symbols are often shortend to nterm. + +Let's analyze the program above as bison does. + +| |token kind|yylval.ID|yylval.NUM|parse |S/R| +|:-:|:--------:|:-------:|:--------:|:-----------------------------------|:-:| +| 1 | FC | | |FC | S | +| 2 | ( | | |FC( | S | +| 3 | NUM | | 1.0 |FC(NUM | S | +| | | | |FC(expression | R | +| 4 | , | | |FC(expression, | S | +| 5 | NUM | | 0.0 |FC(expression,NUM | S | +| | | | |FC(expression,expression | R | +| 6 | , | | |FC(expression,expression, | S | +| 7 | NUM | | 0.0 |FC(expression,expression,NUM | S | +| | | | |FC(expression,expression,expression | R | +| 8 | ) | | |FC(expression,expression,expression)| S | +| | | | |primary_procedure | R | +| | | | |statement | R | +| | | | |program | R | +| 9 | PD | | |program PD | S | +| | | | |program primary_procedure | R | +| | | | |program statement | R | +| | | | |program | R | +|10 | ID |distance | |program ID | S | +|11 | = | | |program ID= | S | +|12 | NUM | | 100.0 |program ID=NUM | S | +| | | | |program ID=expression | R | +| | | | |program primary_procedure | R | +| | | | |program statement | R | +| | | | |program | R | +|13 | ID | angle | |program ID | S | +|14 | = | | |program ID= | S | +|15 | NUM | | 90.0 |program ID=NUM | S | +| | | | |program ID=expression | R | +| | | | |program primary_procedure | R | +| | | | |program statement | R | +| | | | |program | R | +|16 | FD | | |program FD | S | +|17 | ID |distance | |program FD ID | S | +| | | | |program FD expression | R | +| | | | |program primary_procedure | R | +| | | | |program statement | R | +| | | | |program | R | +|18 | TR | | |program TR | S | +|19 | ID | angle | |program TR ID | S | +| | | | |program TR expression | R | +| | | | |program primary_procedure | R | +| | | | |program statement | R | +| | | | |program | R | + +The right most column shows shift/reduce. +Shift is appending an input to the buffer. +Reduce is substituing a higher nterm for the pattern in the buffer. +For example, NUM is replaced by expression in the forth row. +This substitution is "reduce". + +Bison repeats shift and reduction until the end of the input. +If the result is reduced to `program`, the input is syntactically valid. +Bison executes an action whenever reduction occers. +Actions build a tree. +The tree is analyzed and executed by runtime routine . + +Bison source files are called bison grammar files. +A bison grammar file consists of four sections, prologue, declarations, rules and epilogue. +The format is as follows. + +~~~ +%{ +prologue +%} +declarations +%% +rules +%% +epilogue +~~~ + +### Prologue + +Prologue section consists of C codes and the codes are copied to the parser implementation file. +You can use `%code` directives to qualify the prologue and identifies the purpose explisitely. +The following is an extract from `turtle.y`. + +@@@if gfm +~~~bison +@@@elif html +~~~{.bison} +@@@else +~~~ +@@@end +%code top{ + #include + #include + #include + #include "turtle.h" + + /* error reporting */ + static void yyerror (char const *s) { /* for syntax error */ + g_print ("%s from line %d, column %d to line %d, column %d\n",s, yylloc.first_line, yylloc.first_column, yylloc.last_line, yylloc.last_column); + } + /* Node type */ + enum { + N_PU, + N_PD, + N_PW, + ... ... ... + }; +} +~~~ + +The directove `%code top` copies its contents to the top of the parser implementation file. +It usually includes `#include` directives, declarations of functions and definitions of constants. +A function `yyerror` reports a syntax error and is called by the parser. +Node type identifies a node in the tree. + +Another directive `%code requires` copies its contents to both the parser implementation file and header file. +The header file is read by the scanner C source file and other files. + +@@@if gfm +~~~bison +@@@elif html +~~~{.bison} +@@@else +~~~ +@@@end +%code requires { + int yylex (void); + void init_parse (void); + int yyparse (void); + void run (void); + + /* semantic value type */ + typedef struct _node_t node_t; + struct _node_t { + int type; + union { + struct { + node_t *child1; + node_t *child2; + node_t *child3; + } child; + char *name; + double value; + } content; + }; +} +~~~ + +The contents of this directive are used by the other files. + +- `yylex` is shared by parser implemetation file and scanner file. +- `init_parse`, `yyparse` and `run` is called by `run_cb' in `turtleapplication.c`. +- `node_t` is the type of the semantic value of nterms. +The header file defines `YYSTYPE`, which is the semantic value type, with all the token and nterm value types. +The following is extracted from the header file. + +~~~ +/* Value type. */ +#if ! defined YYSTYPE && ! defined YYSTYPE_IS_DECLARED +union YYSTYPE +{ + char * ID; /* ID */ + double NUM; /* NUM */ + node_t * program; /* program */ + node_t * statement; /* statement */ + node_t * primary_procedure; /* primary_procedure */ + node_t * primary_procedure_list; /* primary_procedure_list */ + node_t * procedure_definition; /* procedure_definition */ + node_t * parameter_list; /* parameter_list */ + node_t * argument_list; /* argument_list */ + node_t * expression; /* expression */ +}; +~~~ + +Other useful macros and declarations are put into the `%code` directive. + +~~~ +%code { +/* The following macro is convenient to get the member of the node. */ + #define child1(n) (n)->content.child.child1 + #define child2(n) (n)->content.child.child2 + #define child3(n) (n)->content.child.child3 + #define name(n) (n)->content.name + #define value(n) (n)->content.value + + /* start of nodes */ + static node_t *node_top = NULL; + /* functions to generate trees */ + static node_t *tree1 (int type, node_t *child1, node_t *child2, node_t *child3); + static node_t *tree2 (int type, double value); + static node_t *tree3 (int type, char *name); +} +~~~ + +### Bison declarations + +Bison declarations defines terminal and nonterminal symbols. +It also specifies some directives. + +~~~ +%locations +%define api.value.type union /* YYSTYPE, the type of semantic values, is union of following types */ + /* key words */ +%token PU +%token PD +%token PW +%token FD +%token TR +%token BC +%token FC +%token DP +%token IF +%token RT +%token RS + /* constant */ +%token NUM + /* identirier */ +%token ID + /* non terminal symbol */ +%nterm program +%nterm statement +%nterm primary_procedure +%nterm primary_procedure_list +%nterm procedure_definition +%nterm parameter_list +%nterm argument_list +%nterm expression + /* logical relation symbol */ +%left '=' '<' '>' + /* arithmetic symbol */ +%left '+' '-' +%left '*' '/' +%precedence UMINUS /* unary minus */ +~~~ + +`%locations` directive inserts the location structure into the header file. +It is like this. + +~~~ +typedef struct YYLTYPE YYLTYPE; +struct YYLTYPE +{ + int first_line; + int first_column; + int last_line; + int last_column; +}; +~~~ + +This type is shared by the scanner file and the parser implementation file. +The error report function `yyerror` uses it so that it can inform the location that error occers. + +`%define api.value.type union` generates semantic value type with tokens and nterms and inserts it to the header file. +The inserted part is shown in the previous section as the extracts that shows the value type (YYSTYPE). + +`%token` and `%nterm` directives define tokens and directives respectively. + +~~~ +%token PU +... ... +%token NUM +~~~ + +These directives define a token `PU` and NUM. +The values of token kinds `PU` and `NUM` are defined as an enumeration constant in the header file. + +~~~ + enum yytokentype + { + ... ... ... + PU = 258, /* PU */ + ... ... ... + NUM = 269, /* NUM */ + ... ... ... + }; + typedef enum yytokentype yytoken_kind_t; +~~~ + +In addition, the type of the semantic value of NUM is defined as double in the header file because of `` tag. + +~~~ +union YYSTYPE +{ + char * ID; /* ID */ + double NUM; /* NUM */ + ... ... +} +~~~ + +All the nterm symbols have the same type `* node_t` of the semantic value. + +`%left` and `%precedence` directives define the precedence of operation symbols. + +~~~ + /* logical relation symbol */ +%left '=' '<' '>' + /* arithmetic symbol */ +%left '+' '-' +%left '*' '/' +%precedence UMINUS /* unary minus */ +~~~ + +`%left` directive defines the following symbols as left-associated operator. +If an operator `+` is left-associated, then + +~~~ +A + B + C = (A + B) + C +~~~ + +That is, the calculation is carried out the left operator first, then the right operator. +If an operator `*` is right-assiciated, then: + +~~~ +A * B * C = A * (B * C) +~~~ + +The definition above decides the behavior of the parser. +Addition and multiplication hold associative law so the result of `(A+B)+C` and `A+(B+C)` are equal in terms of mathematics. +However, the parser will be confused if left (or right) assosiativity is not specified. + +`%left` and `%precedence` directive show the precedence of operators. +Later declared operators have higher precedence than former declared ones. +The declaration above says, for example, + +~~~ +v=w+z*5+7 is the same as v=((w+(z*5))+7) +~~~ + +Be careful. +The operator `=` is a logical equal operator, not assignment operator. +Assignment is not expression in turtle language. +It is primary_procedure. +Therefore, if `=` appears in an expression, it is *NOT* an assignment. + +### Grammar rules + +Grammar rules section defines the syntactic grammar of the language. +It is similar to BNF form. + +~~~ +result: components { action }; +~~~ + +- result is a nterm. +- components are list of tokens or nterms. +- action is C codes. It is executed whenever the components are reduced to the result. +Action can be left out. + +The following is the first eleven lines of the grammar rule in `turtle.y`. + +~~~ +program: + statement { node_top = $$ = $1; } +| program statement { + node_top = $$ = tree1 (N_program, $1, $2, NULL); + } +; + +statement: + primary_procedure +| procedure_definition +; +~~~ + +The first six lines show that `program` is `statement` or `program` followed by `statement`. + +- Whenever `statement` is reduced to `program`, an action `node_top=$$=$1;` is executed. +- In the same way, whenever `program` followed by `statment is reduced to `program`, +`node_top=$$=tree1(N_program,$1,$2,NULL);` is executed. +- `node_top` is a static variable. +It points the top node of the tree. +- `$$` is a semantic value of the result, which is `program` in the example above. +The semantic value of `program` is a pointer to `node_t` type structure. +It was defined in the declaration section. +- `$1` is a semantic value of the first component, which is `statement` in the example. +The semantic value of `statement` is also a pointer to `node_t`. +- The function `tree1` generates tree node. + + + +### Epilogue diff --git a/src/sec6.src.md b/src/sec6.src.md index aff18e7..0f3e8ea 100644 --- a/src/sec6.src.md +++ b/src/sec6.src.md @@ -27,7 +27,7 @@ gtk_application_new (const gchar *application_id, GApplicationFlags flags); This flag is described in the GApplication section in GIO API reference. -~~~C +~~~ GApplicationFlags' Members G_APPLICATION_FLAGS_NONE Default. (No argument allowed) diff --git a/src/sec9.src.md b/src/sec9.src.md index f453f20..3261dd6 100644 --- a/src/sec9.src.md +++ b/src/sec9.src.md @@ -84,7 +84,7 @@ It is an old and widely used program. Make analyzes Makefile and executes compilers. All instructions are written in Makefile. -~~~Makefile +~~~makefile sample.o: sample.c gcc -o sample.o sample.c ~~~ diff --git a/src/turtle/meson.build b/src/turtle/meson.build index 7b4df83..9477a15 100644 --- a/src/turtle/meson.build +++ b/src/turtle/meson.build @@ -15,5 +15,5 @@ turtlelexer = custom_target('turtlelexer', input: 'turtle.lex', output: 'turtle_ sourcefiles=files('turtleapplication.c', '../tfetextview/tfetextview.c') -executable('turtle', sourcefiles, resources, turtleparser, turtlelexer, turtleparser[1], dependencies: [mathdep, gtkdep], export_dynamic: true) +executable('turtle', sourcefiles, resources, turtleparser, turtlelexer, turtleparser[1], dependencies: [mathdep, gtkdep], export_dynamic: true, install: true) diff --git a/src/turtle/turtle.lex b/src/turtle/turtle.lex index f182eb7..1edd9e0 100644 --- a/src/turtle/turtle.lex +++ b/src/turtle/turtle.lex @@ -1,4 +1,5 @@ %top{ +#include #include #include "turtle.h" @@ -12,12 +13,11 @@ %option noyywrap -DIGIT [0-9] REAL_NUMBER (0|[1-9][0-9]*)(\.[0-9]+)? IDENTIFIER [a-zA-Z][a-zA-Z0-9]* %% /* rules */ -#.*\n nline++; ncolumn = 1; /* comment */ +#.* ; /* comment. Be careful. Dot symbol (.) matches any character but new line. */ [ ] ncolumn++; \t ncolumn += 8; /* assume that tab is 8 spaces. */ \n nline++; ncolumn = 1; diff --git a/src/turtle/turtle.y b/src/turtle/turtle.y index de21bf4..7ad4538 100644 --- a/src/turtle/turtle.y +++ b/src/turtle/turtle.y @@ -16,10 +16,8 @@ /* error reporting */ static void yyerror (char const *s) { /* for syntax error */ - g_print ("%s\n",s); + g_print ("%s from line %d, column %d to line %d, column %d\n",s, yylloc.first_line, yylloc.first_column, yylloc.last_line, yylloc.last_column); } - static void runtime_error (char *format, ...); - /* Node type */ enum { N_PU, @@ -97,7 +95,7 @@ int yyparse (void); void run (void); - /* node type */ + /* semantic value type */ typedef struct _node_t node_t; struct _node_t { int type; @@ -122,14 +120,13 @@ #define value(n) (n)->content.value /* start of nodes */ - node_t *node_top = NULL; + static node_t *node_top = NULL; /* functions to generate trees */ - node_t *tree1 (int type, node_t *child1, node_t *child2, node_t *child3); - node_t *tree2 (int type, double value); - node_t *tree3 (int type, char *name); + static node_t *tree1 (int type, node_t *child1, node_t *child2, node_t *child3); + static node_t *tree2 (int type, double value); + static node_t *tree3 (int type, char *name); } -%defines %locations %define api.value.type union /* YYSTYPE, the type of semantic values, is union of following types */ /* key words */ @@ -188,11 +185,13 @@ primary_procedure: | TR expression { $$ = tree1 (N_TR, $2, NULL, NULL); } | BC '(' expression ',' expression ',' expression ')' { $$ = tree1 (N_BC, $3, $5, $7); } | FC '(' expression ',' expression ',' expression ')' { $$ = tree1 (N_FC, $3, $5, $7); } + /* assignment */ | ID '=' expression { $$ = tree1 (N_ASSIGN, tree3 (N_ID, $1), $3, NULL); } + /* control flow */ | IF '(' expression ')' '{' primary_procedure_list '}' { $$ = tree1 (N_IF, $3, $6, NULL); } | RT { $$ = tree1 (N_RT, NULL, NULL, NULL); } | RS { $$ = tree1 (N_RS, NULL, NULL, NULL); } - /* procedure call */ + /* user defined procedure call */ | ID '(' ')' { $$ = tree1 (N_procedure_call, tree3 (N_ID, $1), NULL, NULL); } | ID '(' argument_list ')' { $$ = tree1 (N_procedure_call, tree3 (N_ID, $1), $3, NULL); } ; @@ -239,6 +238,9 @@ expression: %% +/* Declaration of the runtime error function */ +static void runtime_error (char *format, ...); + /* Dinamically allocated memories are added to the single list. They will be freed in the finalize function. */ GSList *list = NULL; diff --git a/src/turtle/turtle_doc.md b/src/turtle/turtle_doc.md index 6cbd79d..7ffa36c 100644 --- a/src/turtle/turtle_doc.md +++ b/src/turtle/turtle_doc.md @@ -12,18 +12,19 @@ You need: - gtk4 It is easy to compile the source file of turtle. +If you have installed gtk4 with an option `--prefix=$HOME/local`, put the same option to meson so that you can install `turtle` under the directory `$HOME/local/bin`. The instruction is: ~~~ -$ meson _build +$ meson --prefix=$HOME/local _build $ ninja -C _build +$ ninja -C _build install ~~~ -Then, the executable file _build/turtle is generated. Type the following command then turtle shows the following window. ~~~ -$ _build/turtle +$ turtle ~~~ ![Screenshot just after it's executed](image/turtle1.png) @@ -36,7 +37,7 @@ Write turtle language in the text editor and click on `run` button, then the pro ![Tree](image/turtle_tree.png) If you add the following line in `turtle.h`, then codes to inform the status will also be compiled. -However, the speed will be quite slow because of the output message. +However, the speed will be quite slow because of the output messages. ~~~ # define debug 1 @@ -80,7 +81,7 @@ fd 100 ~~~ The command `tr` is "Turn Right". -The argument is angle with degree. +The argument is angle with degrees. Therefore, `tr 90` means "Turn right by 90 degrees". If you click on `run`button, then two line segment appears. One is vertical and the other is horizontal. @@ -128,6 +129,7 @@ Statements are executed in the order from the top to the end ## Comment and spaces Characters between `#` (hash mark) and `\n` (new line) inclusive are comment. +Characters between `#` and `EOF` (end of file) are also comment. Comments are ignored. ~~~ @@ -139,11 +141,8 @@ tr 120 fd 100 # Now a triangle appears. ~~~ -The comments in the line 1, 2 and 3 are correct. -But there is a bug in the line 6. -The characters after `#` is not recognized as a comment because no new line follows. -You need to press the enter-key and put new line before the end of file. -This is a difficult bug to find. +\ and \ are newline code and end of file respectively. +The comments in the line 1, 2, 3 and 6 are correct syntactically. Spaces (white space, tab and new line) are ignored. They are used only as delimiters. @@ -172,7 +171,7 @@ distance = 100 fd distance ~~~ -A value 100 is assigned tp the variable `distance` in the first line. +A value 100 is assigned to the variable `distance` in the first line. Assignment is a statement. Most of statements begin with commands like `fd`. Assignment is the only exception. @@ -180,14 +179,14 @@ Assignment is the only exception. This program draws a line segment of 100 pixels long. You can use variables in any places in expressions. -There are 8 kinds of calculation available. +There are 8 kinds of calculations available. - addition: x + y - subtraction: x - y - multiplication: x * y - division: x / y - unary minus: - x -- logical equal: x = y This symbol `=` works as `==` in C language. +- logical equal: x = y. This symbol `=` works as `==` in C language. - greater than: x > y - less than: x < y @@ -262,9 +261,9 @@ a () This is a correct program. -- 1: Define a procedure `a`. +- 1: Defines a procedure `a`. A variable `a` is in its body. -- 2: Assign 100 to a variable `a`. +- 2: Assigns 100 to a variable `a`. - 3: Procedure `a` is called. However, using the same name to a procedure and variable makes confusing. @@ -315,7 +314,7 @@ It is the first stage. The second stage adds two shorter line segments at the endpoint of the original segment. The new segment has 70 percent length to the original segment and the orientation is +30 or -30 degrees different. The third stage adds two shorter line segments to the second stage line segments. -And repeat this several times. +And repeats this several times. This repeating is programmed by recursive call. Two more examples are shown here. @@ -381,7 +380,7 @@ Comments and spaces: These characters are used to separate tokens explicitly. They doesn't have any syntactic meaning and are ignored by the parser. -## Syntax +## Grammar ~~~ program: diff --git a/test/test_lib_mktbl.rb b/test/test_lib_mktbl.rb new file mode 100644 index 0000000..a51f047 --- /dev/null +++ b/test/test_lib_mktbl.rb @@ -0,0 +1,4 @@ +require_relative '../lib/lib_mktbl.rb' + +test + diff --git a/test/test_lib_src2md.rb b/test/test_lib_src2md.rb index 9d1d9f6..7273a16 100644 --- a/test/test_lib_src2md.rb +++ b/test/test_lib_src2md.rb @@ -147,7 +147,7 @@ Otherwise (latex) it remains. ![Screenshot of the box](../../image/screenshot_lb4.png){width=6.3cm height=5.325cm} EOS -sample_md = <