===================
Source Code Filters
===================

.. contents::

A `Source Code Filter` transforms the input character stream to an in-memory 
output stream before parsing. A filter can be used to provide templating 
systems or preprocessors.

To use a filter for a source file the *shebang* notation is used::

  #! stdtmpl(subsChar = '$', metaChar = '#')
  #proc generateXML(name, age: string): string =
  #  result = ""
  <xml>
    <name>$name</name>
    <age>$age</age>
  </xml>

As the example shows, passing arguments to a filter can be done
just like an ordinary procedure call with named or positional arguments. The
available parameters depend on the invoked filter.


Pipe operator
=============

Filters can be combined with the ``|`` pipe operator::

  #! strip(startswith="<") | stdtmpl
  #proc generateXML(name, age: string): string =
  #  result = ""
  <xml>
    <name>$name</name>
    <age>$age</age>
  </xml>


Available filters
=================

**Hint:** With ``--verbosity:2`` (or higher) Nim lists the processed code
after each filter application.

Replace filter
--------------

The replace filter replaces substrings in each line.

Parameters and their defaults:

  ``sub: string = ""``
    the substring that is searched for
  
  ``by: string = ""``
    the string the substring is replaced with


Strip filter
------------

The strip filter simply removes leading and trailing whitespace from
each line.

Parameters and their defaults:

  ``startswith: string = ""``
    strip only the lines that start with *startswith* (ignoring leading
    whitespace). If empty every line is stripped.

  ``leading: bool = true``
    strip leading whitespace
  
  ``trailing: bool = true``
    strip trailing whitespace


StdTmpl filter
--------------

The stdtmpl filter provides a simple templating engine for Nim. The
filter uses a line based parser: Lines prefixed with a *meta character*
(default: ``#``) contain Nim code, other lines are verbatim. Because
indentation-based parsing is not suited for a templating engine, control flow
statements need ``end X`` delimiters.

Parameters and their defaults:

  ``metaChar: char = '#'``
    prefix for a line that contains Nim code
  
  ``subsChar: char = '$'``
    prefix for a Nim expression within a template line
    
  ``conc: string = " & "``
    the operation for concatenation
    
  ``emit: string = "result.add"``
    the operation to emit a string literal
    
  ``toString: string = "$"``
    the operation that is applied to each expression

Example::

  #! stdtmpl | standard
  #proc generateHTMLPage(title, currentTab, content: string,
  #                      tabs: openArray[string]): string = 
  #  result = ""
  <head><title>$title</title></head>
  <body>
    <div id="menu">
      <ul>
    #for tab in items(tabs):
      #if currentTab == tab:
      <li><a id="selected" 
      #else:
      <li><a
      #end if
      href="${tab}.html">$tab</a></li>
    #end for
      </ul>
    </div>
    <div id="content">
      $content
      A dollar: $$.
    </div>
  </body>

The filter transforms this into:

.. code-block:: nim
  proc generateHTMLPage(title, currentTab, content: string,
                        tabs: openArray[string]): string = 
    result = ""
    result.add("<head><title>" & $(title) & "</title></head>\n" & 
      "<body>\n" & 
      "  <div id=\"menu\">\n" & 
      "    <ul>\n")
    for tab in items(tabs):
      if currentTab == tab:
        result.add("    <li><a id=\"selected\" \n")
      else:
        result.add("    <li><a\n")
      #end
      result.add("    href=\"" & $(tab) & ".html\">" & $(tab) & "</a></li>\n")
    #end
    result.add("    </ul>\n" & 
      "  </div>\n" & 
      "  <div id=\"content\">\n" & 
      "    " & $(content) & "\n" & 
      "    A dollar: $.\n" & 
      "  </div>\n" & 
      "</body>\n")

  
Each line that does not start with the meta character (ignoring leading
whitespace) is converted to a string literal that is added to ``result``. 

The substitution character introduces a Nim expression *e* within the
string literal. *e* is converted to a string with the *toString* operation
which defaults to ``$``. For strong type checking, set ``toString`` to the
empty string. *e* must match this PEG pattern::

  e <- [a-zA-Z\128-\255][a-zA-Z0-9\128-\255_.]* / '{' x '}'
  x <- '{' x+ '}' / [^}]*

To produce a single substitution character it has to be doubled: ``$$``
produces ``$``.

The template engine is quite flexible. It is easy to produce a procedure that
writes the template code directly to a file::

  #! stdtmpl(emit="f.write") | standard
  #proc writeHTMLPage(f: File, title, currentTab, content: string,
  #                   tabs: openArray[string]) = 
  <head><title>$title</title></head>
  <body>
    <div id="menu">
      <ul>
    #for tab in items(tabs):
      #if currentTab == tab:
      <li><a id="selected" 
      #else:
      <li><a
      #end if
      href="${tab}.html" title = "$title - $tab">$tab</a></li>
    #end for
      </ul>
    </div>
    <div id="content">
      $content
      A dollar: $$.
    </div>
  </body>
