Toxic – A General Purpose Template Engine

The following is a template for a Java class with properties. Comment lines like // >>> ph-name <<< represent a placeholder named ph-name where the generator code can place some dynamic content – something that is computed by the generator. Pairs of comment lines starting with // >>> tmpl-name >>> and ending at // <<< tmpl-name <<< are named sub-templates. Sub-templates are not part of their containing templates but can be placed there for convenience. This way the whole template file can resemble a real Java code file.

E.g. the sub-template property contains line 8–10 and 27–28, where line 10 is a placeholder named setter. One can use the sub-templates pojo or bean to generate the content for setter.

In this example, the lines of the template are not taken literally from the template but are parsed by the TextTemplateParser with the '`' character (backtick) to mark placeholders (see Text Templates). I.e. line 9 contains three placeholders type, Name and name which will be replaced by the code generator. Note: Name and name are used to reflect Java naming conventions.

 1:  // >>> copyright <<<
 2:  // >>> package <<<
 3:  
 4:  // >>> import <<<
 5:  
 6:  public class `classname` {
 7:  // >>> property >>>
 8:     
 9:     public `type` get`Name`() { return `name`; }
10:     // >>> setter <<<
11:     // >>> pojo >>>
12:     
13:     public void set`Name`( `type` value )
14:     {
15:        this.`name` = value;
16:     }
17:     // <<< pojo <<<
18:     // >>> bean >>>
19:  
20:     public void set`Name`( `type` value )
21:     {
22:        final `type` old = get`Name`();
23:        this.`name` = value;
24:        pcs.firePropertyChange( "`name`", old, this.`name` );
25:     }
26:     // <<< bean <<<
27:  
28:     private `type` `name``init`;
29:  // <<< property <<<
30:  // >>> bean >>>
31:     
32:     private final PropertyChangeSupport pcs = new PropertyChangeSupport( this ); 
33:  // <<< bean <<<
34:  // >>> classbody <<<
35:  }

An now, here is the code to generate a Java class from that template. We will start with the plain POJO class:

 1:  Template tClass = bups.get( tp, UTF8, "class" );
 2:  Bount bClass = new Bount( tClass );
 3:  bClass.bind( "copyright", "// Copyright © 2012 Marcus Perlick" );
 4:  bClass.bind( "package", "package toxic.example;" );
 5:  bClass.bind( "import", "import java.io.File;" );
 6:  bClass.bind( "classname", "SomeNewClass" );
 7:  
 8:  Template tProp = bups.get( tp, UTF8, "class", "property" );
 9:  Template tSetter = bups.get( tp, UTF8, "class", "property", "pojo" );
10:  Bount bProp = new Bount( tProp );
11:  bProp.bind( "setter", new Bount( tSetter ) );
12:  tProp = bProp.fix();
13:  bProp = new Bount( tProp );
14:  bProp.bind( "name", "foo" );
15:  bProp.bind( "Name", "Foo" );
16:  bProp.bind( "type", "String" );
17:  bProp.bind( "init", " = \"\"" );
18:  
19:  bClass.bind( "classbody", bProp );
20:  bClass.write( System.out );

In line 1 and 2 we use a TemplateSet to get the class template from the file, which is to top-level template content. Templates are designed to be reused with different content for the placeholders. So we create a Bount (bound template) object that is used to bind content to the templates placeholders. In line 3–6 we bind different strings as content to all placeholders but classbody.

The interesting this is, that a template that has all its placeholders bound to some content can not only be written as output but also is content itself. So we will use the property sub-template to generate a property into the class body:

In line 8 and 9 we load the template property and its sub-template pojo to generate a property with a simple POJO setter. In line 10 we create another bound template bProp where we bind the setter placeholder to a bound object for the POJO setter template. Note that the POJO setter has unbound placeholders and that we don't keep its Bount object. Instead we call the fix() method of the bound property template which creates us a new flattened template that has all the placeholders that were not yet bound. I.e. after line 12 tProp holds a template like this:

public `type` get`Name`() { return `name`; }

public void set`Name`( `type` value )
{
   this.`name` = value;
}

private `type` `name``init`;

In line 13–17 we bind some reasonable strings to the still unbound placeholders and the output is:

// Copyright © 2012 Marcus Perlick
package toxic.example;

import java.io.File;

public class SomeNewClass {
   
   public String getFoo() { return foo; }
   
   public void setFoo( String value )
   {
      this.foo = value;
   }

   private String foo = "";
}

To use just the bean setter from our original template we have to replace some lines in our generator code:

 1:  Template tProp = bups.get( tp, UTF8, "class", "property" );
 2:  Template tSetter = bups.get( tp, UTF8, "class", "property", "bean" );
 3:  Bount bProp = new Bount( tProp );
 4:  bProp.bind( "setter", new Bount( tSetter ) );
 5:  tProp = bProp.fix();
 6:  bProp = new Bount( tProp );
 7:  bProp.bind( "name", "foo" );
 8:  bProp.bind( "Name", "Foo" );
 9:  bProp.bind( "type", "String" );
10:  bProp.bind( "init", " = \"\"" );
11:  Bount bBean = new Bount( bups.get( tp, UTF8, "class", "bean" ) );
12:  
13:  bClass.bind( "classbody", concat( bProp, bBean ) );
14:  bClass.write( System.out );
15:  System.out.flush();

But then we would get the result:

// Copyright © 2012 Marcus Perlick
package toxic.example;

import java.io.File;

public class SomeNewClass {
   
   public String getFoo() { return foo; }

   public void setFoo( String value )
   {
      final String old = getFoo();
      this.foo = value;
      pcs.firePropertyChange( "foo", old, this.foo );
   }

   private String foo = "";   
   private final PropertyChangeSupport pcs = new PropertyChangeSupport( this ); 
}

XML templates are inspired by web frameworks like Apache Wicket or Scala's Lift where templates are normal XML documents peppered with some attributes where the template engine can be hooked in:

 1:  <?xml version="1.0" ?>
 2:  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
 3:            "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 4:  <html xmlns="http://www.w3.org/1999/xhtml"
 5:        xmlns:toxic="http://fractalqb.de/toxic/xml-template">
 6:  <head>
 7:  <title>Beschreibung der Seite</title>
 8:  </head>
 9:  <body>
10:  <table>
11:  <tr><th>Name</th><th>#</th><th>Price</th><th>Total</th></tr>
12:  <tr toxic:replace="body" toxic:template="row">
13:    <td toxic:fill="name">product name</td>
14:    <td toxic:fill="count">number of products</td>
15:    <td toxic:fill="ppu">price per unit</td>
16:    <td toxic:fill="total">total price</td>
17:  </tr>
18:  </table>
19:  </body>
20:  </html>

Loading this template into a browser looks like this. What we want to do with this template is to take the page including the table and its heading from the template and then to add an arbitrary number of rows. To generate the rows we use the one sample row that is also part of the template.

Like in the previous example we have nested templates:

• The main template is the document itself containing lines 1–11, 18–20 and a little of line 12.
• The nested template called row containing lines 12–17.

The attribute toxic:replace in line 12 introduces a placeholder named body into the global template that replaces the complete element of the toxic:replace attribute. I.e. the example row is not part of the global template but instead of it we can fill in some content.

The other attribute toxic:template in line 12 makes a template of the <tr> element that we can use to create table rows to fill the body placeholder in the global template¹.

In line 13–16 we see another kind of placeholder definition, the toxic:fill attribute. In contrast to toxic:replace this one does not discard its complete element but only the child nodes of the element. Also all non-toxic attributes of the elements are preserved. This way we can replace the text content of the <td/> elements with some other content. Replacing attribute content is described in the documentation of XML templates.

To fill in some lines into this table template we use this code:

 1:  public static void main( String[] args ) throws Exception
 2:  {
 3:      TemplateSet tset = mkTSet(); // creates a template set
 4:      XmlTemplateParser xtp = new XmlTemplateParser();
 5:      Template tPage = tset.get( xtp, Charset.forName( "UTF-8" ), "templates", "xml", "table1" );
 6:      Template tRow = tset.get( xtp, Charset.forName( "UTF-8" ), "templates", "xml", "table1", "row" );
 7:      Bount bPage = new Bount( tPage );
 8:      BeanContent<BillPos> bRow = new BeanContent<>( tRow, BillPos.class,
 9:                                                     "article" ,"name",
10:                                                     "number", "count",
11:                                                     "pricePerUnit", "ppu",
12:                                                     "totalAmount", "total" );
13:      bRow.escape( XmlEscAsciiBased.FACTORY, "name" );
14:      ListContent<BillPos> bTable = new ListContent<>( bRow );
15:      bPage.bind( "body", bTable );
16:      List<BillPos> model = new ArrayList<>();
17:      for ( int i = 0; i < 10; i++ )
18:          model.add( new BillPos() );
19:      bTable.link( model );               
20:      bPage.write( System.out );
21:      System.out.flush();
22:  }

Here we use two templates tPage for the HTML document and tRow for a single row in the table. To demonstrate linking between Java objects and templates we use a Java Bean BillPos that has all the data we need to create a single line of the table. In line 8-12 we use a BeanContent object to bind the bean properties article, number, pricePerUnit and totalAmount to the placeholders name, count, ppu and total. To make sure that the strings that go into the name placeholder are escaped for XML output, we add an escape object to that placeholder. One could also escape the other placeholders but we are sure that the numbers from the bean don't need to. Because we want to generate several table rows we use a List<BillPos> as our model (line 16-18). Such a model can be used as content through a ListContent<> object as shown in line 14, 15 and 19. The xHTML that is produced by this example looks like this.

A more exotic application for a template engine might be to remove SQL strings from your Java code when working with plain JDBC. Let's say you start with a simple DAO class like this:

public class PersonDao
{    
    private DataSource dataSource;

    public Person getPersonById( int id ) throws SQLException
    {
        try ( Connection db = dataSource.getConnection() ) {
            PreparedStatement stmt = db.prepareStatement(
                     "select called, surname "+
                     "from test.person "+
                     "where id = ?" );
            stmt.setInt( 1, id );
            ResultSet ress = stmt.executeQuery();
            if ( ress.next() ) {
                return new Person( ress.getString( 2 ),
                                   ress.getString( 1 ) );
            }
            else {
                return null;
            }
        }
    }
}

Besides the question if it wouldn't be better to use some sophisticated ORM like JPA or JDO or even MyBatis there may be simple applications that only have to read some records from a database and nothing else for which the full featured frameworks would be overkill. If you don't believe that something like that could exist… you are not forces to read on.

However, the problem with code like the example above is that it is not "easy" to maintain. I.e. if one changes the SQL string the position parameters in the stmt.setXXX() and ress.getXXX() calls have to be adopted. Especially for more complex statements this is a tedious and error prone task.

First we create a sql file that will contain all our SQL statements for the PersonDao class. We call it PersonDao.sql and put it into the same folder as the class, so that the DAO class can easily load it as a resource:

-- >>> select-by-id >>>
select :<called:, :<surname:
from :schema:.person
where id = :>id:
-- <<< select-by-id <<<

To parse the SQL file we use the same combination of template parsers as in the Nested Code Templates example. As delimiters for the placeholders in the TextTemplateParser we now use ':' instead of '`'. Placeholder names starting with '>' are our input parameters to the statement, those starting with '<' are output parameters and the rest is not handled by the SqlTool. Note that we use such a placeholder schema to select the schema that holds the table in the DAO not in the template.

But now let's see how the DAO code changes to make use of such a template. Actually it will be more code because we have to initialize some data about the SQl statement:

1:  public class PersonDao
2:  {
3:      private final String sqlSelectById;
4:      private final int[] inSelectById_id;
5:      private final int outSelectById_called;
6:      private final int outSelectById_surname;
7:      
8:      private DataSource dataSource;
9:  ...

First we need some members where we can keep the data.

sqlSelectById

holds the SQL statement for the selectById() method.

inSelectById_id

holds all the indices where we have to set the id parameter.

outSelectById_called

holds the index to get the content of column called.

outSelectById_surname

holds the index to get the content of column surname.

For brevity we do all the initialization in the constructor:

10:  ...
11:      public PersonDao() throws IOException
12:      {
13:          TemplateParser tp = SqlTool.makeTemplateParser();
14:          InputStream is = getClass().getResourceAsStream( getClass().getSimpleName()+".sql" );
15:          Template sqt = tp.read( "select-by-id", is, null, Charset.defaultCharset(), 0, "select-by-id" );
16:          is.close();
17:          Bount sqb = new Bount( sqt );
18:          sqb.bind( "schema", "testdb" );
19:          sqt = sqb.fix();
20:          SqlTool sqlTool = new SqlTool( sqt );
21:          sqlSelectById = sqlTool.getSql();
22:          inSelectById_id = sqlTool.getInIndices( "id" );
23:          outSelectById_called = sqlTool.getOutIndex( "called" );
24:          outSelectById_surname = sqlTool.getOutIndex( "surname" );
25:      }
26:  ...

In line 13 we create a new TemplateParser to load the template. In line 14 we open the template file as input stream. Line 15 reads the template. Line 16–18 do the dynamic SQL thing so that the table is located in the DB schema testdb.

In line 20–24 we use the SqlTool to get the SQL statement and all the indices of the in- and output parameters.

Member	Value
sqlSelectById	select called, surname from testdb.person where id = ?
inSelectById_id	[ 1 ]
outSelectById_called	1
outSelectById_surname	2

With these things at hand we can rewrite the getPersonById() method to something more maintainable:

27:  ...
28:      public Person getPersonById( int id ) throws SQLException
29:      {
30:          try ( Connection db = dataSource.getConnection() ) {
31:              PreparedStatement stmt = db.prepareStatement( sqlSelectById );
32:              SqlTool.bind( stmt, inSelectById_id, id );
33:              ResultSet ress = stmt.executeQuery();
34:              if ( ress.next() ) {
35:                  return new Person( ress.getString( outSelectById_surname ),
36:                                     ress.getString( outSelectById_called ) );
37:              }
38:              else {
39:                  return null;
40:              }
41:          }
42:      }
43:  }

There is nothing very exiting about this except that we

• don't have the SQL string in the code.
• don't have to do tedious position counting of column manes or parameter indices.
• can also use the templates for dynamic SQL.

One thing to mention is line 32 which is nothing else than some convenience function for:

for ( int i : inSelectById_id )
    stmt.setInt( i, id );