Mar 25, 2011

Source code generation with Template Toolkit

Source code generation is very powerful and saved me from wasting hours and hours to change a recurrent string in my code.

It is not always the best solution, there is also inheritance, but, in my case I have an abstract class, called PNI::Node, that will be the model for 90% of my modules. By now I have no more than 50 PNI nodes, but there will be more and more nodes and I can't open every .pm file to change, for example, a licence in the pod section.

This is a very simple implementation, for sure you can find a better way to do what you need : ok, let's go on!

For instance, I'm writing a CPAN dist called PNI, using ExtUtils::MakeMaker .

My template processing system is the Template Toolkit that is really great and has a lot of feature.

I just used the most common ones with this naming convention:

    path/to/my/file.ext.tt2 is the template for path/to/my/file.ext

so there can be a path/to/some/other/template.tt2 that don't have an associated output file and can be used in some INCLUDE, PROCESS or WRAPPER directive, and I still can have a path/to/other/file.ext that will not be processed .

All paths are relative to my CPAN dist dir, for example, lib/PNI/ is the template for lib/PNI/ . I created a script called process_templates in my CPAN dist dir and must be launched from there . Since I'm writing Perl code, the script uses Perl::Tidy to indent modules and tests .

Don't forget to add a MANIFEST.SKIP file containg the following lines


so when you will make your dist you will not include all that stuff .

Here is the code for the script, I included it just as an example ...

it just works (for me :-) !

use strict;
use warnings;
use File::Find;
use Template;
use Perl::Tidy;

my $template_config = {
    STRICT => 1,
    DEBUG => 1,
    TRIM => 1

my $template = Template->new($template_config);

sub find_templates {
    my @dirs = @_;
    my @template_files;
            wanted => sub {
                return unless /\.tt2$/;
                push @template_files, $File::Find::name;
    return @template_files;

for my $template_path ( find_templates(qw(lib t)) ) {

    print $template_path , "\n";

    open my ($in), '<', $template_path;
    my @template_content_rows = <$in>;
    chomp @template_content_rows;
    my $template_content = join "\n", @template_content_rows;

    # naming convention: /path/to/my/file.ext.tt2
    # is template file for /path/to/my/file.ext

    my $output_path = $template_path;
    $output_path =~ s/\.tt2$//;

    if ( -e $output_path ) {

        print $output_path, "\n";
        my $output_content = '';    # cannot reference undef
        # process templates
            \$template_content, {}, \$output_content,
            { binmode => 1 }
          or warn $template->error
          # but if there is an error don't commit changes
          and next;

        # clean ^M chars ... i'm on Windows
        my $m_char = chr(13);
        $output_content =~ s/$m_char//g;
        # if it is a module or a test, tidy it
        if ( $output_path =~ /\.(pm|t)/ ) {
            my $output_content_tidy = '';
                source => \$output_content,
                destination => \$output_content_tidy,
                argv => []

            # replace content with its tidy version
            $output_content = $output_content_tidy;

        open my ($out), '>', $output_path;
        print $out $output_content;
        close $out;
    close $in;


It is just a starting point, and I know I'm reinventing the wheel for sure . There are a lot of Perl-men that use something like this, even more sophisticated .

I had to thank leader Marcos Rebelo,
obrigado !
who shared with me his solution ( which also uses JSON to store metadata ) .

No comments:

Post a Comment