Mar 25, 2011

Source code generation with Template Toolkit

Source code generation is very powerful and saved me from wasting hours and hours to change a recurrent string in my code.

It is not always the best solution, there is also inheritance, but, in my case I have an abstract class, called PNI::Node, that will be the model for 90% of my modules. By now I have no more than 50 PNI nodes, but there will be more and more nodes and I can't open every .pm file to change, for example, a licence in the pod section.

This is a very simple implementation, for sure you can find a better way to do what you need : ok, let's go on!

For instance, I'm writing a CPAN dist called PNI, using ExtUtils::MakeMaker .

My template processing system is the Template Toolkit that is really great and has a lot of feature.

I just used the most common ones with this naming convention:

    path/to/my/file.ext.tt2 is the template for path/to/my/file.ext

so there can be a path/to/some/other/template.tt2 that don't have an associated output file and can be used in some INCLUDE, PROCESS or WRAPPER directive, and I still can have a path/to/other/file.ext that will not be processed .

All paths are relative to my CPAN dist dir, for example, lib/PNI/Node.pm.tt2 is the template for lib/PNI/Node.pm . I created a script called process_templates in my CPAN dist dir and must be launched from there . Since I'm writing Perl code, the script uses Perl::Tidy to indent modules and tests .

Don't forget to add a MANIFEST.SKIP file containg the following lines

^process_templates$
\.tt2$

so when you will make your dist you will not include all that stuff .

Here is the code for the script, I included it just as an example ...

it just works (for me :-) !

use strict;
use warnings;
use File::Find;
use Template;
use Perl::Tidy;

my $template_config = {
    STRICT => 1,
    DEBUG => 1,
    TRIM => 1
};

my $template = Template->new($template_config);

sub find_templates {
    my @dirs = @_;
    my @template_files;
    find(
        {
            wanted => sub {
                return unless /\.tt2$/;
                push @template_files, $File::Find::name;
              }
        },
        @dirs
    );
    return @template_files;
}

for my $template_path ( find_templates(qw(lib t)) ) {

    print $template_path , "\n";

    open my ($in), '<', $template_path;
    my @template_content_rows = <$in>;
    chomp @template_content_rows;
    my $template_content = join "\n", @template_content_rows;

    # naming convention: /path/to/my/file.ext.tt2
    # is template file for /path/to/my/file.ext

    my $output_path = $template_path;
    $output_path =~ s/\.tt2$//;

    if ( -e $output_path ) {

        print $output_path, "\n";
        my $output_content = '';    # cannot reference undef
        # process templates
        $template->process(
            \$template_content, {}, \$output_content,
            { binmode => 1 }
          )
          or warn $template->error
          # but if there is an error don't commit changes
          and next;

        # clean ^M chars ... i'm on Windows
        my $m_char = chr(13);
        $output_content =~ s/$m_char//g;
        # if it is a module or a test, tidy it
        if ( $output_path =~ /\.(pm|t)/ ) {
            my $output_content_tidy = '';
            perltidy(
                source => \$output_content,
                destination => \$output_content_tidy,
                argv => []
            );

            # replace content with its tidy version
            $output_content = $output_content_tidy;
        }

        open my ($out), '>', $output_path;
        print $out $output_content;
        close $out;
    }
    close $in;
}

Conclusion

It is just a starting point, and I know I'm reinventing the wheel for sure . There are a lot of Perl-men that use something like this, even more sophisticated .

I had to thank milan.pm leader Marcos Rebelo,
obrigado !
who shared with me his solution ( which also uses JSON to store metadata ) .

Mar 19, 2011

Deploy a Perl application on Windows

Suppose you have a Perl application for an end user but

  • probably your user don't know Perl exists
  • your user uses Windows
  • your user want just to click on a .exe to launch your program

What you will read it worked for me, feel free to improve it, adapt it to fit your needs and/or automate it as you like.
The only requirement needed is Strawberry Perl
Let's define some names so I can explain with my example how to deploy your Perl program on Windows: my program is called pni.exe, has a red mushroom icon,



and it is based on a CPAN module I wrote, called PNI::GUI::Tk.

PNI stands for Perl Node Interface, and PNI::GUI::Tk is a Tk based GUI for PNI with a pniguitk.pl script to launch it. It is really similar to a so called App, except that it is not under the App:: namespace.

Prepare to build you Perl app

First of all, get a Windows machine, ( this is the most difficult part for a lot of people that is allergic to Microsoft :-) then install the Strawberry Perl distribution, it is a really good way to feel unix-like on Windows.

Now, type
cpan PNI::GUI::Tk
to install the module and its dependencies that are Tk, PNI::Node::Tk and the PNI core module.

Now create a folder, called for instance PerlNodeInterface-windows-0.11, put the pniguitk.pl script inside, a README.txt file to be kind ( don't forget the .txt extension, since you are on Windows ) and a lib folder containing all dependencies. This is how it looks like, see below how to compile pni.exe.


There is a pniguitk.pl which launches the app:

#! env perl
use strict;
use warnings;
use PNI::GUI::Tk::App;
my $app = PNI::GUI::Tk::App->new;
$app->run;

Don't forget the auto folder for Tk since it is an XS module.

The dir tree it looks something like this:
lib
lib\auto\Tk\*
lib\PNI.pm
lib\PNI\*
lib\Tk.pm
lib\Tk\*
pni.exe
pniguitk.pl
README.txt

Everything was there, from the beginning

Yes, but you know, most of the people in the world never typed
perldoc ExtUtils::Embed
I started from there, and I found all the informations to compile pni.exe ... so here we go!
perl -MExtUtils::Embed -e xsinit
will produce perlxsi.c, rename it pni.c and add a main function, like the interp.c from
perldoc perlembed
Then change the arguments in the main function and hardcode a -Ilib and the name of the script .

Something like this :

//argv[0] stores the relative path of the executable
argv[1] = "-Ilib";
//argv[2] is for programfile
argv[2] = "pniguitk.pl";

 Get a cool mushroom icon :-) and create a text file pni.rc containing this line
ID ICON "pni.ico"
then launch
windres pni.rc -o coff -o pnirc.o
Now you can compile pni.exe with gcc using options
-Wall -mwindows -o pni pnirc.o pni.c
and options coming from the output of
perl -MExtUtils::Embed -e ccopts
and
perl -MExtUtils::Embed -e ldopts

You can automate compilation with few lines of code:

use ExtUtils::Embed;
use PNI;
print "\nBuilding PNI $PNI::VERSION\n";
my $gcc_cmd = join( ' ' , 'gcc -Wall -mwindows -o pni pnirc.o pni.c' , &ccopts , &ldopts );
print STDOUT $gcc_cmd , "\n";
system( $gcc_cmd );

Conclusion

Finally zip your folder, since zip is supported natively on Windows, and you will get a PerlNodeInterface-windows-0.11.zip ready for an user-friendly deploy.
If this procedure works and make sense for most of the perlish folks, Trawberry Perl could become a sort of JRE for Perl on Windows.
There are a lot of users out there that need our Perl programs, but they don' t need all the complexities behind.