views:

1405

answers:

8

Does anyone have any suggestions for a good approach to finding all the CPAN dependencies that might have arisen in a bespoke development project. As tends to be the case your local development environment rarely matches your live one and as you build more and more projects you tend to build up a local library of installed modules. These then lead to you not necessarily noticing that your latest project has a requirement on a non-core module. As there is generally a requirement to package the entire project up for deployment to another group (in our case our operations team), it is important to know what modules should be included in the package.

Does anyone have any insights into the problem.

Thanks

Peter

+7  A: 

In the past I have used Devel::Modlist which is reasonably good allowing you to go

perl -d:Modlist script.pl

To get a list of the required modules.

Vagnerr
+8  A: 

You can use online web-service at deps.cpantesters.org that will provide you many useful dependency data. All modules on CPAN already have the link to the dependency site (on the right side of the module page).

+3  A: 

The 'obvious' way - painful but moderately effective - is to install a brand new build of base Perl in some out of the way location (you aren't going to use this in production), and then try to install your module using this 'virgin' version of Perl. You will find all the missing dependencies. The first time, this could be painful. After the first time, you'll already have the majority of the dependencies covered, and it will be vastly less painful.

Consider running your own local repository of CPAN modules - so that you won't always have to download the code. Also consider how you clean up the out of date modules.

Jonathan Leffler
+3  A: 

I have a Make-based build system for all my C/C++ applications (both PC-based and for various embedded projects), and while I love being able to do a top-level build on a fresh machine and verify all dependencies are in place (I check my toolchains in to revision control :D), I've been frustrated at not doing the same for interpreted languages that currently have no makefile in my build system.

I'm tempted to write a script that:

  • searches my revision control repository for files with the .pl or .pm extension
  • runs perl -d:Modlist on them (thanks Vagnerr!)
  • concatenating it to the list of required modules
  • and finally comparing it to the list of installed modules.

I'd then execute that script as part of my top-level build, so that anyone building anything will know if they have everything they need to run every perl script they got from revision control. If there is some perl script they never run and don't want to CPAN install what's required to run it, they'd have to remove the unwanted script from their harddrive, so the dependency checker can't find them. I know how to modify a perforce client to leave out certain subdirectories when you do a 'sync', I'll have to figure that out for subversion...

I'd suggest making the dependency checker a single script that searches for pl files, as opposed to an individual makefile to check dependencies for each script, or based on a hard-coded list of script names. If you choose a method that requires user action to have a script checked for dependencies, people will forget to perform that action, since they will be able to run the script even if they don't do the dependency check.

Like I said, I haven't implemented the above yet, but this question has prompted me to try to do so. I'll post back with my experience after I'm done.

KeyserSoze
+2  A: 

Its a "horse that's bolted" answer but I've got into the habit of creating a Bundle file with all my dependencies. Thus when I go to a new environment I just copy it over and install it.

For eg. I have a Baz.pm

package Bundle::Baz;
$VERSION = '0.1';
1;
__END__
=head1 NAME
Bundle::Baz
=head1 SYNOPSIS
perl -MCPAN -e 'install Bundle::Baz'
=head1 CONTENTS
# Baz's modules
XML::Twig
XML::Writer
Perl6::Say
Moose

Put this in ~/.cpan/Bundle/ (or wherever your .cpan lives) and then install 'Bundle::Baz' like a normal CPAN module. This then installs all the modules listed under "=head1 CONTENTS".

/I3az/

draegtun
+5  A: 

I've had this problem myself. Devel::Modlist (as suggested by this answer) takes a dynamic approach. It reports the modules that were actually loaded during a particular run of your script. This catches modules that are loaded by any means, but it may not catch conditional requirements. That is, if you have code like this:

if ($some_condition) { require Some::Module }

and $some_condition happens to be false, Devel::Modlist will not list Some::Module as a requirement.

I decided to use Module::ExtractUse instead. It does a static analysis, which means that it will always catch Some::Module in the above example. On the other hand, it can't do anything about code like:

my $module = "Other::Module";
eval "use $module;";

Of course, you could use both approaches and then combine the two lists.

Anyway, here's the solution I came up with:

#! /usr/bin/perl
#---------------------------------------------------------------------
# Copyright 2008 Christopher J. Madsen <perl at cjmweb.net>
#
# This program is free software; you can redistribute it and/or modify
# it under the same terms as Perl itself.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See either the
# GNU General Public License or the Artistic License for more details.
#
# Recursively collect dependencies of Perl scripts
#---------------------------------------------------------------------

use strict;
use warnings;
use File::Spec ();
use Module::CoreList ();
use Module::ExtractUse ();

my %need;
my $core = $Module::CoreList::version{'5.008'};

# These modules have lots of dependencies.  I don't need to see them now.
my %noRecurse = map { $_ => 1 } qw(
  Log::Log4perl
  XML::Twig
);

foreach my $file (@ARGV) {
  findDeps($file);
}

foreach my $module (sort keys %need) {
  print "  $module\n";
}

#---------------------------------------------------------------------
sub findDeps
{
  my ($file) = @_;

  my $p = Module::ExtractUse->new;

  $p->extract_use($file);

  foreach my $module ($p->array) {
    next if exists $core->{$module};
    next if $module =~ /^5[._\d]+/; # Ignore "use MIN-PERL-VERSION"
    next if $module =~ /\$/;        # Run-time specified module

    if (++$need{$module} == 1 and not $noRecurse{$module}) {
      my $path = findModule($module);
      if ($path) { findDeps($path) }
      else       { warn "WARNING: Can't find $module\n" }
    } # end if first use of $module
  } # end foreach $module used
} # end findDeps

#---------------------------------------------------------------------
sub findModule
{
  my ($module) = @_;

  $module =~ s!::|\'!/!g;
  $module .= '.pm';

  foreach my $dir (@INC) {
    my $path = File::Spec->catfile($dir, $module);
    return $path if -f $path;
  }

  return;
} # end findModule

You'd run this like:

perl finddeps.pl scriptToCheck.pl otherScriptToCheck.pl

It prints a list of all non-core modules necessary to run the scripts listed. (Unless they do fancy tricks with module loading that prevent Module::ExtractUse from seeing them.)

cjm
+2  A: 
use Acme::Magic::Pony;

Seriously. It will auto-install Perl modules if they turn up missing. See the Acme::Magic::Pony page in CPAN.

JDrago
Interesting but if you try and use that in a live commercial environment you operations team is going to be a little upset with you and want to see if you are aware of the concepts of stability and security :-)
Vagnerr
+1  A: 

Here is a quickie bash function (using the excellent ack):

# find-perl-module-use <directory> (lib/ by default)
function find-perl-module-use() {
    dir=${1:-lib}
    ack '^\s*use\s+.*;\s*$' $dir | awk '{ print $2 }' | sed 's/();\?$\|;$//' | sort | uniq
    ack '^\s*use\s+base\s+.*;\s*$' $dir | awk '{ print $3 }' | sed 's/();\?$\|;$//' | sort | uniq
}
Robert Krimen