What dynamic programming features of Perl should I be using?

Question

I am pretty new to scipting languages (Perl in particular), and most of the code I write is an unconscious effort to convert C code to Perl.

Reading about Perl, one of the things that is often mentioned as the biggest difference is that Perl is a dynamic language. So, it can do stuff at runtime that the other languages (static ones) can only do at compiletime, and so be better at it because it can have access to realtime information.

All that is okay, but what specific features should I, with some experience in C and C++, keep in mind while writing code in Perl to use all the dynamic programming features that it has, to produce some awesome code?

Answer 1

At least IME, the "dynamic" nature isn't really that big of a deal. I think the biggest difference you need to take into account is that in C or C++, you're mostly accustomed to there being only a fairly minor advantage to using library code. What's in the library is already written and debugged, so it's convenient, but if push comes to shove you can generally do pretty much the same thing on your own. For efficiency, it's mostly a question of whether your ability to write something a bit more specialized outweighs the library author's ability to spend more time on polishing each routine. There's little enough difference, however, that unless a library routine really does what you want, you may be better off writing your own.

With Perl, that's no longer true. Much of what's in the (huge, compared to C) library is actually written in C. Attempting to write anything very similar at all on your own (unless you write a C module, of course) will almost inevitably come out quite a bit slower. As such, if you can find a library routine that does even sort of close to what you want, you're probably better off using it. Using pre-written library code is much more important than in C or C++.

Answer 2

I think the biggest hurdle will not be the dynamic aspect but the 'batteries included' aspect.

I think the most powerful aspects of perl are

• hashes : they allow you to easily express very effective datastructures
• regular expressions : they're really well integrated.
• the use of the default variables like $_
• the libraries and the CPAN for whatever is not installed standard

Something I noticed with C converts is the over use of for loops. Many can be removed using grep and map

Another motto of perl is "there is more than one way to do it". In order to climb the learning curve you have to tell yourself often : "There has got to be a better way of doing this, I cannot be the first one wanting to do ...". Then you can typically turn to google and the CPAN with its riduculous number of libraries.

The learning curve of perl is not steep, but it is very long... take your time and enjoy the ride.

Answer 3

Good programming practices arent specific to individual languages. They are valid across all languages. In the long run, you may find it best not to rely on tricks possible in dynamic languages (for example, functions that can return either integer or text values) as it makes the code harder to maintain and quickly understand. So ultimately, to answer your question, I dont think you should be looking for features specific to dynamicly typed languages unless you have some compelling reason that you need them. Keep things simple and easy to maintain - that will be far more valuable in the long run.

Answer 4

This question is more than enough to fill a book. In fact, that's precisely what happened!

Mark Jason Dominus' excellent Higher-Order Perl is available online for free.

Here is a quote from it's preface that really grabbed me by the throat when I first read the book:

Around 1993 I started reading books about Lisp, and I discovered something important: Perl is much more like Lisp than it is like C. If you pick up a good book about Lisp, there will be a section that describes Lisp’s good features. For example, the book Paradigms of Artificial Intelligence Programming, by Peter Norvig, includes a section titled What Makes Lisp Different? that describes seven features of Lisp. Perl shares six of these features; C shares none of them. These are big, important features, features like first-class functions, dynamic access to the symbol table, and automatic storage management.

Answer 5

A list of C habits not to carry over into Perl 5:

• Don't declare your variables at the top of the program/function. Declare them as they are needed.
• Don't assign empty lists to arrays and hashes when declaring them (they are empty already and need to be initialized).
• Don't use if (!(complex logical statement)) {}, that is what unless is for.
• Don't use goto to break deeply nested loops, next, last, and redo all take a loop label as an argument.
• Don't use global variables (this is a general rule even for C, but I have found a lot of C people like to use global variables).
• Don't create a function where a closure will do (callbacks in particular). See perldoc perlsub and perldoc perlref for more information.
• Don't use in/out returns, return multiple values instead.

Things to do in Perl 5:

• Always use the strict and warnings pragmas.
• Read the documentation (perldoc perl and perldoc -f function_name).
• Use hashes the way you used structs in C.

Answer 6

There are many things you can do only with dynamic language but the coolest one is eval. See here for more detail.

With eval, you can execute string as if it was a pre-written command. You can also access variable by name at runtime.

For example,

$Double = "print (\$Ord1 * 2);";

$Opd1 = 8;
eval $Double;  # Prints 8*2 =>16.

$Opd1 = 7;
eval $Double;  # Prints 7*2 =>14.

The variable $Double is a string but we can execute it as it is a regular statement. This cannot be done in C/C++.

The cool thing is that a string can be manipulated at run time; therefore, we can create a command at runtime.

# string concatenation of operand and operator is done before eval (calculate) and then print.
$Cmd = "print (eval (\"(\$Ord1 \".\$Opr.\" \$Ord2)\"));";

$Opr  = "*";
$Ord1 = "5";
$Ord1 = "2";

eval $Cmd;  # Prints 5*2 => 10.

$Ord1 = 3;
eval $Cmd;  # Prints 5*3 => 15.

$Opr = "+";
eval $Cmd;  # Prints 5+3 => 8.

eval is very powerful so (as in Spiderman) power comes with responsibility. Use it wisely.

Hope this helps.

Answer 7

Use the features that solve your problem with the best combination of maintainability, developer time, testability, and flexibility. Talking about any technique, style, or library outside of the context of a particular application isn't very useful.

Your goal shouldn't be to find problems for your solutions. Learn a bit more Perl than you plan on using immediately (and keep learning). One day you'll come across a problem and think "I remember something that might help with this".

You might want to see some of these book, however:

• Higher-Order Perl
• Mastering Perl
• Effective Perl Programming

I recommend that you slowly and gradually introduce new concepts into your coding. Perl is designed so that you don't have to know a lot to get started, but you can improve your code as you learn more. Trying to grasp lots of new features all at once usually gets you in trouble in other ways.

Answer 8

Two points.

First, In general, I think you should be asking yourself 2 slightly different questions:

1) Which dynamic programming features of Perl can be used in which situations/to solve which problems?

2) What are the trade-offs, pitfalls and downsides of each feature.

Then the answer to your question becomes extremely obvious: you should be using the features that solve your problem better (performance or code maintainability wise) than a comparable non-DP solution, and which incurs less than the maximum accaptable level of downsides.

As an example, to quote from FM's comment, string form of eval has some fairly nasty downsides; but it MIGHT in certain cases be an extremely elegant solution which is orders of magnitude better than any alternate DP or SP approach.

---

Second, please be aware that a lot of "dynamic programming" features of Perl are actually packaged for you into extremely useful modules that you might not even recognize as being of the DP nature.

I'll have to think of a set of good examples, but one that immediately springs to mind is Text template modules, many of which are implemented using the above-mentioned string form of eval; or Try::Tiny exception mechanism which uses block form of eval.

Another example is aspect programming which can be achieved via Moose (I can't find the relevant StackOverflow link now - if someone has it please edit in the link) - which underneath uses access to symbol table featrue of DP.

Answer 9

Most of the other comments are complete here and I won't repeat them. I will focus on my personal bias about excessive or not enough use of language idioms in the language you are writing code in. As a quip goes, it is possible to write C in any language. It is also possible to write unreadable code in any language.

I was trained in C and C++ in college and picked up Perl later. Perl is fabulous for quick solutions and some really long life solutions. I built a company on Perl and Oracle solving logistics solutions for the DoD with about 100 active programmers. I also have some experience in managing the habits of other Perl programmers new and old. (I was the founder / ceo and not in technical management directly however...)

I can only comment on my transition to a Perl programmer and what I saw at my company. Many of our engineers shared my background of primarily being C / C++ programers by training and Perl programmers by choice.

The first issue I have seen (and had myself) is writing code that is so idiomatic that it is unreadable, unmaintainable, and unusable after a short period of time. Perl, and C++ share the ability to write terse code that is entertaining to understand at the moment but you will forget, not be around, and others won't get it.

We hired (and fired) many programmers over the 5 years I had the company. A common Interview Question was the following: Write a short Perl program that will print all the odd numbers between 1 and 50 inclusive separated by a space between each number and terminated with a CR. Do not use comments. They could do this on their own time of a few minutes and could do it on a computer to proof the output.

After they wrote the script and explained it, we would then ask them to modify it to print only the evens, (in front of the interviewer), then have a pattern of results based on every single digit even, every odd, except every seventh and 11th as an example. Another potential mod would be every even in this range, odd in that range, and no primes, etc. The purpose was to see if their original small script withstood being modified, debugged, and discussed by others and whether they thought in advance that the spec may change.

While the test did not say 'in a single line' many took the challenge to make it a single terse line and with the cost of readability. Others made a full module that just took too long given the simple spec. Our company needed to delver solid code very quickly; so that is why we used Perl. We needed programmers that thought the same way.

The following submitted code snippets all do exactly the same thing:

1) Too C like, but very easy to modify. Because of the C style 3 argument for loop it takes more bug prone modifications to get alternate cycles. Easy to debug, and a common submission. Any programmer in almost any language would understand this. Nothing particularly wrong with this, but not killer:

for($i=1; $i<=50; $i+=2) {
    printf("%d ", $i);
} 
print "\n";

2) Very Perl like, easy to get evens, easy (with a subroutine) to get other cycles or patterns, easy to understand:

print  join(' ',(grep { $_ % 2 } (1..50))), "\n"; #original
print  join(' ',(grep { !($_ % 2) } (1..50))), "\n"; #even
print  join(' ',(grep { suba($_) } (1..50))), "\n"; #other pattern

3) Too idiomatic, getting a little weird, why does it get spaces between the results? Interviewee made mistake in getting evens. Harder to debug or read:

print "@{[grep{$_%2}(1..50)]}\n";   #original
print "@{[grep{$_%2+1}(1..50)]}\n"; #even - WRONG!!!
print "@{[grep{~$_%2}(1..50)]}\n"; #second try for even

4) Clever! But also too idiomatic. Have to think about what happens to the annon hash created from a range operator list and why that creates odds and evens. Impossible to modify to another pattern:

print "$_ " for (sort {$a<=>$b} keys %{{1..50}}), "\n"; #orig
print "$_ " for (sort {$a<=>$b} keys %{{2..50}}), "\n"; #even
print "$_ " for (sort {$a<=>$b} values %{{1..50}}), "\n"; #even alt

5) Kinda C like again but a solid framework. Easy to modify beyond even/odd. Very readable:

for (1..50) { 
    print "$_ " if ($_%2); 
    }              #odd
print "\n";

for (1..50) { 
    print "$_ " unless ($_%2); 
    } #even
print "\n";

6) Perhaps my favorite answer. Very Perl like yet readable (to me anyway) and step-wise in formation and right to left in flow. The list is on the right and can be changed, the processing is immediately to the left, formatting again to the left, final operation of 'print' on the far left.

print map { "$_ " } grep { $_ & 1 } 1..50;  #original
print "\n";
print map { "$_ " } grep { !($_ & 1) } 1..50;  #even
print "\n";
print map { "$_ " } grep { suba($_) } 1..50;  #other
print "\n";

7) This is my least favorite credible answer. Neither C nor Perl, impossible to modify without gutting the loop, mostly showing the applicant knew Perl array syntax. He wanted to have a case statement really badly...

for (1..50) { 
    if ($_ & 1) { 
        $odd[++$#odd]="$_ ";
        next;
    } else {    
        push @even, "$_ ";
    }
}   
print @odd, "\n";
print @even;

Interviewees with answers 5, 6, 2 and 1 got jobs and did well. Answers 7,3,4 did not get hired.

Your question was about using dynamic constructs like eval or others that you cannot do in a purely compiled language such as C. This last example is "dynamic" with the eval in the regex but truly poor style:

$t='D ' x 25;
$i=-1;
$t=~s/D/$i+=2/eg;
print "$t\n";     # don't let the door hit you on the way out...

Many will tell you "don't write C in Perl." I think this is only partially true. The error and mistake is to rigidly write new Perl code in C style even when there are so many more expressive forms in Perl. Use those. And yes, don't write NEW Perl code in C style because C syntax and idiom is all you know. (bad dog -- no biscuit)

Don't write dynamic code in Perl just because you can. There are certain algorithms that you will run across that you will say 'I don't quite know how I would write THAT in C' and many of these use eval. You can write a Perl regex to parse many things (XML, HTML, etc) using recursion or eval in the regex, but you should not do that. Use a parser just like you would in C. There are certain algorithms though that eval is a gift. Larry Wall's file name fixer rename would take a lot more C code to replicate, no? There are many other examples.

Don't rigidly avoid C stye either. The C 3 argument form of a for loop may be the perfect fit to certain algorithms. Also, remember why you are using Perl: assumably for high programmer productivity. If I have a completely debugged piece of C code that does exactly what I want and I need that in Perl, I just rewrite the silly thing C style in Perl! That is one of the strengths of the language (but also its weakness for larger or team projects where individual coding styles may vary and make the overall code difficult to follow.)

By far the killer verbal response to this interview question (from the applicant who wrote answer 6) was: This single line of code fits the spec and can easily be modified. However, there are many other ways to write this. The right way depends on the style of the surrounding code, how it will be called, performance considerations, and if the output format may change. Dude! When can you start?? (He ended up in management BTW.)

I think that attitude also applies to your question.