Should I aggressively release memory while reading a file, line by line? An example:
while (<FILE>) {
my $line = $_;
<process line>
undef($line);
}
"Undefing" the $line variable is a good option for reducing memory consumption?
views:
173answers:
3
+10
A:
That should make no difference as the variable will be released as soon as the next loop iteration starts, whether you explicitly undef it or not.
Nate C-K
2009-12-05 23:39:51
not quite true; the variable's string buffer will remain allocated and be reused (or expanded if needed) the next iteration if it isn't undef'd. Most of the time this will greatly reduce the time spent allocating memory.
ysth
2009-12-06 01:49:39
This is flat wrong. (ysth is being gentle.) Memory used by lexical (`my`) variables isn't released until the program ends. It will be reused within it's block, though. Using `undef` on global variables can allow the memory to be reused in other parts of the program, but probably isn't worth it unless you have large global data stores. In general, memory allocated to a program can't be returned to the OS.
Michael Carman
2009-12-06 02:36:23
Michael: Then you want undef($var) instead of $var = undef!
tsee
2009-12-06 10:03:06
@tsee: From what I've gathered from people with knowledge of the internals that only works for package variables, not lexical ones.
Michael Carman
2009-12-06 14:46:32
Umm. I'm not convinced. Either your sources are wrong or we're misunderstanding each other. Cf. the discussion that ensued from http://www.nntp.perl.org/group/perl.perl5.porters/2009/11/msg153267.html (e.g. http://www.nntp.perl.org/group/perl.perl5.porters/2009/11/msg154075.html). Hence, it is like I said for lexicals and package variables. You can convince yourself using perl -MDevel::Peek -Mstrict -we '{my $x = "1"x100; Dump $x; $x = undef; Dump $x; undef($x); Dump $x}' and the equivalent for package vars.
tsee
2009-12-07 10:46:12
My source is an old (and possibly outdated) thread on c.l.p.moderated, and this post in particular: http://groups.google.com/group/comp.lang.perl.moderated/msg/4c967366e3273474. I'm quite happy to be proven wrong, but if I am the FAQ should be updated.
Michael Carman
2009-12-08 15:53:18
Michael: That is indeed old. I'm confident Ilya was right at the time, but perl's probably changed in this regard. Regarding updating the FAQ: you know where to find brian :)
tsee
2009-12-08 20:10:05
Michael Carman is correct as far as I can tell, although in saying that I'm "flat out wrong" I think he was stretching a bit. While it's not correct that the undef here makes no difference, it is correct that each iteration of the loop is not allocating additional variable space on top of the one from the previous loop iteration (unless the newest $line is bigger than the rest). If all ${line}s are of similar size, the undef here will just hurt performance.
Nate C-K
2009-12-12 19:52:05
Here's a more recent link, which also points out that the Perl docs contradict this in some places and should be fixed: http://www.nntp.perl.org/group/perl.perl5.porters/2006/03/msg111095.html (you have to read the whole thread to get the info)
Nate C-K
2009-12-12 19:52:47
+9
A:
No. See perlfaq3 for more on what you should (and shouldn't) do with regards to memory usage in Perl.
Michael Carman
2009-12-05 23:59:47
+5
A:
No. Unless the lines in your file are of wildly different lengths, and some of those lengths are enormous (10 megabytes and up let's say), why even worry about it?
Is there evidence that loop is consuming too much memory?
Schwern
2009-12-06 21:40:12