On Linux and Mac OS X, I can run a command from the CLI using the "time" command (not the bash shell built-in command time) and get not only the elapsed run time from start to finish, but many other statistics as well. For example on Mac OS X:
% /usr/bin/time -lp /usr/bin/wc PS1-01.m4v
6166365 46283357 1532034853 PS1-01.m4v
real 27.09
user 24.65
sys 1.17
1433600 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
361 page reclaims
1 page faults
0 swaps
0 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
4 voluntary context switches
21604 involuntary context switches
On Ubuntu 10.4, "/usr/bin/time -v " gives similar but not identical output. It is definitely good enough for my purposes, though.
I can also get similar results on Windows XP + Cygwin. On Cygwin, if the command being measured is a "Cygwin binary", it works as I want. But if the command being measured is not a "Cygwin binary" (e.g. in my case I'm running the Java Virtual Machine from the command line, specifying the program class file to run on the command line), then it seems that the time command is measuring statistics for a "hidden" shell process, not the Windows binary. At least, that would make sense given what I have seen. My belief that it is a shell process that is being measured comes from this email from someone who has investigated the matter carefully several years back, and later in the same email thread a Cygwin developer confirms his findings:
http://cygwin.ru/ml/cygwin/2001-09/msg00202.html
The statistics I am most interested in are what is called the "max resident set size" on Linux and Mac OS X (sometimes abbreviated Max RSS or MRSS), but might be called the "working set size" on Windows. Another I'd like to see is the percentage of time that each CPU core is busy (or idle -- I can do 100-x on my own), with a separate measurement for each core in a multi-core system. It would be nice to also get the user and system CPU times used, which would correspond to breaking down the CPU time into Windows kernel vs. user application run time, but having both of those combined into one number would be OK, too. Note that such numbers could be longer than the elapsed wall clock time on a multi-core system, if the program uses more than one in parallel.
I have seen other mentions on Stack Overflow of timeit.exe and timethis.exe. Those seem to give the elapsed time only, as far as I can tell. There are many tools that I think can give me the current "working set size" of a process, but the only ones I know of so far show that number in a GUI window. It is really only useful to me if I can get the information using a batch method, because I want to do performance testing of many different programs run from the CLI, one at a time from a script.