views:

666

answers:

6

I'm evaluating Windows PowerShell as a replacement for cmd.exe for basic process automation for command-line code (e.g. setup, execution, and post-processing of large numbers of Fortran jobs.) I know enough perl and shell to achieve what I want but I'm trying to stay within the Windows toolchain rather than needing to train my coworkers on Perl, Python, Ruby, Cygwin, etc. I work at an engineering firm so we have no need for .NET or extensive frameworks for Windows development. In short: We crunch a lot of numbers with in-house code and I need a reasonable means of automating the repetitive manual tasks.

The problem is that I'm having trouble evaluating the language because I can't find a terse printable language reference manual for PowerShell,

Yes, I've looked at Microsoft's site but it's so disorganized and choked with irrelevant information as to be unusable. Google tends to point me back to that site and hasn't been terribly helpful.

Here are some of the topics I'm interested in (unix & Perl analogues noted):

  • Intrinsic functions
    • Basic test/branch/loop functions and flow control
  • Intrinsic types of variables (boolean, integer, floating-point, string)
  • Intrinsic data structures (scalar, list, associative array, object)
  • Operators and operator precedence
  • Functions specific to a task automation language:
    • Process control (exec, fork, wait, kill)
    • System monitoring (ps, uptime, lsof, fuser, netstat, df)
    • File and directory handling (find, locate, which, cd, mkdir, touch, rm, rmdir)
    • I/O (pipes, cat, tee, xargs, read, write, overwrite, append, seek)
    • Text-processing functions (awk, sed, grep, split, join, cut, fmt, strings)

Other questions that I'd expect to be answered:

  • Is PowerShell strongly-typed (char, string, int, long, float, double) or weakly-typed (variant)?
  • Are common data structures available (scalar, list, associative array) and are they first-class elements?
  • Is there a library or module system?
    • If so, what is in the standard library?
    • Are there third-party libraries I can use if I don’t find what I need?
  • Is this a procedural, functional, or object-oriented language – or to what extent each?
  • What are the coding conventions? (Perl Best Practices)
  • Are there formatting expectations – line-oriented or free-form?
  • Is whitespace or case significant?
  • Is there a character set limitation?
    • Is the language ’8-bit clean’?
    • Does it support UTF-8, Unicode, etc.?

I'm very interested in finding the M in RTFM, but note that I'm in the evaluation stage so dropping $50 or so on a tome from Microsoft Press is a non-starter.

A: 

A Quick Search turned up this Powershell Owners Manual (it's an online manual)

But if you want to stick with tools that are already installed on Windows machines, then PowerShell isn't really an option unless you plan to support only Windows7 and Server 2008 and later.

But Windows Script Host is on every version of Windows since Win98, it supports scripts in JScript or VBScript or any mixture of the two.

John Knoeller
I've seen the PowerShell Owner's Manual and it's not terribly helpful. Thanks though.PowerShell is actively developed and supported on XP SP3 forward and it's not much of a bother to install. I'm curious if the advantages of an OO scripting language outweigh the problems. I'll check if the documentation situation for VBScript is any better than that of PowerShell. I'm leery of anything with 'VB' in the title, but it's worth a shot.Also, isn't Windows Script Host a large source of security issues?
arclight
You can use WSH with VBScript, but you can also use it with JScript, personally I prefer JScript. As for security issues. WSH was one of the attack vectors used by some early email viruses because you could trick user into excuting code by handing them a wsh script. Blaming WSH is a bit like blaming .bat or .exe files for security issues.
John Knoeller
+3  A: 

First of all - there is very very useful documentation available from PowerShell commandline. Just type:

PS> help about
Name                         Category  Synopsis
----                         --------  --------
about_Alias                  HelpFile  Using alternate names for cmdlets and commands in Windows PowerShell
about_aliases                HelpFile  Describes how to use alternate names for cmdlets and commands in Windows
about_Arithmetic_Operators   HelpFile  Describes the operators that perform arithmetic in Windows PowerShell.
about_Array                  HelpFile  A compact data structure for storing data elements
about_arrays                 HelpFile  Describes a compact data structure for storing data elements.
about_Assignment_Operators   HelpFile  Describes how to use operators to assign values to variables.
....

It will answer most of your questions.

Note that you can call exe files without any change as you are used to (just call netstat and you will see). So for some of the tasks you can use the tools you know and like.

As for operators precedence I saw only one blog post by Kirk Munro: Essential PowerShell: Know your operator and enclosure precedence

Books (but only for PowerShell V1)

The others questions are very well described in a book by Bruce Payette: PowerShell in Action. It covers V1, but you can acces MEAP with many chapters from V2 version.

Or if you just evaluate, you can have a look at some free books:

For some best practises you may later look at Keith's book: Effective Windows PowerShell

stej
Thanks for the references - those help a lot! The online documentation available via get-Help is voluminous and detailed, but still incredibly disorganized (i.e. you need to know the magic keywords to find what you're looking for.) It's great for experienced people, semi-useless for evaluation purposes. get-Help returns a flat help namespace vs a hierarchal set of topics that get more detailed the further you descend into the tree; keyword search is available but still has the problem of knowing the PowerShell lingo which may not map to typical CS or unix nomenclature.
arclight
+9  A: 

Intrinsic functions

Not functions per-se. Native PowerShell commands are called cmdlets—they are .NET classes, have a common parameter/argument system and play well with the pipeline. You can now do the same with advanced functions directly in the PowerShell language, though.

There's plenty of built-in cmdlets for file I/O, process control, WMI access and shell-specific stuff such as managing variables, aliases, modules, &c.

Furthermore you have the complete .NET framework at your disposal where you can find the various math functions for example and everything else not easily possible with PowerShell alone.

Basic test/branch/loop functions and flow control

The usual C-like statements: if, switch, for, while, do, foreach. Note that curly braces delimit blocks and all those statements expect blocks (not blocks or a single statement as in many other languages). Details can be found with Get-Help about_If (and about_Switch, about_for, about_while, about_do, about_foreach). Especially switch is insanely powerful, allowing for regular expression matching and other niceties which often can serve as a simple makeshift parser.

Intrinsic types of variables (boolean, integer, floating-point, string)

Assign $true or $false to a variable and it is a boolean, assign an integer to a variable and it becomes Int32, Int64, Decimal or Double, depending on how large it is. You won't surprisingly land at −231 because of overflow. Floating point numbers and strings can be similarly constructed with the usual notation. You can always declare an explicit type if you don't like the default choice:

PS> [long]$a = 5  # 5 would usually result in [int], not [long]

Intrinsic data structures (scalar, list, associative array, object)

You can easily have scalars:

PS> $a = "foo"

or lists (well, they are arrays underneath):

PS> $b = 5, 2, "bar"

or hash tables (associative arrays):

PS> $c = @{ 'a' = 5; 7 = 'foo' }

Both arrays and hash tables are by default based on Object. To get stronger type constraints here you need to jump through a few hoops.

Operators and operator precedence

Operators can be found in Get-Help about_Operators.

Arithmetic (see about_arithmetic_operators)

+  -  *  /  %  ++  --

Assignment (see about_assignment operators)

=  +=  -=  *=  /=  %=

Logical (see about_logical_operators)

-and  -or  -xor  -not  !

Pretty straightforward; ! is just another name for -not which may be of help to those rooted in C-like languages.

Bitwise

-band  -bor  -bxor  -bnot

Comparison operators

Since PowerShell is a shell they can't simply use > and < which are reserved for redirection, hence the slightly more cumbersome variants:

-lt  -le  -eq  -ge  -gt  -ne

Others

There are also two flavors of string matching operators: -match for regular expression matching and -like for wildcard matching; their negated variants: -notmatch, -notlike; and their explicitly case-sensitive and case-insensitive cousins (default is case-insensitive): -cmatch, -imatch, -cnotmatch, -inotmatch, -clike, -ilike, -cnotlike, -inotlike. See about_comparison_operators)

Then there are a few more special operators, such as redirection operators (see about_redirection):

>  >>  2>  2>>  2>&1

Split and Join (new in PowerShell v2):

-split  -join

Those can split a string at a regular expression or join an array with a specified delimiter.

The replace operator -replace which also allows for regular expressions.

The format operator -f:

PS> "{0:N3} {1} {2:yyyy-MM-dd}" -f 123.456789,"foo",(Get-Date)
123,457 foo 2010-02-25

Range operator ..:

PS> 1..10 -join ", "
1, 2, 3, 4, 5, 6, 7, 8, 9, 10

As well as a bunch of others, such as the call operator &, dot sourcing with ., subexpressions with $( ), array subexpressions with @( ) and the array construction operator ,. This is getting out of hand here, though, so look these up yourself.

As for precedence, I doubt that's something you really need to know. If in doubt, use parentheses. The arithmetic operators use the usual rule (mul/div before add/sub). I haven't had much surprises with precedence myself so I didn't bother looking it up so far.

Functions specific to a task automation language

  • Process control (exec, fork, wait, kill)

    It's a shell, so exec is kinda implied. You can directly run other programs without problems. fork is not a Windows thing, so no luck there. For wait there is Wait-Process which I believe does the same (if I'm reading the manpage right).

  • System monitoring (ps, uptime, lsof, fuser, netstat, df)

    • ps is an alias to Get-Process.
    • The system uptime can be queried via WMI and packaged in a handy function if needed:

      function Get-SystemUptime
      {
          $os = Get-WmiObject Win32_OperatingSystem
          (Get-Date) - [Management.ManagementDateTimeConverter]::ToDateTime($os.LastBootUpTime)
      }
      

      which returns a TimeSpan object.

    • lsof, fuser might be grabbed from the process objects somehow; never did that, though. netstat is still available as a native program you can simply run.

    • df can be kinda emulated with

      Get-PSDrive -PSProvider FileSystem
      

      If you need it for more than just to look at it, WMI is probably the more sane way to go.

  • File and directory handling (find, locate, which, cd, mkdir, touch, rm, rmdir)

    • find and locate can usually be achieved via Get-ChildItem and subsequent filtering of the results with Where-Object.
    • which maps to Get-Command which, as of PowerShell v2 displays commands in the order in which they are considered for execution:

      PS Home:\> gcm where
      CommandType   Name        Definition
      -----------   ----        ----------
      Alias         where       Where-Object
      Application   where.exe   C:\Windows\system32\where.exe
      
    • cd is an alias for Set-Location. Possible locations not only include the file system but also the registry, variables, aliases, certificate store and others.

    • mkdir is a function which ultimately maps to New-Item with appropriate arguments.
    • touch can be achieved with Set-ItemProperty:

      Set-ItemProperty foo.txt LastWriteTime (Get-Date)
      

      You can put that in a function named touch if you like.

    • rm and rmdir are aliases for Remove-Item.

  • I/O (pipes, cat, tee, xargs, read, write, overwrite, append, seek)

    • Pipelines use objects instead of bytes, which should be the most significant different in comparison to other shells. Apart from that not too many surprises. Many cmdlets accept pipeline input.
    • cat is an alias for Get-Content.
    • tee is an alias for Tee-Object. It not only allows writing to a file but you can also capture it in a variable for example:

      Get-Process
      | Tee-Object -Variable proc
      | Select-Object ProcessName,CPU
      
    • xargs can be replaced by ForEach-Object as far as I can see.

    • Byte-oriented file I/O is probably better done with normal .NET objects and method calls. It's easy enough to get a complete file as a string with Get-Content but beyond that it's a little cumbersome.
  • Text-processing functions (awk, sed, grep, split, join, cut, fmt, strings)

    Most text-processing functions are not that desperately needed like on UNIX systems as you won't need them to dissect command output, for example—at least not when you're dealing with cmdlets which return objects that allow for far easier access of their properties.

    Beyond that there are various tools for text-processing. Depending on what you need to do switch -regex and the -match, -replace and -split operators can make up for most uses of sed, awk, cut, split, join I've seen.

    No strings or fmt.

Is PowerShell strongly-typed (char, string, int, long, float, double) or weakly-typed (variant)?

PowerShell uses the .NET type system. Variables are implicitly strongly-typed, but not declared. They can be given an explicit type, though.

Are common data structures available (scalar, list, associative array) and are they first-class elements?

You can therefore use any .NET data structure from the System.Collections namespace. Generic collections are a little painful to set up but it can be done. As mentioned before, lists of elements and hash tables are language elements as well.

Is there a library or module system?

PowerShell v2 does have modules which are collections of cmdlets of related functionality. See Get-Help about_Modules for more details.

If so, what is in the standard library?

More than would be sensible to describe here. You can call Get-Command to see what's available. Get-Help as usual helps in finding out what any given thing does.

Are there third-party libraries I can use if I don’t find what I need?

You can dynamically load any .Net assembly using

[System.Reflection.Assembly]::LoadFrom(...)

or (in PowerShell v2) with Add-Type:

Add-Type -AssemblyName ...

Is this a procedural, functional, or object-oriented language—or to what extent each?

Wikipedia says Multi-paradigm: imperative, pipeline, object-oriented, functional, reflective. The object pipeline aspect is a nice way of achieving elegant solutions for some problem domains.

It's object-oriented in the way that you primarily use objects to achieve your goal, you rarely model something with objects in PowerShell. You can't create own classes (not counting Add-Type which allows you to write, compile and import C# or VB code on the fly) so in terms of abstraction for own work within the PowerShell language you'll probably be more on a procedural level.

What are the coding conventions? (Perl Best Practices)

The language is still very young so there is not yet such a thing (at least I'm not aware of it). However, when trying to code-golf PowerShell code it became apparent that it's hard to create utterly unreadable messes. The usual rules for curly-brace languages work quite well and for longer pipelines breaking them into several lines often helps readability.

Are there formatting expectations—line-oriented or free-form?

Not sure how to answer that: Mostly line-oriented, but piped commands may be better readable with every command in a single line.

If you call a built-in, you can use ` (grave accent) as line continuation mark.

Is whitespace or case significant?

No. Except for line breaks which usually terminate a command, unless preceded by the escape character or when they occur within blocks or other surrounding structures.

Is there a character set limitation?

No.

Is the language “8-bit clean”?

Yes. We're no longer in Ye Olde Ages of the 70es. You can also easily name variables, functions as you wish with no constraints. I named a function (U+0004), for example, which allows me to hit Ctrl+D, Enter to close a session.

Does it support UTF-8, Unicode, etc.?

As PowerShell uses .Net, you have full Unicode support. Input and output streams can be set to various encodings.

devio
Ok, is the expectation that one knows .NET in detail before using PowerShell?
arclight
If you know .NET, it will help you for sure. You may use some advanced features. But if you don't, it should not be problem. PowerShell was designed primarily as administration console that uses .NET, but only few administrators are programmers as well.
stej
I took the liberty of expanding a bit and answering some of the other questions the OP had as well. Will expand further in a moment :)
Joey
@Johannes, you are incredible! ;)
stej
@stej: Some of that I have already explained before on Rosetta Code where I'm responsible for the majority of PowerShell examples :-)
Joey
@Johannes, I haven't heard about it yet. Is it http://rosettacode.org/wiki/Category:PowerShell ? Mentioned it at http://twitter.com/stejcz/status/9623171281 you will get more fans ;)
stej
@stej: Ah well, this answer is only about 2/3 by me. devio already provided a pretty nice one; I just explained the rest that wasn't already mentioned ;). And yes, the Rosetta Code link is what I meant. The site is generally a nice thing if you want to compare languages and how they differ. A bit confusing at times, though, due to the sheer volume of languages they cover.
Joey
Yup, when I have time, I'll have a look at comparison Python-PowerShell. Quite curious about where they match and where don't.
stej
This is by far one of the most well written answers I've ever seen, ever, anywhere.
Mark Tomlin
A: 

Sapien Press have released an updated Windows PowerShell v2.0:TFM by Don Jones and Jeff Hicks for PoSh v2. The original containing PoSh v1 ref materials is now a free download and still very relevant. Great resources for all.

The free dl version is at this location: www.primaltools.com/downloads/communitytools/

I can still only drop 1 link per post <- yea, I'm newb, so... :P

kthxbye

-DBofTLP

dboftlp
A: 

This e-Book is extremly helpful for me while doing my first steps with Powershell: http://powershell.com/cs/blogs/ebook

but I'm also looking for a useable reference...

hgignore