Multithreading Help w/Powershell

Question

So I have a script that will go through and ping all the servers from a list that is stored in SQL Server. The script works fine but it does it all sequentially (lame).

Can someone help me out as to how I would change this to use multithreading instead of a foreach loop?

    $Server = "ServerName"
$Database = "DatabaseName"

$con = "server=$Server;database=$Database;Integrated Security=sspi"
$cmd = "SELECT ServerName FROM dbo.vwServerListActive"

  $da = new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $con)

  $dt = new-object System.Data.DataTable

  $da.fill($dt) | out-null


  foreach ($srv in $dt)
    {

    $ping = new-object System.Net.NetworkInformation.Ping
    $Reply = $ping.send($srv.ServerName)

    $ServerName = $srv.ServerName 

    $ServerName
    $Reply.status

    if ($Reply.status –eq “Success”)
    {
        $sql = "UPDATE dbo.ServerList SET GoodPing = 1 WHERE GoodPing <> 1 AND ServerName = '$ServerName'"

    }
    else
    {
        $sql = "UPDATE dbo.ServerList SET GoodPing = 0 WHERE GoodPing <> 0 AND ServerName = '$ServerName'"
    }

    $Reply = ""

    invoke-sqlcmd -serverinstance $Server -database $Database -query $sql


    }

Answer 1

Here's a page with a script which might be useful for you. I haven't used it myself yet, so I can't comment on it beyond that.

Answer 2

Powershell doesn't really do multithreading at all. I've managed to crowbar it into place by faking it with a fire-and-forget script kicked off with "start [powershell path] scriptname.ps1". It'll fire off multiple insances, but you can't get data back from them without doing an end-run by way of a database or other message passing mechanism. Tracking when the child processes terminate is tricky as well.

cmd /c "start /min /low C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -command .\evtlogparse.ps1  "

In your case as you're setting a SQL string as part of the foreach loop, you can try to put the DB update code into a second script that you fire off. You'll potentially have many different processes attempting to update the same database table, so the potential for timing issues is pretty large.

Also, since you're kicking off umpty new powershell instances you're going to eat a lot of memory to make it work. The foreach loop may just be faster than kicking off a bunch of processes.

So, it can be done, but it isn't anything even resembling pretty.

Answer 3

First, I suggest to only create once the variable $ping outside of the 'foreach..'.
Maybe a simpler solution... now that you're using SQL 2008, why not using the SMO method 'enumAvailableSqlServers: "...SMOApplication]::EnumAvailableSqlServers($false)". This will give you a list of all available server on the network. Here's the Microsoft MSDN link so you can read about it: http://msdn.microsoft.com/en-us/library/ms210350.aspx

Answer 4

(Edited as per Chad Miller's Suggestion + Throttling Requirement + Wait-Job fix + STA fix)

Support.ps1

powershell -File "Main.ps1" -Sta

Main.ps1

$Server = "ServerName"   
$Database = "DatabaseName"   

$con = "server=$Server;database=$Database;Integrated Security=sspi"   
$cmd = "SELECT ServerName FROM dbo.vwServerListActive"   

$da = New-Object System.Data.SqlClient.SqlDataAdapter -ArgumentList $cmd, $con  

$dt = New-Object System.Data.DataTable   

$da.Fill($dt) | Out-Null  

$ThrottleLimit = 10 
$activeJobs = New-Object 'System.Collections.Generic.List[Int32]' 

$JobStateChanged = { 
    param ( 
        [System.Object]$Sender, 
        [System.Management.Automation.JobStateEventArgs]$EventArgs 
    ) 

    switch ($EventArgs.JobStateInfo.State) 
    { 
        Blocked { return } 
        Completed { $activeJobs.Remove($Sender.Id); break } 
        Failed { $activeJobs.Remove($Sender.Id); break } 
        NotStarted { return } 
        Running { return } 
        Stopped { $activeJobs.Remove($Sender.Id); break } 
    }

    Unregister-Event -SourceIdentifier ("{0}.StateChanged" -f $Sender.Name)
} 

foreach ($srv in $dt)   
{ 
    while ($true) 
    { 
        if ($activeJobs.Count -lt $ThrottleLimit) 
        { 
            $job = Start-Job -InitializationScript {   
                Add-PSSnapin -Name SqlServerCmdletSnapin100   
            } -ScriptBlock {  
                param (  
                    [String]$Server,  
                    [String]$Database,  
                    [String]$ServerName  
                )  

                if (Test-Connection -ComputerName $ServerName -Quiet)   
                {   
                    $sql = "UPDATE dbo.ServerList SET GoodPing = 1 WHERE GoodPing <> 1 AND ServerName = '$ServerName'"  
                }   
                else   
                {   
                    $sql = "UPDATE dbo.ServerList SET GoodPing = 0 WHERE GoodPing <> 0 AND ServerName = '$ServerName'"  
                }  

                Invoke-SqlCmd -ServerInstance $Server -Database $Database -Query $sql  
            } -ArgumentList $Server, $Database, $srv.ServerName  

            $activeJobs.Add($job.Id) 

            Register-ObjectEvent -InputObject $job -EventName StateChanged -SourceIdentifier ("{0}.StateChanged" -f $job.Name) -Action $JobStateChanged 

            break 
        } 
    } 
} 

Get-Job | Where-Object { $_.State -eq "Running" } | Wait-Job
Get-Job | Remove-Job

Answer 5

so close.... this is what I've got

add-pssnapin SqlServerCmdletSnapin100

$Server = "ServerName"
$Database = "DatabaseName"

$con = "server=$Server;database=$Database;Integrated Security=sspi"  
$cmd = "SELECT ServerName FROM dbo.vwServerListActive"  

$da = New-Object System.Data.SqlClient.SqlDataAdapter -ArgumentList $cmd, $con 

$dt = New-Object System.Data.DataTable  

$da.Fill($dt) | Out-Null 


foreach ($srv in $dt)  
{     
    Start-Job -ScriptBlock { 
        param ( 
            [String]$Server, 
            [String]$Database, 
            [String]$ServerName 
        ) 


    if (Test-Connection -ComputerName $ServerName -quiet)  
            {  
                $sql = "UPDATE dbo.ServerList SET GoodPing = 1 WHERE GoodPing <> 1 AND ServerName = '$ServerName'" 
            }  
            else  
            {  
                $sql = "UPDATE dbo.ServerList SET GoodPing = 0 WHERE GoodPing <> 0 AND ServerName = '$ServerName'" 
            } 

           Invoke-SqlCmd -ServerInstance $Server -Database $Database -Query $sql 
    } -ArgumentList $Server, $Database, $srv.ServerName 
} 

and it looks to be starting multiple jobs... but my table never gets updated. If I remove the "Start-Job" stuff and arguement list and use $srv.ServerName then it works sequentially as it did before. Any ideas? (Thanks so much BTW for all the responses)

Answer 6

If have PowerShell 2.0 you could make use of background jobs. You'll need to break up your server list into "groups". Given a source table with serverName and groupName:

CREATE TABLE [dbo].[vwServerListActive](
    [serverName] [varchar](50) NULL,
    [groupName] [char](1) NULL
) 

A slight modification to your script (save as forum.ps1):

param($groupName)

$Server = "$env:computername\sql2k8"
$Database = "dbautility" 

$con = "server=$Server;database=$Database;Integrated Security=sspi" 
$cmd = "SELECT ServerName FROM dbo.vwServerListActive WHERE groupName ='$groupName'" 

  $da = new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $con) 

  $dt = new-object System.Data.DataTable 

  $da.fill($dt) | out-null 


  foreach ($srv in $dt) 
    { 

    $ping = new-object System.Net.NetworkInformation.Ping 
    $Reply = $ping.send($srv.ServerName) 

    new-object PSObject -Property @{ServerName=$($srv.ServerName); Reply=$($Reply.status)} 

    } 

You can then call the script for different groups:

#groupName A
start-job -FilePath .\forum.ps1 -Name "Test" -ArgumentList "A"
#groupName B
start-job -FilePath .\forum.ps1 -Name "Test" -ArgumentList "B"

Get-Job -name "test" | wait-job | out-null
Get-Job -name "test" | receive-job

#get-job -name "test" |remove-job

If you're using PowerShell V1 or sqlps you could use System.Diagnostics.ProcessStartInfo to start separate powershell.exe processes and pass the group name.

param($groupName)

    $StartInfo = new-object System.Diagnostics.ProcessStartInfo
    $StartInfo.FileName = "$pshome\powershell.exe"
    $StartInfo.Arguments = " -NoProfile -Command C:\scripts\forum.ps1 $groupName"
    $StartInfo.WorkingDirectory = "C:\scripts"
    $StartInfo.LoadUserProfile = $true
    $StartInfo.UseShellExecute = $true
    [System.Diagnostics.Process]::Start($StartInfo) > $null

Answer 7

Here's a script from Jim Truher for background jobs in PowerShell v1.0:

http://jtruher.spaces.live.com/blog/cns!7143DA6E51A2628D!130.entry

PowerShell v2.0 has background jobs built-in:

http://technet.microsoft.com/en-us/library/dd347692.aspx

-Oisin