tags:

views:

64

answers:

2

I have this service that, when request is received, runs a powershell command and returns result. Here is the invoker class code:

public class PowerShellScript {

    public PowerShellScript() {
    }

    public Object[] Invoke( String strScriptName, NameValueCollection nvcParams ) {
        Boolean bResult = true;
        int n = 0;
        Object[] objResult = null;
        PowerShell ps = PowerShell.Create();
        String strScript = strScriptName;

        for (n = 0; n < nvcParams.Count; n++) {
            strScript += String.Format( " -{0} {1}", nvcParams.GetKey( n ), nvcParams[n] );
        }

        //ps.AddScript( @"E:\snapins\Init-profile.ps1" );
        ps.AddScript( strScript );
        Collection<PSObject> colpsOutput = ps.Invoke();
        if (colpsOutput.Count > 0)
            objResult = new Object[colpsOutput.Count];

        n = 0;
        foreach (PSObject psOutput in colpsOutput) {
            if (psOutput != null) {
                try {
                    objResult[n] = psOutput.BaseObject;
                }
                catch (Exception ex) { 
                    //exception should be handeled properly in powershell script
                }
            }
            n++;
        }
        colpsOutput.Clear();
        ps.Dispose();

        return objResult;
    }
}

Method Invoke returns all results returned by powershell script.

All fine and well. As long as this runs in a single thread. As some powershell scripts we invoke can take up to an hour to complete and we don't want for service to do nothing in that time, we decided to go multi-threaded. Unfortunately Powershell class is not thread safe, resulting in sever memory leaks and cpu burn rate. However, if I use lock on Invoke method, this would mean that the entire idea why we went multithreaded will go down the drain.

Any ideas how to solve this?

+1  A: 

You can use BeginInvoke() method of PowerShell class instead of Invoke() that you use. In this case you execute your script asynchronously and do not block the calling thread. But you have to review your whole scenario as well. Your old synchronous method returns results that can be easily consumed right after the call. In new asynchronous approach this is not possible in the same way.

see http://msdn.microsoft.com/en-us/library/system.management.automation.powershell.begininvoke

Roman Kuzmin
Excellent idea. Will try it out ASAP.Mind you, looking at the example, it shouldn't make that much difference
Vladimir Kocjancic
Nope. Still a huge memory leak and CPU burn rate. And even when only a single thread access it. Thing is, while running it in non threaded evnironment, it worked without a glitch
Vladimir Kocjancic
Hmm, then perhaps I have misunderstood what you are going to get. Resources (memory and CPU) are going to be consumed by your tasks on any approach, multithreaded or not. Actually multithreaded approach will consume even more resources if you run more than one task simultaneously. BTW, what do you mean by "memory leak" exactly?
Roman Kuzmin
The process is like this. service runs listens for requests. Request comes in, service starts a new thread, which somwhere wants to execute powershell command. When thread exits and service returns to listen mode, suddenly I've got 9MB memory leak and CPU usage is up to 5% despite the service is only listening.If I comment out ps.Invoke() there is no memory leak and CPU usage is near 0% in listen mode.
Vladimir Kocjancic
I cannot say anything about that extra 5% of CPU after Invoke(). But I have doubts about memory leaks. Though memory leaks are possible in this case, what you observe is not enough to say that they exist. Some memory is allocated ahead for future use by the CLR runtime and normally never freed; it is pretty normal. Initialized PowerShell engine consumes some memory too after the first call, it's normal. You can really suspect memory leaks only if used memory constantly increases after each call of your tasks. If it just holds that extra 9 MB (or even more) then it’s rather fine.
Roman Kuzmin
Yes, I know. However, I have used System Monitor and not Task Manager to follow the memory leak. I used Private bytes of the service in question. Also, CPU % was measured there. Before first invoke it is 0%ish, after first 5%ish, after second 7%, after third 10% and so on. And I checked which process uses that much amount of CPU and it is my service. As I said. If I comment out Invoke command, no such issues appear.
Vladimir Kocjancic
You said: "All fine and well. As long as this runs in a single thread". Now it sounds different or am I wrong? If you call Invoke() but do not call your scripts in it (use some dummy trivial command) - do you see same issues?
Roman Kuzmin
Yeap. It doesn't matter if I call a script that all it does is print $true to output.
Vladimir Kocjancic
Alas, I give up. Your scenario is not quite standard and it introduces some unusual issues (remember another issue with Write-Progress from another topic?). It would be interesting to find out what happens there exactly...
Roman Kuzmin
That one is still up for discovery :)
Vladimir Kocjancic
A: 

Anyway... I gave up on multithreading when executing powershell commands. I created a small program that is able to execute powershell scripts. Then, each thread creates new process for that program. I know it is a bit of an overhead, but it works.

Basically, Powershell classes are not thread safe. Except static variables (http://msdn.microsoft.com/en-us/library/system.management.automation.powershell%28VS.85%29.aspx).

Hence, an attempt to call multiple scripts via separate threads results in memory leakage and some unexplained CPU usage. My wild guess is that it doesn't close properly. Changing from multi-threaded to multi-process environment should sort things out. However, that means a major politics change.

Vladimir Kocjancic