SharePoint Solution Package Deploy Random Failures

Question

I keep seeing this error sporadically during solution deployment and/or feature activation...

Copying of this file failed . This operation uses the SharePoint Administration service (spadmin), which could not be contacted. If the service is stopped or disabled, start it and try the operation again.

The following is the PowerShell script used to deploy:

$programfiles_path = $null
$stsadm_path = $null
$url = $null
$solution = $null
$feature = $null
$appPoolName = $null



$url = "http://" + ${env:COMPUTERNAME}
$programfiles_path = ${env:ProgramW6432}

if ($programfiles_path -eq $null)
{
    $programfiles_path = ${env:ProgramFiles}
}

$stsadm_path = $programfiles_path + "\common files\microsoft shared\web server extensions\12\bin\stsadm.exe"

if(!(test-path $stsadm_path))
{
    echo "stsadm.exe was not found: $stsadm_path"
    exit -1
}


ECHO "***********************************"
ECHO "*** Contoso.Common.SolutionPackage ***"
ECHO "***********************************"

$solution = "Contoso.Common.SolutionPackage.wsp"

ECHO "*** add the solution ***"
& $stsadm_path -o addsolution -filename $solution 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

ECHO "*** deploy the solution ***"
& $stsadm_path -o deploysolution -name $solution -immediate -allowgacdeployment -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.Common.Infrastructure"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.CustomCAS"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.Common.WebParts"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.ExcelExtender"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }



ECHO "*********************************"
ECHO "*** Contoso.Something ***"
ECHO "*********************************"

$solution = "Contoso.Something.wsp"

ECHO "*** add the solution ***"
& $stsadm_path -o addsolution -filename $solution 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

ECHO "*** deploy the solution ***"
& $stsadm_path -o deploysolution -name $solution -immediate -allowgacdeployment -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.Something"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }




ECHO "*********************************"
ECHO "*** Contoso.Nothing ***"
ECHO "*********************************"

$solution = "Contoso.Nothing.wsp"

ECHO "*** add the solution ***"
& $stsadm_path -o addsolution -filename $solution 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

ECHO "*** deploy the solution ***"
& $stsadm_path -o deploysolution -name $solution -immediate -allowgacdeployment -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = $null










ECHO "*****************************************"
ECHO "*** Contoso.Everything ***"
ECHO "*****************************************"

$solution = "Contoso.Everything.wsp"

ECHO "*** add the solution ***"
& $stsadm_path -o addsolution -filename $solution 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

ECHO "*** deploy the solution ***"
& $stsadm_path -o deploysolution -name $solution -immediate -allowgacdeployment -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.Everything"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }




ECHO "*****************************"
ECHO "*** Contoso.CashMoneyMillionaires ***"
ECHO "*****************************"

$solution = "Contoso.CashMoneyMillionaires.wsp"

ECHO "*** add the solution ***"
& $stsadm_path -o addsolution -filename $solution 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

ECHO "*** deploy the solution ***"
& $stsadm_path -o deploysolution -name $solution -immediate -allowgacdeployment -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.CashMoneyMillionaires"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }





ECHO "************************************"
ECHO "*** Contoso.DollaDollaBill ***"
ECHO "************************************"

$solution = "Contoso.DollaDollaBill.wsp"

ECHO "*** add the solution ***"
& $stsadm_path -o addsolution -filename $solution 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

ECHO "*** deploy the solution ***"
& $stsadm_path -o deploysolution -name $solution -immediate -allowgacdeployment -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }

$feature = "Contoso.DollaDollaBill"

ECHO "*** activate feature(s) ***"
& $stsadm_path -o activatefeature -name $feature -url $url -force 
if ($? -eq $false) { exit }

ECHO "*** execute admin jobs ***"
& $stsadm_path -o execadmsvcjobs     
if ($? -eq $false) { exit }



ECHO "************************"
ECHO "*** IIS APPPOOL MGMT ***"
ECHO "************************"

ECHO "*** query wmi for apppool ***"
$appPoolName = "SharedServices1"
$appPool = get-wmiobject -namespace "root\MicrosoftIISv2" -class "IIsApplicationPool" | Where-Object {$_.Name -eq "W3SVC/APPPOOLS/$appPoolName"}
if ($? -eq $false) { exit }

ECHO "*** recycle apppool ***"
$appPool.Recycle()
if ($? -eq $false) { exit }

I have an equivalent batch file which also randomly fails. It fails on different servers. I am quite baffled as to the cause and solution.

Any help would be appreciated.

Answer 1

You need to pause between deploying your solution and trying to activate your features - it's a bit of a race condition.

More: If there are several web front ends, and connectivity is slow, or the servers are busy, or any number of other things that cause this sort of thing to take some time - then you run into trouble. Batch files can't deal with it - a Powershell script could use the object model to check if a solution is truly deployed before moving on.

Answer 2

The best thing that I have found to do is have a Power Shell that will wait for a pending timer job to complete before moving on to the next task, you can but a pause in however that requires you to check to make sure the time job is finished.

The trick is to poll the server using stsadm -o displaysolution -name . if the wsp is installed on the farm the command returns some xml, if there is a timer job for deployment there is a "SolutionDeploymentJob" tag in the xml. With a simple loop you can wait for a time job to finish

Function WaitForJobOn($name){
    [xml]$val = stsadm -o displaysolution -name $name
    while($val.Solution.SolutionDeploymentJob) {
     $log.Info("waiting for a job on $name")
     sleep 5
     [xml]$val = stsadm -o displaysolution -name $name
    }
    $log.Info("Job Compleated")
}

usage: WaitForJobOn somefeature.wsp

it should be noted that the function will fail if the wsp is not added to the farm.

Answer 3

I am also running into this problem. The only way I've gotten around it is to restart the Windows SharePoint Services Administration service each time before a deployment/retraction.

UPDATE: I rebooted the WFE server and this problem has disappeared. I wish I had more info why, but the classic Microsoft reboot solution prevails again.

It is very annoying.

ANOTHER UPDATE: I just released an stsadm command called smartexecjobdefs that addresses this issue. In this blog post, I talk about a Job Definition Executor that monitors for the completion of running jobs before returning control. This stsadm command simply wraps this class.

Answer 4

I've also had this problem. Every time I tried to deploy the solution some of the files were copied then it stopped with this error before copying all of them. I don't have any evidence but I believe the anti-virus software on the server was locking the files and interfering.

It eventually worked after I attempted to deploy the solution several times (each time copying a few more files than the time before).

Answer 5

I find Windows SharePoint Administration Service (SPAdmin) fails sporadically in sp1 when used in a SharePoint farm with multiple web front end servers.

I suggest disabling spadmim and run stsadm -o execadmsvcjobs using remote execution tool such as psexec.

Example:

psexec.exe @serverlist.txt "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\BIN\stsadm.exe" -o execadmsvcjobs