Powershell Copy-Item but only copy changed files...

Question

I am trying to recurse through a directory and copy it from A to B. That can be done with the following:

Copy-Item C:\MyTest C:\MyTest2 –recurse

I want to be able though to only copy new files (ones that exist in src but not dest) and also only copy files that may have changed based off a CRC check and not a datetime stamp.

$file = "c:\scripts"
param
(
$file
)

$algo = [System.Security.Cryptography.HashAlgorithm]::Create("MD5")
$stream = New-Object System.IO.FileStream($file, [System.IO.FileMode]::Open)

$md5StringBuilder = New-Object System.Text.StringBuilder
$algo.ComputeHash($stream) | `
% { [void] $md5StringBuilder.Append($_.ToString("x2")) }
$md5StringBuilder.ToString()

$stream.Dispose()

This code gives me a CRC check on a specific file...I am just not sure how to put the two scripts together to really give me what I need. I also don't know if the CRC check above is actually the correct way of doing this.

Does anyone have any insight?

Answer 1

I found a solution...but not sure it is the best from a performance perspective:

$Source = "c:\scripts"
$Destination = "c:\test"
###################################################
###################################################
Param($Source,$Destination)
function Get-FileMD5 {
Param([string]$file)
$mode = [System.IO.FileMode]("open")
$access = [System.IO.FileAccess]("Read")
$md5 = New-Object System.Security.Cryptography.MD5CryptoServiceProvider
$fs = New-Object System.IO.FileStream($file,$mode,$access)
$Hash = $md5.ComputeHash($fs)
$fs.Close()
[string]$Hash = $Hash
Return $Hash
}
function Copy-LatestFile{
Param($File1,$File2,[switch]$whatif)
$File1Date = get-Item $File1 | foreach-Object{$_.LastWriteTimeUTC}
$File2Date = get-Item $File2 | foreach-Object{$_.LastWriteTimeUTC}
if($File1Date -gt $File2Date)
{
Write-Host "$File1 is Newer... Copying..."
if($whatif){Copy-Item -path $File1 -dest $File2 -force -whatif}
else{Copy-Item -path $File1 -dest $File2 -force}
}
else
{
#Don't want to copy this in my case..but good to know
#Write-Host "$File2 is Newer... Copying..."
#if($whatif){Copy-Item -path $File2 -dest $File1 -force -whatif}
#else{Copy-Item -path $File2 -dest $File1 -force}
}
Write-Host
}

# Getting Files/Folders from Source and Destination
$SrcEntries = Get-ChildItem $Source -Recurse
$DesEntries = Get-ChildItem $Destination -Recurse

# Parsing the folders and Files from Collections
$Srcfolders = $SrcEntries | Where-Object{$_.PSIsContainer}
$SrcFiles = $SrcEntries | Where-Object{!$_.PSIsContainer}
$Desfolders = $DesEntries | Where-Object{$_.PSIsContainer}
$DesFiles = $DesEntries | Where-Object{!$_.PSIsContainer}

# Checking for Folders that are in Source, but not in Destination
foreach($folder in $Srcfolders)
{
$SrcFolderPath = $source -replace "\\","\\" -replace "\:","\:"
$DesFolder = $folder.Fullname -replace $SrcFolderPath,$Destination
if(!(test-path $DesFolder))
{
Write-Host "Folder $DesFolder Missing. Creating it!"
new-Item $DesFolder -type Directory | out-Null
}
}

# Checking for Folders that are in Destinatino, but not in Source
foreach($folder in $Desfolders)
{
$DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:"
$SrcFolder = $folder.Fullname -replace $DesFilePath,$Source
if(!(test-path $SrcFolder))
{
Write-Host "Folder $SrcFolder Missing. Creating it!"
new-Item $SrcFolder -type Directory | out-Null
}
}

# Checking for Files that are in the Source, but not in Destination
foreach($entry in $SrcFiles)
{
$SrcFullname = $entry.fullname
$SrcName = $entry.Name
$SrcFilePath = $Source -replace "\\","\\" -replace "\:","\:"
$DesFile = $SrcFullname -replace $SrcFilePath,$Destination
if(test-Path $Desfile)
{
$SrcMD5 = Get-FileMD5 $SrcFullname
$DesMD5 = Get-FileMD5 $DesFile
If(Compare-Object $srcMD5 $desMD5)
{
Write-Host "The Files MD5's are Different... Checking Write
Dates"
Write-Host $SrcMD5
Write-Host $DesMD5
Copy-LatestFile $SrcFullname $DesFile
}
}
else
{
Write-Host "$Desfile Missing... Copying from $SrcFullname"
copy-Item -path $SrcFullName -dest $DesFile -force
}
}

# Checking for Files that are in the Destinatino, but not in Source
foreach($entry in $DesFiles)
{
$DesFullname = $entry.fullname
$DesName = $entry.Name
$DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:"
$SrcFile = $DesFullname -replace $DesFilePath,$Source
if(!(test-Path $SrcFile))
{
Write-Host "$SrcFile Missing... Copying from $DesFullname"
copy-Item -path $DesFullname -dest $SrcFile -force
}
}

Answer 2

Here is some of the guidelines how you can your script to be more maintainable.

Conver the original script as a filter.

filter HasChanged { 
    param($file)

    # if $file's MD5 has does not exist
    # then return $_
}

Then simply filter all files that are updated and copy them.

# Note that "Copy-Item" here does not preserve original directory structure
# Every updated file gets copied right under "C:\MyTest2"
ls C:\MyTest -Recurse | HasChanged | Copy-Item -Path {$_} C:\MyTest2

Or you can create another function that generates sub directory

ls C:\MyTest -Recurse | HasChanged | % { Copy-Item $_ GenerateSubDirectory(...) }