Sunday, April 8, 2007

Script: Exchange 2007 backups with NTBackup

NTBackup is a great solution for Exchange disaster recovery without buying expensive additions to your current backup software. We use NTBackup to backup each Exchange database to its own backup file. Afterwards our primary backup software will send each of those files to tape. Here is a script we use to dynamically backup all the databases on our Exchange servers using NTBackup.

Optimizing NTBackup


Technet has a great article on how Microsoft IT uses NTBackup to backup their Exchange 2003 clusters. This article recommends some registry entries to help optimize performance as well as details about an enhanced NTBackup version that is included with Windows 2003 Service Pack 1. These performance enhancements cut our backup times by more than half. To start the script I set these registry values.

#Add registry keys to enhance NTBackup performance
New-ItemProperty -Path:"HKCU:Software\Microsoft\Ntbackup\Backup Engine" -Name:"Logical Disk Buffer Size" -Value:"64" -PropertyType:"String" -Force
New-ItemProperty -Path:"HKCU:Software\Microsoft\Ntbackup\Backup Engine" -Name:"Max Buffer Size" -Value:"1024" -PropertyType:"String" -Force
New-ItemProperty -Path:"HKCU:Software\Microsoft\Ntbackup\Backup Engine" -Name:"Max Num Tape Buffers" -Value:"16" -PropertyType:"String" -Force


Detect Databases on Each Server


Next, the script will detect all the mailbox databases on the server. The same steps will be performed for public folder databases.

Get-MailboxDatabase -Server:$ServerName | foreach{
...


Creating the Backup Command File


Normally a ‘backup command’ or .bks file will be created by the NTBackup GUI. The script will create a .bks file for each database with the correct JET syntax and file encoding.

#Create the backup JET file for NTBackup
$JetBackupSyntax = "JET " + $_.ServerName + "\Microsoft Information Store\" + $_.StorageGroupName + "\" + $_.AdminDisplayName + "`n"
$BksFileName = $BackupScriptPath + $_.ServerName + "" + $_.StorageGroupName + "" + $_.AdminDisplayName + ".bks"
$JetBackupSyntax | out-file -filepath $BksFileName -encoding unicode


Invoking NTBackup


The ‘&’ character is an alias for the Invoke-Expression cmdlet. I use ‘cmd /c’ to ensure that the script waits for the completion of the database backup before continuing.

#Call NTBackup to backup the database
$BksDescriptiveName = $_.ServerName + "-" + $_.StorageGroupName.Replace(" ","_") + "-" + $_.AdminDisplayName.Replace(" ","_")
&cmd /c "C:\WINDOWS\system32\ntbackup.exe backup `"@$BksFileName`" /n $BksDescriptiveName /d $BksDescriptiveName /v:no /r:no /rs:no /hc:off /m normal /fu /j `"$BksDescriptiveName`" /l:s /f $BackupPath$BksDescriptiveName.bkf"


Compiling a Single Log File


Each backup process will create a log file in the format of backup##.log within the %userprofile%\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\ directory. The script will add the contents of the latest backup file to the backup log for the server.

#Append database backup log to server backup log
&type (get-childitem "$Home\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\" | Sort -Property:LastWriteTime -Descending)[0].FullName >> $BackupLog


At the end of the script, this file can be formatted and emailed to the system administrators.

#Add line breaks to format the backup log
$BackupLogFormatted = ""
Get-Content $BackupLog | foreach { $BackupLogFormatted += $_ + "`n" }

#Email the backup log
$smtp = New-Object Net.Mail.SmtpClient -arg $EmailSMTPServer
$smtp.Send($EmailReportFromAddress,$EmailReportTo,"Exchange Backup Results for " + $ServerName + ": " + $(Get-Date).ToString('MM/dd/yyyy'),$BackupLogFormatted)


Cluster Version


To backup our cluster we created separate cluster groups for our backup disk resource. This cluster group is moved to the corresponding active server to write the backup files. Afterwards it is moved to a passive server that sends the backup files to tape. We schedule this script to be run at the same time on all cluster nodes. It tests if a file path exists (that would be owned by an exchange virtual server) and if so perform the backup for the corresponding server node.

if (Test-Path G:\)
{
write-host ""
write-host "STARTING BACKUP (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
write-host "** Getting Backup Disk..."
CLUSTER GROUP "EXEVS1 Backups" /MOVETO:$env:COMPUTERNAME
write-host "** Performing Backup..."

#Backup databases on EXEVS1 to the R:\ drive
perform_backup "EXEVS1" "R:\"

write-host "** Reassign Backup Disk to Server hit by TSM..."
CLUSTER GROUP "EXEVS1 Backups" /MOVETO:$BackupsServer
write-host ""
write-host "BACKUP COMPLETE (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
}
elseif (Test-Path H:\)
{
write-host ""
write-host "STARTING BACKUP (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
write-host "** Getting Backup Disk..."
CLUSTER GROUP "EXEVS2 Backups" /MOVETO:$env:COMPUTERNAME
write-host "** Performing Backup..."
perform_backup "EXEVS2" "S:\"
write-host "** Reassign Backup Disk to Server hit by TSM..."
CLUSTER GROUP "EXEVS2 Backups" /MOVETO:$BackupsServer
write-host ""
write-host "BACKUP COMPLETE (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
}
elseif (Test-Path I:\)
{
write-host ""
write-host "STARTING BACKUP (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
write-host "** Getting Backup Disk..."
CLUSTER GROUP "EXEVS3 Backups" /MOVETO:$env:COMPUTERNAME
write-host "** Performing Backup..."
perform_backup "EXEVS3" "T:\"
write-host "** Reassign Backup Disk to Server hit by TSM..."
CLUSTER GROUP "EXEVS3 Backups" /MOVETO:$BackupsServer
write-host ""
write-host "BACKUP COMPLETE (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
}
else
{
write-host "--- NO BACKUP REQUIRED ---";
}


Scheduling the Task


Because this script uses PoweSshell with Exchange cmdlets you can’t just schedule the .ps1 file. You must run PowerShell, add in the Exchange PSConsoleFile, and invoke the script as a command.

C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -PSConsoleFile "C:\Program Files\Microsoft\Exchange Server\bin\exshell.psc1" -command "c:\util\Backup-Databases.ps1"

The Script



function perform_backup ([string] $ServerName, [string] $BackupRootDrive) {

### Start User Defined Variables ###
#Set the backup file paths
$BackupScriptPath = $BackupRootDrive + "BackupFiles\"
$BackupPath = $BackupRootDrive + "DailyBackups\"

#Set email options
$EmailReportTo = "user@domain.com"
$EmailReportFromAddress = "user@domain.com"
$EmailSMTPServer = "smtp.domain.com"

$BackupLog = $BackupScriptPath + "ExchangeBackupLogs.log"

### End User Defined Variables ###

#Create an empty backup log file
new-item $BackupLog -type file -force

#Add registry keys to enhance NTBackup performance
New-ItemProperty -Path:"HKCU:Software\Microsoft\Ntbackup\Backup Engine" -Name:"Logical Disk Buffer Size" -Value:"64" -PropertyType:"String" -Force
New-ItemProperty -Path:"HKCU:Software\Microsoft\Ntbackup\Backup Engine" -Name:"Max Buffer Size" -Value:"1024" -PropertyType:"String" -Force
New-ItemProperty -Path:"HKCU:Software\Microsoft\Ntbackup\Backup Engine" -Name:"Max Num Tape Buffers" -Value:"16" -PropertyType:"String" -Force

Get-MailboxDatabase -Server:$ServerName | foreach{

#Create the backup JET file for NTBackup
$JetBackupSyntax = "JET " + $_.ServerName + "\Microsoft Information Store\" + $_.StorageGroupName + "\" + $_.AdminDisplayName + "`n"
$BksFileName = $BackupScriptPath + $_.ServerName + "" + $_.StorageGroupName + "" + $_.AdminDisplayName + ".bks"
$JetBackupSyntax | out-file -filepath $BksFileName -encoding unicode

#Call NTBackup to backup the database
$BksDescriptiveName = $_.ServerName + "-" + $_.StorageGroupName.Replace(" ","_") + "-" + $_.AdminDisplayName.Replace(" ","_")
&cmd /c "C:\WINDOWS\system32\ntbackup.exe backup `"@$BksFileName`" /n $BksDescriptiveName /d $BksDescriptiveName /v:no /r:no /rs:no /hc:off /m normal /fu /j `"$BksDescriptiveName`" /l:s /f $BackupPath$BksDescriptiveName.bkf"

#Append database backup log to server backup log
&type (get-childitem "$Home\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\" | Sort -Property:LastWriteTime -Descending)[0].FullName >> $BackupLog
}
Get-PublicFolderDatabase -Server:$ServerName | foreach{

#Create the backup JET file for NTBackup
$JetBackupSyntax = "JET " + $_.ServerName + "\Microsoft Information Store\" + $_.StorageGroupName + "\" + $_.AdminDisplayName + "`n"
$BksFileName = $BackupScriptPath + $_.ServerName + "" + $_.StorageGroupName + "" + $_.AdminDisplayName + ".bks"
$JetBackupSyntax | out-file -filepath $BksFileName -encoding unicode

#Call NTBackup to backup the database
$BksDescriptiveName = $_.ServerName + "-" + $_.StorageGroupName.Replace(" ","_") + "-" + $_.AdminDisplayName.Replace(" ","_")
&cmd /c "C:\WINDOWS\system32\ntbackup.exe backup `"@$BksFileName`" /n $BksDescriptiveName /d $BksDescriptiveName /v:no /r:no /rs:no /hc:off /m normal /fu /j `"$BksDescriptiveName`" /l:s /f $BackupPath$BksDescriptiveName.bkf"

#Append database backup log to server backup log
&type (get-childitem "$Home\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\" | Sort -Property:LastWriteTime -Descending)[0].FullName >> $BackupLog
}

#Add line breaks to format the backup log
$BackupLogFormatted = ""
Get-Content $BackupLog | foreach { $BackupLogFormatted += $_ + "`n" }

#Email the backup log
$smtp = New-Object Net.Mail.SmtpClient -arg $EmailSMTPServer
$smtp.Send($EmailReportFromAddress,$EmailReportTo,"Exchange Backup Results for " + $ServerName + ": " + $(Get-Date).ToString('MM/dd/yyyy'),$BackupLogFormatted)

}

### Start of backup script ###

#Define backup server
#The cluster backup resource will be moved to this server upon completion
$BackupsServer = "BackupServerName"

if (Test-Path G:\)
{
write-host ""
write-host "STARTING BACKUP (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
write-host "** Getting Backup Disk..."
CLUSTER GROUP "EXEVS1 Backups" /MOVETO:$env:COMPUTERNAME
write-host "** Performing Backup..."

#Backup databases on EXEVS1 to the R:\ drive
perform_backup "EXEVS1" "R:\"

write-host "** Reassign Backup Disk to Server hit by TSM..."
CLUSTER GROUP "EXEVS1 Backups" /MOVETO:$BackupsServer
write-host ""
write-host "BACKUP COMPLETE (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
}
elseif (Test-Path H:\)
{
write-host ""
write-host "STARTING BACKUP (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
write-host "** Getting Backup Disk..."
CLUSTER GROUP "EXEVS2 Backups" /MOVETO:$env:COMPUTERNAME
write-host "** Performing Backup..."
perform_backup "EXEVS2" "S:\"
write-host "** Reassign Backup Disk to Server hit by TSM..."
CLUSTER GROUP "EXEVS2 Backups" /MOVETO:$BackupsServer
write-host ""
write-host "BACKUP COMPLETE (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
}
elseif (Test-Path I:\)
{
write-host ""
write-host "STARTING BACKUP (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
write-host "** Getting Backup Disk..."
CLUSTER GROUP "EXEVS3 Backups" /MOVETO:$env:COMPUTERNAME
write-host "** Performing Backup..."
perform_backup "EXEVS3" "T:\"
write-host "** Reassign Backup Disk to Server hit by TSM..."
CLUSTER GROUP "EXEVS3 Backups" /MOVETO:$BackupsServer
write-host ""
write-host "BACKUP COMPLETE (" $(get-date).Tostring("yyyy-MM-dd HH:mm:ss") ")"
write-host ""
}
else
{
write-host "--- NO BACKUP REQUIRED ---";
}


Download the clustered and standalone server scripts here.

--Nick

38 comments:

Anonymous said...

Thanks, I'm really happy with this script! Works great.

Anonymous said...

Excellent :) one thing though - not sure if i'm doing this correctly, but i found that $home was not available when running the script as a domain user using Windows Task Scheduler. As a result the backup logs were not being included in the email report.

Got around this by adding this to the script:-

$prof = $env:USERPROFILE

... and using $prof instead of $home

Anonymous said...

Hi Nick

Not sure if i'm missing something - but the transaction logs don't get truncated when running the script? Is this because the DB's are being backed up individually? What do you do in your case?

Thanks
Ray

RB said...

Hi Nick,

I have PoSh v1.0 and the "get-command -verb get" results in nothing Exchange related ... how/where do I get the Exchange bits?

Thanks,

Ron

Anonymous said...

RE: I have PoSh v1.0 and the "get-command -verb get" results in nothing Exchange related ... how/where do I get the Exchange bits?

Please ignore this newby request. I now have it. Duh!

Anonymous said...

You ROCK! This script saved me heaps of time. I had to make a few tweaks to suit our site, but it is where we were going with ours.

Thanks heaps from sunny Ngaruawahia!

Anonymous said...

This is great, move backup to a different machine, installed Exchange 2007 32bit management tools and editited your script. Worked like a charm. Took 30mins for a complete backup where on the exchange server it was taking 3 hours.

Now Exchange isn't bogged down even during backup.

Ben

Anonymous said...

Do you think it would be possible to backup Exchange 2007 SP1 running on Windows 2008 Server remotely to a machine running Windows 2003 Server installed with Exchange System Manager using the 2003 NTBACKUP utility?

Have you ever tried this?

Anonymous said...

Hi Nick,

Thanks for this great script.

I'm getting the following error on completion of the backup

"System error 1753 has occurred (0x000006d9).
There are no more endpoints available from the endpoint mapper."

I'm not running this in a clustered environment, juts a single server, but I am trying to copy the .bkf's to a network location.

Would you be able to give me any pointers?

Thanks,
Matt

Nick Smith said...

Matt,

It sounds like a RPC problem. Can you expand on what step in the script you get the error?

--Nick

Matt said...

Hi Nick,

Thanks for the quick reply.

I've tried stepping through the script with PowerGUI, but I can't see exactly where it fails.

If you don't mind, this might be easier if I send you my version of the script so you can see how I modified it.

Would you mind doing this? I could mail it to your Gmail account.

Nick Smith said...

Matt,

Yes, I'd be happy to look at your copy of the script. Please send it to me at nick at knicksmith.com.

--Nick

Unknown said...

What about running E2K7 on W2K8? Is this script still valid?

Nick Smith said...

Ash,

Windows Server 2008 does not support NTBackup. I have not yet investigated adapting the script to use Windows Server Backup.

--Nick

Anonymous said...

Thanks very much. Just what I needed.
FYI. It was announced this week at TECHED that an Exchange 2007 plugin for NT Backup on W2K8 would be "made available" in the very near future... no ETA was given.

Anonymous said...

Looks great but I am unsure how I set the path to where the files are backed up to? I obviously am new at this!

Anonymous said...

Me again - to clarify my question is it possible for me to send the backup file to a UNC drive? To my untrained eye it looks like the backup path variable sends this to the local disk, which for us wouldn't work because we have no disk space.

Thanks again!

Nick Smith said...

Fred,

I have not tried it, but you should be able to change the input string when calling the function to save the files to a UNC path. For example change:

perform_backup "EXEVS1" "R:\"

to

perform_backup "EXEVS1" "\\servername\share\"

--Nick

Anonymous said...

Is it possible to get this script to work on Windows Small Business Server 2003, with exchange 2003 ???

Dennis Dupont

Unknown said...

Hi nick
thanks for the script
I have a question for some reason the log file that gets mailed to me at the end of the backup is always empty

the only thing i change was to make it run daily
and moved the backup to a D-Drive
any idea's ?

Nick Smith said...

Pete,

I exerpienced the same issue when the login name for the backup service account was changed and no longer matched the profile directory. I would suggest hardcoding the directory where NTBackup log files are stored instead of using: "$Home\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\"

--Nick

Anonymous said...

Hi Nick,

just wanted to thank you for your script.
I have tested on a test machine.
Everything ran smoothly, except one error as follows:

Unable to index into an object of type System.IO.FileInfo.
At C:\its\backup wct05.ps1:47 char:144
+ &type (get-childitem "$Home\Local Settings\Application Data\Microsoft
\Windows NT\NTBackup\data\" | Sort -Property:LastWriteTime -Descending)[0 <<<<
].FullName >> $BackupLog

And when I tried to restore one of database for testing, it failed. Is it because the error above causing it failed? Please excuse me, I am just learning as I go.

Anyway, really appreciate your script and help

Cheers

Mike Vu

Virtually Now said...

Can this script be used with Exchange 2003????? Please???

Jim Shilliday said...

Thank you -- I adapted this script for our setup in about 20 minutes and saved hours of work. This is a well-written and well-designed piece of work. My next step will be adding code to ship the backup files to Amazon cloud storage.

Thanks again --

Alex said...

My nephew John told me about his complicated trouble. I knew that he had problems with Exchange data. I suggested him one determintion, which one of the best in the Inet in my opinion. What is more such application is able to help in this trouble - recover edb files.

Tad Sherrill said...

This Backup script got me to %90 of where I needed to be... I did a non-custer re-write of it, would you mind if I post the results to my blog, referencing you?

thanks!

Michael Burutzis said...

I keep getting an error:
You cannot call a method on a null-valued expression.
At C:\Backup\exchange-2007-bkp.ps1:24 char:72
+ $BksDescriptiveName = $_.ServerName + "-" + $_.StorageGroupName.Replace <<<< (" ","_") + "-" + $_.AdminDisplayName.Re
place(" ","_")
+ CategoryInfo : InvalidOperation: (Replace:String) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull


--- And in the log file:
Invalid command line parameter '/N'.
Usage: Specify a new media name for command line switch /n.
----

Any idea what I am doing wrong?

Roshan said...

Hi Nick,

I used this script for about 2 yrs now, recently my exchange backup is taking about 24 hrs to complete. Earlier it took only about 8 hrs. Any Idea why ? Appreciate your comment. thanks
ROshan

Unknown said...

Hi,

Really amazing and useful post for me. thanks welcome to visit our links,and for more information's Laptop encryption software uk :)

Regards
Kim Roddy

mary tharpe said...

Exchange 2007 backups with NTBackup is very nice and useful.
backup software for windows free

Unknown said...

Thanks for informing about NTBackup. Keep your system updated with crack software keys free

annette cooper said...

NTBackup is very easy solution for exchange data recovery.
software windows crack

annette cooper said...
This comment has been removed by the author.
herin woods said...

Exchange 2007 backups with NTBackup is very easy solution for data recovery. thanks for sharing very nice and informative article.
ddl file software

Unknown said...

thanks a very useful information for NTBackup. good sharing!
software center downloading

Anonymous said...

Nice article! valuable sharing about NTBackup.
Crack Download Software

Unknown said...

It is helpful post. tanks for giving nice information.
free crack software | full version software

Unknown said...

good work..!!!
Photoscape Latest Version | Solidworks Portable Download Gratis