Monday, November 8, 2010

NetBackup 7 Restore Blues, Catalog Woes

I recently ran into an issue where a customer couldn't restore files from a backup. In fact the NetBackup GUI immediately reported "There are no files matching the specified criteria." Normally this would implicate my search criteria; perhaps I specified an incorrect policy type or media server. However, I repeatedly got the same error even when all I changed was the ending search date (specifying an earlier backup).

In the NetBackup "Backup, Archive, and Restore" applet I could generate a backup listing for backups prior to 9/15/2010, so I knew they were there. Merely moving the end date earlier than 9/15 generated the above error.

In fact, after a little testing I found I could generate the error for all backup images for all clients prior to a certain date. Command line attempts to find the file (via bplist with the appropriate start/end dates) immediately resulted in status code 227 (no entity was found), so I was pretty sure it was a catalog problem.

The customer had recently upgraded their master server from 6.5.6 to 7.0, so it was tempting to assume the upgrade had somehow corrupted part of the catalog. However, the timing didn't fit the circumstances.

Analysis of extant backup images via bpimagelist (using the -L flag) showed an interesting coincidence. All backups prior to 9/15 (and some taken early the morning of) were marked "DB Compressed: Yes" in the bpimagelist -L output.

Apparently the customer had enabled catalog compression, which dutifully compressed catalog entries as directed, but prior to the NBU7 upgrade. The NetBackup administrator guide for UNIX, volume 1 states enabling catalog compression reduces disk consumption at the expense of performance when a user lists or restores files.

Now things started settling into place. The customer transitioned from SPARC Solaris to RedHat Linux 5.4 as part of the NetBackup upgrade. By default, RedHat Linux installations do not include ncompress. Installing the ncompress RPM onto the master server immediately resolved the issue.

No comments:

Post a Comment