bplabel -erase didn't actually leave assigned tapes alone. Ouch.
This happened. It is kind of a sad story with a moral lesson and a warning to all.
So there was this media pool with some 50+ VTL tapes.
(Yes, I know, VTLs are a dying breed.)
Most of the tapes in the pool were assigned and full.
The rest were unassigned because all backup images inside them had expired.
I wanted to do a quick erase on each of the unassigned tapes in the pool.
This is a common practice for VTLs because only then would an unassigned tape actually has its data deleted inside the VTL.
Some VTLs do this automatically but our's isn't one that does.
For more information about erasing unassigned VTL tapes and why it is necessary:
Anyhow, the Netbackup admin console GUI doesn't actually allow you to erase an assigned tape,
so I thought its command-line equivalent would behave the same.
If you do this to an assigned tape:
bplabel.exe -erase -m vt0001 -d hcart3 -o -p VTL_pool -host nbumaster
It will say "Media is assigned, and will not be labeled".
And if you check the Activity Monitor, you will find a Media Erase job that says the following:
Info bptm(pid=1076) INF - Waiting for mount of media id VT0001 on server nbumaster for reading.
mounted; mount time: 00:00:07
Info bptm(pid=1076) EXITING with status 0 <----------
Error bplabel(pid=3196) Media is assigned, and will not be labeled
end Erase; elapsed time: 00:00:14
cannot overwrite media, data on it is protected(168)
So it won't let you erase assigned tapes by mistake.
If you still don't believe it, check the catalog, you will find that all backup images are still there on the tape.
So bplabel -erase does protect you from erasing assigned media.
Except when it doesn't.
What would actually happen - at least in my case - is that even though Netbackup would keep its word on not touching the tape,
the VTL would somehow still receive the erase command and delete all data on the tape. Magic.
You'll know when you try to restore some files from the tape and couldn't.
You'll get some kind of read timeout error.
The Netbackup catalog-data about the backup images on the tape becomes irrelevant,
because the data will not be on the tape anymore.
Now, when you are ready for this:
I scheduled and ran a daily-recurring script that does bplabel -erase against all tapes in the VTL media pool.
It should leave the assigned tapes alone right?
The script didn't get to run the second time when we figured out what happened.
In short: Netbackup kept its promise (apparently) while the VTL destroyed all.
I couldn't even import the tapes because our VTL apparently does a very good job in securely deleting the data associated with the tapes when it is told to erase them.
A stupid destructive script was clearly ran against the media pool,
and Netbackup had clearly skipped all assigned tapes and protected all catalog-data about the backup images on the tapes.
Our VTL vendor says that there is nothing they could do about it because the VTL was clearly told to erase the tapes.
Only that it shouldn't have been told to.
We haven't logged a support case with Symantec yet, because we're pretty sure that "the VTL shouldn't have done it!".
I know, we probably should. But not before we are sober.