My BE 2010R3 --> 2012 Upgrade Experience
I hope this will help out other users and also the Symantec tech support team:
I would like to preface this by saying that I have been working on trying to get BE2012 installed on my system since it came out. I waited for SP1 with abated breath hoping it would solve my issues, no luck. I went to the forums, emailed tech support, spoke with tech support…all of that, no magic bullet. One of the reasons it has taken me this long was because I didn’t have the amount of consecutive time needed to troubleshoot such an issue. It wasn’t until September that I was able to start dedicating real time to this. We were also having some licensing woes, seems that Dell way over allocated our licensing that I had to take care of. All in all, between the licensing and this upgrade issue, I needed to invest about 60 hours of time and it actually may be closer to 80. Symantec Tech Support was of assistance and they were helpful (after explaining the situation what seemed like a dozen times to different people), but unfortunately they couldn’t solve my problem.
Here is what started it all, DLO. BE2012 upgrade refused to install if I had the DLO option selected. We don’t use DLO, it was a lingering part of our license that we tested once and found it less useful than anticipated for our situation (using Win7 backup to a network drive and backing up to tape that resultant Win7 backup structure.) OK, so it was unselected, uninstalled, DBs renamed and removed, Symantec had me run a DLO removal tool for x86 and x64, and still, no luck. Registry keys were reviewed, file/folder permissions audited, and yet nothing would work. For months I was going back and forth with a Symantec Tech Support person that PM’d me in the forums. His final response was to wait for 2012SP2, which was to fix some of the upgrade issues. Unfortunately we were using back channels and it ended up needing to stop, I needed to get an official case created.
I finally ran that MS Installer Cleanup tool to see if DLO was in there, it wasn’t. However, I did notice 2 BE installs showing up when only one was in Add/Remove programs, R2 and R3. We (Symantec Support and I) also saw this in the registry, however, they told me it wasn’t causing any issues.
After working several hours over the phone over several days, the final response was to uninstall BE 2010R3. Hurumph, I didn’t care for that solution, because it isn’t a solution, it is an avoidance. So, I took matters into my own hands. I fancy myself as an excellent black-box troubleshooter but with my job responsibilities I need to unload that work to our various vendors’ tech support, that’s why we pay the support after all.
Anyhow, I took the plunge and purged that lingering R2 install using the Installer cleanup. I recalled that the upgrade from SP2 to SP3 would not work using LiveUpdate. I needed to download the manual installer. Even then, the SP2->SP3 update was less than smooth. I suspect this is what caused the whole headache in the first place. So after the cleanup, I ran the upgrade process again and eureka, DLO was no longer showing as an installed product. Hurray!!! The upgrade began to move forward.
After about 2 minutes of celebrating, there was another problem. It appeared that nearly all the install registry keys for SP3 had blown ACLs. BE2012 told me it couldn’t access those keys and then would rollback the update to 2010R3. Now, I didn’t realize it was nearly ALL of the keys, I had hopefully thought that it was a manageable dozen or so…Nope, there are 100’s, maybe even over 1,000. Luckily, it was the same key just in lots of places. It appeared that every single BE2010 file had a corresponding installer registry key. So like a good soldier, I began searching for the keys, assigning ownership and then assigning privs., dozens of times. When do you reach the middle of the forest??? After several hours and seemingly not making a dent (or getting to the proverbial middle of the forest), I went and started to look for alternative methods of changing these registry keys.
At the same time, I knew because of this registry issue there was a good chance that a repair or uninstall wouldn’t work, so I marched forward with finding a way to discretionarily change reg keys in bulk. However, that search in itself was a challenge. So, I hung my head in defeat and decided to try and uninstall the product and install BE2012. As I suspected, the same keys that were preventing the upgrade also prevent an uninstall, go figure. I was less than hopeful about the repair process and sure enough, the repair failed as well, it just couldn’t read those registry keys. Tried them both twice.
So, back to the search for a bulk-registry acl changer. This is the particular key that was causing the problem, it may be different on other systems: D92D4B60317D3FE43B994C43F68BF43F This is the registry path to where all those sub-keys live, but under other keys of course:
This is a short list of tools I attempted/used to get everything fixed. I tried other things as well, however the below is the most notable.
RegEdit would have taken me too long. It is a basic and great tool that works, but in this instance, I would have needed a dozen people finding and fixing ACLs to get it finished in any amount of acceptable time.
Powershell (http://technet.microsoft.com/en-us/library/bb978526.aspx) was too cumbersome and I couldn’t get everything just right, although there are some good sites that describe how to use Powershell for situations similar to mine. It probably would have worked if I knew it a little better, but don’t quote me on that. We then thought if PS could generate a list of broken keys, we could create a VBScript or other program to methodically go in and fix it for us, however I didn’t want to spend that kinda dough nor did we have an effective way to test it out.
Subinacl (http://www.microsoft.com/en-us/download/details.aspx?id=23510) an older WinNT tool, would be a great option, for 32-bit only. It can’t be fully discretionary, but I can at least start further into the hive so I don’t change too many keys. Problem with this tool is that it does not access the 64-bit registry, where the necessary BE keys live. I found that out after a couple of iterations of trying this solution.
SetACL (http://helgeklein.com/setacl/) is a Godsend. It may not be able to support discretionary changes the way I wanted it, but it sure gets the job done. I was able to change the owner and the permissions on all the keys under the …S-1-5-18\Components. The trick with any blown ACL is to set the owner first. This is an important step because if you don’t change the owner FIRST, all other security will fail. In fact, you will likely get an error even when you set the owner first, but that error is a farce. You can check yourself. After the owner is set, you are free to modify the ACL. Be aware you will likely need to add yourself (or the admin group) to the ACL before you actually can open up the key(or folder or file). So, here is the code I ran. NOTE: the SetACL.exe and the two .CMD files I list below need to be in the same directory. You could modify the code to fit your needs if you want a different setup. The options are robust, proceed with caution. ALSO, THESE SCRIPTS ARE FOR MY MACHINE IN PARTICULAR. THEY MAY WORK ON YOUR MACHINE, BUT YOU NEED TO FULLY AUDIT THE CODE BEFORE RUNNING IT ON YOUR SYSTEM.
First, I ran the command to set the ownership of all the keys referenced above. I created a txt file, named it owner.cmd and put the below code in. Please note, when running this command, you need to be in a CMD shell that was ‘run as administrator’ and it would be a good idea if the user context you are running this under is also a member of the local server administrators group.
SetACL -on HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Components -ot reg -actn setowner -ownr "n:Administrators" -rec yes -log setacllog_owner.txt
Then, I ran the command to set the ACLs. I created a txt file, named it acl.cmd and put the below code in.
SetACL -on HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Components -ot reg -actn ace -ace "n:Administrators;p:full;m:grant" -ace "n:DOMAIN\DOMAINADMIN;p:full;m:grant" -ace "n:SYSTEM;p:full;m:grant" -ace "n:Users;p:read;m:grant" -rec yes -log setacllog_ACL.txt
Replace DOMAIN\DOMAINADMIN with your particular domain admin user account.
After running these 2 scripts, I went and verified I could access those keys in RegEdit, they were accessible.
So now, I didn’t want to uninstall, I didn’t want to repair, I just wanted to run through the upgrade. I started the upgrade process. One thing I did not mention before was the DeDupe problem. I could NOT carry over my licenses because as with DLO, the DeDupe option would not fit our needs so we did not license it. BE2012 doesn’t like to upgrade if DeDupe isn’t part of the carry over license. So I selected the trial version of BE2012 to install as mentioned in several other Symantec forums. The upgrade went flawlessly after that!!! Once it was installed, I could enter in all my keys without any problems and deselect that DeDupe option (finally).
I sit here with a successful BE2012 updated server now running my first round of full backups. BE2012 is certainly way different. All I need to do now is figure out how to review what the actual job exceptions were and I would have successfully mapped out all the functionality I need to know about. There seems to be some granularity lost with the job setup, for instance, I can’t seem to find how to configure a full SQL Instance backup (we use Simple recovery mode) during an incremental backup as I could before and a few other things, but I think I like the new layout. What I don’t like is having to create a backup job for every server AND every method. I liked the 2010 and earlier way of doing it, just create a job for every method and add multiple servers to it.
I hope this is helpful to all those experiencing upgrade woes!