Wise Package Studio, Part 11: Testing Our Packages

When we build packages in Wise package studio 7 that we will deploy to our company we have to test the package. Often we just don't know where to start. In this article I will help you further on this road that is not the easiest, but still the most important part of our packaging work.

Our package should be tested for various things.

• Installation
• Behavior
• Working with other packages

So where do you need to start when you have finished the package?

Our first step is to test the installations. For this we need several computers.

The first computer is just a Windows XP computer. We start the installation and we answer to the various screens. After the installation is finished we check if the application is working like we think it should. For an application like Abiword is very easy. Install it, start it, and see if we can do some things with it.

If everything works like we think it should, we can go to the second step in our testing procedure.

Now we take a computer that contains our corporate image. There are already several applications on this image. Login as an administrator on that computer. Now we install the application, and we start it again. This is most likely the same as we did in the previous test. Again we create a document and we click around in the application to see if we can find an error message.

If you get an error message, then there can be two things wrong. The package you just created is not complete, or another package disrupts the new package. Now we have to see what can be the problem.

Do the same things you did when you got the error message on the first computer. If it works there, you can be very sure that the error is created by another package on that computer. Now you have to find out what can cause this. Two very helpful tools in trying to find this are Sysinternal's Filemon and Sysinternal's Regmon. Both pieces of software can be found at http://live.sysinternals.com/, and are also explained on the Juice.

If everything works fine, we are going to do the third check.

Install the software on a computer that has your company image on it. Do this as an administrator. But do not start the package. Now logoff from the computer and logon as a user that has the same rights those other users have. This should always be a test user account that is never used to do company work. If you take a user's account that is also used to work with, the user account will have settings, and that can influence you testing.

Again you have to start the application and see if everything works.

Most applications that are built in automated package streets will fail if they are not very well checked in this test. Automated package streets will assume that the user that starts the application is an administrator or has admin rights. In our companies this will never be the situation. Our users will be restricted users that can do only things that we have granted them.

Most packages I saw that are built by package streets will work perfectly, only applications that write registry settings or write INI files will stop working. The registry settings in package streets are always admin created. In our environments that means that we need the user to take over the admin properties to make him able to write some user specific settings into the registry. Mostly this error will occur because the registry settings are written into the HKEY Local machine registry setting.

User will not have permissions to write values over there. Users need to write registry settings into the Current User HKEY inside the registry. The HKEY Current User registry will be exported into the profile settings when a user logs off of the machine.

When this error occurs, you have to send the package back to let it be fixed. Solving this error is a lot of work and it needs a lot of knowledge about the registry.

Also one thing we see in this third test is that the user that logs on to the computer cannot start the application at all. Simply because during the packaging the option "Install only for this user" was selected. The user was not the user that installed the software so he or she will not get the shortcuts, and definitely will not have the correct registry settings that are necessary for the application to work.

The package is not good and you have to reject it and send it back to the department or company that built it to get it rebuilt.

Our fourth test is to uninstall it. Is the application uninstalled correctly? Are all the files and registry keys gone? Are the classes uninstalled correctly?

If you remove Abiword from a computer, we wish to use a different program to open the documents with the extension .doc. So we double click on a document. If Abiword still starts, we know that the uninstall is not correct and it does not remove the classes where we register the application extensions. This is a very common problem. Abiword will start because it is a very simple program. If we have this problem with for instance Adobe Acrobat, then it will give the user an error message.

That is behavior we definitely do not want. A well configured and built uninstall removes all the files, the registry settings and also the extension registration. Test this firmly. When you are aware of an error, and you believe it is OK because you just want this particular software package to run on the computers inside your company, it will cause you a lot of trouble when you want to upgrade to a newer version or you just want to retire the software package. You can better prevent this behavior than spending hours and hours trying to find a solution because the package is on hundreds or even thousands of machines.

Final test know is to check if all other applications still work? When you deploy Abiword, will Office still do the job? You need to be sure before you deploy to hundreds of machines and then see that WinWord does not work any more. So start other applications also. Test if you can open the applications and do several tests inside them. Don't just think, "It is OK. The package works and I'm going to deploy." This will get you in so much trouble.

And now if everything is tested, you can do the deployment?

Wrong, Wrong, Wrong.

There is still a test that needs to be done. For Abiword this will not be necessary, but for other applications that are much more complex this is a step you never will forget when it goes wrong once.

Deploy the package to a series of validated users that know that they are working like test dummies for you. This series of validated users needs to be between 5 and 10 users that have a good understanding in computing. That know how to work with applications and that know that error reporting is not just calling the helpdesk and telling to them "an unknown error occurred, Can you solve it in 5 minutes"?

These users are selected carefully. Check if they are users that can sort out errors. They should be able to explain what they did when the error occurred. They should also be able to support the helpdesk telling and explaining them why they did the action that triggered the error.

And then finally you are ready to go.

The time between the request for the new application and the delivery of that new application to all company computers can be more then 4 weeks. This is the timeline you should keep in mind when a user requests new software or an update of existing software. Users will think they can get the new software in 1 or 2 days.

Yes, they can. But when you want to be safe and do this correctly you will need at least 4 weeks.

Welcome to my world! Welcome into the world of an application packager.

With kind regards,
Erik Westhovens

Wise Package Studio, Part 10: Orca, What is It and Why Do We Need It?