There is no native functionality in Windows Installer to handle validation of product keys, so a custom action is always required.
You can certainly write a custom action to do this, which I hope answers your question - as many others have done this already.
Since any MSI can be opened and edited using standard MSI editors, the custom action should be written in a language that can be compiled to a DLL or EXE, otherwise it will be very easy for a hacker to figure out your serial number validation logic, and therefore bypass the process.
You should also ensure that your install will not complete successfully if the serial number validation custom action is removed or bypassed in the MSI - so perhaps it should write a licence key to the install folder which the EXE then checks for.
No doubt there are other solutions - have you tried using google to search for ideas?

EdT,

Thanks for your valuable comments.

As Wise Installation Studio uses the similar algorithm used in "Customer Information" template of Setup and Deployment project (Visual Studio 2005), Can you pls clarify the differences between these two algorithm.

Thanks.

Sorry, I am not a programmer, so have no experience of the projects within Visual Studio 2005.
However, Wise Installation Studio integrates with Visual Studio 2005, if is it installed AFTER VS2005, (or you can run a repair if VS 2005 was installed after Wise), and therefore you can use WIS7 to directly package VS2005 projects.

Therefore if you are able to create your serial number validation solution in Visual Studio, this should be possible to package as an MSI directly from VS.

EdT,

Thanks for your reply.

Can you explain how to interact with an assembly to validate the product id (entered by the user) and blocking the next successive dialog if the validation fails.

As I explained in a previous posting, I am not a programmer, so cannot advise you on how to interact with an assembly to validate the product ID.
Why would you want to do this anyway? What if the assembly you require is not present on the target machine? You can't install it easily during the user interface sequence.
From what I've seen in the past, your best option is to code the serial number validation solution in a language such as C++ which does not depend on any runtime code being present on the target workstation.