Gacutil must not be called from a custom action. Gacutil is not designed to be used during installation.
Gacutil.exe works, but it is a developer tool, and developer tools go into SDKs and not runtime packages typically. It isn't really appropriate to put more tools into the runtime because that causes it to get larger, which makes it more difficult for applications to redistribute because of increased download size, etc.
In general, installing an assembly to the GAC is an application deployment activity, and is most often done during application setup. One should use Windows Installer to install your application. Starting with version 2.0, Windows Installer has built-in functionality to install assemblies to the GAC - the MsiAssembly and MsiAssemblyName tables in particular.
Its always better to use a MSI as an installer and directly authoring files, registry and GAC installation steps using built-in Windows Installer functionality instead of using a batch script, regsvr32 and gacutil. We can gain a lot of benefits from using an MSI (clean rollback, uninstall, serviceability, deployment options, etc).
Therefore we should use this built-in functionality to handle GAC installation and uninstallation.
This means that if we want to obtain Windows Vista Logo certification for our application, we should use Windows Installer and the built-in GAC installation functionality for our setup.
Source: Aaron Stebner's WebLog