Gacutil must not be called from a custom action. Gacutil is not designed to be used during installation.
Gacutil.exe works, but it is a developer tool, and developer tools go into SDKs and not runtime packages typically. It isn't really appropriate to put more tools into the runtime because that causes it to get larger, which makes it more difficult for applications to redistribute because of increased download size, etc.
In general, installing an assembly to the GAC is an application deployment activity, and is most often done during application setup. One should use Windows Installer to install your application. Starting with version 2.0, Windows Installer has built-in functionality to install assemblies to the GAC - the MsiAssembly and MsiAssemblyName tables in particular.
Its always better to use a MSI as an installer and directly authoring files, registry and GAC installation steps using built-in Windows Installer functionality instead of using a batch...