but i dont delete the "unnecessary" installer files.  reason being that those files are there in case something needs to have a repair install executed or in some cases files provided for software uninstall.  yes these files may take alot of space and yes you could probably get rid of most of them but if your image needs one of these files later due to an update, upgrade or hotfix your end user will be constantly prompted for source files making your life more difficult

Not looking to remove any "necessary" cache directories.  Just trying to identify what is "safe" for removal, if there is anything safe for removing.

In other postings, I see folks comenting their image is 1.5 - 2.5 GB.  I find that hard to believe unless it's just Windows and no other applications or updates included.  

By all means clean up the temp folders, clear the prefetch cache, get rid of the backup folders for windows updates (those starting with a $ under the windows folder), and empty the C:\Windows\SoftwareDistribution\Download folder where updates appear to be cached after download.

Leave any files in C:\windows\installer untouched.

Make sure your policy is set to delete the swap file on shutdown - otherwise this can take up a chunk of space in your image.
If you have System Restore enabled by default, disable it then re-enable it just before imaging - this gets rid of the existing system restore points which can also chew up a bunch of space on your hard disk.

if your using sysprep the swap file is removed by default.  my basic images containing windows XP, drivers and office 2007 using fast compression are just under 3gb.  like EdT said i remove temp files and the software distribution folder (used for downloaded windows updates) but i leave the installer cache.  if you have a bunch of software installed your images could get much much larger.