Ghost Solution Suite

Being an Altiris/Symantec Management Platform customer we have access to the Ghost.exe in the imaging part of the product. I have noticed that there is a slightly newer version of the main ghost.exe in the solution, version 12.0.0.X. It appears that something is slight optimized when using this ghost32.exe or ghost64.exe vs. the ghost.exe that comes with GSS 2.5.1. which is at version 11.5.1. I can upload and download ghost images much faster.

Could anybody answer why this is?

Would it be worthwhile to use this updated ghost32.exe or ghost64.exe with the GSS 2.5.1 solution or would it fail when trying to roll out images with tasks in GSS since there might be something different with the Ghost.exe with command line switches being different or the file format being different?

Thanks

Brad Staupp

JCCC

Hey, Brad. Was great to meet you at ManageFusion that time, pity we weren't allowed off the reservation after that.

It's hard to say what the main factor is, because there's a *lot* of work done in 2008/2009/2010 that's not in the GSS product line. Basically, the shipping GSS that's out now still comes from the Jan 2008 release stream of the code line and although a lot was done subsequently most of the changes we did for the 3.0 release of GSS before it was cancelled were pretty big and so couldn't be backported.

In 2008 and 2009 Ghost for the announced 3.0 release was enhanced to also include the RapiDeploy model of image distribution, where multicast could still occur despite not using GhostCast, with images stored using the slow way of putting them on file shares (DS is a really basic platform and isn't smart enough to schedule GhostCast, so RapiDeploy's no-server model of multicast was the only way to make DS perform well, and so Ghost32.exe had to support it to perform well in DS). There's also an HTTP client that was baked into Ghost32.exe around that time too, again mainly to target use in DS, although unlike the RapiDeploy support which was basically finished and QA'd when our NZ development site was closed, that hadn't really been QA'd at all and I don't think it had any perf advantage.

So, that's one possibility; certainly RapiDeploy-style multicast can make a very big, very noticeable difference. It's entirely possible that the Ghost32.exe build you have was cherry-picked from the unreleased 3.0 version of the GSS code line and if so it would have picked up that feature.

There were also some small perf regressions that I remember Rob Chester resolving in 2010 not long before he and I were laid off; we'd had a visit from one of the DS field engineers who had found a case where ImageX was doing a DS job about 10% better end-to-end than Ghost32 from the 2.5 release using a set of images he had, which Rob took and used to figure something out from - I don't know what that was though, although I can ask him since we're still in regular e-mail contact with each other.

In terms of dropping it back in to 2.5, while I suspect it'd still work, it depends which of the changes is the source of the big difference you're seeing; RapiDeploy-style multicast doesn't give any benefit in GSS if you are using GhostCast, only if you're sending images to and from fileshares. Otherwise you might get a small bump from some other the other perf tuning that was done, but I didn't think it was all that dramatic (maybe 12-15%, on that scale).

By the way just as a generic thing we were very rigorous in terms of how we approached the file format and network protocols to make sure that we stayed compatible in *both* directions - because we knew it would be a real pain point for customers having to upgrade their entire environment simultaneously, so in practice they'd want to deploy images taken with later versions using older toolchains (just think of all the boot disks that would have to be remastered!)

Obviously since there's no continuity between the development teams then and now I can't personally take responsibility for future releases, but the folks in Pune we handed over to had a thorough and meticulous approach to QA so I'm confident they'll keep things pretty stable version to version.

Got an answer from Rob on the performance regression versus ImageX, his recollection was that it was an SMB networking quirk that he did some changes to work around since DS was only running on Windows PE - SMB fileshare networking being a minefield for performance over the years as different editions of the DOS network redirectors required different funky API call sequences to avoid strange slowdowns, and as that code was shared between DOS and Windows PE, it was always hard to code things in a way that was maintainable and got good performance everywhere.

So the best guess is the changes you noticed in DS probably won't affect performance in Ghost Solution Suite if you use GhostCast, only if you're using SMB fileshares as image stores for some reason - basically the same advice as for the RapiDeploy multicast feature, assuming that's enabled and being used in current DS.

Nigel, Thanks for the quick response! Yes it is very sad that we have not been able to meet up again since ManageFusion. Any chances they would let out at all to come of Symantec Vision at all in Las Vegas? It is nowhere near as good as ManageFusion use to but that is all they have now that Altiris and Ghost are all under Symantec. I will be there next month and it would be great to meet up again with you!

It does sound like that you have put much blood, sweat and tears into GSS only to see none of it see the light of day. It was my guess that the ghost32.exe that comes with SMP 7.1 SP1 must have some changes done to it, probably something that makes it more aware of the advanced hard drives that have come out over the past few years. We are seeing a pretty massive improvement over the GSS 2.5.1 build of ghost32.exe. GSS build is around 1000 MB/per min. download in our environment and the SMP 7.1 SP1 build is almost 3 times of that around 2800-3000 MB/per min. I was just blown away at the increased speed that got the brain thinking about how to put a brand new engine in a 10 year old car.

Although when it comes to multicast, I have yet to see the same performance. When you are referring to Mutlicast with ghost32.exe in a Rapideploy style is that using the ghost32.exe in “master” mode while the “slaves” listen for the same packet? It is quite different from using the GhostCast server for multicast and I find it hard to believe that it could even preform as well. Could it? It does not really seem like a “real” multicast when you compare it the ghostcast method.

I just noticed your latest update that you posted about talking to Rob about the performance regression versus ImageX and it would make sense that changes were made that help with Windows PE – SMB share, that is how we are primarily using it in a unicast method, however when it comes to multicast, using GSS with an automated task, using the newer build of the ghost32.exe (12.0.0.X) was the question if it would work at all or better, since the ghostcast server is accessing the ghost image via SMB share as well.

Is there a performance increase or worse a decrease when it comes so accessing ghost images via HTTP? I would think a decrease, however I never really attempted…yet with the SMP 7.1 solution. Could it be better than SMP?

Well, Symantec laid me off 18 months ago, 18 months after my colleagues and friends were laid off when our site in NZ closed. So no, I can't see getting to Vision! Not that I could have anyway, MF was a one-off and it's a bit of a strange story how I ended up there in fact. But it certainly was nice getting some of that Altiris attitude, they'd done all the right things in terms of relating to their customers - it's a shame that Greg in particular moved on and so that aspect of the Altiris culture didn't get to take root.

probably something that makes it more aware of the advanced hard drives that have come out over the past few years

[ SSDs were a slightly different matter as TRIM support appeared on drives in 2010 too and that meant adding extra code to detect whether the command was supported by the drive and send it. Advanced Format was easier in that respect as Vista had anticipated what those drives needed and it just worked. ]

is that using the ghost32.exe in “master” mode while the “slaves” listen for the same packet? [...] It does not really seem like a “real” multicast when you compare it the ghostcast method.

It's basically that, yeah; the thing is that RapiDeploy customers in DS 6.9 were used to that as it was the only mode RD had, and in particular the fact that no multicast traffic ever crossed subnet boundaries, and so we wanted to make sure they had the option of keeping something that had the exact same usage pattern on their network infrastructure as RD in DS6.9 had been (the question of whether the DS team could get 7.x to feature parity with 6.9 to get those people to upgrade being a separate question; our goal was to make our piece work, we had no control of influence over DS).

As for the perf in that mode, it usually wasn't as good as GhostCast but then GhostCast itself has very serious design problems - UDP packets are small and have a high overhead getting them from the program into and out of the operating system compared to TCP streams, especially in GbE and 10GbE where there is also TCP/IP offload in the NIC you can take advantage of. There's also a 64kb fixed window in GhostCast just like classic TCP for communication with the "master" client that GhostCast has chosen, so at GbE speeds just one router hop means that the max speed is limited by the latency between ends because that 64kb fills almost instantaneously.

Now, a fresh modern design can avoid all these problems, and provide other benefits too, but that's a topic for another day. I have not been idle, let's just say that.

since the ghostcast server is accessing the ghost image via SMB share as well.

Ah, interesting. But no, my expectation would be that using the 12.x build of Ghost32.exe wouldn't affect things. The changes for SMB perf were unique to the Ghost client executable - and the GhostCast protocol is usually the limiting factor on GbE networks (due to the small send window and other factors, I never ever saw it get above 25% utilitization of a GbE link). It's certainly possible that there's something that could be gained there - complex systems are always full of surprises - but I wouldn't predict it from first principles.

Since you did remind me of the issue of 4kb drives then maybe if you are deploying XP images there will be some benefit for those, but since I don't have access to the 12.x builds I can't be sure if it realigns them for you or not.

Is there a performance increase or worse a decrease when it comes so accessing ghost images via HTTP?

It could go either way in principle, and as we hadn't tried it out that much before the site closure we didn't have either much data to extrapolate from or time to do any performance tuning around that.

The primary goal in adding it was that again RD had something like it, and in any case it was a feature we'd wanted for *years* to do. We'd had customers in Australia who had these massive, ruinously expensive HTTP caching proxies and wanted to run their imaging traffic over them to their remote offices in places like Perth, and prior to coming under Altiris we just could never get approval to add that. Because RD had that feature, suddenly the political obstacle vanished, as did many others at least until Greg & co left and then we went back to being under hostile VPs again.