Ghost Solution Suite

The following information has been placed on the clipboard. Visit the Ghost forums at https://forums.symantec.com/syment/board?board.id=109 and supply this information to gain help with this error.

Symantec Ghost Client Agent

An exception has occurred of type C0000005

C:\Program Files\Symantec\Ghost - C:\Program Files\Symantec\Ghost\ngctw32.exe 11.5.1.2266

0xFFB8FFFF

r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gc.cpp#664:Library::Gc::GCSpace::scanMarked

r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gc.cpp#1984:Library::Gc::GCGeneration::collectSpace

unsigned int required = 4112

bool matureSpace = false

int save = 0

bool pageInfo = true

unsigned int * temp = 0x0

r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gc.cpp#2221:Library::Gc::GCGeneration::collect

unsigned int required = 4112

r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gc.cpp#1395:Library::Gc::GCGeneration::fullAllocObject

unsigned int size = 4112

bool collected = false

unsigned int totalWaste = 704

r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\inc\gc\gc#363:Library::Gc::GCGeneration::allocObject

@1r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gcobjs.cpp#316:Library::Gc::SBlob::make

unsigned int size = 0x489DA8

r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\ichange.cpp#371:Library::Gc::InSession::readFrom

unsigned int tag = 32

unsigned int length = 2

unsigned int value = 0

unsigned int size = 0

bool gss202 = true

.... 

Was there a question you wanted to ask, or were you just reporting an error on your machine?

On the assumption that you are seeking help with this error, I will direct you to this thread, which at the time of posting was 5 postings below your own:

https://www-secure.symantec.com/connect/forums/error-c000005

This will give you a lot of background from Nigel Bree, one of the Ghost Developers, plus a strategy for getting a solution.

Aside from the general information about the error code and what it means in the thread EdT linked to, this class of error is somewhat different to most others and so it's useful to understand what it means.

0xFFB8FFFF r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gc.cpp#664:Library::Gc::GCSpace::scanMarked r:\depot_cont\ghost\gsstrunk\ghost\configureagent\library\src\gc\gc.cpp#1984:Library::Gc::GCGeneration::collectSpace

Now, these 3 lines show what the program was doing at the time of the fault; the topmost line is the context for the code running at the time of the failure, while the lines below it show going back in time the path by which it got there. The top line is just a number because it's actually outside the program completely, somewhere it *really* shouldn't be.

So, how did it end up there? Well, the lines below are part of a subsystem in all the GSS management pieces; the GC stands for Garbage Collection, a technique for automatically managing memory in complex programs. One of the things this collector does is work out what in the program can be reclaimed, and it does this with a small program that inspects the memory use of the entire program (starting with a small set of "root" objects, it computes the transitive closure of everything that is still in use; complementing that set is what can be treated as free and re-used).

Now, one of the properties of the GC in the GSS system is that, as a consequence of how I designed it, it also ends up verifying that everything that is still potentially usable is structurally valid, auditing it. The trace above is a consequence of that - some piece of the program's memory that it supposed to be a valid object evidently isn't - and so this fault is telling us that the audit failed, drawing attention to the fact that there is some latent error in the program. Note that the garbage collector itself isn't the problem; it's just what alerted us that a different kind of problem is lurking, as part of a design technique called being fail-fast.

This doesn't tell us *why* some piece of the program's data has gone bad, but it is intended to be combined with the text dump as a more complete piece of evidence; when the program stopped, the entire state of it was written to a file called NGERROR.DMP for post-mortem analysis (assisted by a CPU emulation extension to Visual Studio to help work out the data state). Basically that dumpfile can be loaded into a debugger by the current developers who can then work out more of what the program was being asked to do at the time, and look at the configuration of the program's memory to form a theory as to what might have gone wrong.

[ The other thing that goes into analysing this is whether the problem is repeatable or not; in general, this type of problem is now incredibly rare (because the GSS management components do this kind of fail-fast self-auditing, and have survived so many hundreds of millions of hours of operation, most of these category of bugs have been reported and fixed in years past) but since not everyone uses GSS the same way or there can be odd aspects of a particular machine or install, if a problem like this shows up repeatably in one customer installation that's important data too. ]

I used to just collect this failure data and analyse it myself, but now I'm not at Symantec, in order to get the data to the maintenance team it's probably necessary to go through the tedium of reporting the case to Technical Support, providing the error files to the support rep and hoping that it eventually gets escalated to someone who can actually look at this, and that way you can also inform them if it's a problem you can repeatably provoke somehow or not.

If I understand the current rules correctly, you need a support agreement in order to get this level of support. The contact details for support are in a sticky posting at the top of the forum.

@Nigel - wow, that's some clever coding!

Well, one of the things about resolving situations like this is that solving problems relies on getting good data; customers with paid maintenance do tend to go right to support  (where their reports have an easier time getting through the "being escalated" part of the process) rather than posting here but that's a smaller customer base to draw data from, and the nature of things is that things escalated through this route don't always have the right diagnostic data attached with the report (and by the time the escalation process does its thing, often the data isn't available). More reports, especially ones where the customers know to save the all-important NGERROR.DMP and be prepared to supply it, means a better chance that the critical data will get where it needs to go.

That's part of why I had a habit of leaping on forum reports and getting data mailed right to me, and I had the full support of my direct managers in doing so; problems that affect customers that aren't paying maintenance are pretty much always affecting the customers with maintenance too. The trick is getting the critical data needed to analyse the cause and develop a fix -  by and large, thanks to the forum customers, I'd have what I needed and a fix developed and in testing, then later on the same underlying problem would appear via the escalation process (but without the detailed data required to solve the case sometimes no practical way to get it).

Resolving problems in complex systems relies on sourcing and analyzing quality data about the specifics of a problem (and often, the key data is contextual). Process memory dump files, Process Monitor traces, Wireshark packet captures - it's hard to overstate just how valuable these things are. Even though a report like this won't necessarily get escalated **by itself**, if it fits into a problem that others have reported the Support folks can pick up on that and feed that all-important data into an investigation.

clever coding!

Well, sort of. The conservative generational GC (and reflection-based scripting runtime built on top of it) I built up slowly over some years; an x86/x64 CPU emulator is relatively simple by comparison. Neither thing in and of itself is particularly impressive - I'd expect a decent graduate student to be able to put either together. Mind you, that still doesn't stop me wishing I hadn't lost all access to my work as a result of being laid off!

The real artistry in software is making such things small, and maintainable, and fast - software shares with mathematics an aesthetic of minimalism, for good reason. So, what I do feel a degree of pride in is that the GC+scripting runtime+x64/x86 emulator all fit into ~140Kb of compiled code, and the design for the management system they supported was similarly compact. One can debate the extent to which that kind of behind-the-scenes engineering ultimately affects customer experience, but given Symantec's low level of investment in Ghost generally it's hard to see how the management platform in GSS could have survived let alone thrived had it been built any other way.

Given your oft stated comments on the parlous lack of funding by Symantec for any support activities within the Ghost team, I may have wrongly assumed that support would turn away any non revenue earning calls.

As for coding - there aren't enough "decent" grads out there, and at the time I graduated, paper tape and punched cards were still being used and a minicomputer with 32 K of core memory and 2.5Mb of disk storage was considered powerful....

The support organization (or organizations, really, as it has its own complex structure) is its own thing in Symantec; it's not part of any product team, which is part of why there are odd bits of turf warfare internal to Symantec. If, for instance, a customer with maintenance had opened a case that involved a support rep, and the same customer then also reported their problem here on the forums - and got them to e-mail me and I just fixed it, that would *and did* generate complaints from the support managers to my manager.

Basically, the breach of protocol involved in *appearing* to bypass a support rep dealing with a customer (even when I had no reason to know the customer had already opened a case) was apparently a Big Enough Deal that regular complaints about it would be made. Fortunately, my managers were also New Zealanders and we all cared about, y'know, actually looking after the interests of customers and shareholders. So I got the cover I needed to actually do what was right rather than worry about not offending someone but still ... ugh. That's one of the things like hours wasted on expense reporting I don't miss.

[ This said, by the way, with respect for the support job and some great folks in the NA support org like Greg Haldeman who did (and may still be doing) great work. It's just that some folks in the support org took "thou shalt never be seen to contradict a statement made by a support rep" as holy writ rather than being a guideline to be overruled *when actual customer care is being compromised by it*. ]

Just as another point, Ghost was always based in New Zealand. The support org, aside from having its own many layers and separate management chain, is based primarily in North America where the customers are, with parallel support orgs in the regions. However, there never was anyone in the support org ever co-located with the Ghost dev + QA teams even though we had always wanted that and had tried to find a way to make it happen. I believe support folk were moved out to Utah after the Powerquest reverse takeover so it did get co-located with *their* dev, but that didn't exactly do any good for the real Ghost product.

that support would turn away any non revenue earning calls.

Calls don't really earn revenue; contracted support is its own thing different from sales revenue with a different accounting treatment (accrual, for starters) and who on earth knows what the Finance people did in terms of nominally allocating it.

The thing is that - nonwithstanding the turf-sensitivities alluded to above - the support org is full of people who a) are interested in actually supporting customers, and b) run some good process in terms of call logging and various other things. If a customer call came in *and* it matched an open case that had been referred to dev, then there's at least a decent chance that it could be matched up somehow, whether in the support org or dev person searching the support org's case-logging system.

So, to get that escalation to dev, a problem case probably needs to come from someone with contracted support or be affecting a wide swathe of customers. However, as part of the escalation process there's some evaluation that gets done in the support org as to problem severity and mining the case-logging system for similar issues. Unfortunately the folks doing initial customer contact don't always know off the bat what questions to ask or what technical data to gather, but if you push information at them, it probably will go into their notes where it can be mined out and (hopefully) get into what's presented to dev when it gets escalated.