Deployment Solution

Presently we are running ITMS 7.6 and my WinPE is version 4.0.

The upgrade to 7.6 had its hiccups, but were sorted through easily and all was well. However, I've come across a new problem in the midst of slowly migrating our imaging process from DS6.9 to ITMS 7.6.

We PXE boot our machines into WinPE 4.0 with the x64 setup. This worked before and only when I rebuilt the WinPE did I run into trouble. My first headache was caused by the SYMLINKs being broken that BDC utilizes.

For reference you will see the links in question listed in this location on main NS server: C:\Program Files\Altiris\Notification Server\NSCap\bin\Deployment 

The location will have 2 SYMLINKs and 2 actual directories. The SYMLINKs directories should be BDC & DirversDB. The other two are Installs & Packages.

I corrected that issue alone with ONLY Step 3 in the article: http://www.symantec.com/docs/TECH211409

[2015/03/29 15:15:45.288 1416:1420 2] Log File Created.
[2015/03/29 15:15:45.585 1416:1420 2] AgentGlobals.h:139 RegisterLibrary: x:\program files\Symantec\Deployment\Client Task Agent.dll has been registered sucessfully
[2015/03/29 15:15:45.601 1416:1420 2] AgentGlobals.h:139 RegisterLibrary: x:\program files\Symantec\Deployment\AeXNetComms.dll has been registered sucessfully
[2015/03/29 15:15:45.601 1416:1420 2] AgentGlobals.h:112 RegisterLibrary: LoadLibrary Failed to load x:\program files\Symantec\Deployment\AeXBasicInventory.dll
[2015/03/29 15:15:45.601 1416:1420 0] AgentGlobals.h:114 Agent Exception, Could not laod file ErrorCode=0, SystemError=6
[2015/03/29 15:15:45.601 1416:1420 2] PECTAgent.cpp:152 Stopping PECTAgent. Please check for Network configuration or server name resolving problem and try again.

<event date='03/29/2015 15:15:45.2880000 -08:00' severity='4' hostName='minint-rq4cmo9' source='PectAgent' module='PECTAgent.exe' process='PECTAgent.exe' pid='1416' thread='1420' tickCount='22015' >
  <![CDATA[Version 7.06.1395]]>
</event>
<event date='03/29/2015 15:15:45.6010000 -08:00' severity='1' hostName='minint-rq4cmo9' source='PectAgent' module='PECTAgent.exe' process='PECTAgent.exe' pid='1416' thread='1420' tickCount='22328' >
  <![CDATA[AgentGlobals.h:114 Agent Exception, Could not laod file ErrorCode=0, SystemError=6]]>
</event>

A couple tests I've run since my original posting have been:

I've been running a WinPE 4.0 build for PXE. To rule out WinPE direclty, I added WinPE 5.1 to my list of available WinPE configurations. After updating the WIM from 5.0 to 5.1, I moved the file in place of the WinPE.WIM file for x64. As before the system came up the same as WinPE4.0 and presented the exact same errors in the PECTAgent.log file and the SMP.log file.

Another items that was asked was if DNS was working. I am able to ping and mapdrives by the DNS shortnames of the servers. So DNS appears to not be the issue either.

Another step I have taken is to perform the full process below, which part of the below linked technical article:

http://www.symantec.com/docs/TECH211409

If you think there's corruption, we have some suggestions that may resolve things.  Missing files or links or missing "anything" really can indicate corruption, and the following whould be followed.  Remember though to separate what is under the AGENT with what is under altiris\Deployment and/or altiris\Notification Server.  The latter two are server-side issues which we'll help fix with the steps below.  The fomer folder / the agent folder / even on the SMP, is a completely different set of processes.

1. Using the SIM, repair DS. This is to ensure that nothing in the file system is missing from the SMP. For instance, we have seen BDC folders with only a partial installation. The \program files\altiris\deployment\BDC folder should have well in excess of 1000 files. If not, you MAY need to copy down a version of that folder from another working system (e.g. from Support or a lab server)
2. NOTE: Running a repair WILL break the DS solution. If you do not then follow-up with the next step, you can expect things to fail. For instance, it is well known that the symbolic links under NSCap will be removed for BDC and DA.
3. Reconfigure DS using AEXCONFIG against the Altiris.Deployment.Config file located under program files\altiris\deployment\config (example: D:\Program Files\Altiris\Notification Server\Bin\aexconfig /configure "D:\Program Files\Altiris\Deployment\Config\altiris.deployment.config"). At this point, you should confirm that the BDC package in the console looks healthy. Updating distribution points on that package will ensure the package servers get a current copy of that package.
4. Wait. It takes a bit for services and policies to get all caught up with themselves. You might be able to hurry things along by:
   1. Running a complete collection update
   2. Running a package refresh
   3. Ensuring that the SMP and Site Servers report up inventory
   4. Enabling the Site Service policies and subagents, and ensuring that site servers get them, AND send up basic inventory.
5. Ensure you have at least one configured NBS server in Site Server settings. If not, BootWiz will run, but obviously never build anything for PXE. Ensure that it is running, the services installed and "still running) before moving on.
6. Build or Rebuild a Preboot configuration. This is what actually triggers the download of BootWiz to an NBS server. Whether on the SMP or a site server, this will take time. It has to first download BootWiz and the Imaging tools to the Agent location (not the Package Server location), and once complete, it will THEN run.
   1. If after about 15 minutes, BootWiz never launched on the target NBS server, check the Altiris Agent \ Deployment \ logs folder for the DSTasks.TXT file. This is a log that should show what happened when you attempted to rebuild. It can be rather cryptic, so Support may need to look at that for you. This is one place we are seeing a known issue that we have not resolved. You can also check to see if there is a Deployment \ SBS \ BootWiz \ ... folder structure there. In the Bootwiz \ GUID folder, there should be a Snapshot.XML, Snapdata.XML and log.xml. These SHOULD be updated to the time/date stamp of today/now/when you ran.
   2. If you find that the BootWiz package IS present, that BootWiz when launched from the Agent folders works and shows the preboot environments, and that the NBS site server reports in the console as active, and BootWiz STILL will not run, check HOWTO93837 for a way to manually run BootWiz, but you should still call this in to support.

Any remaining errors you are runnning into after this needs to be reported to Support.

I'm open to suggestions, but even this has not corrected the issue.

What I do find interesting is that the "Product Version" shows a different number than the listed DS Version number under Symantec Installation Manager (SIM). As noted in the below imaged. I am not sure if they are related in some way or give an idea to look at another piece of ITMS that would be the cause of this issue.

I may just have to open a case with Symantec tomorrow to get any further with this. Any additional help or thoughts are welcome and appreciated.

Follow up...

I have opened a case with Symantec regarding the issue. I will update as more information comes in, but appears to be a bug with AeXBasicInventory.DLL.

I should note to anyone that comes across this, you may not realize you had this issue until you recreate your PreBoots. Your PreBoots will techinically still be the older 7.5 version until the recreation. At which point, the new DLLs are brought in during the next build recreation cycle you initiate.

Mine worked posted upgrade to 7.6 until I updated the WinPE to include BGInfo and ran the recreation set.

Im seeing the same problem in 7.5.  I replaced the .wim file with one we created for PE 5.1.  

After some more playing around, I gave up and put the factory .wim file back in place.  Still have the same problem.

I will be opening a case with support today.  Do you mind to send me your case number that I can refernce it with mine?

Phish,

I moved your post under Deployment Solution. You might get more answers there.

Regards,

Triinu Gross
 

Hello Phish,

Did you try with x86 WinPE ?
Are you facing same problem with 32-bit WinPE?

Regards,
Anil

JeffCarlson - PM'd the case number to you.

As for Anil's suggestion of building an x86, I am in the process of doing that now. In fact I changed the active WinPE version to 4.0. Typically, I don't run the x86 as there is no point. All the programs I need can and do run x64, but this would rule out if it is a 32 or 64 bit issue.

I'll report back when the Site Server is done with Bootwiz.

-Phish

I forgot the x86 is not longer an option since Windows 8 came out. First OS to only do x64. Thus, I cannot get a WinPE Driver pack to make the x86 method work. Remember, WinPE 5.0/5.1 is Windows 8/8.1

I'll try and see if choosing x86/x64 options to see if dual support fixes the issue.

Yeah, it didn't like that method either. All in all, still stuck with waiting on the backend guys to make progress on my case. Support thought it was similar to an issue they found in 7.5. The point fix would not work for instance, as this is 7.6.

Hello Phish,

You tried with new fresh x86 WinPE image right?

Please attach full SMP.log and PECTAgent.log files

Thank you
 

As I noted before, I cannot boot into WInPE x86 with these machines. Under a x86 PE boot, I get a Windows Boot Error Status code: 0xc000035a.

So... I can't pull either of those logs from that particular boot environment all due to driver issues.

Hello Phish,

Please provide full SMP.log and PECTAgent.log files for WinPE x64 after reproducing the issue.

Thank you

The complete logs were posted for this issue in my first post within this thread. Though here is a fresh set...

PECTAgnent.log

[2015/04/06 18:22:07.577 336:944 2] Log File Created.
[2015/04/06 18:22:07.874 336:944 2] AgentGlobals.h:139 RegisterLibrary: x:\program files\Symantec\Deployment\Client Task Agent.dll has been registered sucessfully
[2015/04/06 18:22:07.906 336:944 2] AgentGlobals.h:139 RegisterLibrary: x:\program files\Symantec\Deployment\AeXNetComms.dll has been registered sucessfully
[2015/04/06 18:22:07.906 336:944 2] AgentGlobals.h:112 RegisterLibrary: LoadLibrary Failed to load x:\program files\Symantec\Deployment\AeXBasicInventory.dll
[2015/04/06 18:22:07.906 336:944 0] AgentGlobals.h:114 Agent Exception, Could not laod file ErrorCode=0, SystemError=6
[2015/04/06 18:22:07.906 336:944 2] PECTAgent.cpp:152 Stopping PECTAgent. Please check for Network configuration or server name resolving problem and try again.

SMP.log

<event date='04/06/2015 18:22:07.5770000 -08:00' severity='4' hostName='minint-69p1p2b' source='PectAgent' module='PECTAgent.exe' process='PECTAgent.exe' pid='336' thread='944' tickCount='18343' >
  <![CDATA[Version 7.06.1395]]>
</event>
<event date='04/06/2015 18:22:07.9060000 -08:00' severity='1' hostName='minint-69p1p2b' source='PectAgent' module='PECTAgent.exe' process='PECTAgent.exe' pid='336' thread='944' tickCount='18671' >
  <![CDATA[AgentGlobals.h:114 Agent Exception, Could not laod file ErrorCode=0, SystemError=6]]>
</event>

Hi Phish,

assuming you are using a site server, please use the reference below for troubleshooting:

1 - Check all the necessary plug ins are present at the Site Server:

Altiris Base Task Handlers    7.6.1383    C:\Program Files\Altiris\Altiris Agent\Agents\Agent Tasks\BaseTaskHandlers.dll

Altiris Client Task Agent    7.6.1383    C:\Program Files\Altiris\Altiris Agent\Agents\Client Task Agent\client task agent.dll

Altiris Client Task Server Agent    7.6.1383    C:\Program Files\Altiris\Altiris Agent\Client Task Server\CTServerAgent_x64.dll

Altiris Client Task Server Agent    7.6.1383    C:\Program Files\Altiris\Altiris Agent\Client Task Server\CTServerAgent.dll

Deployment NBS Plug-in    7.6.1395    C:\Program Files\Altiris\Altiris Agent\Agents\Deployment\DeploymentSolutionAgent.dll

Deployment Package Server    7.6.1395    C:\Program Files\Altiris\Altiris Agent\Agents\Deployment\PSComponent

Deployment Task Server Handler    7.6.1395    C:\Program Files\Altiris\Altiris Agent\Agents\Deployment\Task Handler

Package Server    7.6.1383    C:\Program Files\Altiris\Altiris Agent

Symantec Management Agent    7.6.1383    C:\Program Files\Altiris\Altiris Agent

2 - Check integrity of the Site Server IIS:

3 - Make sure the "automation" .DLLs are being copied to the Site Server and and processed by Bootwiz correctly for creating boot.wim(automation):

(Automation .DLLs may have different versions if compared with the "production" .DLLs - SMA plugins)

• compare the Notification Server (SMP) "automation" plug-in with image below: 

SMP:

\Program Files\Altiris\Deployment\BDC\bootwiz\oem\DS\winpe\x64\Base\Program Files\Symantec\Deployment\

Program Files\Altiris\Deployment\BDC\bootwiz\oem\DS\winpe\x86\Base\Program Files\Symantec\Deployment

• If you notice an issue and you replace any of them, follow up by updating the distribution point for the BDC package at the console: Settings -> all Settings -> Settings -> Deployment and Migration -> Packages -> BDC
• click "update" on the Site Server Symantec Management Agent -> the package 374E1C49-4F58-4F5C-8D51-07A30F0D44AD will start to re-download
• When the download is finished, check if replicated OK at Site Server location below:

Site Server:

\Program Files\Altiris\Altiris Agent\Package Delivery\{374E1C49-4F58-4F5C-8D51-07A30F0D44AD}\cache\bootwiz\oem\DS\winpe\x64\Base\Program Files\Symantec\Deployment

\Program Files\Altiris\Altiris Agent\Package Delivery\{374E1C49-4F58-4F5C-8D51-07A30F0D44AD}\cache\bootwiz\oem\DS\winpe\x86\Base\Program Files\Symantec\Deployment

• If not replicating, delete the corresponding GUID snapshot at location below on SMP

C:\ProgramData\Symantec\SMP\Snapshots

• Re-create the pre-boot environment

Boot-wiz will copy the files to:

\Program Files\Altiris\Altiris Agent\Agents\Deployment\SBS\Bootwiz\{374E1C49-4F58-4F5C-8D51-07A30F0D44AD}\cache\bootwiz\oem\DS\winpe\x64\Base\Program Files\Symantec\Deployment

\Program Files\Altiris\Altiris Agent\Agents\Deployment\SBS\Bootwiz\{374E1C49-4F58-4F5C-8D51-07A30F0D44AD}\cache\bootwiz\oem\DS\winpe\x86\Base\Program Files\Symantec\Deployment

• When Bootwiz has compleated, run the test

Mauro,

Looks like the only item that was missing. of all you listed for me to verify, was the IIS site. Below is a screenshot, which shows the "Deployment" section the only difference between your IIS and mine. How do I go about fixing that?

Hi, 

The deployment folder in IIS is needed.

Please try to install it manually as from instructions at link below:

(on Site Server, run via Command Prompt as Admin the file Install_DS_PS_Component.bat file)

https://www-secure.symantec.com/connect/blogs/ds-75-sp1-hfxx-issue-installing-dsps-component-site-server-microsoft-vbscript-runtime-error-pa

or:

http://www.symantec.com/docs/TECH227569

Well the solution to the topic of this posting was actually resolved very simply via conversation with Blake that had my case. We did a simple drop an replace of the affected files, which resolved the issue.

Specifically, we replaced the directory OEM directory at

C:\Program Files\Altiris\Deployment\BDC\bootwiz\oem

I recommend the following procedure and get this from another good server source or symantec. I'd share the folder I was given, but mine is specific to 7.6 and future users that find this post would not find the files useful with the progression of time. Though the process will be useful.

1. Access your SMP server (not the Site Server).
2. Copy your the entire OEM folder to a safe location to revert back, in case it doesn't work.
3. Make sure you get a copy of the pectagent.ini file as well (location: C:\Program Files\Altiris\Deployment\BDC\bootwiz\oem\DS\winpe\x86\Base\Program Files\Symantec\Deployment\pectagent.ini) The file is specific to your environment.
4. Next, extract the OEM folder over your current OEM folder (not the backup location).
5. Copy the pectagent.ini file into the directory noted in step 2 where it was originally found.
6. Proceed into the web console to the PDC Package (found at: Settings > All Settings > Deployment and Migration > Packages > BDC).
7. Click the  "Update Distribution Points" button and wait for confirmation.
8. If you wish to accelerate the update process, then open the run command (Windows +R)
9. Type taskschd.msc
10. Select "Task Scheduler Library" to present the list of scheduled tasks.
11. Find NS.Package Distrubution Point Update Schedule from the list.
12. Right-click and select RUN.
13. Confirm the process was success in that row with the Last Run Result column and the Last Run Time columns.
14. Repeat steps 10 thru 12 with NS.Package Refresh scheduled task, which will take at least a minute to convert back to successful from running.
15. Open your Site Server, if you have a specific one you want to test on.
16. Open the Symantec Agent in the System Tray.
17. Click Update Configuration.
18. Verify the BDC Package was updated recent after to clicking update.
19. Go back to the SMP Server (not the Site Server)
20. Open "PreBoot Configuration" (under: Settings > All Settings > Deployment)
21. Select a WinPE configuration followed by clicking "Recreate Preboot Environment"
22. Repeat step 21 for each WinPE build line, if you have multiple.
23. Go back to the Site Server you had issues on.
24. Open the Symantec Agent.
25. Click Update Configuration.
26. After a minute or so, you can check Task Manager and see if the BootWiz.exe file is listed in the running programs, which indicates WinPE is rebuilding. (suggested to wait until all WinPEs are rebuild.
27. PXE boot a machine and test an Job or Task against it.

If all went well a new machine or current machine should respond to jobs you send to it, which is what occurred with my systems.

As an aside, this was fixed last week. I only did the IIS fix today, which apparently is an unrelated item that was not affecting me. Though I am happy to fix an issue that I was not aware of.

Thanks all and to Blake at Symantec.

-Phish