NNTASK PANIC due to Insufficient memory on 32-bit Windows operating system

Article:TECH199197  |  Created: 2012-10-31  |  Updated: 2013-01-23  |  Article URL http://www.symantec.com/docs/TECH199197
Article Type
Technical Solution


Issue



The Domino server generates an NSD file with a FATAL THREAD of NNTASK. Examination of the stack shows that when NNTASK requested memory, the operating system had none left to assign. Therefore, the operating system returned "Insufficient memory", which prompted the Domino server to exit. Further investigation shows that more than 2GB of physical memory is installed.


Error



NSD file contains excerpts similar to the following:

OS Version      : Windows/2003 5.2 [32-bit] (Build 3790), PlatID=2, Service Pack 2 (8 Processors)

Domino Version   : Release 8.5.1FP5 HF261 (32-bit server)

<@@ ------ System Data -> Memory Usage (Time 09:04:42) ------ @@>

 
Total Physical Memory: 24.0G
Avail Physical Memory: 20.3G
Memory Usage        : 15%
Total Paging File   : 28.5G
Avail Paging File   : 25.0G
Total Virtual Memory:  2.0G
Avail Virtual Memory:  2.0G
Avail Extended Virtual Memory:  0.0K
############################################################
### thread 2/12: [  nntask:  0958:  11a4] FATAL THREAD (Panic)
### FP=0x403ddb84, PC=0x7c82845c, SP=0x403ddb14
### stkbase=0x403e0000, total stksize=262144, used stksize=9452
### EAX=0x00000001, EBX=0x403de754, ECX=0x00000001, EDX=0x7c82845c
### ESI=0x00000ae8, EDI=0x00000000, CS=0x0000001b, SS=0x00000023
### DS=0x00000023, ES=0x00000023, FS=0x0000003b, GS=0x00000000 Flags=0x00000293
############################################################
[ 1] 0x7c82845c ntdll.KiFastSystemCallRet+0 (ae8,927c0,0,403de334)
[ 2] 0x77e61c8d kernel32.WaitForSingleObject+18 (ae8,927c0,403deac8,403de76c)
@[ 3] 0x601b4685 nnotes.FRSendCommandToService+789 (403de354,403de754,403de76c,0)
@[ 4] 0x601b52bf nnotes.OSRunExternalScript@8+1055 (258,1)
@[ 5] 0x601b583f nnotes.FRTerminateWindowsResources+975 (1,1010,1,0)
@[ 6] 0x601b5c68 nnotes.OSFaultCleanupExt@24+984 (e86a68,1010,0,0,0,403dedf0)
@[ 7] 0x601b5cea nnotes.OSFaultCleanup@12+26 (0,1010,0)
@[ 8] 0x601c1334 nnotes.OSNTUnhandledExceptionFilter@4+276 (403dfe28)
@[ 9] 0x6018350d nnotes.Panic@4+589 (60d242ef)
@[10] 0x601835ec nnotes.Halt@4+28 (107)
@[11] 0x6010b745 nnotes.AccessAllProtected@0+85 ()
@[12] 0x600461ae nnotes.AccessAll@8+46 (1,1)
@[13] 0x600477a3 nnotes.ProcessGlobalEvent@4+19 (1442ee8)
[15] 0x77e6482f kernel32.GetModuleHandleA+223 (6010cc40,0,0,c8)
 
############################################################
### PASS 2 : FATAL THREAD (Panic) with STACK FRAMES [  nntask:  0958:  11a4]
############################################################
            403df1e0 496E7375 66666963 69656E74 206D656D | Insu ffic ient  mem |
            403df1f0 6F72792E 00000000 00000000 00000000 | ory. .... .... .... |

Environment



  • Microsoft Windows 32-bit operating system
  • IBM Domino 32-bit server
  • Symantec Mail Security for Domino 32-bit
  • More than 2GB physical memory

 


Cause



Windows 32-bit operating system has a maximum address space of 2GB by default.

It is possible for Windows 32-bit operating system, Domino server, and SMSDOM to all function by design and yet exhaust this amount of memory.


Solution



 

Upgrade to Windows 64-bit operating systems, Domino Server 64-bit, and SMSDOM 64-bit.

 

If you are not able to upgrade to Windows 64-bit operating system, Domino Server 64-bit, and SMSDOM 64-bit all at once, some or all of the following measures can reduce the amount of memory consumed. Any single measure listed below may or may not keep your Domino server stable until you reach the date of your upgrade.

  • Upgrade Windows 64-bit operating system only. This permits Windows itself to allocate memory above 2GB. SMSDOM 32-bit is Large Address Aware and can at least work within 4GB addressable memory space under these conditions.
  • Raise additional Domino Messaging servers and split mailboxes between them. Alternatively, for applications, raise additional Domino Application Servers and split the client load between them. This approach requires effective synchronization between hub and spoke servers.
  • Eliminate spam email more aggressively upstream of the hub mail server. Typical antispam solutions tend to drop 90% inbound messages as spam with a one in a million False positive rate. At the mail gateway level, mail administrators can use solutions such as Symantec Email Security.cloud or Symantec Traffic Shaper to drop smtp connections before a message is sent via the SMTP DATA command. If upstream measures drop spam effectively, disable the Premium Anti-Spam feature of SMSDOM to save resources, then remove npas from the ServerTasks= line of the notes.ini.
  • Check whether realtime filesystem antivirus excludes the Data folder for Domino and the Temporary Scanning folder for SMSDOM. Realtime filesystem antivirus can touch the same files as SMSDOM. At worst, this can result in data loss when the antivirus encounters a false positive within an .nsf file. At best, permitting the filesystem antivirus to scan within the Domino data folder and the SMSDOM temporary scanning folder will result in the double-handling of files and additional use of memory.
  • Check whether Symantec Endpoint Protection (SEP) has Email Tools installed. Email Tools inflates memory usage and should not be installed on a Domino server, especially a Domino server running the smtp server task.
  • Within SMSDOM, optimize fulll scans during the work week. By splitting a full scan into multiple incremental scans and setting a start and end time, SMSDOM administrators can reduce the resources SMSDOM uses during production hours, including the amount of files SMSDOM retains in its memory cache. When combined with a scheduled stop and start of SMSDOM's NNTASK process, the entirety of SMSDOM's memory cache is flushed following the completion of the scan.
  • Within SMSDOM, reduce complexity by lowering container limits. The container limits within SMSDOM permit administrators to control how deeply within attachments SMSDOM will scan. Historically, container limits were implemented to prevent various security products from hanging when encountering an archive file with pointers which form a circular structure (aka "the zip of death"). By reducing container limits, you also reduce memory consumed from mail scans and document writes.
  • Within SMSDOM, disable in-memory scanning.
  • Schedule a stop and start of the NNTASK process for SMSDOM at a time outside of the regular office hours for the production environment.
  • Within notes.ini, move nntask to the front of the ServerTasks= line. If npas is also present, move it to immediately to the right of the nntask process.
    Example of not recommended ServerTasks= line:

    ServerTasks=Replica,Router,Update,AMgr,Adminp,Sched,CalConn,RnRMgr,ntask,npas,HTTP


    Example of recommended ServerTasks= line:

    ServerTasks=ntask,npas,Replica,Router,Update,AMgr,Adminp,Sched,CalConn,RnRMgr,HTTP


     
  • Set Domino settings within note.ini to limit memory usage:
    NSF_Buffer_Pool_Size_MB=400
    SERVER_NOTEOPEN_USE_SHARED=1
    SERVER_MAX_NOTEOPEN_MEMORY_KB=200000
     




Article URL http://www.symantec.com/docs/TECH199197


Terms of use for this information are found in Legal Notices