Tuning DMP memory pool sizes when using native DMP 5.1 support for Logical Volume Manager volume groups on AIX

Article:HOWTO36201  |  Created: 2010-11-29  |  Updated: 2011-04-08  |  Article URL http://www.symantec.com/docs/HOWTO36201
Article Type
How To



Introduction:

Storage Foundation 5.1SP1 for AIX has formally introduced support for creation of Logcal Volume Manager (LVM) volume groups (VGs) on top of Dynamic Multi-Pathing (DMP) pseudo devices. This extends the proven benefits of DMP, previously only available when using Veritas Volume Manager disk groups, to LVM logical volume groups. In addition, it is also possible to migrate an existing systems rootvg (root volume group) and all logical volumes it contains away from traditional hdisk devices onto DMP metanodes. The machine will then boot from DMP devices, with these devices able to take advantage of the features offered by the DMP product. Such support is termed 'DMP rootability'.

DMP Memory Pool:

When building LVM volume groups on top of  DMP devices, it is required that the DMP driver is able to allocate memory to service I/O requests coming from LVM devices. If DMP cannot allocate memory for a given I/O request, it follows that it will not be able to service that I/O request.

To try and ensure that the DMP driver is always able to allocate memory for incoming I/O requests, an initial pool of DMP memory is assigned to the driver at system boot time. The DMP driver can then make allocations as required from this pool. Likewise should the initially allocated pool not be of sufficient size to handle all concurrent I/O requests, the pool can be dynamically grown to a pre-determined maximum size. Note that when using DMP 5.1SP1 or later, this pool is created using native AIX ldata pools. Prior to this release the pool is created as a slab via the slab allocator. Regardless of the method of creation used, the pool and tunables discussed by this document operate in a similar manner on all releases.

There are two tunables to manage the pool created for use by the DMP driver:

dmpslab_minsz - specifies the size of pool in 4Kb pages which should be created for the DMP driver at boot time
dmpslab_maxsz - specifies the maximum size in 4Kb pages to which the pool can dynamically grow 

Note that dmpslab_maxsz must be greater than or equal to dmpslab_minsz. dmpslab_maxsz defaults to 5% of physical memory or 146Mb/37500 pages (whichever is smaller), and dmpslab_minsz defaults to 192Kb/48 pages from 5.1SP1RP1 onwards.

On a system with 12Gb of physical memory, the tunables will have the following default values as shown via the lsattr command:

# lsattr -El vxdmp | grep dmpslab
dmpslab_maxsz 37500 N/A True
dmpslab_minsz 48   N/A True

On a machine using the above settings we can say that an initial pool of (4Kb * 48) 192Kb will be allocated to the DMP driver at boot time, and that this pool can dynamically grow to a maximum of (4Kb * 37500) 150000Kb whilst the system is running. Note that the pool only needs to be large enough for the DMP driver to make allocations to handle all concurrent I/O requests (IOPS) taking place on the system. As such, on a system with relatively small I/O load it is expected that the pool may not grow significantly larger than initially allocated.

Tuning dmpslab_maxsz and dmpslab_minsz:

dmpslab_minsz is set to 48 pages or 192Kb which is sufficient to serve 512 IOPS.

dmpslab_maxsz is set to 146Mb or 5% of system memory (whichever is smaller). Considering a machine with 12Gb of physical memory 5% of physical memory is 614Mb hence dmpslab_maxsz will be set to 146Mb. 146Mb (37500 pages) is sufficient to serve 400,000 IOPS.

Once the initial dmppool is allocated, depending on the number of outstanding I/O requests on the system, the dmppool is grown dynamically until it reaches dmpslab_maxsz.

To determine the values of dmpslab_minsz, dmpslab_maxsz, and current dmppool size, the vxmemstat command can be used as shown below. Note that this command is only available from the 5.1SP1 release onwards and shows values in Kb:

# vxmemstat

...

Pool       DG       Min       Max         In          Allocated   Max       Waiting
                     Size      Size        Use                     Used           
----------  -------  --------  ----------  ----------  ----------  ---------  -------
...

dmppool     -           196608    153600000     -             196608     -          -

...

dmpslab_minsz and dmpslab_maxsz can be modified via smitty in the following locations:

smitty vxvm =>  Change / Show DMP Tunables => Total Maximum Pages for dmp slab (dmpslab_maxsz)
smitty vxvm =>  Change / Show DMP Tunables => Total Minimum Pages for dmp slab (dmpslab_minsz)

Alternatively the tunables can also be modified using the chdev command:

# chdev -P -l vxdmp -a dmpslab_maxsz=newsize
# chdev -P -l vxdmp -a dmpslab_minsz=newsize

According to design, the new tunable value should be (current value of tunable value + 8 * (positive integer)). Note that regardless of whether using smitty or chdev to modify tunables all values are given in number of 4Kb pages to allocate. Changes made to the tunables are persistent and stored in the AIX ODM.

Once modified, the machine must be rebooted to use the new values assigned to the tunables.


Further Information

For further information or assistance on tuning the dmpslab_minsz or dmpslab_maxsz variables, please contact Symantec customer support.



Article URL http://www.symantec.com/docs/HOWTO36201


Terms of use for this information are found in Legal Notices