PGP Universal Server Large Database Joins

Article:TECH149925  |  Created: 2010-08-23  |  Updated: 2012-08-14  |  Article URL http://www.symantec.com/docs/TECH149925
Article Type
Technical Solution


Problem



In PGP Universal Server environments with very large databases, database joins can take an extraordinarily long time. In some cases it make take over 24 hours to complete the join. Furthermore, when multiple joins are run in a series, the joins can take days to complete.
 


Solution




Use the following steps when performing joins on a very large database. 

  1. From the command line on the sponsor server, run manual-join-prepare.sh. This generates the join data file /var/tmp/join-data.sql.gz.
     
  2. Copy the join data file to the same location on each of the # joiners, using scp, ftp, or another tool.
     
  3. On each joiner, in parallel or in series, run manual-join.sh.

    Each execution ends by generating a SQL statement (INSERT or # UPDATE). This is saved to a file called `hostname`.sql. Furthermore, the script prints out the same query, as a psql command, to the standard output.
     
  4. On the sponsor, execute each of the SQL statements generated on the joiners. You can do this by copying the .sql files and running them from the command line using "psql -U postgres oviddb -f <file>, or by pasting the printed text into the sponsor terminal window.

Accessing the PGP Universal Server command line for read-only purposes (such as to view settings, services, logs, processes, disk space, query the database, etc) is supported. However, performing configuration modifications or customizations via the command line may void your Symantec Support agreement unless the following procedures are followed.

Any changes made to the PGP Universal Server via the command line must be: 

  • Authorized in writing by Symantec Technical Support or published as an approved and documented process on the Symantec Knowledge Base.
  • Implemented by a Symantec Partner, reseller or Symantec Technical Support.
  • Summarized and documented in a text file in /etc/pso on the PGP Universal Server itself.


Changes made through the command line may not persist through reboots and may be incompatible with future releases. Symantec Technical Support may also require reverting any custom configurations on the PGP Universal Server back to a default state when troubleshooting new issues.


 



Legacy ID



2201


Article URL http://www.symantec.com/docs/TECH149925


Terms of use for this information are found in Legal Notices