Managing the Broker Persistent Store

The persistent store is used for storing unit-of-work messages to disk. This means message and status information can be recovered after a hardware or software failure to the previous commit point issued by each application component.

Under Linux, the broker persistent store can be implemented with:

  • the Adabas database of Software AG

  • the c-tree © database of FairCom Corporation ®

This document covers the following topics:

See also Concepts of Persistent Messaging.


Implementing an Adabas Database as Persistent Store

Introduction

EntireX provides an Adabas persistent driver. This enables Broker unit of work (UOW) messages and their status to be stored in an Adabas file. It is designed to work with Adabas databases under z/OS, Linux, Windows and BS2000, and can be used where the database resides on a different machine to the Broker kernel. For performance reasons, we recommend using EntireX Broker on the same machine as the Adabas database.

Adabas Persistent Store Parameters

Parameters are supplied using the Adabas-specific Attributes in the platform-independent Administration documentation. See excerpt from the attribute file:

DEFAULTS=BROKER
  STORE                     = BROKER
  PSTORE-TYPE               = ADABAS
  PSTORE                    = COLD
DEFAULTS=ADABAS
  DBID                      = dbid
  FNR                       = fnr

Configuring and Operating the Adabas Persistent Store

Selecting the Adabas Persistent Store Driver

The Adabas persistent store driver is contained in module etbnuc and is activated by specifying the Broker attribute PSTORE-TYPE=ADABAS.

Use the supplied script persistence.fdu.v6 for PSTORE-VERSION=6 and above in the bin directory to create a persistent store file in your Adabas database. This script uses the Adabas FDT definition found in file persistence.fdt.v6 in the etc directory.

The script persistence.fdu.v6 can be executed like this:

 persistence.fdu.v6 <dbid> <base_fnr> <lob_fnr>

Note:
You can customize the supplied script and FDT file in accordance with your site requirements. See the Adabas Utilities manual where necessary, specifically ADAFDU (File Definition Utility).

Start of instruction setTo run the script file

  1. Ensure that you execute the script file on the same machine that the target Adabas is running on. (The database can be either active or inactive at the time you execute it.)

  2. Ensure that Adabas environment variables (such as ACLDIR, ADATOOLS, ADABIN and ADALNK) are set up. These environment variables are set by sourcing the corresponding environment scripts. See your Adabas documentation for details.

  3. Set your working directory to the one where the fdt file is located.

  4. Execute the fdt file, passing it three parameters:

    1. the DBID where the persistent store file is to be created

    2. the base file number

    3. the LOB file number

  5. Option: If the DBID is less than 3 characters long, include leading zeros. For example:

    persistence.fdu.v6 001 19 20

    Result: Creation of base file number 19 and LOB file number 20 in database 1.

Defining an Adabas FDT for EntireX File

ADACMP FNDEF='01,WK,21,A,DE'
ADACMP FNDEF='01,WJ,126,B,MU'
ADACMP FNDEF='01,WI,126,B,DE,NU'
ADACMP FNDEF='01,WL,96,A,DE,NU'
ADACMP FNDEF='01,WP,96,A,DE,NU'
ADACMP FNDEF='01,L1,0,A,LB,NV,NU,NB'

Restrictions

If a HOT start is performed, the Broker kernel must be executed on the same platform on which also the previous Broker executed. This is because some portions of the persistent data are stored in the native character set and format of the Broker kernel. It is also necessary to start Broker with the same Broker ID as the previous Broker executed.

If a COLD start is executed, a check is made to ensure the Broker ID and platform information found in the persistent store file is consistent with the Broker being started (provided the persistent store file is not empty). This is done to prevent accidental deletion of data in the persistent store by a different Broker ID. If you intend to COLD start Broker and to utilize a persistent store file which has been used previously by a different Broker ID, you must supply the additional Adabas-specific attribute FORCE-COLD=YES.

Recommendations

  • Perform regular backup operations on your Adabas database. The persistent store driver writes C1 checkpoint records at each start up and shut down of Broker.

  • For performance reasons, execute Broker on the same machine as Adabas.

Broker Checkpoints in Adabas

During startup, Broker writes the following C1 checkpoint records to the Adabas database. The time, date and job name are recorded in the Adabas checkpoint log. This enables Adabas protection logs to be coordinated with Broker executions. This information can be read from Adabas, using the ADAREP utility with option CPLIST:

Broker Execution Name Broker Execution Type Adabas
ETBC Broker Cold Start Normal Cold Start
ETBH Broker Hot Start Normal Hot Start
ETBT Broker Termination Normal Termination

Adabas DBA Considerations

BLKSIZE: Adabas Persistent Store Parameter for Broker

Caution should be exercised when defining the block size (BLKSIZE) parameter for the Adabas persistent store. This determines how much UOW message data can be stored within a single Adabas record. Therefore, do not define a much larger block size than the size of the maximum unit of work being processed by Broker. (Remember to add 41 bytes for each message in the unit of work.) The advantage of having a good fit between the unit of work and the block size is that fewer records are required for each I/O operation.

It is necessary to consider the following Adabas parameters and settings when using Adabas for the persistent store file:

Table of Adabas Parameter Settings

Topic Description
Allowing Sufficient Adabas UQ Elements Allow sufficient Adabas user queue (UQ) elements each time you start Broker. The Broker utilizes a number of user queue elements equal to the number of worker tasks (NUM-WORKER), plus two. Adabas timeout parameter (TNAE) determines how long the user queue elements will remain. This can be important if Broker is restarted after an abnormal termination, and provision must be made for sufficient user queue elements in the event of restarting Broker.

Use either the Adabas utility ADAOPR or the Adabas DBA workbench to clean-up any user queue element belonging to the previous Broker job.

Setting Size of Hold Queue Parameters Consideration must be given to the Adabas hold queue parameters NISNHQ and NH. These must be sufficiently large to allow Adabas to add/update/delete the actual number of records within a single unit of work.

Example: where there are 100 message within a unit of work and the average message size is 10,000 bytes, the total unit of work size is 1 MB. If, for example, a 2 KB block size (default BLKSIZE=2000) is utilized by the Adabas persistent store driver, there will be 500 distinct records within a single Adabas commit (ET) operation, and provision must be made for this to occur successfully.

Setting Adabas TT Parameter Consideration must be given to the Adabas transaction time (TT) parameter for cases where a large number of records is being updated within a single unit of work.
Defining LWP Size Sufficient logical work pool (LWP) size must be defined so that the Adabas persistent store can update and commit the units of work. Adabas must be able to accommodate this in addition to any other processing for which it is used.
Executing Broker Kernel and Adabas Nucleus on Separate Machines If Broker kernel is executed on a separate machine to the Adabas nucleus, with a different architecture and codepage, then we recommend running the Adabas nucleus with the UEC (universal conversion) option in order to ensure that Adabas C1 checkpoints are legible within the Adabas checkpoint log.
Setting INDEXCOMPRESSION=YES This Adabas option can be applied to the Adabas file to reduce by approximately 50% the amount of space consumed in the indexes.
4-byte ISNs If you anticipate having more than 16 million records within the persistent store file, you must use 4-byte ISNs when defining the Adabas file for EntireX.
Specification of Adabas LP Parameter
Warning:
This parameter must be specified large enough to allow the largest UOW to be stored in Adabas.

If this is not large enough, Broker will detect an error (response 9; subresponse - 4 bytes - X'0003',C'LP') and Broker will not be able to write any further UOWs.

See the description of the LP parameter under ADARUN Parameters in theDBA Reference Summary of the Adabas documentation.

Tips on Transports, Platforms and Versions

  • Entire Net-Work
    If you intend to use Adabas persistent store through Entire Net-Work, see the Entire Net-Work documentation for installation and configuration details.

  • Adabas Versions
    Adabas persistent store can be used on all Adabas versions currently released and supported by Software AG.

  • Prerequisite Versions of Entire Net-Work with Adabas
    See the Adabas and Entire Net-Work documentation to determine prerequisite versions of Entire Net-Work to use with Adabas at your site.

Copying the Persistent Store from/to another Adabas File or Database

The DBA can perform an UNLOAD of the Adabas file in which the persistent store is located (this must be done when Broker is not running). This allows the persistent store to be LOADed into another Adabas file, in the same or in another Adabas database. Broker can then be restarted (PSTORE=HOT) with the attributes specifying the new location of the persistent store file. See Table of Adabas Parameter Settings above. See separate Adabas documentation for details of Adabas utilities for UNLOAD and LOAD operations.

The persistent store file can only be reloaded into another Adabas database running on the same platform type as the Adabas database from which it was unloaded.

c-tree Database as Persistent Store

EntireX provides a c-tree© persistent driver based on the c-tree© User API of the FairCom Corporation®. This driver manages a fast and reliable embedded local database.

In order to operate EntireX using the c-tree persistent store option, you must assign Broker attribute PSTORE-TYPE=CTREE. No other attributes are required. However, several attributes are supported to set additional optional attributes for the c-tree store. See c-tree-specific Attributes for details.