Examining GT.M's IPC Resources

GT.M uses UNIX IPC resources as follows:

The following exercise demonstrates how GT.M utilizes IPC resources in a multisite database replication configuration. The task is to set up a replicated GT.M database configuration on two servers at 2 different locations.

The steps of this task are as follows:

  1. Create 2 databases-America and Brazil-on 2 different servers ( Server_A and Server_B) and deploy them in a multisite database replication configuration so that America is the primary site and Brazil is the secondary site. Ensure that no GT.M processes exist on either server.

  2. In Server_A and in the directory holding database files for America give the following commands (note that because the default journal pool size is 64MB, a value of 1048576 bytes - GT.M's minimum size of 1MB for this exercise):

    $ export gtm_repl_instance=multisite.repl 
$ mupip replicate -instance_create -name=America 
$ mupip set -replication=on -region "*" 
$ mupip replicate -source -start -buf=1048576 -secondary=Server_B:1234 -log=A2B.log -instsecondary=Brazil

  3. Now execute the following command:

    $ gtm_dist/ftok mumps.dat multisite.repl

  4. This command produces the "public" (system generated) IPC Keys (essentially hash values) for mumps.dat and its replication instance multisite.repl. It produces a sample output like the following:

      mumps.dat :: 721434869 [ 0x2b0038f5 ] 
multisite.repl :: 721434871 [ 0x2b0038f7 ]

  5. The keys starting with 0x2b (Hexadecimal form) are the keys for the semaphores used by replication instance America with the high order hexadecimal 0x2b replaced by 0x2c for the replication instance file (GT.M's standard prefix for semaphores for journal pools is 0x2c and that for database files is0x2b). You can observe this with the ipcs command:

    ------ Semaphore Arrays --------
key  semid owner perms nsems
0xd74e4524 196608 welsley 660 1
0x2c0038f7 983041 welsley 777 3
0x00000000 1015810 welsley 777 5
0x2b0038f5 1048579 welsley 777 3
0x00000000 1081348 welsley 777 3
    [Note]Note

    You can expect files in separate file systems to share the same public ftok. This is a normal behavior for large systems with multiple installations and does not affect GT.M operations in any way. This is because GT.M does not assume the semaphore has a one-to-one relationship with the resource and startup/shutdown operations are relatively rare, so the interference among resources have a minimal or no impact. However, the private semaphore (with the 0 key) is unique for a database and is used while a process is actively using the resource.

  6. Execute the following command and note down the shared memory id and private semaphore id on instance America.

    $ mupip ftok mumps.dat

    This command identifies the "private" (GT.M generated) semaphores that a process uses for all "normal" access. The sample output of this command looks like the following:

    File  ::   Semaphore Id   ::   Shared Memory Id  :: FileId
---------------------------------------------------------------------------------------------------------------
mumps.dat ::  1081348 [0x00108004] :: 2490370 [0x00260002] :: 0xf53803000000000000fe000000000000ffffffd2

  7. Now, execute the following command and note down the shared memory and private semaphore id for journal pool.

    $ mupip ftok -jnl multisite.repl

    The sample output of this command looks like the following:

    File   :: Semaphore Id     ::   Shared Memory Id  :: FileId
---------------------------------------------------------------------------------------------------------------
multisite.repl :: 1015810 [0x000f8002]  ::  2457601 [0x00258001] :: 0xf73803000000000000fe000000000000ffffffd2

    Note that the Semaphore id 1015810 and Shared Memory ID 2457601 are in the sample output of the ipcs -a command below.

  8. Now execute the command ipcs -a to view the current IPC resources. This command produces an output like the following:

    ------ Shared Memory Segments --------
key  shmid owner perms bytes nattch status
0x00000000 0  root  777 122880 1
0x00000000 2457601 welsley 777 1048576 1
0x00000000 2490370 welsley 777 2633728 1
0x00000000 2523139 welsley 600 393216 2  dest
0x00000000 2555908 welsley 600 393216 2  dest
0x00000000 1048583 welsley 600 393216 2  dest
0x00000000 1081352 welsley 600 393216 2  dest
0x00000000 1114121 welsley 666 376320 2
0xd74e4524 1146890 welsley 660 64528 0
0x00000000 1933323 welsley 666 62500 2
0x00000000 1966092 welsley 666 1960000 2
------ Semaphore Arrays --------
key  semid owner perms nsems
0xd74e4524 196608 welsley 660 1
0x2c0038f7 983041 welsley 777 3
0x00000000 1015810 welsley 777 5
0x2b0038f5 1048579 welsley 777 3
0x00000000 1081348 welsley 777 3
 
------ Message Queues --------
key  msqid owner perms used-bytes messages

    Using the following formula, where n is the number of regions, to calculate GT.M's IPC resources in a multisite replication configuration:

    IPCs = (n regions * (1 shm/region + 1 ftok sem/region + 1 private sem/region)) 
+ 1 sem/journal-pool + 1 sem/receiver-pool

    In this case, America has one region and no receiver-pool so:

    1 region * 3 IPCs/region + 1 IPC/journal-pool = 4 IPCs

    Therefore, assuming that instance America has 1 region, the total IPC utilized by GT.M is: 4 [1 * 3 + 1 +0]. Note that there is no receiver pool for instance America.

    [Note]Note

    For MUPIP RECOVER operations the total number of IPC resources are 3n (As there is no Journal Pool or Receiver Pool) where n is the number of regions.

  9. Now connect to Server_B and give the following commands in the directory holding database files for Brazil:

    $ export gtm_repl_instance=multisite1.repl 
$ mupip replicate -instance_create -name=Brazil $ mupip rundown -region "*"
$ mupip set -journal="enable,before,on" -replication=on -region "*"
$ mupip replicate -source -start -passive -buf=1048576 -log=B2dummy.log -inst=dummy 
$ mupip replicate -receive -start -listenport=1234 -buf=1048576 -log=BFrA.log

    Now execute the command:

    $gtm_dist/ftok mumps.dat multisite1.repl

    This command produces the "public" (system generated) IPC Key of mumps.dat and its replication instance multisite1.repl. It produces a sample output like the following:

      mumps.dat :: 722134735 [ 0x2b0ae6cf ]
multisite1.repl :: 722134737 [ 0x2b0ae6d1 ]

    Note that keys starting with 0x2b in the output of the ipcs -a command are the public IPC keys for the semaphores of the database file on replication instance Brazil.

  10. Then, execute the following command and note down the shared memory id and private semaphore id on instance Brazil.

    $ mupip ftok mumps.dat

    This command identifies the "private" (GT.M generated) semaphores that a process uses for all "normal" access. The sample output of this command looks like the following:

    File :: Semaphore Id  :: Shared Memory Id :: FileId
--------------------------------------------------------------------------------------------------------------
mumps.dat :: 327683 [0x00050003] :: 11665410 [0x00b20002]:: 0xcfe63400000000000a0000000000000000000000

  11. Now, execute the following command and note down the shared memory and private semaphore id for journal pool.

    $ mupip ftok -jnl multisite1.repl

    The sample output of this command looks like the following:

    File  :: Semaphore Id  :: Shared Memory Id :: FileId
---------------------------------------------------------------------------------------------------------------
multisite1.repl :: 262145 [0x00040001] :: 11632641[0x00b18001]:: 0xd1e63400000000000a0000000000000000000

    Note that the Semaphore id 262145 and Shared Memory ID 11632641 are in the sample output of the ipcs -a command below.

  12. Now, execute the command ipcs -a to view the IPC resources of Brazil.

    This command produces a sample output like the following:

    ------ Shared Memory Segments --------
key  shmid owner perms bytes nattch status
0x00000000 11632641 gtmuser 777 1048576 3
0x00000000 11665410 gtmuser 777 2629632 2
0x00000000 11698179 gtmuser 777 1048576 2
------ Semaphore Arrays --------
key  semid owner perms nsems
0x2c0ae6d1 229376 gtmuser 777 3
0x00000000 262145 gtmuser 777 5
0x2b0ae6cf 294914 gtmuser 777 3
0x00000000 327683 gtmuser 777 3
0x00000000 360452 gtmuser 777 5
------ Message Queues --------
key  msqid owner perms used-bytes messages

Brazil has 1 region and its receiver server is listening to America, therefore as per the formula for calculating GT.M IPC resources, the total IPCs utilized by GT.M is: 5 [1 * 3 + 1 + 1].

PrevA UpA Next
A HomeA