GT.M components use the $gtm_db_startup_max_wait environment variable to determine how long to wait for a resolution of any resource conflict when they first access a database file. GT.M uses semaphores maintained using UNIX Inter-Process Communication (IPC) services to ensure orderly initialization and shutdown of database files and associated shared memory. Normally the IPC resources are held in an exclusive state only for minuscule intervals. However, under unusual circumstances that might include extreme large numbers of simultaneous database initializations, a long-running MUPIP operation involving standalone access (like INTEG -FILE or RESTORE), an OS overload or an unpredicted process failure the resources might remain unavailable for an unanticipated length of time. $gtm_db_startup_max_wait specifies how long to wait for the resources to become available:

The default value for the wait if $gtm_db_startup_max_wait is not defined is 96 seconds. Previously, a process waited 60 seconds for one semaphore and immediately tried to acquire the other semaphore. If either of these attempts failed, it variously gave CRITSEMFAIL or REQRUNDOWN in addition to the above mentioned errors. [UNIX](GTM-4337)

MUPIP JOURNAL -ROLLBACK recognizes the -ONLINE qualifier which specifies that ROLLBACK run without requiring standalone access to the database and the replication instance file. GT.M provides the $ZONLNRLBK intrinsic special variable which it increments every time the process detects a concurrent MUPIP JOURNAL -ONLINE -ROLLBACK. If the logical state of the database after the completion of MUPIP JOURNAL -ONLINE -ROLLBACK matches the logical state of the database at the start of MUPIP JOURNAL -ONLINE .ROLLBACK, that is: only removes or commits an uncommitted TP transaction or non-TP mini-transaction, any transaction (TP or Non-TP) incurs a restart and nothing more. If the MUPIP JOURNAL -ONLINE -ROLLBACK changes the logical state of the database GT.M affects concurrent processing by incrementing $ZONLNRLBK and, In a TP transaction including trigger code within a transaction: restarts the transaction, or in a Non-TP mini-transaction, including within an implicit transaction caused by a trigger: issues a DBROLLEDBACK error, which, in turn, invokes the error trap if $ETRAP or $ZTRAP are in effect. Any utility/command attempted while MUPIP JOURNAL -ONLINE -ROLLBACK operates waits for ROLLBACK to complete; the $gtm_db_startup_max_wait environment variable (see GTM-4337) configures the wait. The default behavior of MUPIP JOURNAL -ROLLBACK is -NOONLINE, which requires standalone access on the database and replication instance file. Previously, ROLLBACK always required standalone access. At this time, FIS considers MUPIP JOURNAL -ROLLBACK -ONLINE to be of field test grade quality; please do not use this in production without doing comprehensive evaluation and testing. We would especially appreciate any feedback on this feature. In its current form, we expect its main usefulness would be to automatically rollback a replicating instance in response to a rollback on the originating instance.

If ROLLBACK (either- NOONLINE or -ONLINE) terminates abnormally (say because of a kill -9), it leaves the database in a potentially inconsistent state indicated by the FILE corrupt field in the database file header. When a ROLLBACK terminates leaving this field set, all other processes receive DBFLCORRP errors any time they attempt to interact with the database. You can clear this condition as following in descending order of risk:

However, the MUPIP and DSE actions do not ensure that the database has consistent state; check for database integrity with MUPIP INTEG.

The project to introduce this feature also addressed the following changes:

GT.M manages database and replication locks more carefully. Previously, in some rare cases, a conflict between internally managed locks for a region and the replication [journal] pool could result in a deadlock.

MUPIP now correctly manages control structures if it doesn't succeed in allocating shared memory. Previously such an unusual failure leaked a small amount of memory.

MUPIP RESTORE now ensures that until the last of the incremental BACKUP files are restored, the process continues to operate in a standalone mode. Previously, it was possible for a concurrent process to sneak in while restore was switching input files and start accessing the database causing inconsistency.

MUPIP now appropriately interprets MUPIP INTEG -ONLINE as equivalent to MUPIP INTEG -ONLINE -REGION and treats the response at the "File or Region:" prompt as a region. Previously, MUPIP treated MUPIP INTEG -ONLINE as MUPIP INTEG -FILE and assumed any input provided for "File or Region:" to be a file rather than a region.

If the target database is frozen when MUPIP JOURNAL -RECOVER and -NOONLINE -ROLLBACK start, they issue a DBFRZRESETFL informational message and unfreeze the database. Previously, RECOVER did not issue any such informational message when it performed the unfreeze.

MUPIP REPLIC -START -LISTENPORT -RECEIV now supports a new optional qualifier -AUTOROLLBACK[=VERBOSE] which allows the Receiver Server to run ONLINE FETCHRESYNC ROLLBACK on receiving a REPL_ROLLBACK_FIRST message from the Source Server. The optional value keyword VERBOSE allows the ONLINE FETCHRESYNC ROLLBACK to provide more diagnostic information. Previously, on receiving REPL_ROLLBACK_FIRST message, the Receiver Server halted and required FETCHRESYNC ROLLBACK to be run before restarting the Receiver Server.

DSE ALL -WCREINIT and DSE WCREINIT now sends an operator log - DSEWCREINIT - to record this unusual event. Previously, no such message was sent to the operator log.

For more information on the areas affected by ONLINE ROLLBACK, refer to Documentation Addendum. [UNIX] (GTM-6306)

GT.M now uses a revised filename-to-key (ftok) algorithm to locate shared resources on all UNIX platforms. The algorithm takes into account the identity of the file (inode number, generation number) and the identity of the file system containing it (major device, minor device). The previous algorithm had a higher probability of collisions between databases on different devices, with a consequent higher probability of synchronization problems.

A prior GT.M release using the old algorithm may conflict with the current release if they concurrently attempt to initialize shared resources. Take care to avoid using prior and current GT.M versions at the same time on the same database. As with any hash algorithm,there is a very small probability of collisions with the new algorithm too; this can be detected using the "ftok" utility included with GT.M. In the event more than one database file produces the same ftok hash, copy the database file to a new location and use the copy, which will have a different key, instead of the original.

In addition, MUPIP RUNDOWN on Unix now outputs a VERMISMATCH error after two shared memory size messages when an older GT.M is using the database. Previously, MUPIP would have only output the shared memory size messages, or in some cases (AIX or Solaris with a 32-bit prior version) succeeded, leading to database corruption.

	It is still possible to corrupt a database by running an older version of MUPIP RUNDOWN while a current GT.M has the database open. Be certain to shut down all GT.M processes before switching GT.M versions. [UNIX] (GTM-6811)

The encryption reference plug-in supports a method you can employ if you want to pass an obfuscated password between unrelated processes (e.g., a child process with a different userid invoked via a sudo mechanism), or even from one system to another (e.g., over an ssh connection). When the environment variable gtm_obfuscation_key names a file readable by the process, the plug-in uses an SHA-512 hash of the file's contents as the XOR mask for the obfuscated password in the environment variable gtm_passwd. When gtm_obfuscation_key does not point to a readable file, the plug-in creates an XOR mask based on the userid and inode of the mumps executable, as it did in previous versions, and then computes an SHA-512 hash of the XOR mask to use as a mask. In addition, the maskpass program handles passphrases containing blanks. Previous versions did not support blanks in the password, did not hash the mask before performing the XOR obfuscation and used a mask that was unique to the process and so could not be used to pass the password except to another process with the same user on the same GT.M installation. [UNIX] (GTM-6953)

If the duplicate SET optimization is enabled, and if a SET causes no change in the value of a preexisting node, that update now invokes any associated triggers. Previously it did not invoke the triggers in this case. It is now up to the trigger code to check if the value of the node did or did not change due to this update and take an appropriate course of action. This also makes trigger invocation behavior consistent between an originating and its replicating instances. Previously each GT.M process on the originating primary used to be able to control whether triggers were invoked or not for duplicate sets whereas the Update Process on the replicating instance (which performs updates on behalf of all the GT.M processes on the originating primary) inherited only one setting for trigger invocation (in case of duplicate sets) and hence did not always mirror the GT.M process in terms of trigger invocations. This could lead to potential out-of-sync situations between the originating instance and its replicating instance due to different trigger code being executed on the two instances and in addition caused a huge journal file on the replicating instance compared to the journal file on the replicating instance.

The environment variable gtm_gvdupsetnoop is no longer honored. By default, the duplicate set optimization is now enabled (i.e. equivalent to gtm_gvdupsetnoop previously being set to 1). The only way to control this feature is with a VIEW command (View "GVDUPSETNOOP":0 disables the optimization, View "GVDUPSETNOOP":1 enables it). (GTM-7018)

Trigger updates from MUPIP TRIGGER and $ZTRIGGER() now handle concurrency conflicts with appropriate transaction restarts. Previously concurrency conflicts with these update could cause misleading TRIGDEFBAD errors [UNIX] (GTM-7020)

GT.M now correctly allows all processes with read-write access to a database file to switch journal files as long as they have read-write access to the journal file directory. Previously, in an edge case, GT.M did not permit a process with read-write access to a database file to perform a needed journal file switch. As part of this change, we have generated documentation to enumerate the various cases for ownership and permissions of shared resources. This documentation is reproduced in the Shared Resource Authorization Permissions section. (GTM-7122)

MUPIP TRIGGER, $ZTRIGGER(), and the ZTRIGGER command now operate correctly on a read-only database or in an environment where some databases are read-only and others are not. Previously, these functions ignored the read-only setting of the database updating shared memory anyway. If other processes had the database open read-write, the changes could be flushed to the database. The ZTRIGGER command assumed it had read-write access which could cause flushing issues. [UNIX] (GTM-7158)

TROLLBACK to a $TLEVEL other than zero (0) now leaves $REFERENCE empty which is the same behavior as a full TROLLBACK to $TEVEL=0. Previously such a TROLLBACK left $REFERENCE with a value that was a function of the last rolled back action. (GTM-4033)

Interrupted GT.M HANG commands now account for the time spent handling the interrupt. Previously, the HANG command resumed using the full specified wait time thus potentially hanging indefinitely if hit with a stream of interrupts. (GTM-5996)

Interrupted timed GT.M LOCK commands now account for the time spent waiting before the interrupt when the LOCK resumes its wait. The time spent handling the interrupt is ignored thus the locks specified in the command may be waited on during several intervals which add up to the wait time specified in the LOCK command. Note that while handling the interrupt, the lock requests (and any partial reservations) are released. Previously, the LOCK command restarted the full wait time thus never completing if additional interrupts arrived. (GTM-5997)

GT.M now evaluates command postconditionals for invalid commands at runtime before generating a GTM-E-INVCMD error. If GT.M exaluates the postconditional to false, GT.M does not raise an error and continues execution. GT.M generates compile time GTM-E-INVCMD messages as before, though if the command includes a postconditional the compilation continues. Previously, GT.M considered the invalid command a run time error regardless of the command postconditional. (GTM-6185)

GT.M properly manages stack frames when fielding CTRAP interrupts from the console. Previously, if a $ETRAP did not handle a CTRAP error, GT.M mishandled the re-throw of the error in frames associated with an indirection, for example DO @X, subsequently causing a GTMASSERT or address exception (UNIX SIG-11 or OpenVMS ACCVIO). (GTM-6228)

<CTRL-D> at the GT.M Direct Mode prompt on an empty line terminates GT.M and returns control to the shell. FIS thanks Sam Habiel for this enhancement. Previously GT.M ignored <CTRL-D> in this circumstance. As in prior versions, if there is a character at the cursor when you enter <CTRL-D> in direct mode, GT.M erases it; if there are characters on the line but the character position is at the end of the line, GT.M ignores <CTRL-D>. Note: to delete the prior character on the line, use the key defined in your TERMINFO as key_backspace. [UNIX] (GTM-6449)

There are a few very small and infrequent windows in GT.M code (expected to be under a microsecond on contemporary hardware) during which a process should not be terminated (hence the recommendation to use kill -9 on GT.M processes only as a last resort). When a MUPIP STOP is received within such a window, GT.M now defers process termination until it is outside the window. If, for some reason, the process is stuck within such a window of protected code, and receives three MUPIP STOP signals, it stops immediately. Previously, GT.M processes responded to MUPIP STOP signals immediately, and if one received a MUPIP STOP within such a window, it might generate a GTMASSERT or other failure while attempting to terminate. This change should require no operational changes in GT.M because of the tiny windows within which MUPIP STOP is deferred---the three STOP technique being required only to recover from unanticipated out-of-design conditions. [UNIX] (GTM-6569)

GT.M ignores SIGINT (control-C) if $PRINCIPAL is not a terminal. Previously receipt of this signal blocked subsequent out-of-band events such as MUPIP INTRPT or TPTIMEOUT. [UNIX] (GTM-6614)

GT.M better handles some error processing cases. Previously, GT.M used to fail with addressing exceptions (UNIX SIG-11 or OpenVMS ACCVIO) while capturing error context information for $STACK(). Other changes have addressed the known errors in that category but this improves the robustness of GTM by giving it a better chance to continue operating even in adverse circumstances. (GTM-6622)

When returning from code invoked by MUPIP INTRPT (clearing $ZININTERRPT), GT.M implicitly clears any error(s) detected while 1=$ZININTERRUPT and sends a JIUNHNDINT error notification to the operator log, for example:

Sep 26 11:41:09 flyingv GTM[15119]: %%GTM-E-JIUNHNDINT, An error during $ZINTERRUPT 
processing was not handled: 150379874,jobintr2+4^jobintr,
%%GTM-E-ERRWZINTR, Error while processing $ZINTERRUPT
-GTM-E-LABELMISSING, Label referenced but not defined: 
nonexistantrtn -- generated from 0x00007FEFD25B831A.

Prior versions followed the basic error handling pattern of re-throwing the most recent unhandled error at each level as long as $ECODE is not equal to the empty string. [UNIX] (GTM-6711)

GT.M now provides better management of timed events by deferring their processing during certain brief sections of code. Previously, GT.M could not keep timers alive while preventing various timer-driven events from interfering with the executing code; this limitation resulted in a number of very rare failures, caused by either a lost timer or a timed event delivered at an inappropriate moment, resulting in symptoms such as hung processes and GTMASSERTs. In addition, GT.M now raises the COREINPROGRESS error if an attempt to generate a core fails and triggers an attempt to generate another core. Previously, under certain very rare conditions, GT.M could get into a core generation loop potentially eating up disk space and process slots. [UNIX] (GTM-6719)

The installation kit now includes a random string generator utility function called %RANDSTR (strlen,charranges,patcodes,charset). This generates a random string of length strlen from an alphabet defined by charset or by charranges and patcodes. If charset specifies a string of non-zero length, %RANDSTR generates the random string using the characters in charset, otherwise it takes its alphabet as specified by charranges and patcodes. charranges uses the same syntax used for FOR loop ranges, for example, 48:2:57 to select the even decimal digits or 48:1:57,65:1:70 to select hexadecimal digits. patcodes specifies pattern codes used to restrict the characters to those that match the selected codes. If charset is of zero length, and is passed by reference, %RANDSTR() initializes it to the alphabet of characters defined by charranges and patcodes. If not specified, strlen defaults to 8, charranges defaults to 1:1:127 and patcodes to "AN". (GTM-6738)

GT.M now defers TPTIMEOUT interrupts as follows:

GT.M delivers interrupts including TPTIMEOUTs at the beginning of the next line, the beginning of the next FOR iteration or while waiting in a command with a timeout. Previously GT.M delivered TPTIMEOUT as described in the prior sentence without regard to the error, interrupt or transaction state. In addition, GT.M does not adjust state information related to transaction retry on any error, including on the error used to signal a TPTIMEOUT. This reduces the possibility of indefinite transaction restarts. This also means that the Principal device of a process in a transaction entering direct mode now displays a TPNOTACID error. Note that GT.M does not support transaction activity in Direct Mode. FIS recommends restricting Direct Mode interaction with transactions to investigating state for debugging purposes. Previously, GT.M released the database critical section on an error or interrupt. [UNIX] (GTM-6759)

ZMESSAGE now treats any non-GT.M error as a trappable or fatal event. In prior releases, a ZMESSAGE argument with low order bits indicating success or information in the GT.M error designation scheme, for example ZMESSAGE 1, went to $PRINCIPAL without activating an error trap. [UNIX] (GTM-6771)

Label invocations using DO no longer require parentheses for calls with no actuallist. If DO or a $$ that does not specify an actuallist invokes a label with a formallist, the missing parameters are undefined in the called routine. As a result, a GT.M process no longer issues ACTLSTEXP and FMLLSTPRESENT errors, regardless of the arguments passed and parameters expected. WARNING: If DO or $$ specifies a routine but no label using an actuallist, then whether that routine's top label has a formallist or not, the actuallist applies to it directly, whereas before the actuallist would "fall through" to the first label with executable code. Previously, excluding parentheses from a DO invocation of an entryref with a formallist resulted in FMLLSTPRESENT error for local labels, and ACTLSTEXP error for labels in external routines and indirect expressions. (GTM-6813)

The gtm_trace_gbl_name environment variable enables GT.M tracing at process startup. Setting gtm_trace_gbl_name to a valid global variable name instructs GT.M to report the data in the specified global when a VIEW command disables the tracing, or implicitly at process termination. This setting behaves as if the process issued a VIEW "TRACE" command at process startup. However, gtm_trace_gbl_name has a capability not available with the VIEW command, such that if the environment variable is defined but evaluates to zero (0) or, only on UNIX, to the empty string, GT.M collects the M-profiling data in memory and discards it when the process terminates (this feature is mainly used for in-house testing). Note that having this feature activated for process that otherwise don't open a database file (such as GDE) can cause them to encounter an error. Previously, the only way to start tracing was with a VIEW command. As with the VIEW command, tracing initiated with gtm_trace_gbl_name can be redirected or turned off by a subsequent VIEW command.

In addition, if a process issues a malformed VIEW command that attempts to turn tracing off, GT.M issues an error but retains all accumulated profiling data and continues tracing. If the tracing is still enabled at the process shutdown and the trace start specified a reporting location, GT.M attempts to place the trace data there. Note that if there is a problem updating the specified trace-reporting global variable, GT.M issues an error at process termination. Previously, any VIEW command that unsuccessfully attempted to transfer tracing data to a global variable turned tracing off and discarded all collected data. On UNIX, for each line executed, GT.M tracing now reports the elapsed time in addition to CPU times (user, system, and combined).

It also reports two aggregate entries for UNIX and one on OpenVMS, "*RUN" for the current process and, on UNIX, "*CHILDREN" for all of child processes spawned by the current process, each containing user, system, and combined CPU times. Previously, the trace facility did not report these details of the child processes for the ZSYSTEM command and PIPE devices that are closed before the trace data is reported. The "CHILD" category data excludes processes that result from the JOB command, PIPE devices OPENed with the INDEPENDENT device parameter and processes from PIPE devices that are still active. Note that M-profiling is currently disabled for DAL calls on OpenVMS. (GTM-6819)

GTM now correctly processes WRITE mnemonics for GTM socket devices. Previously,a WRITE mnemonic usage such as WRITE /LISTEN or WRITE /WAIT without the depth value (for /LISTEN) or without a timeout (for /WAIT) could cause either a INVCTLMNE error or an addressing exception (signal-11 (UNIX) or ACCVIO (VMS)). (GTM-6836)

The GT.M parser now appropriately recognizes the end of a source line. Previously it preprocessed the next character, which resulted in an inappropriate SPOREOL error if the two last lines in a routine had certain odd characteristics. The workaround was to add another line to the routine - either empty or comment lines work if a QUIT is not appropriate (GTM-6845)

$ZDATE() generates a ZDATEBADDATE for input date values greater than 31-Dec-999999 (364570088) or less than 01-JAN-1840 (-365), a ZDATEBADTIME for time values greater than a second before midnight (86399) or less than 0 (zero). Previously if the input was too large (either positive or negative), the process might produce meaningless output or terminate with a segmentation violation (UNIX SIG-11 or OpenVMS ACCVIO). Because invalid conditions that used to be ignored are now treated as errors, this change might require revisions to existing application code. (GTM-6851)

GT.M now evaluates functions with side effects - extrinsic functions ($$), external functions ($&) and $INCREMENT() in left-to-right order when in full_boolean mode. In V5.4-002 it evaluated those functions early to ensure they were not skipped by the default GT.M short-circuit optimization, however this could cause their side-effects to occur out of order. (GTM-6854)

GT.M correctly compiles expressions like if a<$$x(b-c) when specifying FULL_BOOLEAN. Previously this combination caused a GTMASSERT. (GTM-6856)

MERGE now gives no error when the source variable and the destination variable are identical. Previously it would give a MERGEDESC error message. (GTM-6935)

$ZTRNLNM() now recognizes all the documented keywords including all the OpenVMS keywords. FULL and VALUE (the default) return the translation and LENGTH returns the length. Other keywords select default values. Many of the keywords return the empty string if the item to translate has no value. Previously it only recognized "FULL". Note that on UNIX $ZTRNLNM() does not do iterative translation. [UNIX] (GTM-6957)

64-bit GT.M editions - currently those except HP Alpha (OpenVMS and Tru64), HP PA-RISC (HP-UX), and 32-bit x86 (Linux) - now pass 64-bit long and ulong values between C and M though the call-out and call-in interfaces. GT.M represents values that fit in 18 digits as numeric values, and values that require more than 18 digits as strings. M code may use these strings in a numeric context, but the numeric result has less precision than the original value. Note that this change may expose latent problems if a call-in definition file contains incorrectly typed values. Prior versions truncated long and ulong values passed between C and M through the these interfaces to 32-bit int values even on 64-bit platforms. (GTM-6994)

A ZSHOW "D" that includes one or more SOCKET devices manages memory in a stable fashion. In prior versions, each such ZSHOW leaked a small amount of memory. (GTM-6996)

The INDEPENDENT device parameter now has the correct behavior for all shells. Previously, the subprocess started by the PIPE device would not continue to run after the PIPE device was closed for some shells, such as the Bourne shell. The RENAME device parameter now correctly renames a sequential file if it is followed by another device parameter. Previously, the new file name of the sequential file could contain information from the following parameter. [UNIX] (GTM-7002)

GT.M now handles nesting of XECUTE strings that contain subscripted FOR control variables. In V5.4-002x such constructs could result in a memory segmentation violation (UNIX SIG-11 or OpenVMS ACCVIO). (GTM-7007)

GTM correctly handles an indirect DO without arguments (e.g. DO @x) inside an XECUTE argument. In prior versions, this construct caused a GTMASSERT or an addressing exception. (GTM-7008)

GT.M now issues a DEVOPENFAIL error for a PIPE device if either the SHELL or STDERR device parameters are set to the null string. Previously, the open error was ignored and any subsequent ZSHOW "D" caused a SIG-11. In addition, the DEVOPENFAIL error message for a PIPE device now outputs the device name. Previously, it output the device type. [UNIX] (GTM-7123)

GT.M uses an additional form of GTMASSERT error - GTMASSERT2, which adds information about the failing condition. This additional information may only be useful to FIS or others familiar with GT.M internals. For example:

%GTM-F-GTMASSERT2, GT.M V5.5-000 Linux x86 - Assert failed /usr/library/V994/src/mdb_condition_handler.c 
line 490 for expression (TPRESTART_STATE_NORMAL == tprestart_state)
%GTM-F-GTMASSERT2, GT.M V5.5-000 Linux x86 - Assert failed /usr/library/V994/src/gds_rundown.c 
line 543 for expression (NULL != csa->wbuf_dqd)

GTMASSERT and GTMASSERT2 indicate GT.M has detected a condition that seems to violate a design assumption and should be reported to your GT.M support channel. Prior versions did not provide the additional diagnostic information reported by GTMASSERT2. (GTM-7078)

ZBREAK and ZPRINT() now issue errors when invoked with an indirect routine name that evaluates to the empty string other than a string literal, for example, the return from a $GET(). Previously this construction caused a memory segmentation violation (UNIX SIG-11 or OpenVMS ACCVIO). (GTM-7176)

The MERGE command now counts global gets as well as updates in the database counters displayed with ZSHOW "G" and DSE. Previously MERGE only counted the update portion of its work. (GTM-7179)

FOR now evaluates a termination value specified with a subscripted local variable at loop initiation. Previously it incorrectly reevaluated the termination value at each iteration. (GTM-7185)

On platforms where the information is unavailable error messages that report the address of an event now report a hexadecimal -1 (all F's). This information in a message is typically of use only to FIS or others who work with the GT.M implementation code. Previously GT.M reported a zero (all 0's) address in such cases, which called into question whether the error message was properly constructed. [UNIX] (GTM-7188)

M-profiling no longer leaks memory. Previously, processes using M-profiling experienced a memory leak and could run out of memory when running code with repetitive function invocations. Also, M-profiling no longer receives a SIGSEGV signal during process termination. Previously, a tracing-enabled GT.M process interrupted at an inopportune time could die with a SIGSEGV condition. In addition, M-profiling correctly reports each label count entry. Previously, such it did not report entries for labels invoked with a DO command inside a FOR construct that used a variable rather than a literal for the termination value. (GTM-7196)

GT.M completely removes all trigger artifacts during an unlink using the form ZGOTO 0:entryref. Previously, GT.M removed the code but left some trigger-related structures behind, which could cause memory segmentation violations (SIG-11) or other bad behavior. [UNIX] (GTM-7206)

In OpenVMS, GT.M more reliably reports the name of an undefined unsubscripted local variable. Previously, depending on circumstances, the UNDEF error message sometimes failed to report the variable name. [OpenVMS] (GTM-7210)

The GT.M LOCK manager now guards against unlikely, but possible, mismanagement of LOCK resources. This is not an issue that we are aware of anyone ever encountering, but is analogous to a relatively uncommon issue we saw with local variables on certain x86 Linux releases. Previously GT.M could, at least in theory, fail to properly maintain resource names. (GTM-7214)

At runtime, gtm_zquit_anyway environment variable causes code of the form QUIT <expr> execute as if it were SET <tmp>=<expr> QUIT:$QUIT tmp QUIT, where <tmp> is a temporary local variable in the GT.M runtime system that is not visible to application code. This mode no longer relies on the GT.M compiler. Previously, the gtm_zquit_anyway environment was used when compiling code, and caused code of the form QUIT <expr> to be compiled as QUIT:$QUIT <expr> QUIT. While FIS normally avoids changing GT.M behavior in any way that is not upward compatible, in this instance FIS has worked with users of gtm_zquit_anyway to determine that this changed behavior is more appropriate to existing applications. (GTM-7215)

The LOCK (and ZALLOCATE and ZDEALLOCATE) commands now manage LOCK resource names more efficiently in LOCK_SPACE. Previously they used more space than they really needed and could lose efficiency or generate an error when it was unnecessary. (GTM-7228)

A JOB command with no parameters created by a JOB'd process with one or more parameters receives no parameters. Previously the JOB command with no parameters inappropriately received the first parameter of the JOB'd parent process. [UNIX] (GTM-7234)

MUPIP REORG -TRUNCATE[=percentage] specifies that REORG, after it has rearranged some or all of a region's contents, should attempt to reduce the size of the database file and return free space to the file system. The optional percentage (0-99) provides a minimum amount for the reclamation; in other words, REORG won't bother performing a file truncate unless it can give back at least this percentage of the file; the default (0) has it give back anything it can. TRUNCATE always returns space aligned with bit map boundaries, which fall at 512 database block intervals. TRUNCATE analyses the bit maps, and if appropriate, produces before image journal records as needed for recycled (formerly used) blocks; The journal extract of a truncated database file may contain INCTN records having the inctn opcode value 9 indicating that the specific block was marked from recycled to free by truncate. TURNCATE only works on V5 databases - it does not work on database files that contain unconverted V4 format blocks. TRUNCATE does not complete if there is a concurrent online BACKUP or use of the snapshot mechanism, for example by INTEG. [UNIX] (GTM-3873)

The NOREPLCTDREG error message now correctly identifies the reported context as the instance file. Previously since the introduction of multi-site replication on UNIX that error mislabeled the file as a global directory. The label has not changed on OpenVMS, where it is correct. [UNIX] (GTM-4202)

MUPIP REORG now takes an optional -REGION qualifier. MUPIP REORG -REGION <region-list> reorganizes each global in the specified regions. Note that a global must be in the global directory in order for MUPIP REORG to process it. Previously, REORG would reorganize every global in the global directory unless specific globals were selected with -SELECT or EXCLUDE. Note that MUPIP REORG previously supported a -REGION qualifier, but only in conjunction with the -UPGRADE and -DOWNGRADE qualifiers. (GTM-5797)

GT.M replication supports Supplementary Instances - instances which both receive updates from some other originating instance and local updates. Please refer to the Supplementary Instance Technical Bulletin for detailed information.

The project to introduce this feature also addressed the following changes most, but not all, of which apply only to UNIX editions:

The Source Server exits with error JNLNOREPL if it opens a journal file and finds that replication was not on for that journal. Also it retries any journal file read in the current journal file that returns an incomplete record; in UNIX, if it reads an incomplete record from a previous generation journal file, it exits with a JNLRECINCMPL error. Previously, these two conditions caused a GTMASSERT. Also, in UNIX, a Source Server does not switch to a new journal if it fails to open the currently active journal file. Additionally, MUPIP SET JOURNAL reports the PREVJNLLINKSET message in the operator log in addition to the operator terminal. Previously, those were only reported to the terminal. (GTM-6736)

A Receiver Server started with the -UPDATERESYNC qualifier starts every Source Server connection handshake during its own lifetime with that characteristic. Previously, a Receiver Server based only the initial handshake for its first connection on the -UPDATERESYNC qualifier, which meant any errors that closed the connection caused it to reconnect in a way that likely caused a REPLINSTNOHIST error. (GTM-6650)

A MUPIP action (RECOVER or ROLLBACK) that restores V4 format blocks to a database that had completed a transition from V4 to V5 format (using MUPIP REORG -UPGRADE) sets the indicator that the database still contains blocks in V4 format. Previously, such a sequence caused the database logic to incorrectly stop checking for V4 format blocks, which could, in rare cases, cause database corruption. The workaround was to take a BACKUP with fresh journal files immediately after completing the MUPIP REORG -UPGRADE. (GTM-6725)

A replication filter no longer needs to package a simple update as a transaction. This change enables the use of filters such as the UNIX tee utility. GT.M also no longer requires the external filter pipe buffer be large enough to accommodate the entire largest transaction. If an external filter returns a bad record, GT.M issues a FILTERBADCONV error. Previously, GT.M required the external filter to return all transactions wrapped as TP transactions, even if they were originally non-TP single updates (mini-transactions), would hang if a transaction exceeded the buffer size for the pipe, and would loop indefinitely if the filter returned a defective record. Note that GT.M still requires that a filter maintain a one-to-one relationship between input and output transactions, as documented. [UNIX] (GTM-6826)

MUPIP JOURNAL -ROLLBACK -FETCHRESYNC (or -RESYNC) now correctly recovers the database and creates a correct unreplicated (lost) transaction file in cases where a second command specifies a later point in time than the prior interrupted command. In prior versions, such a change sequence might create an incorrect recovery. In addition, MUPIP RECOVER -BACKWARD restricts processing that identifies broken transactions to the most recent journal file(s). Previously, backward recovery extended transaction validity checking all the way to the turnaround point at which it starts forward processing, which unnecessarily required more memory and potentially failed if that memory was not available. Also, now the -RESYNC ROLLBACK is more memory efficient and uses approximately same amount of memory as -FETCHRESYNC ROLLBACK uses. Previously -RESYNC required more memory than -FETCHRESYNC to accomplish the same thing. Note that -RESYNC is intended for use when advised by your GT.M support channel and is not recommended for routine normal use. (GTM-6834)

The Source Server no longer hangs while waiting for a the message used in the initial handshake between a Source and Receiver Server to determine if compression can be turned on to send journal records across. In previous versions of GT.M, if a Receiver Server sent a flow control messages just before sending the REPL_CMP_SOLVE message (which appears in the Receiver Server log), the Source Server hung. [UNIX] (GTM-6841)

Cross-endian replication now works correctly in various scenarios. Previously, any abnormal events on the receiver side (e.g. draining the replication pipe in case of Update Process crash or a bad transaction etc.) could result in a hang on the Receiver Server. Note that cross-endian replication is handled robustly by GT.M V5.5-000 only if both the source and receiver sides run that version. If not, only a few combinations are supported. Below is the list.

KILLs of non-existent global nodes (henceforth referred to as duplicate KILLs), which are possible for example if the global mapping is changed in the gld file on the replicating instance, are now handled robustly on the Update Process side by maintaining the journal sequence numbers of the originating instance and its replicating instances in sync. Previously, in this scenario, the Update Process printed a "Number of transactions from Primary which did NOT update JNLSEQNO on Secondary" message in its log indicating the sequence number at which the duplicate KILL occurred but continued processing even though the two instance sequence numbers were no longer in sync. This gave a false impression the two instances were in sync when they actually were not. Additionally, if an update that is played on the Update Process side does not increment the journal sequence number of the replicating instance, the Update Process prints the "Number of transactions from Primary which did NOT update JNLSEQNO on Secondary" message in its logfile and terminates. Previously, it used to continue processing thereby making it very difficult to get the two instances back in sync. (GTM-6898)

MUPIP INTEG and MUPIP JOURNAL -ROLLBACK or -RECOVER handle database files larger than one terabyte. Previously GT.M gave a DBFILOPERR error with a secondary error of invalid argument (ENO22 on UNIX) when they attempted to access data higher than 1TB (GTM-6958).

GT.M ensures not even a single update can occur during a MUPIP FREEZE. This issue was primarily addressed by C9J01-003076 in V5.3-004, however, we discovered one subtle case where under high update contention, a single update from MUPIP REORG might squeeze in right after a FREEZE. (GTM-7004)

MUPIP LOAD can now read from the standard input by specifying the -STDIN qualifier instead of a file-name. [UNIX] (GTM-7016)

MUPIP JOURNAL RECOVER or ROLLBACK -BACKWARD now processes multi-region TP transactions appropriately even if they are broken. Also, the Source Server avoids writes to the database and/or journal as much as possible. In versions starting with V5.4-000, RECOVER or ROLLBACK terminated with a GTMASSERT error if they encountered a complete transaction after a broken transaction. Previously the Source Server could write an INCTN journal record in cases where the database cache integrity was suspect. Both of these issues arose after processes were terminated abnormally, typically with a kill -9 - FIS strongly recommends against using kill -9 on GT.M processes, as it can cause database damage when used to terminate a process that is actively updating a database. (GTM-7034)

MUPIP EXTRACT -FORMAT=BIN now handles database blocks of maximum size. Previously it failed with a "%SYSTEM-E-ENO14, Bad address" error in UNIX or a "%RMS-F-RSZ, invalid record size" error in OpenVMS" (GTM-7077)

MUPIP JOURNAL -SHOW=ALL output now shows counts of all Record Types correctly up to 1 billion. Previously, a Record Type count more than 1 million showed as '*******'. (GTM-7079)

With a "-stdout" (case ignored) as the argument of its -EXTRACT qualifier, the MUPIP JOURNAL command now writes its output to standard output instead of creating a file with the name "-stdout". Although this alters existing GT.M behavior, FIS believes that using "-stdout" to direct output to stdout is more useful than creating a file with a name that makes it awkward to read and process. In addition, MUPIP JOURNAL -EXTRACT now ends with the last extracted journal record. Previously it placed a blank line at the end of the file. [UNIX] (GTM-7082)

The Update Process and journal recover/rollback now correctly handle a NULL type journal record. Previously, they skipped a check on the maximum available space in the current journal file (based on the journal file autoswitchlimit) while writing the NULL journal record. This resulted in the journal file space getting filled up so eventually there is no room left in it for writing even one journal record. At that point, a journal file switch is unavoidable because there is no space but is also impossible because the switching routine needs to write one journal record to the current journal file. This caused them to go into an indefinite recursion eventually running out of C-stack space and terminating abnormally as well as leaving the journal file in an out-of-design state. Any other process that next does a global reference in this database ended up with the same situation leading to a cascade of process terminations and an unusable journal file. (GTM-7230)

DSE FIND -SIBLING now shows the current block in addition to any left and right siblings - all designated in hexadecimal. Previously the command only reported the two siblings when they were in the same branch of the tree, and while the representation was also in hexadecimal notation, it could be ambiguous. In addition, DSE FIND now correctly represents blocks in the global directory tree when they are selected. Previously FIND did not show block one and could represent other directory tree blocks as if they had been in a global tree. (GTM-2296)

GDE and MUPIP now allow the user to set the journal extension size to a maximum of 1073741823 blocks on Unix; the VMS maximum remains at 65535 blocks. This feature required a change to the Global Directory (GLD) file format; the file must be upgraded using GDE. (Refer to the section on Upgrading Your Global Directory.) Note that older versions of GT.M cannot read the upgraded GLD files. Previously, GDE and MUPIP limited the journal extension size to 65535 blocks on all platforms.

GDE now supports AUTOSWITCHLIMIT as an option for the -JOURNAL region qualifier. GDE stores the value in the global directory, and MUPIP now uses the value from the global directory when initializing a region in place of the old default. However, the default AUTOSWITCHLIMIT value for GDE is the same as the old default for MUPIP CREATE, so the default result is unchanged. [UNIX]

When you specify values for ALLOCATION, EXTENSION, and AUTOSWITCHLIMIT for a region such that (ALLOCATION+EXTENSION>AUTOSWITCHLIMIT), either using GDE or MUPIP SET -JOURNAL, the utility sets ALLOCATION to match the AUTOSWITCHLIMIT, and outputs a GDE-I-JNLALLOCGROW or GTM-I-JNLALLOCGROW message, respectively. [UNIX]

At journal extension time, including journal autoswitch time, if (ALLOCATION+EXTENSION>AUTOSWITCHLIMIT) for a region, GT.M now uses the larger of EXTENSION and AUTOSWITCHLIMIT as the increment to warn of low available journal disk space. Otherwise, it uses the EXTENSION, as before. [UNIX]

The GDE CHANGE -REGION -JOURNAL command no longer forces the EXTENSION to be the same as the specified ALLOCATION if the EXTENSION is not also specified. The previous behavior was documented for OpenVMS, but not for UNIX. (GTM-6224)

Corrected typos in the DBCDBNOCERTIFY and JOBEXAMFAIL error messages, a message from MUPIP RESTORE and, on OpenVMS, to the gtmkitinstal.com. (GTM-6849)

On platforms where GT.M supports object code in shared libraries, the configure script which installs GT.M now automatically creates a shared library libgtmutil.so containing the object modules of the routines distributed with GT.M that are written in M (GDE and utility programs). The "configure" installation script asks a question as to whether to delete the .o files as part of the installation. They can safely be deleted as long as the $gtmroutines environment variable ($ZROUTINES intrinsic special variable) specifies the libgtmutil.so file rather than the installation directory ($gtm_dist). The "gtminstall" installation script includes a --keep-obj command line switch and associated gtm_keep_obj environment variable (supporting "Y" and "N" values) that instructs the configure script not to delete object files. Note that some platforms use a .sl or .dll extension for shared libraries, rather than .so. The gtmprofile script that you can source to set up a default GT.M environment automatically uses libgtmutil.so if it exists. In addition, if there are any shared libraries in the $gtmdir/$gtmver/o directory ($gtmdir/$gtmver/o/utf8 for processes in UTF-8 mode), the gtmprofile script automatically includes them in the gtmroutines environment variable. Previously gtminstall, gtmprofile and configure scripts did not deal with shared object libraries. Furthermore, the gtmprofile script now provides a framework to extend GT.M with plug-ins. Place the M code in $gtm_dist/plugin/r, the M mode object code in the $gtm_dist/plugin/o directory and UTF-8 mode object modules in the $gtm_dist/plugin/o/utf8 directory. On platforms where GT.M supports object code in shared libraries, you can create shared libraries instead of .o files. The gtmprofile script now automatically includes these in the gtmroutines environment variable. [UNIX] (GTM-6664)

DSE DUMP -FILE -ALL output prints journal record types in hexadecimal. Since V4.4-002 these numbers have been printed in decimal which, if the numbers exceeded seven characters, overflowed the format and filled the output field with asterisks (*). (GTM-6860)

%RO now leaves trailing spaces intact and strips comments correctly. Previously it would strip trailing spaces from the end of each line in selected files, and it would mistake semicolons within quotes for comments. (GTM-6939)

GDE now generates a FILENOTFND message when it fails to open a command file, and a CRYPTNOMM message when it attempts to save a global directory with both encryption and MM access method set. Previously it would generate the undocumented FNF and ENCNOMM messages, respectively. (GTM-7005)

The size of the installation kit has been reduced by removing the file GTMDefinedTypesInit.o and creating a new GTMDefinedTypesInit for the distribution. This new kit contains the file GTMDefinedTypesInit.m from which GTMDefinedTypesInit.o can be created as needed. Previously, these were included in the installation kit. As the GTMDefinedTypesInit module is not needed for normal GT.M operation and is intended for use in diagnostic purposes under the guidance of FIS GT.M support, you do not need to download and install it except when so directed by FIS GT.M support. In addition, certain shell scripts have been removed from $gtm_dist. These scripts are not documented in GT.M user documentation but were inadvertently included in GT.M distributions. Also, when lowercase utility programs are requested during a UTF-8 mode GT.M installation, the configure script now supplies the programs in $gtm_dist/utf8 using symbolic links to the files in $gtm_dist, as it does for the upper case names. Previously, the lowercase .m files were copied from $gtm_dist. [UNIX] (GTM-7021)

LKE now issues NOLOCKMATCH when a non-existing lock is searched with "show l[ock]=<resource>" or attempted to be released using "clear l[ock]=<resource>". Previously, it issued the following text: "No locks were found in <region>" (GTM-7027)

GCC 4.6, which is part of the toolchain on Ubuntu 11.10, passes additional flags to the linker which prevent linking of libraries without direct dependencies. Previously, if a library was specified on the command line without direct dependencies, but that library's dependencies resolved a program dependency, the library would still be included. GT.M links libgtmshr against librt to get semaphore support, but semaphores are actually provided by libpthread, which is a dependency of librt. This caused builds to fail on Ubuntu 11.10 with a message that "sem_post" is not found. This change is of interest only for users who build GT.M from source. [UNIX] (GTM-7029)

LOCK commands which find no available lock space send a LOCKSPACEFULL message to the operator log. To prevent flooding the operator log, GT.M suppresses further such messages until the lock space usage drops below 75% full. In addition, LKE SHOW displays lock space usage with a message in the form of: "%GTM-I-LOCKSPACEUSE, Estimated free lock space: xxx% of pppp pages." If the lock space is full, it also displays a LOCKSPACEFULL error. Previously, GT.M did not issue such messages. (GTM-7073)

GT.M utilities now give a CLIERR error when an equal-sign (=) follows a qualifier without an immediately following value. Previously GT.M handled this case either with a different error or by ignoring the command without reporting any error. [UNIX] (GTM-7110)

All GT.M distribution kits now include a README.txt file which clarifies license related text in the header of each source file and its relationship to the COPYING file. Previously, this information was not provided. Output from the configure script has been changed to omit the word "National" and to refer appropriately to UTF-8. [UNIX] (GTM-7224)