Copyright (C) 2011 Fidelity National Information Services, Inc.
Copyright (C) 2011 Fidelity Information Services, Inc. All Rights Reserved
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts.
GT.M™ is a trademark of Fidelity Information Services, Inc. Other trademarks are the property of their respective owners.
This document contains a description of GT.M and the operating instructions pertaining to the various functions that comprise the system. This document does not contain any commitment of FIS. FIS believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. FIS is not responsible for any errors or defects.
May 17, 2011
|Revision 1.0||May 17, 2011|
|First published version.|
Table of Contents
This document provides supplementary information about GT.M as of release V5.4-002A for programmers and system managers. There are three sections:
General: Describes GT.M's philosophy for supporting software releases and platforms. Also includes terms and definitions used in this document.
ISO/IEC-10756:1999 Conformance Considerations: Describes GT.M Language features where the standard explicitly grants implementation-specific behavior or where the implemented features do not fully conform with the ISO/IEC-10756:1999 standard.
Other Considerations: Includes known bugs, misfeatures, and potential surprises.
The FIS GT.M team assigns a reference number to new functionality, known bugs, misfeatures, and surprises and tracks them as change requests. All change requests received from our supported customers have leading "S" where as all internal change requests have leading "C". Every new release of the GT.M software is an extensively tested implementation of a set of change requests. For every GT.M release, the change requests are considered after careful analysis and continuous prioritization. Therefore, all change requests listed in this document may or may not be considered for an upcoming release of GT.M. If you have a particular need, please let us know so we can account for it in our prioritization.
This section defines the terms used in this document.
UNIX: The term UNIX is used here in the general sense of all platforms for which GT.M uses a POSIX API. As of this date, this includes: AIX; HP-UX on IA64 and PA-RISC; GNU/Linux on IA64, x86, x86_64, and System z; Solaris on SPARC; z/OS.
Command Syntax: UNIX syntax (that is, lowercase text and "-" for flags/qualifiers) is used throughout this document. OpenVMS accepts both lowercase and uppercase text; flags/qualifiers on OpenVMS should be preceded with "/".
Program Names: When referring to a GT.M program or function, the reference is in upper case, e.g, MUPIP BACKUP. When a specific example is provided, the lower case UNIX command names are used, e.g., mupip backup -database ACN,HIST /backup
Reference Number: Reference numbers used to track software enhancements and customer support requests appear in parentheses ().
Platform Identifier: If a new feature or software enhancement does not apply to all platforms, the relevant platform or platforms appear in brackets .
Henceforth, the term "originating instance" is used where previously "primary instance" or "originating primary instance" were used, and the term "replicating instance" is used where previously "secondary instance" and "originating secondary instance" were used. Since it is easier to change documentation than it is to change software, and given our tradition of maintaining compatibility especially for existing programs and scripts, the terms "primary" and "secondary" will remain in the software for a long time to come, even as they are supplemented with the new terms in the code, and replaced in the documentation.
GT.M runs on a wide variety of UNIX/Linux implementations as well as OpenVMS. Consult FIS for currently supported versions. Each GT.M release is extensively tested by FIS on a set of specific versions of operating systems on specific hardware architectures (the combination of operating system and hardware architecture is referred to as a platform). This set of specific versions is considered Supported. There will be other versions of the same operating systems on which a GT.M release may not have been tested, but on which the FIS GT.M support team knows of no reason why GT.M would not work. This larger set of versions is considered Supportable. There is an even larger set of platforms on which GT.M may well run satisfactorily, but where the FIS GT.M team lacks the knowledge to determine whether GT.M is Supportable. These are considered Unsupported. Contact FIS GT.M Support with inquiries about your preferred platform.
System requirements vary widely depending on application needs, and should be empirically determined for each situation.
FIS usually supports new operating system versions six months after stable releases are available and we usually support each version for a two year window. GT.M releases are also normally supported for two years after release. While FIS will attempt to provide support to customers in good standing for any GT.M release and operating system version, our ability to provide support is diminished after the two year window.
GT.M cannot be patched, and bugs are only fixed in new releases of software.
This section describe the change requests for known M Language features that deviate from the ISO/IEC-10756:1999 Standard. The Limits section describes GT.M internal limits that you should consider while writing M program and scripts.
Using a subscripted control variable with a FOR command and assigning to it an expression containing an extrinsic function generates a FORCTRLINDX error. The workaround is to call the function and assign the result to a variable before using it in the FOR command, or otherwise avoiding this odd construct. (C9L03-003392)
GT.M provides an option to not short-circuit Boolean expressions. For more information refer to GT.M V5.4-002 series Release Notes.
An intrinsic function containing extrinsic functions in its arguments may not evaluate according to the standard unless the extrinsic functions have no side effects on the other arguments to the intrinsic function, or a sole extrinsic function call is in the first argument to the intrinsic function. If your code must use extrinsic functions with side effects, the workaround is to use precalculated temporary variables to avoid using extrinsic functions as arguments to intrinsic functions. (C9B10-001766)
After a complete ROLLBACK, $REFERENCE evaluates to an empty string in conformance with the standard. However, after an incremental ROLLBACK (invoked by TROLLBACK intexpr), $REFERENCE evaluates to the last global reference whether or not the incremental ROLLBACK action discarded that last global reference. In the future, FIS plans to make the $REFERENCE evaluate to an empty string after all TROLLBACK commands. (C9B12-001841)
For SOCKET devices, certain attributes (WIDTH, LENGTH, [NO]WRAP, EXCEPTION, FILTER, IPCHSET, OPCHSET) and certain status information ($KEY, $DEVICE, $X, $Y, $ZEOF, $ZA, $ZB) apply to the device and are shared if there are multiple sockets associated with the device. Workarounds include using separate devices for each socket, or capturing socket state before a USE of another socket on the same device. (C9C05-001988)
GT.M mishandles a pattern match literal of more than 14 characters, producing inappropriate results including possible process failure. The workaround is to break long literals in a pattern match into multiple elements of 14 characters or less. (C9C10-002148)
GT.M does not accept Intrinsic Special Variables (ISVs) as indirect left-hand arguments to a SET command. Workarounds include avoiding indirection or using indirection on the entire SET argument. (C9D01-002226)
$QUERY() on global variables using the default legacy GT.M null subscript collation does not work if the last subscript specified in the global variable is a null subscript, because the legacy GT.M collation places the null subscript between numeric and string subscripts. If a global variable must have null subscripts, FIS strongly recommends that you specify standard null collation, which places a null subscript before all other subscripts. For more information on enabling standard collation for null subscripts, refer to GT.M Null Subscripts and Null Subscriptions Collation. FIS maintains the legacy behavior as the default to avoid breaking applications that rely on it. (C9E11-002667)
GT.M treats something like SET indr="label+1" DO @indr as a missing label rather than a syntax error. The workarounds are to avoid invalid syntax like this or be aware the resulting error is less helpful than it might be. Note that $TEXT() using this syntax simply returns an empty string result. (C9E12-002682)
For a SOCKET device, if WRAP is enabled for the device (as it is by default) and the device encounters a WRITE or series of WRITEs that produce an exact multiple of WIDTH characters, it can exhibit inappropriate behavior. On UNIX, if those preconditions are met and if the DELIMITER device parameter is specified (say in the OPEN command, or a USE command) and subsequently a USE command specifies NODELIMITER, GT.M may write invalid characters on the output socket at the WRAP boundary. On OpenVMS if those preconditions are met and NODELIMITER is specified, or DELIMITER is not specified in the OPEN or USE command, GT.M incorrectly issues a SOCKWRITE error. Also in OpenVMS, every subsequent WRITE causes the same error, and so, nothing more can be successfully sent until the SOCKET is reinitialized. In both cases, The workaround is to explicitly WRITE # every WIDTH characters. (C9F03-002711)
Since GT.M compiles, rather than interprets code, when it encounters an invalid argument, even one for a command followed by a postconditional that is literally zero (FALSE), it issues a compilation warning. If there is a need for such arguments, the workaround is to keep them after an IF with an argument that evaluates to FALSE. Note invalid arguments to commands bypassed by postconditions do not cause run-time errors, only compile-time errors. (C9I05-002991)
A SET where the left-hand side of the argument is an indirection that resolves to a subscripted variable and the right-hand side contains an extrinsic function or special variable that modifies a subscript of the left-hand target, uses the modified subscript while it should use the unmodified subscript. The workaround is to avoid the indirection if possible, indirect the entire argument, or assign the result of the extrinsic to a temporary variable and use the temporary variable in the original right-hand argument. (C9J06-003133)
While writing M program and scripts, you should consider the following GT.M internal limits:
Numbers in GT.M are nominally accurate to 18 significant digits.
As long as the integer part is less than or equal to 15 digits, GT.M ensures accuracy to at least three decimal places. The range of numbers represented is 1E-43 through 1E47. When application code results in a number that is in-range but with more than 18 significant digits, GT.M silently retains the most significant 18 digits.
Local and global variable names, label names, routine names and M lock resource names
GT.M distinguishes these names using the first 31 characters (not counting any leading caret). GT.M silently uses only the first 31 characters of longer names internally.
Source and object file names (not including the extension)
31 characters. When forming routine names, the GT.M compiler truncates default filenames to 31 characters. For example, the compiler considers the routine name for a source file Adatabaseenginewithscalabilityproven.m to be Adatabaseenginewithscalabilityp.m. This silent truncation can work, but is best avoided as it can cause frequent recompilation and problems should the recognized name length increase.
Local variable string
GT.M silently uses only the first 28 characters of longer names internally.
Replication instance name
The maximum key size is 255 bytes. When determining the maximum key size, applications should consider the following:
For example, the key ^ACN ("Name", "Type") internally occupies 17 bytes.
The maximum record size (in bytes) is:
block_size - block_header_size - reserved_bytes
where block_header_size is 16 bytes.
The maximum block size is 65,024 bytes.
GT.M does not compile a routine containing more than 4096 [AIX, HPUX, Tru64 Unix, OpenVMS] or 16384 [Linux, Solaris] unique local variable names. If a routine contains more local names than this limit, GT.M terminates with a GTMASSERT error. Note that this restriction does not apply to local variable names used in indirection code. The workaround is to break the large routine into two or more smaller routines containing fewer local names. (S9E11-002507)
GT.M does not support database and journal files which reside on an NFS mounted filesystem. However, you can use an NFS mounted filesystem for keeping source and object code modules and performing sequential file IO.
This section describe the change requests for known bugs, misfeatures, and surprises. It also provides solutions, workarounds, and recommendations to help you manage them.
In order to prevent Direct Mode activity from affecting other processes, a process always releases any critical sections at the time of entering the Direct Mode. For M transactions, this has 2 implications:
Direct mode actions within a TRANSACTION cannot be restarted, which can cause the process to fail.
Direct mode actions (including External interactions) within a transaction violate the Isolation property of wholesome transactions and should be avoided except when there a well understood and thoroughly vetted application design reason.
FIS strongly recommends not including any Direct Mode - or indeed any human interaction - inside a transaction. (S9911-001425)
"Z" pseudo transactions (those bracketed with ZTSTART/ZTCOMMIT commands) are no longer supported - they do not work in a replicated environment. Remove their use from your application code before using replication.
FIS strongly recommends that you avoid suspending a running GT.M process (on UNIX) with <CTRL>-Z or by sending a SUSPEND signal. Suspending a process actively participating in the management of a shared database may result in a database hang if the suspended process holds a critical section. You can redefine ^Z in the shell to avoid sending a SUSPEND. [UNIX] (S9C09-002229)
Using the ^%gbldef function while journaling (or database replication) is in progress can corrupt the journal file. The workaround is to turn off journaling when performing administrative actions with ^%gbldef.(C9B03-001658)
GT.M currently treats all segments whose file specification is of the form VOLUMESET::UCI (access method USER) as remote DDP segments, even if the VOLUMESET is served by the GT.CM DDP server running on the local node. A GT.M client access to a global residing in such a VOLUMESET generates GTM-E-NOCONNECT error. The workaround is to map locally residing globals to local files, and not to a VOLUMESET::UCI pair. FIS has no plans to change this behavior. [OpenVMS]
MERGE ^gvn2=^gvn1 does not work correctly if ^gvn1 and ^gvn2 have different alternative collation (act) settings, The subscripts of the source global may not be transformed according to the destination collation setting before placing them in the destination global. The workaround is to set up a loop to sequence through the global nodes in ^gvn1 and copy each in turn to ^gvn2. (C9E11-002663)
If your user-defined alternate collation routines gtm_ac_xform_1()/gtm_ac_xback_1() use the OpenVMS mailbox facility, you must initialize the dsc64$w_mbo and dsc64$l_mbmo fields in the control structures, as GT.M does not. [OpenVMS]
For the a terminal Principal device ($PRINCIPAL), GT.M works with the terminal characteristics or settings in place when it is invoked, changes terminal settings as specified by deviceparameters on USE commands, and leaves the terminal in its then current state when GT.M terminates. If invoking code and GT.M need different terminal characteristics, they must be explicitly managed by some combination of USE commands and code in the invoking program or script. For example in UNIX, an invoking script might use the "stty sane" command after a GT.M invocation. (S9F01-002524)
In OpenVMS, GT.M truncates all error messages that are longer than 255 characters. This is most likely to happen with long file specifications - a workaround is to shorten file names and paths. [OpenVMS] (C9C04-001974)
Attempts to remove the stack level caused by an implicit Xecute such as $ZTRAP immediately above a Direct Mode frame with a QUIT may not work. The workaround is to use ZGOTO in these cases. (C9C11-002180)
A GTM-E-STACKCRIT error while executing $ZTRAP leaves GT.M in direct mode. In general, such errors in $ZTRAP should cause GT.M to terminate instead of entering direct mode. Error trapping with $ETRAP ensures that the process does not unintentionally enter direct mode. FIS strongly recommends using $ETRAP, or $ZTRAP code that detects the STACKCRIT error and terminates the process or takes corrective action such as a suitable ZGOTO. (C9C11-002181)
ZGOTO with an indirect entry reference argument (i.e. ZGOTO level:@entryref where entryref is of the form "lab^rtn") fails with a LABELMISSING error even if the label is available in the specified routine. This is consistent with the specification for ZGOTO - use indirection of the entire ZGOTO argument instead.
If a routine of more than 31 characters long is auto-compiled by auto-ZLINK or ZLINK, GT.M does not truncate the object file name to the maximum routine length (31 characters). This behavior causes any subsequent auto-ZLINK to fail to locate the object file. The workaround is to compile the routines with long names explicitly with the mumps command before auto-ZLINK attempts to compile. (C9E12-002678)
GT.M needs to be able to execute code in a process' heap/stack space because GT.M is a compiler and M requires code that is able to dynamically compile and execute. As this is not a requirement of typical C/C++ programs, and one of the precepts of good security is to grant no privileges save those that are required, it is increasingly common for default security settings to disable the ability of a process to execute code from heap/stack space.
Different operating systems have different ways to configure this security setting, and there can be multiple ways to configure this functionality on any given operating system. Users may experience unexpected segmentation violation – SEGV (sig-11) errors such as:
%GTM-F-KILLBYSIGSINFO1, GT.M process 23914 has been killed by a signal 11 at address 0x092C57D0 (vaddr 0x092C57D0)
Identify the security configuration setting that prevents GT.M processes from executing code in heap/stack space and configure it to allow GT.M processes to do so. One example of an implementation of such a feature is exec-shield on some distributions of GNU/Linux. Examine the following files to determine the status of exec-shield:
/proc/sys/kernel/exec-shield and /proc/sys/kernel/exec-shield-randomize cat /proc/sys/kernel/exec-shield cat /proc/sys/kernel/exec-shield-randomize
If either value is non-zero, you may need to set it to 0:
echo 0 >/proc/sys/kernel/exec-shield echo 0 >/proc/sys/kernel/exec-shield-randomize
Versions of GT.M starting from V4.2-000 do not process the response to the "What file: " prompt on the initiating terminal when a mumps -run omits routine names on the command line. The workaround is to specify the routine name as an argument to the original command. [UNIX] (C9C04-001965)
MUPIP JOURNAL RECOVER/ROLLBACK -BACKWARD may leave a database with KILL_IN_PROG and "GTM-E-DBMRKBUSY, Block incorrectly marked busy" errors. Although you should address these errors as soon as convenient, they are benign as long as they are the only integ errors in the database (as discussed in the GT.M User documentation). (C9902-000863)
If there is no space left in the disk where a journal file resides, GT.M will stop journaling for that database. If that database is part of a replicated instance, GT.M also turns replication off if it needs to use the missing journal, in other words, if the in-memory journal pool overflows. Depending on the circumstances, recovery may involve recreating secondary instances that replicate from that instance. Please ensure that there is always adequate space on devices where journal files are written.
MUPIP EXTRACT fails to extract globals residing on a local node if the next global in collation sequence resides on a remote node. The workaround is to use a global directory that omits remote globals before performing MUPIP EXTRACT operations. (C9B11-001834)
Always perform a MUPIP RECOVER or ROLLBACK as the first step to restore a journaled database after a crash. Never start GT.M updates or use MUPIP SET -JOURNAL before the recovery, as this affects subsequent recover or rollbacks and prevents them from being successful. GT.M does not strictly enforce this order of steps. (C9C01-001892)
MUPIP INTRPT commands initiated while the target process is already handling an interrupt can cause failure of the target process. The workaround is to avoid using MUPIP INTRPT on any given process too soon after the previous MUPIP INTRPT - a delay of a second is ample in most environments. [OpenVMS] (C9C04-001970)
If the logical names SYS$OUTPUT and/or SYS$ERROR are redefined to a file with a DEFINE /USER command, certain utility commands (for example. MUPIP JOURNAL /SHOW, MUPIP BACKUP, DSE) on a journaled database terminate with a RMS-E-FLK error. The workaround is to use a script that runs only the utility command. The script then can perform, in the beginning, a DEFINE SYS$OUTPUT and DEFINE SYS$ERROR without the /USER qualifier. [OpenVMS] (C9C05-001984)
FENCE=ALWAYS may work incorrectly when multiple regions are involved. FIS strongly recommends migrating your application to standard transaction processing rather than the less capable ZTSTART/ZTCOMMIT approach. (C9C06-002025)
GT.M relies on the monotonic increase of time stored in the journal records to maintain application consistency across multi-region fenced transactions. If the time stamp in later journal records states an earlier time than the time stamp in earlier journal records, GT.M database recovery after a crash may be complicated and even compromised.
When adjusting the system clock to maintain the correct time, FIS strongly recommends "slewing" it rather than "stepping" it. Standard packages such as NTP provide this capability.
Since the system clock is traditionally set to UTC on UNIX systems, the transition from Daylight Savings Time to Standard Time does not involve stepping the clock, and the transition is transparent to GT.M on such systems.
On systems where the system clock is set to local time (which includes OpenVMS systems as well as those UNIX/Linux systems where the clock is not set to UTC), stepping the clock backwards is required when changing from Daylight Saving Time to Standard Time. At such times, as well as when the system clock has drifted so far that slewing it back to the right time is impractical, it may be necessary to step system time backwards.
Use the procedure below when you need to step the time backwards.
Execute MUPIP FREEZE -ON for all database regions where journaling is in use.
Record the current system time.
In each environment, switch to new journal files using MUPIP SET JOURNAL -REGION "*"
Adjust system time
In each environment, switch to new journal files using MUPIP SET JOURNAL -REGION "*"
Execute MUPIP FREEZE -OFF for all database regions where journaling is in use
Whenever the new system time is prior to the recorded system time (that is, the time step was negative), then all future MUPIP RECOVER commands that use time qualifiers, which fall in the time range of adjustment need to use a time that is either less than the start time (for SINCE and AFTER qualifiers), or greater than the end time of the time range (for BEFORE qualifiers). In other words, possible recovery time options should never reference a time that is ambiguous because of the time change, but rather one that is either before any "overlapping time" or after the change. This includes the time in the journal files created by the first command in the instructions, but may include earlier times as well. (C9D06-002282)
MUPIP EXTRACT of a database in binary format on OpenVMS fails when database block sizes are over 32,765. In addition, MUPIP JOURNAL EXTRACT or MUPIP EXTRACT in ZWR or GO format terminate if they produce a record larger than 32,767 bytes, including the record header and terminators. This is due to OpenVMS RMS' limitation of 32767 bytes for the maximum record size of a sequential file. FIS recommends that you use a database block size of 32254 (the largest legal block size smaller than 32768) on OpenVMS. FIS has no plans to change this behavior. [OpenVMS]
If the duplicate SET optimization (see Release Notes for GT.M V4.4-004 for C9D12-002472) is enabled and if the update pattern is such that the last update in a sequence of updates is a duplicate-SET and there are no more updates for quite a while although the updating processes are still attached to the database, it is possible that the database file header does not get flushed in a timely manner. In this case, if before-image journaling is turned ON, it is also possible that EPOCHs are not written to disk in a timely manner. If the processes attached to the database are then killed abnormally due to a system crash or using a kill -9 (UNIX) or STOP/ID (OpenVMS), then a MUPIP JOURNAL ROLLBACK and/or RECOVER /BACKWARD on the same database produces an error as these commands rely on EPOCHs being written in a timely manner. The workaround is to issue a backward recovery with a /SINCE_TIME that matches the time of last update to this database or a rollback with a /RESYNC_SEQNO that matches the last update to this database. (C9E03-002536)
In GT.M versions starting from V4.4-002, the files that are created as a result of MUPIP EXTRACT /FORMAT=ZWR (or /FORMAT=GO) command have a maximum record size of 65,535 which may cause problems if other programs and OpenVMS utilities use them. To resolve this, issue the following DCL command:
set file/attribute=mrs=<longest record length> file.glo
where <longest record length> is the appropriate value from a DIRECTORY/FULL of the file (file.glo above.) [OpenVMS] (C9E03-002549)
If a process containing a HANG command is frequently interrupted by MUPIP INTRPT and the length of that HANG exceeds the period of the interrupt, the HANG command will never complete as it is restarted in its entirely when it is resumed after the process responds to the interrupt. The same is applicable to timeout on locks. Any timeout except READ SOCKET and READ TERMINAL reset when a RETURN from $ZINTERRUPT restarts the command. (C9H04-002848)