Managing core dumps

Managing core dumps
Prev	Appendix B. Monitoring GT.M	Next

When an out-of-design situation or a fatal error causes a GT.M process to terminate abnormally, GT.M attempts to create a GTM_FATAL_ERROR.ZSHOW_DMP_*.txt file containing a dump of the M execution context. On encountering an unexpected process termination, GT.M instructs the operating system to generate a core dump on its behalf at the location determined from the core generation settings of the operating system. GTM_FATAL_ERROR*.txt and core dump files may help GT.M developers diagnose and debug the condition which resulted in an unexpected process termination, and help you get back up and running quickly from an application disruption. In additional to information having diagnostic value, a core dump file may also contain non-public information (NPI) such as passwords, local variables and global variables that may hold sensitive customer data, and so on. If you are an organization dealing with non-public information, you should take additional care about managing and sharing GTM_FATAL_ERROR.ZSHOW_DMP_*.txt and core dump files.

As core dump files may contain non-public information, you might choose to disable core dump generation. In the absence of a core dump file, you may be asked to provide detailed information about your hardware, GT.M version, application state, system state, and possibly a reproducible scenario of the unexpected process termination. Note that unexpected process terminations are not always reproducible. You are likely to spend a lot more time in providing post-mortem information during a GT.M support engagement than what you'd spend when a core dump file is available.

Core file generation and configuration are functions of your operating system. Ensure that core file generation is configured and enabled on your operating system. On Linux platforms, /proc/sys/kernel/core_pattern determines the naming convention of core files and /proc/sys/kernel/core_uses_PID determines whether the process id of the dumped process should added to the core dump file name. A core_pattern value of core creates core dump files in the current directory. Check the man mage for core (on Linux), and chcore (on AIX) for instructions on enabling and configuring core dump file generation according to your requirements.

	Note
As maintainers of FIS GT.M, our goal is to make the product as reliable as it can be, so you should get few if any core files. Before a public release, GT.M goes through several rounds of automated testing which provides a thorough test coverage for new functionality and possible regressions. When new functionality passes our regression testing cycle, we frequently make field test releases so that GTM gets additional testing coverage in customer environments before a public release. While prioritizing fixes for a GT.M public release, we assign a higher priority to unexpected process terminations that our regression testing cycle and supported GT.M customers may report. As part of any fix, we add new test cases that become an integral part of future regression testing cycles. We have followed this practice for the past several years and therefore it is very unusual for a stable production application to generate core files. GT.M supplies a wide range of functionality in ways intended to maximize performance. Nonetheless, GT.M is reasonably complex as the number of possible execution paths is large, and our testing coverage may not include all possible edge cases. If you encounter a core dump because of a GT.M issue, it is likely that it is not part of our test coverage and we may find it hard to reproduce. Core files, especially combined with gtmpcat, are a powerful tool in diagnosing and addressing issues that cause process failures. Note also that user actions can directly cause core files without any contributing GT.M issue (see the following example).

Note

As maintainers of FIS GT.M, our goal is to make the product as reliable as it can be, so you should get few if any core files. Before a public release, GT.M goes through several rounds of automated testing which provides a thorough test coverage for new functionality and possible regressions. When new functionality passes our regression testing cycle, we

frequently

make field test releases so that GTM gets additional testing coverage in customer environments before a public release. While prioritizing fixes for a GT.M public release, we assign a higher priority to unexpected process terminations that our regression testing cycle and supported GT.M customers may report. As part of any fix, we add new test cases that become an integral part of future regression testing cycles. We have followed this practice for the past several years and therefore it is very unusual for a stable production application to generate core files. GT.M supplies a wide range of functionality in ways intended to maximize performance. Nonetheless, GT.M is reasonably complex as the number of possible execution paths is large, and our testing coverage may not include all possible edge cases. If you encounter a core dump because of a GT.M issue, it is likely that it is not part of our test coverage and we may find it hard to reproduce. Core files, especially combined with gtmpcat, are a powerful tool in diagnosing and addressing issues that cause process failures. Note also that user actions can directly cause core files without any contributing GT.M issue (see the following example).

The following suggestions may help with configuring core dump files:

Always put cores in a directory having adequate protection and away from normal processing. For example, the core file directory may have write-only permissions for protection for almost all users.
Set up procedures to remove core dumps and GTM_FATAL_ERROR.ZSHOW_DMP_*.txt when they are no longer needed.
Always configure core file generation in a way that each core gets a distinct name so that new cores do not overwrite old cores. GT.M never overwrites an existing core file even when /proc/sys/kernel/core_uses_pid is set to 0 and /proc/sys/kernel/core_pattern is set to core. If there is a file named core in the target core directory, GT.M renames it to core1 and creates a new core dump file called core. Likewise, if core(n) already exists, GT.M renames the existing core to core(n+1) and creates a new core dump file called core.

Here are the possible steps to check core file generation on Ubuntu_x86 running GT.M V6.1-001_x86_64:

$ ulimit -c unlimited
$ /usr/lib/fis-gtm/V6.1-001_x86_64/gtm
GTM>zsystem "kill -SIGSEGV "_$j
$GTM-F-KILLBYSIGUINFO, GT.M process 24570 has been killed by a signal 11 from process 24572 with userid number 1000
$ ls -l core*
-rw------- 1 gtmnode jdoe 3506176 Aug 18 14:59 core.24573

In order to test your core generation environment, you can also generate a core dump at the GT.M prompt with a ZMESSAGE 150377788 command.

If you do not find the expected dump files and have already enabled core generation on your operating system, check file permissions and quotas settings.
As GT.M core dumps are not configured for use with automated crash reporting systems such as apport, you might want to adjust the core naming conventions settings in such a way core dumps are preserved safely till the time you engage your GT.M support channel.

Before sharing a core dump file with anyone, you must determine whether the files contain NPI and whether the recipient is permitted to view the information in the files. GT.M Support at FIS does not accept NPI. You can use the gtmpcat software under the guidance of FIS GT.M Support to extract meaningful information from core files (by default, gtmpcat extracts meta-data without protected information; although you should always review any gtmpcat output before you send it to FIS). gtmpcat is diagnostic tool available to all customers who have purchased FIS GT.M Support. For more information on the gtmpcat software, refer to Appendix E: “GTMPCAT - GT.M Process/Core Analysis Tool ”.

Prev	Up	Next
Monitoring GT.M Messages	Home	Appendix C. Building Encryption Libraries