- Author: Roman Savochenko
For debugging and testing OpenSCADA it is given considerable time of developers, but due to resource constraints and the inability to cover all configuration and implementation options of OpenSCADA, errors can be occurred as a failure of some functions, incorrectness of their implementation and even crash of the program. Understanding this on the part of the user is very important and it requires goodwill and some time to prepare a report about the problem, with the aim of its subsequent elimination by the developers.
Contents
1 The reporting conditions and variants
Anyone can report for error in the program for be considered by the developers on the general grounds of the free technical support, outside of the agreement of the commercial technical support or execution of this user's work. But the developers will consider only the errors that are the direct errors of OpenSCADA, not the problems of the environment and third-party libraries and are reproduced in the developer in the demo configurations of the last versions of OpenSCADA and the existing equipment of the OpenSCADA project!
The main and only official site for error reporting on the general grounds for free technical support is the section of the forum "Bug tracker", where developers is guaranteed to give it an answer. If you are not sure that this is an error directly with OpenSCADA, then it's better to write a report about it in another section of the OpenSCADA forum, where no warranty is given. Otherwise, after three consecutive incorrect error reports, you will be disconnected from the forum for the bad reception of the technical support and consultation, in vain spending so time of the developers! The error reporting rules are detailed in here.
Before forming a bug report on the general grounds of the free technical support, you need to check the list of known reports on the page of the forum tags:
- confirmed-opened and solving now — "BugConfirmed";
- which are currently in need of feedback from the author — "BugNeedFeedBack";
- execution environment and specific to the user — "BugEnvironment";
- incorrect error reports — "BugWrong".
Reporting of errors in the program based on the commercial technical support or user work can be done in the following ways, according to the developer's priority:
- in the forum section "Tech support" that appears and is available to registered users with an active package of the technological support;
- e-mail to service at oscada.org;
- directly by e-mail, instant messaging and call service, or by telephone of the developer, as well as private forum messages (for registered users).
2 Requirements for the error report
In order to exclude unnecessary overhead questions, or even premature closure of the error in the state "Not a bug", and to speed up the process of localization of the problem, it is recommended to follow the following requirements for the report:
- Specify the environment of the execution of OpenSCADA, that is: Distribution and version of the operating system.
- Specify the version of OpenSCADA, including the SVN revision of the working branch.
- Specify the configuration and execution features.
In the case of free support, it is mandatory to reproduce the problem in standard configurations and in the demo database, especially if it is an emergency crash and it is not possible to generate a crash report or it is indescribable.
- Describe the actions that cause the error.
- Attach the OpenSCADA message protocol for a session with the error.
- Generate a program crash report — unroll the stack at the time of the crash.
- the report is usually generated automatically by generating it from the pre-mortem memory dump; refer to the next section to enable the dump generation and manual generation of the report from it;
- for problems related to the blocking of one or more OpenSCADA threads, it may be useful to manually interrupt the program by the "SIGSEGV" signal, which will cause the formation of the death dump and the fall-down report with the information about the hangover;
- to significantly increase the use of the stack unrolling, you must add debugging information, by setting the "-g" option during compilation, or install the ready debug package openscada-dbg.
3 Obtaining the file of the pre-mortem dump and its processing
In the case of an emergency exit of a program, the kernel of the Linux operating system can form a pre-mortem memory dump of the program. With this dump, you can often find a place in the program that caused the emergency stop. To enable the possibility of generating the pre-mortem memory dump of program by the Linux kernel, you need to execute the commands:
# Check the possibility of generation of memory dumps
# Specifies the "core" if it is enabled
cat /proc/sys/kernel/core_pattern
# Enabling the generation of memory dumps
echo "core" > /proc/sys/kernel/core_pattern
After that, you need to remove the limit on the size of the generated dump file. This limit is currently, by default, removed into OpenSCADA and is enabled with the command line argument --noCoreDump. To remove this limitation, at the operating system level, you can execute, before calling OpenSCADA, a special command ulimit -c unlimited.
Then, you must make sure that the user launching OpenSCADA has the rights to write to the OpenSCADA working directory, which is required in the latest versions of OpenSCADA and provided by the OpenSCADA Project Manager.
Then start OpenSCADA and reproduce the crash, which resulted in the working directory OpenSCADA, creates a file "core".
The latest versions of OpenSCADA provide the automatic generation of the crash report, in the presence of the "core" dump file and the gdb debugger, and this is done at the start of OpenSCADA. Crash report files are named "AGLKS_core_2018-02-16_22.06.crash" and accumulate in the work folder of the OpenSCADA project.
If the "core" memory dump file is generated and the crash report is not automatically generated, then the gdb debugger is probably not installed. In this case, you must install it and call:
- service procedure of the OpenSCADA projects manager:
- openscada-proj proc {ProjName}
- or a direct debugger command from the project's working directory:
- gdb openscada --core core --batch --quiet -ex "thread apply all bt full" -ex "quit" > {ProjName}_core_$(date +%F_%H:%M).crash
- or a direct debugger procedure in the interactive mode, by calling gdb without arguments:
# Go to working folder of the OpenSCADA project
(gdb) cd /var/spool/openscada/{ProjName}
(gdb) cd ~/.openscada/{ProjName}
# Specifying an executable file
(gdb) file /usr/bin/openscada
# Specifying the dump file of the program
(gdb) core-file ./core.26658
# Getting the reversal of the stack execution — the crash report
(gdb) thread apply all bt full
#0 0xb7d104c0 in pthread_cancel () from /lib/librt.so.1
#1 0xb7d1edaa in start_thread () from /lib/libpthread.so.0
#2 0xb7dfcf5e in clone () from /lib/libc.so.6