Synopsis
cubeinspect [-v|--verbose] [--include-pattern=PATTERN]…
[--inspect|--repair-buffer-overrun]
[--output-dir=DIRECTORY [--force-overwrite]]
file | directory…
cubeinspect [-h|--help] [--version] [--sysinfo]
Description
Cubeinspect screens Cube files for conspicuous features that potentially may cause problems when processing the recordings. For some well-defined cases the utility can also be used to (partly) repair the problematic Cube file.
There are two modes of operation. By default, the cubeinspect utility
runs in inspection mode (command line option --inspect). All files
and directories given at the command line are searched recursively for
Cube files, which in turn are scanned one by one for conspicuous features.
To speed up the processing, Cube input can be restricted to contain only
files with a name matching the patterns given by the --include-pattern
option.
After all input files were scanned successfully a report is generated for each file. It contains general information about the Cube hardware as well as some statistics about the recording. Each report ends with a list of detected issues.
The other operational mode is the repair mode. Again the input files
are processed one by one. But this time the indicated modification (option
--repair-buffer-overrun) is applied to each file. The content of the Cube
input is written directly to standard output (i.e. text console) unless
the output directory (option --output-dir) is given as well.
Options
The program pretty much follows expected Unix command line syntax. Some
command line options have two variants, one long and an additional short
one (for convenience). These are shown below, separated by commas.
However, most options only have a long variant. The ‘=’ for options
that take a parameter is required and cannot be replaced by a whitespace.
- -h, --help
-
Summarize the most important command line options and exit.
- --version
-
Print the cubeinspect release information and exit.
- --sysinfo
-
Report system settings relevant for cubeinspect and exit.
- -v, --verbose
-
This option increases the amount of information given to the user during program execution. By default, (i.e. without this option) cubeinspect only reports warnings and errors. (See the diagnostics section below.)
- --include-pattern=PATTERN
-
Only process files whose filename matches the given PATTERN. Files with a name not matching the search PATTERN will be ignored. This option is useful to speed up recursive searches through large subdirectory trees and can be used more than once in the same command line. (In this case files matching any of the provided patterns are included.)
You can use two wild card characters (
*and?) when specifying an include PATTERN. For example using'*.123'selects all files ending with the extension.123. Remember to enclose patterns with single quotes when using wildcards or spaces in the context of an--include-patterncommand line option.There is also a predefined GIPP filter. Using
--include-pattern=GIPPautomatically selects all data files following the default naming convention for files recorded by loggers borrowed from the Geophysical Instrument Pool Potsdam (GIPP).The search PATTERN is only applied to the filename part and not to the full pathname of a file.
- --inspect
-
The utility will inspect files and diagnose problems. After (!) all input files were scanned a short report is generated for each file.
The report contains a SYSTEM section with some basic information about the used recorder (e.g. datalogger ID, installed firmware). It is followed by a DATA section providing some statistics about the samples that were recorded (channel minima/maxima given in counts and how often values were clipped). Finally, the PROBLEMS section summarizes all detected issues such as buffer overruns, clipped samples or if the recorded data is affected by a WNRO event.
Please refer to the Detected Problems section below for more details on each issue.
- --repair-buffer-overrun
-
This option is used to fill gaps in the recorded data stream that were caused by buffer overrun errors. For a more information about this problem please refer to the Buffer Overrun description in the Detected Problems section below.
When running in repair-buffer-overrun mode the cubeinspect utility will read Cube data from a file and fill all buffer overrun gaps with dummy samples (zero amplitude). The resulting repaired/modified file can be processed as usual again.
Repairing buffer overruns cannot magically restore the lost samples! The missing samples are gone for good!
All the program can do is fill the gap with the correct number of "dummy samples" so the file can be processed. - --output-dir=DIRECTORY
-
Save the resulting files to this DIRECTORY instead of writing to the standard output (i.e. text console). The directory must already exist and be writable!
Already existing files in that directory will not be overwritten unless the option
--force-overwriteis used as well.
- --force-overwrite
-
If this option is used, already existing files in the output directory will be overwritten without mercy!
The default behavior, however, is not to overwrite already existing files. Instead, a new file is created with an additional number before the file extension.
Detected Problems
The cubeinspect utility will detect the following problems and errors in Cube files.
Buffer Overrun
A buffer overrun is an internal recorder error that occurs when data is recorded faster than it can be written to the SD card storage. Eventually, the Cube/Nube internal memory buffer overflows and a handful of samples are lost (i.e. not written to SD card).
The result is a data gap in the (no longer) continues sample stream. The
missing data prohibits straightforward conversion to other data formats
(e.g. using cube2mseed, cube2segy, …). Affected files become usable
again when the gap is patched with (dummy) samples.
Unfortunately, the error cannot be avoided by simply switching to a "faster" SD card as the problem is not caused by an insufficient write speed. Most modern SD cards are easily fast enough for the samplings rates selectable in Cube/Nube data loggers.
Instead, the gaps are caused by the increased write latency of the SD card. This happens occasionally, mostly due to wear leveling when the card pauses until it could move data from on area of flash memory to another. Unfortunately, there is no external way to control the internal wear leveling algorithm inside the SD card.
Typically, buffer overrun problem surfaces first at the highest sampling rates while using old, worn out SD cards. In the beginning buffer overrun errors are very rare events. However, with time the frequency of buffer overrun errors increases. There is no known solution other than replacing the worn out SD card by a new, high quality one.
You can use the --repair-buffer-overrun command line option (see
above) to fill the gap with the
correct number of dummy samples.
|
Week Number Rollover (WNRO)
Cube data logger receive the recording time from the Global Positioning System (GPS). Unlike the commonly used Gregorian calendar, GPS (internally) expresses date/time information using two integer numbers. The first number counts the weeks since start of the GPS system on January 6th, 1980. The second number gives the seconds relative to the beginning of the week.
GPS Satellites transmit this week number as a 10-bit long integer, which will become zero again every 1024 weeks. This integer overflow is called week number rollover (WNRO) and is a common issue with all GPS receivers. Sooner or later, every GPS receiver will encounter this problem! A future, modernized GPS is proposed to use a 13-bit counter.
All Cube related GIPPtool utilities contain an algorithm that will detect
WNRO and just correct the date automatically by adding 1024 weeks to
the "wrong" recording time. However, in cases where the automatic detection
and correction fails, the handling of WNRO can be manually controlled
using the command line option --wnro-correction. Please refer to the
documentation of the respective utility for usage details.
| The cubeinspect utility only lists WNRO events as a convenience to the user. There is no need to "repair" Cube files as the issue is already handled transparently by the GIPPtool utilities. |
Clipped Samples
Recorded amplitudes are clipped if the voltage from the sensor exceeds the maximum voltage for the configured gain settings. The clipped value count line in the DATA section of the inspection report shows the number of clipped samples per recording channel. The number in the PROBLEMS section lists the total number of affected samples.
You can change the maximum recorded voltage by adjusting the gain settings.
Environment
The following environment variables can optionally be used to influence the behavior of the GIPPtool utilities.
- GIPPTOOLS_HOME
-
This environment variable can be used to set the GIPPtools installation directory.
In particular, the Java classes that make up the GIPPtools are read from JAR files in the
javasubdirectory located inside GIPPTOOLS_HOME. The start scripts for the individual GIPPtool utilities can be found inside thebinsubdirectory. - GIPPTOOLS_JAVA
-
The utilities of the GIPPtools are written in the programming language Java and consequently need a Java Runtime Environment (JRE) to execute. Use this variable to specify the path to the JRE, which should be used.
- GIPPTOOLS_OPTS
-
You can use this environment variable for additional fine-tuning of the Java runtime environment. It is typically used to set the Java heap size available to GIPPtool programs.
- GIPPTOOLS_LEAP
-
The GIPPtools require up-to-date leap second information to correctly interpret Cube files. Usually, this information is read from the
leap-seconds.listfile located in theconfigsubdirectory of the GIPPtools installation directory (GIPPTOOLS_HOME). This environment variable can be used to provide a more up-to-date leap second list to GIPPtool programs.
It is usually not necessary to define any of those variables as suitable values should be selected automatically. However, if the automatic detection build into the start script fails, or you need to choose between different GIPPtool or Java runtime releases installed on your computer, these environment variables might become helpful to troubleshoot the situation.
Diagnostics
During execution, the cubeinspect utility will produce user feedback. In general, user messages are classified as INFO, WARNING or ERROR.
INFO messages usually report about the progress of the program run,
give statistical information or write a final summary. They are only
displayed when the --verbose command line option is used.
More important are WARNING messages. In general, they warn about any issues that may influence the outcome in unexpected ways. Although the program will continue with execution, you certainly should check the results carefully. You might not have gotten what you (thought you) asked for.
Finally, ERROR messages inform about problems that cannot be resolved automatically. Program execution usually stops and the user must fix the cause of the error first.
Exit codes
Use the following program exit codes when calling cubeinspect from scripts or other programs to see if cubeinspect finished successfully. Any non-zero code indicates an ERROR!
- 0
-
Success.
- 64
-
Command line syntax or usage error.
- 65
-
Input data was incorrectly formatted.
- 66
-
An input file did not exist or was not readable.
- 70
-
Error in internal program logic.
- 74
-
I/O error.
- 99
-
Other, unspecified errors.
Examples
-
Scan a Cube file for conspicuous features.
cubeinspect 12200000.123
The program will output a short report about file
12200000.123recorded by Cube #123. -
Repair buffer overrun errors
A buffer overrun error results in lost samples in the recording. To patch the hole in the otherwise continuous time series dummy samples must be inserted at the location of the gap.
cubeinspect --repair-buffer-overrun < 12200000.123 > 12200000.patched.123
The utility will read in file
12200000.123and replace lost samples due to buffer overrun errors by dummy samples with an amplitude of zero. The modified recording is redirected to file12200000.patched.123.After the file has been repaired, it can take the place of the original file.
The file was not repaired in the sense that the lost samples miraculously were recovered. Only the gaps in the time series were filled with dummy samples!
Files
- $GIPPTOOLS_HOME/bin/cubeinspect
-
The cubeinspect "program". Usually just a copy of or a symbolic link pointing to the standard GIPPtools start script.
- $GIPPTOOLS_HOME/bin/gipptools
-
The standard start script used to run all GIPPtool utilities.
See also
gipptools(1), cube2ascii(1), cube2mseed(1), cube2segy(1), cubeaux(1), cubeevent(1), cubeinfo(1), cubeinspect(1), mseed2ascii(1), mseed2mseed(1), mseed2pdas(1), mseed2segy(1), mseedcut(1), mseedinfo(1), mseedrecover(1), mseedrename(1)
Bugs and caveats
-
The cubeinspect should report more details and to detect more problems!