This manual aims to be a thorough guide to using Dump Test Tools (DTT), leaving all implementation details aside. I would be grateful if you could review the code and provide some feedback.
The DTT package is constituted by the following files:
DTT's goal is to provide a light weight infrastructure to generate severe kernel failures (panics, hangs, exceptions, etc) in a controlled manner. These anomalies in the kernel are caused at so called crash points, which were implemented as hooks (IBM's "Kernel hooks" in particular). I will use "crash point" and "hook" interchangeably throughout this document.
Pursuing to minimize the intrusiveness of DTT, a mechanism was created so that hooks inserted in a kernel module can be registered from user space (thus saving some lines in the module's init and exit functions). Besides, hooks built into the kernel (built-in hooks) are automatically registered at DTT's initialization time.
Finally, it is possible to perform dynamic configuration of the hooks.
Details are offered in subsequent sections.
To apply the kernel patches please follow the steps below:
# cd KERNEL_SRC_BASE # zcat PATH_TO_PATCH/kernelhooks-v1.9-2.6.9-combined.patch.gz | patch -p1 # zcat PATH_TO_PATCH/dumptesttools-core.patch.gz | patch -p1 # zcat PATH_TO_PATCH/dumptesttools-crashpoints.patch.gz | patch -p1Alternatively, the full patch can be used:
# zcat PATH_TO_PATCH/dtt-full.patch.gz | patch -p1
Once the desired patches have been applied we can proceed to configure the kernel as usual making sure that the options below are also enabled.
# make menuconfig Kernel hacking ---> Kernel debugging [*] Kernel Hook support [*] or [M] Crash points [*] or [M]
Kernel build and system reboot:
# make # make modules_install # make install # shutdown -r now
In case DTT was compiled as a module modprobe it:
# modprobe dtt [iter_num={>0}] [debug={0|1}]
The module has the following options:
stack growth >= iter_num * sizeof(buf) (please see "recursive_loop" function at drivers/dtt/khdtt.c)
# tar xzf dtt_tools.tar.gz # cd dtt_tools # make
ttutils command [options]
command = {help|ver|version|list|ls|add|set|reset}
Display usage information.
Example:
# ./ttutils
Usage: ./ttutils {help|ver|version|list|ls|set|reset｜add|rmv} [options]
Display version number of DTT.
Example:
# ./ttutils version
Version: 0.1
Show registered points (crash points)
Example:
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 dummy1 INT_TASKLET_ENTRY 63 kern
3 none FS_DEVRW 0 kern
4 panic MEM_SWAPOUT 7 kern
5 none TASKLET 0 kern
6 none TIMERADD 0 kern
201 none test_cpoint1 0 mod
100 none test_cpoint2 0 dyn
Change the characteristics of a registered hook (crash point).
hook's name
Type of kernel failure to be generated. Currently available types:
If no -t option is specified the hook type remains unchanged.
Note: when a hook is registered its type is set to none by default.
Number of times the crash point is crossed before the failure associated with its type is triggered. The default value is 10.
Example:
# ./ttutils set -p INT_TASKLET_ENTRY -t panic -c 100
Disable a hook. Besides, the associated counter that keeps track of the number of times the crash point was traversed is also reset.
Hook's name.
Reset not only the hook's counter but also revert the hook's type to none.
Example:
# ./ttutils set -p INT_TASKLET_ENTRY -c 150
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 panic INT_TASKLET_ENTRY 145 kern
[without -f option]
# ./ttutils reset -p INT_TASKLET_ENTRY
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 panic INT_TASKLET_ENTRY 0 kern
[using -f]
# ./ttutils reset -p INT_TASKLET_ENTRY -f
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 none INT_TASKLET_ENTRY 0 kern
Register a hook from a kernel module so that it can be actually used.
This is aimed at modules that do not register the hooks inserted in their own code.
Please note that registering does not imply activation. Activation is accomplished using "-set".
Hook's name.
Hook's ID.
Example:
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 none INT_TASKLET_ENTRY 0 kern
201 none test_cpoint1 0 mod
# ./ttutils add -p IDE_CORE_CP -n 50
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 none INT_TASKLET_ENTRY 0 kern
201 none test_cpoint1 0 mod
50 none IDE_CORE_CP 0 dyn
Remove a hook registered with "-add".
Hook's name.
Example:
# ./ttutils set -p IDE_CORE_CP -t dummy1 -c 3
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 none INT_TASKLET_ENTRY 0 kern
201 none test_cpoint1 0 mod
50 dummy1 IDE_CORE_CP 3 dyn
# eject
# ./ttutils ls
id crash type crash point name count location
1 none INT_HARDWARE_ENTRY 0 kern
2 none INT_TASKLET_ENTRY 0 kern
201 none test_cpoint1 0 mod
50 dummy1 IDE_CORE_CP 2 dyn
# ./ttutils rmv -p test_cpoint2
# ./ttutils ls
1 none INT_HARDWARE_ENTRY 0 kern
2 none INT_TASKLET_ENTRY 0 kern
201 none test_cpoint1 0 mod