"make check" is a target in the top level makefile. It takes care of running a number of unit and system tests to confirm that FreeSWAN has been compiled correctly, and that no new bugs have been introduced.
As FreeSWAN contains both kernel and userspace components, doing testing of FreeSWAN requires that the kernel be simulated. This is typically difficult to do as a kernel requires that it be run on bare hardware. A technology has emerged that makes this simpler. This is User Mode Linux.
User-Mode Linux is a way to build a Linux kernel such that it can run as a process under another Linux (or in the future other) kernel. Presently, this can only be done for 2.4 guest kernels. The host kernel can be 2.2 or 2.4.
"make check" expects to be able to build User-Mode Linux kernels with FreeSWAN included. To do this it needs to have some files downloaded and extracted prior to running "make check". This is described in the UML testing document.
After having run the example in the UML testing document and successfully brought up the four machine combination, you are ready to use "make check"
"make check" works by walking the FreeSWAN source tree invoking the
"check" target at each node. At present there are tests defined only
for the klips directory. These tests will use the UML
infrastructure to test out pieces of the klips code.
The results of the tests can be recorded. If the environment
variable $REGRESSRESULTS is non-null, then the results of
each test will be recorded. This can be used as part of a nightly
regression testing system, see Nightly testing
for more details.
"make check" otherwise prints a minimal amount of output for each test, and indicates pass/fail status of each test as they are run. Failed tests do not cause failure of the target in the form of exit codes.
Each test consists of a set of directories under testing/
. There are directories for klips, pluto,
packaging and libraries. Each directory has a list
of tests to run is stored in a file called TESTLIST in
that directory. e.g. testing/klips/TESTLIST.
This isn't actually a shell script. It just looks like one. Some tools other than /bin/sh process it. Lines that start with # are comments.
# test-kind directory-containing-test expectation [PR#]
The first word provides the test type, detailed below.
The second word is the name of the test to run. This the directory in which the test case is to be found..
The third word may be one of:
The fourth word may be a number, which is a PR# if the test is failing.
Each test directory has a file in it called testparams.sh
. This file sets a number of environment variables to define the
parameters of the test.
klips/test/fixups) to apply
to sanitize the console output of the machine under test. These are
typically perl, awk or sed scripts that remove things in the kernel
output that change each time the test is run and/or compiled.a file of commands that is fed into the virtual machine's console in single user mode prior to starting the tests. This file will usually set up any eroute's and SADB entries that are required for the test.
Lines beginning with # are skipped. Blank lines are skipped.
Otherwise, a shell prompted is waited for each time (consisting of
\n#) and then the command is sent. Note that the prompt is waited
for before the command and not after, so completion of the last command
in the script is not required. This is often used to invoke a program
to monitor the system, e.g. ipsec pf_key.
a file of commands that is fed into the virtual machine's console in single user mode, before the packets are sent. On single machine tests, this script doesn't provide any more power than INIT_SCRIPT, but is implemented for consistency's sake.
a file of commands that is fed into the virtual machine's console in single user mode after the final packet is sent. Similar to INIT_SCRIPT, above. If not specified, then the single command "halt" is sent. If specified, then the script should end with a halt command to nicely shutdown the UML.
uml_netjig will be printed to stderr during the test. In
addition, the jig will be invoked with --debug, which causes it to log
its process ID, and wait 60 seconds before continuing. This can be used
if you are trying to debug the uml_netjig program itself. The klipstest function starts a program (
testing/utils/uml_netjig/uml_netjig) to setup a bunch of I/O
sockets (that simulate network interfaces). It then exports the
references to these sockets to the environment and invokes (using
system()) a given script. It waits for the script to finish.
The script invoked (testing/utils/host-test.tcl) is a
TCL expect script that arranges
to start the UML and configure it appropriately for the test. The
configuration is done with the script given above for INIT_SCRIPT
. The TCL script then forks, leaves the UML in the background and exits.
uml_netjig continues. It then starts listening to the simulated network
answering ARPs and inserting packets as appropriate.
The klipstest function invokes uml_netjig with
arguments to capture output from network interface(s) and insert
packets as appropriate:
uml_netjig. The klipstest function then uses
tcpdump on the file to produce text output, which is compared to the
file given.uml_netjig. It should contain
"--exitonempty" of uml_netjig should exit when all of the input (
PUBINPUT,PRIVINPUT) packets have been injected.uml_netjig. It should contain "--arpreply"
if uml_netjig should reply to ARP requests. One will
typically set this to avoid having to fudge the ARP cache manually. The basic concept of the mkinsttest test type is that
it performs a "make install" to a temporary $DESTDIR. The resulting
tree can then be examined to determine if it was done properly. The
files can be uninstalled to determine if the file list was correct, or
the contents of files can be examined more precisely.
find $ROOT ( -type f -or -type -l ) will be
done to get a list of a real files and symlinks. The resulting file
will be compared to the file listed by this option.reffile samplefileone record per line. A diff between the provided reference file, and the sample file (located in the temporary installation root) will be done for each record.
The rpm_build_install_test type is to verify that the
proper packing list is produced by "make rpm", and that the mechanisms
for building the kernel modules produce consistent results.
The libtest test is for testing library routines. The library file
is expected to provided an #ifdef by the name of
library
. The libtest type invokes the C compiler to compile this
file, links it against libfreeswan.a (to resolve any other
dependancies) and runs the test with the -r argument to
invoke a regression test.
The library test case is expected to do a self-test, exiting with status code 0 if everything is okay, and with non-zero otherwise. A core dump (exit code greater than 128) is noted specifically.
Unlike other tests, there are no subdirectories required, or other parameters to set.
The umlplutotest function starts a pair of user mode line processes, connecting them as shown in the multi_netjig diagram (not linked here).
The script invoked (testing/utils/2host-test.tcl) is a
TCL expect script that arranges
to start the UML and configure it appropriately for the test. It then
starts listening (using uml_netjig) to the simulated network answering
ARPs and inserting packets as appropriate.
The following are used to control the input and output to the system:
a file of commands that is fed into the virtual machine's console in single user mode prior to starting the tests. This file will usually set up any eroute's and SADB entries that are required for the test. Similar to INIT_SCRIPT, above.
a file of commands that is fed into the virtual machine's console in single user mode, before the packets are sent. This set of commands is run after all of the virtual machines are initialized. I.e. after EAST_INIT_SCRIPT AND WEST_INIT_SCRIPT. This script can therefore do things that require that all machines are properly configured.
a file of commands that is fed into the virtual machine's console in single user mode after the final packet is sent. Similar to INIT_SCRIPT, above. If not specified, then the single command "halt" is sent. Note that when this script is run, the other virtual machines may already have been killed. If specified, then the script should end with a halt command to nicely shutdown the UML.
There are two additional environment variables that may be set on the command line:
The kernel_patch_test function takes some kernel source, copies it with lndir, and then applies the patch as produced by "make kernelpatch".
The following are used to control the input and output to the system:
The module_compile test attempts to build the KLIPS module against a given set of kernel source. This is also done by the RPM tests, but in a very specific manner.
There are two variations of this test - one where the kernel either doesn't need to be configured, or is already done, and tests were there is a local configuration file.
Where the kernel doesn't need to be configured, the kernel source that is found is simply used. It may be a RedHat-style kernel, where one can cause it to configure itself via rhconfig.h-style definitions. Or, it may just be a kernel tree that has been configured.
If the variable KERNEL_CONFIG_FILE is set, then a new directory is created for the kernel source. It is populated with lndir(1). The referenced file is then copied in as .config, and "make oldconfig" is used to configure the kernel. This resulting kernel is then used as the reference source.
In all cases, the kernel source is found the same was for the kernelpatch test, i.e. via KERNEL_VERSION/KERNEL_NAME and KERNEL_${KERNEL_NAME}${KERNEL_VERSION}_SRC.
Once there is kernel source, the module is compiled using the top-level "make module" target.
The test is considered successful if an executable is found in OUTPUT/module/ipsec.o at the end of the test.
If you need to interactively control the UML under test, then you can invoke netjig directly with the right arguments, but give it "sh" as the startup script. An easy way is to do this is to edit "NETJIGDEBUG" in setup.sh to "true" and run the test:
e.g:
cassidy-[nightly/klips/test/east-pass-01] mcr 1009 %NETJIGDEBUG=true ../../utils/runme.sh /c2/freeswan/sandboxes/nightly/testing/utils/uml_netjig/uml_netjig --tcpdump --exitonempty --playprivate ../inputs/01-sunrise-sunset-ping.pcap --recordpublic OUTPUT/spi1-output.pcap --startup expect -f /c2/freeswan/sandboxes/nightly/testing/utils/host-test.tcl /c2/freeswan/sandboxes/nightly/UMLPOOL/east/start.sh spi1.shso, substitute "sh" for the startup script.
/c2/freeswan/sandboxes/nightly/testing/utils/uml_netjig/uml_netjig --tcpdump --exitonempty --playprivate ../inputs/01-sunrise-sunset-ping.pcap --recordpublic OUTPUT/spi1-output.pcap --startup sh sh-2.05a$ env | grep UML_ UML_private_CTL=/tmp/umlz8Xrpn/private/ctl UML_public_DATA=/tmp/umlz8Xrpn/public/data UML_public_CTL=/tmp/umlz8Xrpn/public/ctl UML_private_DATA=/tmp/umlz8Xrpn/private/data
then, in a *new window*, paste those variables in place, and start the UML you need. The start.sh scripts looks for UML_{private,public}_{CTL,DATA} and will connect them to eth0/eth1. Setup the UML as appropriate, etc. When you are ready, exit the above shell, and the uml_netjig will start to inject packets and record them.