Test Suite: Difference between revisions
Jump to navigation
Jump to search
(list changes) |
(fix subnetting) |
||
Line 123: | Line 123: | ||
2001:db8:1:3::209 2001:db8:1:3::33 | 2001:db8:1:3::209 2001:db8:1:3::33 | ||
| | | | | | ||
192.1.3.0/ | 192.1.3.0/24 -----+----------------+--------------+-- 2001:db8:1:3::/64 | ||
| | | | ||
2001:db8:1:3::254 | 2001:db8:1:3::254 | ||
Line 140: | Line 140: | ||
| | | | | | ||
| TEST-NET-1 | | TEST-NET-1 | ||
| 192.0.2.0/ | | 192.0.2.0/24 ----+--- 2001:db8:0:2::/64 | ||
| | | | ||
192.0.1.0/ | 192.0.1.0/24 ---+------------------------------------ 2001:db8:0:1::/64 | ||
== Problems with the existing Network == | == Problems with the existing Network == | ||
Line 168: | Line 168: | ||
192.0.3.254(eth0) | 192.0.3.254(eth0) | ||
ROAD NORTH | ROAD NORTH | ||
192.1.3.209(eth0) | 192.1.3.209(eth0) 192.1.3.33(eth1) | ||
2001:db8:1:3::209 2001:db8:1:3::33 | 2001:db8:1:3::209 2001:db8:1:3::33 | ||
| | | | | | ||
Line 191: | Line 191: | ||
| | | | | | ||
TEST-NET-2 | | | TEST-NET-2 | | | ||
198.51.100.0/ | 198.51.100.0/28 ----------+---+----------- | | ||
2001:db8:51:100::/64 | | | 2001:db8:51:100::/64 | | | ||
| | | | | | ||
Line 197: | Line 197: | ||
| | | | ||
TEST-NET-3 | | TEST-NET-3 | | ||
203.0.113.0/ | 203.0.113.0/28 ---------------------------------------------+---+--- | ||
2001:db8:0:113::/64 | | 2001:db8:0:113::/64 | | ||
${OS}RISE | ${OS}RISE |
Revision as of 14:33, 30 September 2024
Nightly Test Tesults
Libreswan's testsuite is run nightly. The results are published here, with the most recent result here. The tests are categorized as:
- good: these tests are expected to pass (unfortunately, some still have timing problems and occasionally fail)
- wip: these tests require further work, for instance the result may not be deterministic, or the bug they demonstrate hasn't yet been fixed
- skiptest: these tests require manual intervention to run
To run tests locally, read on.
Running tests
The libreswan tests, in testing/pluto, can be run using several different mechanisms:
Framework | Speed | Host OS | Guest OS | initsystem testing (systemd, rc.d, ...) | Post-mortem | Interop testing | Notes |
---|---|---|---|---|---|---|---|
KVM | slower | Fedora, Debian (BSD anyone?) |
Alpine, Fedora, FreeBSD, NetBSD, OpenBSD, Debian | yes | shutdown, core, leaks, refcnt, selinux | strongswan (Linux, FreeBSD), iked (OpenBSD), racoon (NetBSD), racoon2 (NetBSD) | gold standard ideal for BSD builds idea for testing custom kernels used by the Testing machine requires 9p (virtio anyone?) |
Namespaces | fast | linux | uses host's libreswan, kernel, and utilities | no | core, leaks | strongswan (linux)? | ideal for quick tests requires libreswan to be built/installed on the host requires all dependencies to be installed on the host test results sensitive differing kernel and utilities |
Docker | linux | uses host's kernel uses distro's utilities |
? | ? | ? | ideal for cross-linux builds (CentOS 6, 7, 8, Fedora 28 - rawhide, Debian, Ubuntu) sensitive to differing kernel and utilities |
How tests work
All the test cases involving VMs are located in the libreswan directory under testing/pluto/. The most basic test case is called basic-pluto-01. Each test case consists of a few files:
- description.txt to explain what this test case actually tests
- ipsec.conf files - for host west is called west.conf. This can also include configuration files for strongswan or racoon2 for interop testig
- ipsec.secret files - if non-default configurations are used. also uses the host syntax, eg west.secrets, east.secrets.
- An init.sh file for each VM that needs to start (eg westinit.sh, eastinit.sh, etc)
- One run.sh file for the host that is the initiator (eg westrun.sh)
- Known good (sanitized) output for each VM (eg west.console.txt, east.console.txt)
- testparams.sh if there are any non-default test parameters
Once the test run has completed, you will see an OUTPUT/ directory in the test case directory:
$ ls OUTPUT/ east.console.diff east.console.verbose.txt RESULT west.console.txt west.pluto.log east.console.txt east.pluto.log swan12.pcap west.console.diff west.console.verbose.txt
- RESULT is a text file (whose format is sure to change in the next few months) stating whether the test succeeded or failed.
- The diff files show the differences between this testrun and the last known good output.
- Each VM's serial (sanitized) console log (eg west.console.txt)
- Each VM's unsanitized verbose console output (eg west.console.verbose.txt)
- A network capture from the bridge device (eg swan12.pcap)
- Each VM's pluto log, created with plutodebug=all (eg west.pluto.log)
- Any core dumps generated if a pluto daemon crashed
- testing/baseconfigs/
- configuration files installed on guest machines
- testing/guestbin/
- shell scripts used by tests, and run on the guest
- testing/linux-system-roles.vpn/
- ???
- testing/packaging/
- ???
- testing/pluto/TESTLIST
- list of tests, and their expected outcome
- testing/pluto/*/
- individual test directories
- testing/programs/
- executables used by tests, and run on the guest
- testing/sanitizers/
- filters for cleaning up the test output
- testing/utils/
- test drivers and other host tools
- testing/x509/
- certificates, scripts are run on a guest
Network Diagram
- interface-0 (eth0, vio0, vioif0) is connected to SWANDEFAULT which has a NAT gateway to the internet
- the exceptions are the Linux test domains: EAST, WEST, ROAD, NORTH; should they?
- the BSD domains always up inteface-0 so that /pool, /source, and /testing can be NFS mounted
- NIC needs to run DHCP on eth0 manually; how?
- transmogrify does not try to modify interface-0(SWANDEFAULT) (doing so would break established network sessions such as NFS)
- the interface names do not have consistent order (see comment above about Fedora's interface-0 not pointing at SWANDEFAULT)
- Fedora has ethN
- OpenBSD has vioN (different order)
- NetBSD has vioifN (different order)
LEFT RIGHT
192.0.3.0/24 -------------------------------------+-- 2001:db8:0:3::/64 | 2001:db8:0:3::254 192.0.3.254(eth0) ROAD NORTH 192.1.3.209(eth0) 192.1.3.33(eth1) 2001:db8:1:3::209 2001:db8:1:3::33 | | 192.1.3.0/24 -----+----------------+--------------+-- 2001:db8:1:3::/64 | 2001:db8:1:3::254 192.1.3.254(eth2) NIC---swandefault(0) 192.1.2.254(eth1) 2001:db8:1:2::254 | 192.1.2.0/24 ---+------------------+-------------+--- 2001:db8:1:2::/64 | | 2001:db8:1:2::45 2001:db8:1:2::23 192.1.2.45(eth1) 192.1.2.23(eth1) WEST EAST 192.0.1.254(eth0) 192.0.2.254(eth0) 2001:db8:0:1::254 2001:db8:0:2::254 | | | TEST-NET-1 | 192.0.2.0/24 ----+--- 2001:db8:0:2::/64 | 192.0.1.0/24 ---+------------------------------------ 2001:db8:0:1::/64
Problems with the existing Network
The current network has a number of limitations. This section identifies those problems and proposes changes to address them:
- the network addresses being used are not really suitable for use in documentation
- the RFCs recommend a number of addresses to use in documentation, by using these in tests it will be possible to directly lift examples from working test cases
- IPv4 192.0.2.0/24 aka TEST-NET-1, 198.51.100.0/24 aka TEST-NET-2, 203.0.113.0/24 aka TEST-NET-3, see https://www.rfc-editor.org/rfc/rfc5737
- IPv6 2001:DB8::/32 see https://www.rfc-editor.org/rfc/rfc3849
- look for documentation in https://www.rfc-editor.org/rfc/rfc6890
- two hosts, each behind a gateway, where the gateway's are connected using IPsec
- HOST-<unencrypted>-GATEWAY-<IPsec>-GATEWAY-unencrypted>-HOST
- tests try to mimic this by injecting packets into gateway's internal interface, however it isn't the same
- the OS specific VMs (NetBSD, Alpine, Debian, FreeBSD, OpenBSD, Fedora) have no consistent home
- currently they are kind of like east, west, and north
LEFT RIGHT
192.0.3.0/24 -------------------------------------+------ 2001:db8:0:3::/64 | 2001:db8:0:3::254 192.0.3.254(eth0) ROAD NORTH 192.1.3.209(eth0) 192.1.3.33(eth1) 2001:db8:1:3::209 2001:db8:1:3::33 | | 192.1.3.0/254 ---------+----------------+---------+------ 2001:db8:1:3::/64 | 2001:db8:1:3::254 192.1.3.254(eth2) NIC---swandefault(0) 192.1.2.254(eth1) 2001:db8:1:2::254 | TEST-NET-1 | 192.0.2.0/24 -----------------+---------+-------------------+----- 2001:db8:0:2::/64 | | | | 2001:db8:1:2::45 2001:db8:1:2::23 192.1.2.45(eth1) 192.1.2.23(eth1) $(OS)W--if0-swandefault-if0--${OS}E WEST EAST 192.0.1.254(eth0/2) 192.0.2.254(eth0/2) 2001:db8:0:1::254 2001:db8:0:2::254 | | TEST-NET-2 | | 198.51.100.0/28 ----------+---+----------- | 2001:db8:51:100::/64 | | | | ${OS}SET | | TEST-NET-3 | 203.0.113.0/28 ---------------------------------------------+---+--- 2001:db8:0:113::/64 | ${OS}RISE
The changes are as follows:
- behind EAST renumber its network to TEST-NET-3 and add the machine $(OS)RISE
- behind WEST renumber its network to TeST-NET-3 and add the machine $(OS)SET
- between EAST and WEST renumber the network to TEST-NET-1
- behind NIC, consider renumbering the network to one of the reserved for home use / nats 192.168 / or 10.0....