Test Suite: Difference between revisions
Jump to navigation
Jump to search
(add post-mortem) |
(more guest dependencies) |
||
| Line 26: | Line 26: | ||
| fast | | fast | ||
| linux | | linux | ||
| uses libreswan | | uses host's libreswan, kernel, and utilities | ||
| no | | no | ||
| core, leaks | | core, leaks | ||
| strongswan (linux)? | | strongswan (linux)? | ||
| ideal for quick tests <br> requires libreswan to be built/installed on the host <br> requires all dependencies to be installed on the host <br> | | ideal for quick tests <br> requires libreswan to be built/installed on the host <br> requires all dependencies to be installed on the host <br> test results sensitive differing kernel and utilities | ||
|- style="vertical-align:top;" | |- style="vertical-align:top;" | ||
| [[Test Suite - Docker | Docker]] | | [[Test Suite - Docker | Docker]] | ||
| | | | ||
| linux | | linux | ||
| | | uses host's kernel <br> uses distro's utilities | ||
| ? | | ? | ||
| ? | | ? | ||
| ? | | ? | ||
| ideal for cross-linux builds | | ideal for cross-linux builds (CentOS 6, 7, 8, Fedora 28 - rawhide, Debian, Ubuntu) <br> sensitive to differing kernel and utilities | ||
|} | |} | ||
Revision as of 20:00, 5 August 2023
Running tests
The libreswan tests, in testing/pluto, can be run using several different mechanisms:
| Framework | Speed | Host OS | Guest OS | initsystem testing (systemd, rc.d, ...) | Post-mortem | Interop testing | Notes |
|---|---|---|---|---|---|---|---|
| KVM | slower | Fedora, Debian (BSD anyone?) |
Alpine, Fedora, FreeBSD, NetBSD, OpenBSD (Ubuntu anyone?) |
yes | shutdown, core, leaks, refcnt, selinux | strongswan (Linux, FreeBSD), iked (OpenBSD), racoon (NetBSD), racoon2 (NetBSD) | gold standard ideal for BSD builds idea for testing custom kernels used by the Testing machine requires 9p (virtio anyone?) |
| Namespaces | fast | linux | uses host's libreswan, kernel, and utilities | no | core, leaks | strongswan (linux)? | ideal for quick tests requires libreswan to be built/installed on the host requires all dependencies to be installed on the host test results sensitive differing kernel and utilities |
| Docker | linux | uses host's kernel uses distro's utilities |
? | ? | ? | ideal for cross-linux builds (CentOS 6, 7, 8, Fedora 28 - rawhide, Debian, Ubuntu) sensitive to differing kernel and utilities |
How tests work
All the test cases involving VMs are located in the libreswan directory under testing/pluto/. The most basic test case is called basic-pluto-01. Each test case consists of a few files:
- description.txt to explain what this test case actually tests
- ipsec.conf files - for host west is called west.conf. This can also include configuration files for strongswan or racoon2 for interop testig
- ipsec.secret files - if non-default configurations are used. also uses the host syntax, eg west.secrets, east.secrets.
- An init.sh file for each VM that needs to start (eg westinit.sh, eastinit.sh, etc)
- One run.sh file for the host that is the initiator (eg westrun.sh)
- Known good (sanitized) output for each VM (eg west.console.txt, east.console.txt)
- testparams.sh if there are any non-default test parameters
Once the test run has completed, you will see an OUTPUT/ directory in the test case directory:
$ ls OUTPUT/ east.console.diff east.console.verbose.txt RESULT west.console.txt west.pluto.log east.console.txt east.pluto.log swan12.pcap west.console.diff west.console.verbose.txt
- RESULT is a text file (whose format is sure to change in the next few months) stating whether the test succeeded or failed.
- The diff files show the differences between this testrun and the last known good output.
- Each VM's serial (sanitized) console log (eg west.console.txt)
- Each VM's unsanitized verbose console output (eg west.console.verbose.txt)
- A network capture from the bridge device (eg swan12.pcap)
- Each VM's pluto log, created with plutodebug=all (eg west.pluto.log)
- Any core dumps generated if a pluto daemon crashed
- testing/baseconfigs/
- configuration files installed on guest machines
- testing/guestbin/
- shell scripts used by tests, and run on the guest
- testing/linux-system-roles.vpn/
- ???
- testing/packaging/
- ???
- testing/pluto/TESTLIST
- list of tests, and their expected outcome
- testing/pluto/*/
- individual test directories
- testing/programs/
- executables used by tests, and run on the guest
- testing/sanitizers/
- filters for cleaning up the test output
- testing/utils/
- test drivers and other host tools
- testing/x509/
- certificates, scripts are run on a guest
Network Diagrams
Current Network Diagram
- interface-0 (eth0, vio0, vioif0) is connected to SWANDEFAULT which has a NAT gateway to the internet
- the exceptions are the Fedora test domains: EAST, WEST, ROAD, NORTH; should they?
- the BSD domains always up inteface-0 so that /pool, /source, and /testing can be NFS mounted
- NIC needs to run DHCP on eth0 manually; how?
- transmogrify does not try to modify interface-0(SWANDEFAULT) (it breaks established network sessions such as NFS)
- the interface names do not have consistent order (see comment above about Fedora's interface-0 not pointing at SWANDEFAULT)
- Fedora has ethN
- OpenBSD has vioN (different order)
- NetBSD has vioifN (different order)
LEFT RIGHT
192.0.3.0/24 -------------------------------------+-- 2001:db8:0:3::/64
|
2001:db8:0:3::254
192.0.3.254(eth0)
ROAD NORTH
192.1.3.209(eth0) 192.1.3.33(eth1)
2001:db8:1:3::209 2001:db8:1:3::33
| |
192.1.3.0/254 ----+----------------+--------------+-- 2001:db8:1:3::/64
|
2001:db8:1:3::254
192.1.3.254(eth2)
NIC---swandefault(0)
192.1.2.254(eth1)
2001:db8:1:2::254
|
192.1.2.0/24 ---+------------------+-------------+--- 2001:db8:1:2::/64
| |
2001:db8:1:2::45 2001:db8:1:2::23
192.1.2.45(eth1) 192.1.2.23(eth1)
WEST---[swandefault(0)] EAST---[swandefault(0)]
192.0.1.254(eth0/viof2) 192.0.2.254(eth0/viof2)
2001:db8:0:1::254 2001:db8:0:2::254
| |
| 192.0.2.0/255 ---+--- 2001:db8:0:2::/64
|
192.0.1.0/255 --+------------------------------------ 2001:db8:0:1::/64
192.1.4.0/255 --------------------------------------- 2001:db8:1:4::/64
Proposed Network Diagram
Changes:
- add NOC, unlike NIC can run libreswan
- add SUNRISE (behind east) and SUNSET (behind west)
Notes:
- if0 (eth0, vio0, vioif0) is connected to SWANDEFAULT which has a NAT gateway to the internet
- the exceptions are the Fedora test domains: EAST, WEST, ROAD, NORTH; should they?
- the BSD domains use if0 to NFS mount /bench, /pool, /source, and /testing
- NIC needs to run DHCP on eth0 manually; how?
- transmogrify does not try to modify if0(SWANDEFAULT) (it breaks established network sessions such as NFS)
- the interface names still do not have consistent order
- ifN denotes a consistent numbering
- ethN denotes Fedora only assignment
- bsdN denotes a BSD only assignment (FreeBSD has ???N, NetBSD has vioifN, OpenBSD has vioN)
- re-number networks so that they are RFC friendly, these RFCs came up in discussion; alternatively do nothing
192.0.3.0/24 --------------------------------------+---- 2001:db8:0:3::/64
|
2001:db8:0:3::254
192.0.3.254
<eth0>
ROAD NORTH
<eth0> <eth1>
192.1.3.209 192.1.3.33
2001:db8:1:3::209 2001:db8:1:3::33
| |
192.1.3.0/254 ----+---+-------------------------+---+---- 2001:db8:1:3::/64
| |
2001:db8:1:3::254 2001:db8:1:3::253
192.1.3.254(eth2) 192.1.3.253
<if2> <if2>
NIC-<if0>-swandefault NOC-<if0>-swandefault
<if1> <if1>
192.1.2.254 192.1.2.253
2001:db8:1:2::254 2001:db8:1:2::253
| |
192.1.2.0/24 -----+--+-------------------------+--+----- 2001:db8:1:2::/64
| |
2001:db8:1:2::45 2001:db8:1:2::23
192.1.2.45 192.1.2.23
<if1> <if1>
WEST-[<bsd0>-swandefault] EAST-[<bsd0>-swandefault]
<eth0/bsd2> <eth0/bsd2>
192.0.1.254 192.0.2.254
2001:db8:0:1::254 2001:db8:0:2::254
| |
| 192.0.2.0/255 ---------+--+-- 2001:db8:0:2::/64
| |
| 2001:db8:0:2::252
| 192.0.2.252
| <if1>
| SUNRISE-<if0>-swandefault
|
192.0.1.0/255 --+--+------------------------------------- 2001:db8:0:1::/64
|
2001:db8:0:1::252
192.0.1.252
<if1>
SUNSET-<if0>-swandefault
192.1.4.0/255 --------------------------------------- 2001:db8:1:4::/64
Hand Sketch of Current Network
Original Network Diagram
