( we will sort this in categories once we have more )
Module unloading error on shutdown or restart: Module esp4 is in use
A common error to see is:
ERROR: Module xfrm4_mode_tunnel is in use ERROR: Module esp4 is in use FAILURE to unload NETKEY esp4/esp6 module
This happens when libreswan is stopped or restarted. Libreswan supports the alternative KLIPS IPsec stack. This can be configured using protostack=netkey or protostack=klips. See man ipsec.conf for details
The error happens when sockets in the kernel are still holding deleted IPsec SA's. There is no way to "force delete" these. Processes might be holding open sockets that have pinned down those deleted SA's. Although our IKE daemon (pluto) process has ended, these might still be lingering on the system.
The kernel people's answer is don't delete the modules, which is not an option for us because we want to support switching IPsec stacks. An alternative is to use rmmod -w which will cause us to wait, but we have no idea how long that might be.
Regardless, the deleted IPsec SA's are not interfering with the operation of the the XFRM/NETKEY and this error can be ignored as long as you are not switching stacks.