XFRM Interface Development Notes: Difference between revisions
No edit summary |
No edit summary |
||
| Line 132: | Line 132: | ||
* XFRMI interface and multihoming. Yet to test this. It would bet interesting to test. If it all work the idea is mark on incoming ESP, set by iptable rule, would get copied first to incoming clear packet, then to outgoing clear text packet and finally to the outgoing ESP packet. With XFRMi at every stage there are could routlookup involving fwmark with mark. I wonder if it works. Any one tested it? | * XFRMI interface and multihoming. Yet to test this. It would bet interesting to test. If it all work the idea is mark on incoming ESP, set by iptable rule, would get copied first to incoming clear packet, then to outgoing clear text packet and finally to the outgoing ESP packet. With XFRMi at every stage there are could routlookup involving fwmark with mark. I wonder if it works. Any one tested it? | ||
* Simple case IPsec with IP_VTI. Then you can pick any | * Simple case IPsec with IP_VTI, only libreswan using marks on the ssytem. | ||
in libreswan config you set mark=2/0xffffffff. The rest is transparent to users, magic happens. | Then you can pick any mark you want. in libreswan config you set mark=2/0xffffffff. The rest is transparent to users, magic happens. | ||
* | * more complicated case VTI and multihoming | ||
Lets say you have two upstreams | |||
Lets say you have two upstreams. Both route packets to your host. Then you would start to use iptable to mark the incoming traffic, ESP. First the ESP will get decrypted and new clear text packet will retain the mark. The mark will be used to get source address for response packet. Then it will go through xfrm. Get encrypted again retainng the mark. Then get routed to via the interface the the packet came in. | |||
Now with VTI. | |||
A portion of mark clear text outgoing response is over written (using mask). | |||
If you are familiar with the commands bellow you get an idea what is going on. I herd it works. | |||
I wonder if the incoming ESP packet's mark, say set by iptable before hitting XFRM input code, would end up on the outgoing ESP too or just on the clear text response. | |||
<pre> | <pre> | ||
iptables -t mangle -N MARK-ISP1 | iptables -t mangle -N MARK-ISP1 | ||
| Line 152: | Line 159: | ||
ip route add table ISP2 192.0.3.0/24 dev eth2 src 192.0.22.0/24 | ip route add table ISP2 192.0.3.0/24 dev eth2 src 192.0.22.0/24 | ||
</pre> | </pre> | ||
= Track bugs and related issue = | |||
* Fedora support in F29 [https://src.fedoraproject.org/rpms/kernel/c/84dd8fe88279144ddb82168d1cc44117073ff07e F29 commit] enabled it. However not in F28 | |||
* RHEL bugzilla request?? | * RHEL bugzilla request?? | ||
* Where is the debian support? https://salsa.debian.org/kernel-team/linux/tree/master/debian/config/amd64 ?/? | * Where is the debian support? https://salsa.debian.org/kernel-team/linux/tree/master/debian/config/amd64 ?/? | ||
Revision as of 14:31, 19 February 2019
libreswan/pluto design choices
- allow names ipsec0, ipsec1 ... ipsecx.
- Should we allow names other than ipsecXX ?
- initial thought is keep "xfrm interface id" and "xfrm output mark" consistent.
- interface creation is inside pluto.
- create bugzilla entry for RHEL support.
- XFRMi code is compile time option. If the kernel headers do no't support it won't compile. Think RHEL 6 or Debian Weezy.
XFRM INTERFACE
Commit cover letter from Steffen 20180612, Merged in 4.19:
https://patchwork.ozlabs.org/cover/928175/ Steffen Klassert June 12, 2018, 7:56 a.m. This patchset introduces new virtual xfrm interfaces. The design of virtual xfrm interfaces interfaces was discussed at the Linux IPsec workshop 2018. This patchset implements these interfaces as the IPsec userspace and kernel developers agreed. The purpose of these interfaces is to overcome the design limitations that the existing VTI devices have. The main limitations that we see with the current VTI are the following: - VTI interfaces are L3 tunnels with configurable endpoints. For xfrm, the tunnel endpoint are already determined by the SA. So the VTI tunnel endpoints must be either the same as on the SA or wildcards. In case VTI tunnel endpoints are same as on the SA, we get a one to one correlation between the SA and the tunnel. So each SA needs its own tunnel interface. On the other hand, we can have only one VTI tunnel with wildcard src/dst tunnel endpoints in the system because the lookup is based on the tunnel endpoints. The existing tunnel lookup won't work with multiple tunnels with wildcard tunnel endpoints. Some usecases require more than on VTI tunnel of this type, for example if somebody has multiple namespaces and every namespace requires such a VTI. - VTI needs separate interfaces for IPv4 and IPv6 tunnels. So when routing to a VTI, we have to know to which address family this traffic class is going to be encapsulated. This is a lmitation because it makes routing more complex and it is not always possible to know what happens behind the VTI, e.g. when the VTI is move to some namespace. - VTI works just with tunnel mode SAs. We need generic interfaces that ensures transfomation, regardless of the xfrm mode and the encapsulated address family. - VTI is configured with a combination GRE keys and xfrm marks. With this we have to deal with some extra cases in the generic tunnel lookup because the GRE keys on the VTI are actually not GRE keys, the GRE keys were just reused for something else. All extensions to the VTI interfaces would require to add even more complexity to the generic tunnel lookup. To overcome this, we started with the following design goal: - It should be possible to tunnel IPv4 and IPv6 through the same interface. - No limitation on xfrm mode (tunnel, transport and beet). - Should be a generic virtual interface that ensures IPsec transformation, no need to know what happens behind the interface. - Interfaces should be configured with a new key that must match a new policy/SA lookup key. - The lookup logic should stay in the xfrm codebase, no need to change or extend generic routing and tunnel lookups. - Should be possible to use IPsec hardware offloads of the underlying interface.
Initial xfrmi kernel commits
- Steffen Klassert's at Linux IPsec 2018, Dresden https://workshop.linux-ipsec.org/2018/slides/xfrm_interfaces.pdf
XFRM OUTPUT_MARK (required commits to be pulled to RHEL)
Add mask and rename to XFRMA_SET_MARK, Steffen
Which mark?
There are wo marks used in the context of IPsec or xfrm in Linux kernel. XFRMA_MARK and XFRMA_SET_MARK(aka XFRMA_OUTPUT_MARK). Now on I use XFRM_OUTPUT_MARK mark which as of 4.20 mean XFRM_SET_MARK/XFRM_SET_MARK_MASK
One line difference between the two. XFRM_OUTPUT_MARK is for routing a packet after XFRM(think as ESP out or Clear text in), while XFRMA_MARK could be seen as routing a clear text packet into xfrm; XFRMA_MARK is a lookup key of SPDB and SADB.
XFRMA_MARK (along with mask) is used inside xfrm code to find policy and satte on for outgoing, clear text, packet Jamal's notes (clear text packet out only. Then I wonder why set XFRMA_MARK on incoming SA? See libreswan commit. What is the use case of it. Is it supposed to copy some mark on the incoming clear to something? I don't thing so yet!
To be clear, both marks are used for routing a packet at different stages of it while to goes through Linux stack and get transformed inside XFRM. or in a more complicated case where a packet get marked as incoming packet to the host the application replies. Now the magic is the incoming packet's mark get copied to related outgoing packet, use case multi homing. While XFRM_OUTPUT_MARK is used for routing ESP packet on the output side and clear text packet on the input side. You see a bit more details on XFRMA_MARK see Jamal's explanation
XFRMA_MARK - mark(u32)/mask(u32). XFRM_OUTPUT_MARK - mark(u32)/[mask(u32). So one difference is for XFRM_OUTPUT_MARK mask is optional as of 4.19, 4.18 did not support mask for OUTPUT_MARK.
IPsec and routing has to share same mark. That why there is mask. One part for IPsec/XFRM and other part for the rest of the system use. XFRM stack should pass on the mark set by the system when correct mask is used. Masked part is opaque to xfrm.
use case of marks
- Simple use case XFRMI interface.
XFRM_OUTPUT_MARK by libreswan when the the other/peer end is inside the extruded tunnel. In other words. Say /32-to-/32 tunnel without NAT or 0.0.0.0/0 tunnel. Note it is adding rules to rout
ip rule add prio 100 to 192.1.2.23/32 not fwmark 1/0xffffffff lookup 50 ip route add table 50 192.1.2.23/32 dev ipsec0 src 192.1.3.209
- XFRMI interface and multihoming. Yet to test this. It would bet interesting to test. If it all work the idea is mark on incoming ESP, set by iptable rule, would get copied first to incoming clear packet, then to outgoing clear text packet and finally to the outgoing ESP packet. With XFRMi at every stage there are could routlookup involving fwmark with mark. I wonder if it works. Any one tested it?
- Simple case IPsec with IP_VTI, only libreswan using marks on the ssytem.
Then you can pick any mark you want. in libreswan config you set mark=2/0xffffffff. The rest is transparent to users, magic happens.
- more complicated case VTI and multihoming
Lets say you have two upstreams. Both route packets to your host. Then you would start to use iptable to mark the incoming traffic, ESP. First the ESP will get decrypted and new clear text packet will retain the mark. The mark will be used to get source address for response packet. Then it will go through xfrm. Get encrypted again retainng the mark. Then get routed to via the interface the the packet came in.
Now with VTI. A portion of mark clear text outgoing response is over written (using mask). If you are familiar with the commands bellow you get an idea what is going on. I herd it works. I wonder if the incoming ESP packet's mark, say set by iptable before hitting XFRM input code, would end up on the outgoing ESP too or just on the clear text response.
iptables -t mangle -N MARK-ISP1 iptables -t mangle -A MARK-ISP1 -j MARK --set-mark 1 iptables -t mangle -A MARK-ISP1 -j CONNMARK --save-mark iptables -t mangle -N MARK-ISP2 iptables -t mangle -A MARK-ISP2 -j MARK --set-mark 2 iptables -t mangle -A MARK-ISP2 -j CONNMARK --save-mark ip rule add fwmark 1 table ISP1 ip rule add fwmark 2 table ISP2 ip route add table ISP1 192.0.3.0/24 dev eth1 src 192.0.2.0/24 ip route add table ISP2 192.0.3.0/24 dev eth2 src 192.0.22.0/24
- Fedora support in F29 F29 commit enabled it. However not in F28
- RHEL bugzilla request??
- Where is the debian support? https://salsa.debian.org/kernel-team/linux/tree/master/debian/config/amd64 ?/?