Typical and most robust way of configuring connectivity over SCTP (e.g. Diameter or Camel links) is to use two independent IP links (VLANs), bind them on SCTP level and let the protocol do the magic.
So the setup looks more or less like that:
Strangely enough, in one of my installations, I had to connect:
- a single-homed node (“Node A”) with – this node is represented by a single IP address. Obviously, it has redundant network interfaces but logically, it’s just on IP
- a multi-homed node (“Node B”) with classical setup – two interfaces (ports), two IP addresses
Im not going to convince you that this is a good solution. There are better ☺️ But in the given situation, I have had no other choice.
Communication in direction A -> B is easy – the A has both IP3 and IP4 set in the configuration, SCTP INIT is sent to one of the two. In response, INIT ACK contains both IPs of the B.
But here small troubles start: how this poor Node B knows over which port the packet should be sent? Classical routing configuration does not help as Node A is just one port but the B has two possible paths?
Answer is source-based routing. In Linux machine, it’s possible to set up IP routing not only based on the destination but also based on the source address.
Let’s assume that Node B has following interfaces:
$ ifconfig -a eth21 Link encap:Ethernet HWaddr 00:50:56:01:24:FD inet addr:10.131.34.141 Bcast:10.131.34.143 Mask:255.255.255.248 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:97809018 errors:0 dropped:0 overruns:0 frame:0 TX packets:250084 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:9587677300 (8.9 GiB) TX bytes:23053672 (21.9 MiB) eth22 Link encap:Ethernet HWaddr 00:50:56:01:24:FC inet addr:10.131.34.133 Bcast:10.131.34.135 Mask:255.255.255.248 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2792022 errors:0 dropped:0 overruns:0 frame:0 TX packets:1873725 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:366567654 (349.5 MiB) TX bytes:250473782 (238.8 MiB)
Configure Routing Rules
Create rules to forward all the package entering via a particular NIC to go out the appropriate routing table. Tree routing tables table1 Sig-A, table 2 for Sig-B and default (table 254).
Example
$ cat /etc/sysconfig/network-scripts/rule-eth22 iif eth22 table 1 from 10.131.34.133 table 1 $ cat /etc/sysconfig/network-scripts/rule-eth21 iif eth21 table 2 from 10.131.34.141 table 2
Configure Routing File
Create one route-file for each signalling interface, including which routing table to use for different destination addresses.
Node A source-routing in table 1 (Sig A) and table 2 (Sig B).
Node A destination-routing in global routing table (254) with metric for Sig B.
Example:
$ cat /etc/sysconfig/network-scripts/route-eth22 # Source routing for SIGA-Network # Routing table 1 10.131.34.128/29 dev eth22 table 1 217.214.139.72/29 via 10.131.34.129 dev eth22 table 1 217.214.139.80/29 via 10.131.34.129 dev eth22 table 1 # Routing table default 254 217.214.139.72/29 via 10.131.34.129 dev eth22 217.214.139.80/29 via 10.131.34.129 dev eth22
$ cat /etc/sysconfig/network-scripts/route-eth21 # Source routing for SIGB-Network # Routing table 2 10.131.34.136/29 dev eth21 table 2 217.214.139.72/29 via 10.131.34.137 dev eth21 table 2 217.214.139.80/29 via 10.131.34.137 dev eth21 table 2 # Routing table default 254 217.214.139.72/29 via 10.131.34.137 dev eth21 metric 512 217.214.139.80/29 via 10.131.34.137 dev eth21 metric 512 # Routing table default 254 10.131.34.16/28 via 10.131.34.137 dev eth21 10.131.42.16/28 via 10.131.34.137 dev eth21
Verify routing
To show different routing tables
ip route show table [TableNumber] ip route = ip route show table 254
Example:
$ ip route 217.214.139.72/29 via 10.131.34.129 dev eth22 217.214.139.72/29 via 10.131.34.137 dev eth21 metric 512 10.131.34.128/29 dev eth22 proto kernel scope link src 10.131.34.133 217.214.139.80/29 via 10.131.34.129 dev eth22 217.214.139.80/29 via 10.131.34.137 dev eth21 metric 512 10.131.34.136/29 dev eth21 proto kernel scope link src 10.131.34.141 10.131.42.16/28 via 10.131.34.137 dev eth21 10.131.42.0/28 via 10.131.34.129 dev eth22 10.131.34.16/28 via 10.131.34.137 dev eth21 10.131.34.0/28 via 10.131.34.129 dev eth22 $ ip route show table 1 217.214.139.72/29 via 10.131.34.129 dev eth22 217.214.139.80/29 via 10.131.34.129 dev eth22 10.131.34.128/29 dev eth22 scope link $ ip route show table 2 217.214.139.72/29 via 10.131.34.137 dev eth21 217.214.139.80/29 via 10.131.34.137 dev eth21 10.131.34.136/29 dev eth21 scope link
To verify source-routing entery
ip route get [Destination-IP] from [SourceInterface-IP]
ip route get [Destination-IP] = global routing entery
Example:
$ ip route get 217.214.139.74 from 10.131.34.141 217.214.139.74 from 10.131.34.141 via 10.131.34.137 dev eth21 cache ipid 0xc39d mtu 1500 hoplimit 64 $ ip route get 217.214.139.74 from 10.131.34.133 217.214.139.74 from 10.131.34.133 via 10.131.34.129 dev eth22 cache mtu 1500 hoplimit 64 $ ip route get 217.214.139.74 217.214.139.74 via 10.131.34.129 dev eth22 src 10.131.34.133 cache mtu 1500 hoplimit 64
To verify routing trace from a specific source interface
traceroute –s [SourceInterface-IP] [Destination-IP] traceroute [Destination-IP] = global routing trace
Example:
$ traceroute -s 10.131.34.133 217.214.139.74 traceroute to 217.214.139.74 (217.214.139.74), 30 hops max, 60 byte packets 1 10.131.34.129 (10.131.34.129) 0.132 ms 0.124 ms 0.113 ms 2 * * * 3 * * * 4 * * * 5 * * * 6 217.214.139.74 (217.214.139.74) 11.965 ms 14.021 ms 14.000 ms $ traceroute -s 10.131.34.141 217.214.139.74 traceroute to 217.214.139.74 (217.214.139.74), 30 hops max, 60 byte packets 1 10.131.34.137 (10.131.34.137) 0.175 ms 0.194 ms 0.167 ms 2 * * * 3 * * * 4 * * * 5 * * * 6 217.214.139.74 (217.214.139.74) 12.635 ms 12.653 ms 12.803 ms $ traceroute 217.214.139.74 traceroute to 217.214.139.74 (217.214.139.74), 30 hops max, 60 byte packets 1 10.131.34.129 (10.131.34.129) 0.135 ms 0.106 ms 0.082 ms 2 * * * 3 * * * 4 * * * 5 * * * 6 217.214.139.74 (217.214.139.74) 11.883 ms 13.979 ms 11.875 ms
That’s it!
Many thanks to Ola for sorting this out and preparing this procedure.