[LUG] Routing Strangeness

[Theo Zourzouvillys <theo@xxxxxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Argh.

This is driving me *mad*.  I think i'm missing something really simple but 
i've been staring at it too long.

Please excuse me if no one here is routing/network/tcp gods, i'll go beg for 
mercy from linux-net list if there isn't ;)

Imagine a netowrk looking somethign like this:

* start sexy ascii *

          Peer Feeds
              ||
              ||
              ||
           --------
          | ROUTER |==== (to core router # 2)
           --------
              ||
    ---------------------
   | Arrowpoint CSS11500 |
   |Load Balancer/Switch | ====> To Co-Lo Customers
   |                     |
    ---------------------
 eth0 -> ||       || <- eth1
    ----------------------
   |    Linux Firewall    |
    ----------------------
 eth2 -> ||       || <- eth3
         ||       ||
         ||       ||
To Linux Boxes   To Windows Boxes
on 10.2.0.0/16   on 10.1.0.0/16

* end sexy acsii *

Router Internal Address # 1 = 172.16.0.1
Switch VLAN Address = 172.16.0.2

on the linux box:
eth0 = 172.16.0.3
eth1 = 172.16.0.4
eth2 = 10.2.0.1
eth3 = 10.3.0.1

The CS150's default gateway is 172.16.0.3
The Routers default gateway for 123.123.123.0/24 is 172.16.0.3

the linux box's routing table looks like so:

anglerfish:~# ip route show
172.16.0.100 dev eth1  scope link
172.16.0.0/24 dev eth1  proto kernel  scope link  src 172.16.0.4
172.16.0.0/24 dev eth0  proto kernel  scope link  src 172.16.0.3
unreachable 123.123.123.0/24  scope host
123.123.123.0/24 dev eth0  proto kernel  scope link  src 123.123.123.1
10.2.0.0/16 dev eth2  proto kernel  scope link  src 10.2.0.1
10.1.0.0/16 dev eth3  proto kernel  scope link  src 10.1.0.1
default via 172.16.0.1 dev eth1

anglerfish:~# ip rule show
0:      from all lookup local
32765:  from all fwmark        2 lookup gw_arpt
32766:  from all lookup main
32767:  from all lookup default

anglerfish:~# ip route show table gw_arpt
default via 172.16.0.2 dev eth1

and netfilter mangle table looks like this:

anglerfish:~# iptables -L -n -v -t mangle
Chain PREROUTING (policy ACCEPT 814K packets, 368M bytes)
pkts bytes target     prot opt in     out     source               
destination
  12  1078 MARK       all  --  eth3   *       10.1.0.0/16          
123.123.123.0/24    MARK set 0x2
4928 1780K MARK       all  --  eth2   *       10.2.0.0/16          
123.123.123.0/24    MARK set 0x2


the nat table looks a bit like this:

anglerfish:~# iptables -L -n -v -t nat
Chain PREROUTING (policy ACCEPT 82909 packets, 6741K bytes)
pkts bytes target     prot opt in     out     source               
destination
   0    0  DNAT       tcp  --  *      *       0.0.0.0/0            
123.123.123.246     tcp dpt:80 to:172.16.0.100


Chain POSTROUTING (policy ACCEPT 19174 packets, 1119K bytes)
pkts bytes target     prot opt in     out     source               
destination
   0     0 SNAT       all  --  *      *       172.16.0.100         0.0.0.0/0        
 
to:123.123.123.246


172.16.0.100 is a VIP address for a load balanced service on the CS150.  
Essentialy, it monitors specified services for load, keeps track ofg how many 
connection there are, and then when a flow comes in to it's VIP address, it 
DNAT's to the service that is least loaded.

so if i connect from an external source to 123.123.123.246, the packet should 
do this:

- Skips over the net, hits my router.
- router forwards packet to 172.16.0.3 (eth0 on linux firewall/router)
- firewall DNAT's to 172.16.0.100 and sends out of eth1 via 172.16.0.2
- CS150 chooses a service (e.g 10.2.1.7) and DNAT's out to 10.2.1.7 via
  eth0 on router
- linux router then forwards packet to 10.2.1.7 on eth2.

the journey of a packet going back should be:

- 10.2.1.7 replies to the CS150 VIP address (172.16.0.100) via eth2 on
  router
- router forwards back to CS150's VIP via eth1 (packet gets marked by mangle
  and routed as such)
- CS150 maps the flow back to it's original IP source and sends back out to
  172.16.0.3 (eth0 on linux router)
- linux router does the rest of the DNAT magic, and sends back out to client
  via the core router.

ok, so now thats all out the way, and i've lost my brain somewhere, does that 
*sound* like it should work???

~ Theo

hint: it should, and it's not :p

- -- 

Theo Zourzouvillys
http://zozo.org.uk/

One of the most striking differences between a cat and a lie is that a cat has
only nine lives.
               -- Mark Twain, "Pudd'nhead Wilson's Calendar"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8957C448CrwpTn6YRAk1/AKDv145ZRgKYOCK43hk95MfEkOxRjwCgriQH
rBeVsAno9aIt2zrkWC3mhWw=
=77NL
-----END PGP SIGNATURE-----


--
The Mailing List for the Devon & Cornwall LUG
Mail majordomo@xxxxxxxxxxxx with "unsubscribe list" in the
message body to unsubscribe.