2

I am having a dumb, and I am hoping you lovely people can jumpstart my brain.

I am running a service in a local network namespace. The namespace looks like this:

# ip -n ns1 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ns1-int@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6e:35:ab:46:78:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.255.11/24 scope global ns1-int
       valid_lft forever preferred_lft forever
    inet6 fe80::6c35:abff:fe46:7871/64 scope link
       valid_lft forever preferred_lft forever

The namespace routing table looks like:

$ ip -n ns1 route
default via 192.168.255.1 dev ns1-int
192.168.255.0/24 dev ns1-int proto kernel scope link src 192.168.255.11

Where interface ns1-int is one end of a veth pair which is connected to br0 on the host. 192.168.255.1 is the address of br0:

# ip addr show br0
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ce:10:da:46:71:fb brd ff:ff:ff:ff:ff:ff
    inet 192.168.255.1/24 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::cc10:daff:fe46:71fb/64 scope link
       valid_lft forever preferred_lft forever
# ip addr show master br0
5: ns1-ext@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether d6:69:bd:06:9c:fa brd ff:ff:ff:ff:ff:ff link-netns ns1
    inet6 fe80::d469:bdff:fe06:9cfa/64 scope link
       valid_lft forever preferred_lft forever

I'm running a simple web service on port 80; I can access it from the host using the namespace address:

# curl 192.168.255.11
Hostname: node1
IP: 127.0.0.1
IP: ::1
IP: 192.168.255.11
IP: fe80::6c35:abff:fe46:7871
RemoteAddr: 192.168.255.1:58396
GET / HTTP/1.1
Host: 192.168.255.11
User-Agent: curl/8.0.1
Accept: */*

I would like to expose this service on host port 8080, both for remotely originating traffic and for locally originating traffic.

Remote traffic is no problem:

iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.255.11:80

This works just fine; from another host on the network I can now access the same web endpoint on port 8080:

anotherhost$ curl 192.168.121.67:8080
Hostname: node1
IP: 127.0.0.1
IP: ::1
IP: 192.168.255.11
IP: fe80::6c35:abff:fe46:7871
RemoteAddr: 192.168.121.1:47888
GET / HTTP/1.1
Host: 192.168.121.67:8080
User-Agent: curl/8.0.1
Accept: */*

I would also like to be able to access the service locally on port 8080 using e.g. curl localhost:8080. In theory, to redirect locally generated traffic we just need a rule in the OUTPUT chain:

iptables -t nat -A OUTPUT -p tcp --dport 8080 -j DNAT --to-destination 192.168.255.11:80

At this point the complete netfile configuration looks like:

# iptables-save
# Generated by iptables-save v1.8.9 on Sun Jun 11 00:47:49 2023
*filter
:INPUT ACCEPT [751:53502]
:FORWARD ACCEPT [10:937]
:OUTPUT ACCEPT [433:43385]
COMMIT
# Completed on Sun Jun 11 00:47:49 2023
# Generated by iptables-save v1.8.9 on Sun Jun 11 00:47:49 2023
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [21:1580]
:POSTROUTING ACCEPT [22:1640]
-A PREROUTING -p tcp -m tcp --dport 8080 -j DNAT --to-destination 192.168.255.11:80
-A OUTPUT -p tcp -m tcp --dport 8080 -j DNAT --to-destination 192.168.255.11:80
COMMIT
# Completed on Sun Jun 11 00:47:49 2023

Attempts to access localhost:8080 simply get stuck:

$ curl --connect-timeout 60 localhost:8080
curl: (28) Failed to connect to localhost port 8080 after 60001 ms: Timeout was reached

The OUTPUT rule is definitely matching; I can see the hit counter incrementing as a I run that curl command. I feel like I'm missing something simple. What could it be?

1 Answer 1

2

For that DNAT rule in the OUTPUT chain to work correctly, it looks like we need two things.

  1. We need to set the appropriate route_localnet sysctl:

    sysctl -w net.ipv4.conf.br0.route_localnet
    

    With just that change, we'll see the following traffic showing up inside the namespace:

    03:23:02.984603 ns1-int In  IP 127.0.0.1.39656 > 192.168.255.11.80: Flags [S], seq 409674911, win 65495, options [mss 65495,sackOK,TS val 2257175047 ecr 0,nop,wscale 7], length 0
    

    That's problematic because the source address (127.0.0.1) is useless, so...

  2. We need a MASQUERADE rule for traffic from localhost to the namespace:

    iptables -t nat -A POSTROUTING -s 127.0.0.1/32 -d 192.168.255.0/24 -j MASQUERADE
    

With these two rules in place, things work as expected:

# curl localhost:8080
Hostname: node1
IP: 127.0.0.1
IP: ::1
IP: 192.168.255.11
IP: fe80::8054:55ff:fef7:3e76
RemoteAddr: 192.168.255.1:36676
GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/8.0.1
Accept: */*

And inside the namespace, we see that the incoming packets now look like:

03:26:38.660882 ns1-int In  IP 192.168.255.1.38500 > 192.168.255.11.80: Flags [S], seq 1141073629, win 65495, options [mss 65495,sackOK,TS val 2257390724 ecr 0,nop,wscale 7], length 0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.