linux as multi-port (subnet) router using Ubuntu 11.10 server

Discussion in 'Linux Networking' started by lincy, Apr 8, 2012.

  1. lincy

    lincy Guest

    Hi everyone.

    I have an fast-growing network. Around 120-130 pcs in single class c localnet (192.168.1.0/24). Because service lot large file share (multi-nas for multi-media).

    Now I am try to split it by department. So, I am using an 4 ports giga lan machine and install ubuntu 11.10 server. Because I just want to split the network but not to do any security in this machine. So, I just setup 4 subnet and enable the ip forward.

    eth0: 192.168.1.1 netmask 255.255.255.0
    eth1: 192.168.2.1 netmask 255.255.255.0
    eth2: 192.168.3.1 netmask 255.255.255.0
    eth3: 192.168.4.1 netmask 255.255.255.0

    And echo 1 > /proc/sys/net/ipv4/ip_forward in /etc/rc.local

    Now, My problem was.

    I can't ping cross subnet, ex: ping 192.168.2.x from 192.168.1.x
    But I can open web page cross subnet. ex: open 192.168.1.x intranet web server from 192.168.2.x or 192.168.3.x
    I also can't access the windows/share-folder either samba share cross subnet.
    But same subnet can access. ex: 192.168.1.x's pc can access NAS within 192.168.1.x but not NAS in 192.168.2.x.

    Does anyone know what's wrong ?
     
    lincy, Apr 8, 2012
    #1
    1. Advertisements

  2. lincy

    Tauno Voipio Guest


    How is the default gateway in the client machines set?

    My guess is:

    a) The clients in subnets do not know that the route to another
    subnet goes via the Linux machine,

    b) There is a HTTP proxy in the Linux machine forwarding the
    Web requests.

    Another question: Is the way out from the subnets to the big
    Internet via the Linux machine, or something else?

    If this is a basic network tree, you should have in each machine
    the default gateway pointing to the next step toward the Net.
    You have now changed the next step from what it was.
     
    Tauno Voipio, Apr 8, 2012
    #2
    1. Advertisements

  3. lincy

    lincy Guest

    Tauno Voipioæ–¼ 2012å¹´4月8日星期日UTC+8下åˆ10時16分11秒寫é“:
    I have using dnsmasq too, so each subnet's client can get correctly ip and default gw to this machine. ex: 192.168.1.x



    Tauno Voipioæ–¼ 2012å¹´4月8日星期日UTC+8下åˆ10時16分11秒寫é“:


    Tauno Voipioæ–¼ 2012å¹´4月8日星期日UTC+8下åˆ10時16分11秒寫é“:


    Tauno Voipioæ–¼ 2012å¹´4月8日星期日UTC+8下åˆ10時16分11秒寫é“:
    Thanks your reply.
    First, I have 192.168.1.x subnet. so. Have one firewall in front of Internet. It's 192.168.1.254, So, every machine in 192.168.1.x was default gw point to 192.168.1.254

    At the new 4 ports machine. I set the default gw to 192.168.1.254
    Then add static route, 192.168.2.x/192.168.3.x/192.168.4.x point to 192.168..1.1
    This machine no any rule. Just forward enable.
    as above.

    I think, If I don't set any filter rule. This machine should pass any package. But look like it don't. I don't know why http port can work. But not other ports.
     
    lincy, Apr 8, 2012
    #3
  4. lincy

    lincy Guest

    Thanks your reply.
    Sorry, the static route to 192.168.1.1 was setting in firewall machine.
     
    lincy, Apr 8, 2012
    #4
  5. Hello,

    lincy a écrit :
    So subnet 192.168.1.x is special : packets from a host in this subnet to
    another subnet has to go through the firewall first and then
    (hopefully) be forwarded to the Linux router.

    Let's save it for later. What about communications between subnets other
    that 192.168.1.x ?

    Note : packet capture, e.g. with tcpdump, on the Linux router may help.
     
    Pascal Hambourg, Apr 8, 2012
    #5
  6. lincy

    lincy Guest

    Pascal Hambourgæ–¼ 2012å¹´4月9日星期一UTC+8上åˆ12時13分17秒寫é“:
    192.168.2.x/192.168.3.x/192.168.4.x can't ping each other.
    ex: 192.168.2.x can't ping 192.168.3.x
    but 192.168.2.a can ping 192.168.2.b, 192.168.3.a can ping 192.168.3.b

    For web access. 192.168.2.x can access 192.168.3.x web (ex: NAS's web-UI)
    For SMB/Windows access. 192.168.2.x can't access 192.168.3.x share (ex:NAS's windows share)
    But SMB access with same subnet was ok.
     
    lincy, Apr 8, 2012
    #6
  7. lincy a écrit :
    Did you check that /proc/sys/net/ipv4/ip_forward is actually set to 1 ?
    What is the output of "route -n" or "ip route" on the Linux router and
    on a host in each subnet ?
    What is the exact result (output) of the ping commands ?
    What is the output of iptables-save on the Linux router ?
     
    Pascal Hambourg, Apr 8, 2012
    #7
  8. lincy

    lincy Guest

    Pascal Hambourgæ–¼ 2012å¹´4月9日星期一UTC+8上åˆ12時31分47秒寫é“:
    1. yes, It's 1

    2. 4 ports router.
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 192.168.1.254 0.0.0.0 UG 100 0 0 eth0
    192.168.2.0 * 255.255.255.0 U 0 0 0 eth1
    192.168.3.0 * 255.255.255.0 U 0 0 0 eth2
    192.168.4.0 * 255.255.255.0 U 0 0 0 eth3
    192.168.1.0 * 255.255.255.0 U 0 0 0 eth0

    client machine. (windows)
    IP address : 192.168.2.181
    netmask : 255.255.255.0
    gateway : 192.168.2.1
    DHCP server : 192.168.2.1
    DNS Server : 192.168.2.1
    WINS Server : 192.168.1.1
    NetBIOS over tcpip : yes

    3.
    ping 192.168.3.51
    Timeout...
    Timeout...
    .....



    4.
    # Generated by iptables-save v1.4.10 on Mon Apr 9 00:41:02 2012
    *filter
    :INPUT ACCEPT [686:74793]
    :FORWARD ACCEPT [129:25283]
    :OUTPUT ACCEPT [607:68740]
    COMMIT
    # Completed on Mon Apr 9 00:41:02 2012
    # Generated by iptables-save v1.4.10 on Mon Apr 9 00:41:02 2012
    *nat
    :pREROUTING ACCEPT [55:5548]
    :INPUT ACCEPT [41:4300]
    :OUTPUT ACCEPT [73:7833]
    :pOSTROUTING ACCEPT [86:8975]
    COMMIT
    # Completed on Mon Apr 9 00:41:02 2012
    # Generated by iptables-save v1.4.10 on Mon Apr 9 00:41:02 2012
    *mangle
    :pREROUTING ACCEPT [816:100182]
    :INPUT ACCEPT [686:74793]
    :FORWARD ACCEPT [129:25283]
    :OUTPUT ACCEPT [607:68740]
    :pOSTROUTING ACCEPT [1002:127527]
    COMMIT
    # Completed on Mon Apr 9 00:41:02 2012
     
    lincy, Apr 8, 2012
    #8
  9. lincy a écrit :
    What about the host on the other subnet, 192.168.3.51 ?
    I would run tcpdump on the router to see what is going on...
     
    Pascal Hambourg, Apr 8, 2012
    #9
  10. lincy

    Tauno Voipio Guest

    Point 2.

    There is a DHCP server mentioned. Do you run DHCP in the network?

    If you do, the proper location for the server is the Linux machine.
    In other cases, you'll need a DHCP relay in the Linux machine.

    The setup means that machines in 192.168.2.x network must have
    192.168.2.1 as the default gateway, and similarly for networks
    192.168.3.x and 192.168.4.x. The gateway must always be in the
    same local network as the client.

    For clients in the 192.168.1.x network it is different: the
    default gateway must be 192.168.1.254.

    IIRC, the WINS name server should be also in the local network.

    Point 3.

    Weird - if there is not a 'security' module in Windows blocking
    the show. Can you test with a Linux laptop in the subnet?

    As Pascal said, tcpdump or - still better - Wireshark is your
    friend here.

    Point 4.

    A better listing is available from iptables:

    iptables -n -v -L
     
    Tauno Voipio, Apr 8, 2012
    #10
  11. Tauno Voipio a écrit :
    This is better if the hosts have specific routes to the other subnets.
    Otherwise, the default gateway could be the Linux router. It depends
    wheter most of the traffic is going to the internet or to the other subnets.
    No, a WINS server, like a DNS server can be anywhere. You don't need on
    in each subnet.
    I disagree. The output of iptables-save is more compact and complete (by
    default it includes the contents of all the active tables) and the
    output format is closer to the iptables syntax used to create the rules.
     
    Pascal Hambourg, Apr 8, 2012
    #11
  12. lincy

    lincy Guest

    Pascal Hambourgæ–¼ 2012å¹´4月9日星期一UTC+8上åˆ1時54分39秒寫é“:
    IP address : 192.168.3.51
    netmask : 255.255.255.0
    gateway : 192.168.3.1
    I have using tcpdump to listen eth1 (192.168.2.0) and eth2 (192.168.3.0),
    I have saw the 192.168.3.51 have echo icmp request. But this package only reach 192.168.3.1, not going to 192.168.2.1, so, 192.168.2.181 report time out.
     
    lincy, Apr 9, 2012
    #12
  13. lincy

    lincy Guest

    Point 2.
    Yes, I have using dnsmasq in this 4 ports linux box.

    here is the conf.

    except-interface=eth0 # don't dhcp for 192.168.1.0
    dhcp-range=interface:eth1,192.168.2.101,192.168.2.200,255.255.255.0 #the default gw will as eth1 -> 192.168.2.1
    dhcp-range=interface:eth2,192.168.3.101,192.168.3.200,255.255.255.0 # the default gw will as eth2 -> 192.168.3.1
    dhcp-range=interface:eth3,192.168.4.101,192.168.4.200,255.255.255.0 # the default gw will as eth3 -> 192.168.4.1
    dhcp-option=44,192.168.1.1 # Wins point to self (Samba just for WINS Server)

    192.168.1.0 was dhcp by 192.168.1.254 (firewall). All machine in 192.168.1.0 was default gw to 192.168.1.254
    I will try to setup an linux box as client to try this late.
    Chain INPUT (policy ACCEPT 27166 packets, 1905K bytes)
    pkts bytes target prot opt in out source destination

    Chain FORWARD (policy ACCEPT 17895 packets, 10M bytes)
    pkts bytes target prot opt in out source destination

    Chain OUTPUT (policy ACCEPT 29452 packets, 5517K bytes)
    pkts bytes target prot opt in out source destination
     
    lincy, Apr 9, 2012
    #13
  14. lincy a écrit :
    Can you be more precise, or better, provide the output of tcpdump on
    both interfaces ?
     
    Pascal Hambourg, Apr 9, 2012
    #14
  15. lincy

    lincy Guest

    Pascal Hambourgæ–¼ 2012å¹´4月9日星期一UTC+8下åˆ4時45分12秒寫é“:
    Sorry, I was wrong. I was issue some iptable rule. To try and error. Then saw this result. But after clear all rule. It's little difference

    Here is the tcpdump with no any iptable rule.

    dump for eth1 (192.168.2.0)
    21:00:36.593857 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56228, length 40
    21:00:36.681668 IP 192.168.2.181.138 > 192.168.2.255.138: UDP, length 213
    21:00:41.594273 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56229, length 40
    21:00:46.592657 ARP, Request who-has 192.168.2.1 (00:60:e0:4e:f9:e1) tell 192.168.2.181, length 46
    21:00:46.592676 ARP, Reply 192.168.2.1 is-at 00:60:e0:4e:f9:e1, length 28
    21:00:46.593667 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56230, length 40
    21:00:47.417978 IP 192.168.2.181.60120 > 192.168.1.11.161: UDP, length 78
    21:00:51.593160 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56231, length 40
    21:00:56.593645 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56232, length 40
    21:00:58.391788 IP 192.168.2.181.60120 > 192.168.1.11.161: UDP, length 78
    21:01:01.593176 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56233, length 40
    21:01:06.593578 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56234, length 40
    21:01:08.392724 IP 192.168.2.181.60120 > 192.168.1.11.161: UDP, length 78
    21:01:11.594091 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56235, length 40
    21:01:13.093096 ARP, Request who-has 192.168.2.1 (00:60:e0:4e:f9:e1) tell 192.168.2.181, length 46
    21:01:13.093112 ARP, Reply 192.168.2.1 is-at 00:60:e0:4e:f9:e1, length 28
    21:01:16.593682 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56236, length 40
    21:01:19.315536 ARP, Request who-has 192.168.2.1 tell 192.168.2.181, length46
    21:01:19.315552 ARP, Reply 192.168.2.1 is-at 00:60:e0:4e:f9:e1, length 28
    21:01:19.596954 IP 192.168.2.181.50952 > 239.255.255.250.1900: UDP, length 133
    21:01:21.594124 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56237, length 40
    21:01:22.597326 IP 192.168.2.181.50952 > 239.255.255.250.1900: UDP, length 133



    dump for eth2:
    21:00:36.457244 ARP, Request who-has 192.168.3.51 tell 192.168.51.1, length28
    21:00:36.457510 ARP, Reply 192.168.3.51 is-at 70:5a:b6:40:02:d5, length 46
    21:00:36.593887 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56228, length 40
    21:00:41.594300 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56229, length 40
    21:00:46.593692 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56230, length 40
    21:00:51.593187 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56231, length 40
    21:00:56.593677 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56232, length 40
    21:01:01.593200 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56233, length 40
    21:01:06.593605 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56234, length 40
    21:01:11.594113 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56235, length 40
    21:01:16.593717 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56236, length 40
    21:01:21.594152 IP 192.168.2.181 > 192.168.3.51: ICMP echo request, id 3, seq 56237, length 40


    Ok, here is what I try...

    iptables -t nat -A POSTROUTING -s 192.168.2.0/24 -d 192.168.3.0/24 -j SNAT --to 192.168.3.1
    iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -d 192.168.2.0/24 -j SNAT --to 192.168.2.1

    now can ping from 192.168.2.x to 192.168.3.x, vice versa.
    The strange thing is 192.168.3.x to 192.168.2.x SMB function ok, but 192.168.2.x to 192.168.3.x was not.....
    Still trying ....
     
    lincy, Apr 9, 2012
    #15
  16. lincy a écrit :
    It cannot possibly be an issue with some iptables rule, as iptables-save
    showed no rule at all.
    Huh ? "tell 192.168.51.1" ? What is the address configured on eth2 ?
    [...]
    So the echo request is forwarded, but no reply ever comes back.
    I can see only two explanations :
    - host 192.168.3.51 default route is misconfigured or missing
    - or it has a firewall which drops requests from other subnets.
    Ah, SNAT aka masquerading. It is called "masquerading" because it
    masquerades the source address, but I like to say that it is actually
    used to masquerade the real - routing of filtering - problem.
     
    Pascal Hambourg, Apr 9, 2012
    #16
  17. lincy

    lincy Guest

    Pascal Hambourgæ–¼ 2012å¹´4月10日星期二UTC+8上åˆ3時01分22秒寫é“:
    I meaning I am try&error to type iptable rule from command line to test it.I don't save the rule or edit the script file.

    I am reboot machine and capture fresh package with no any rule.

    It's typing error. sorry. should be 192.168.3.1
    The default route of 192.168.3.51 was 192.168.3.1
    I am guess the kernel drop something too. But I don't know why. Because theiptable rule are empty.
    yes, I just try to see masq work or not. But the service won't work. Just can ping. Weird....

    This 4ports linux box was hard to setup more then internet firewall box. >_<

    Firewall rule to reject all then accept what you want was easy understand.

    This machine no any rule, all chain was accept. I can't understand where todrop/reject/block package. so this box don't work as I expected.
     
    lincy, Apr 9, 2012
    #17
  18. lincy a écrit :
    You should not need any iptables rules if everything else is setup
    correctly.
    Looks good.
    It cannot be iptables nor the IP stack. Incoming packets are captured by
    tcpdump before iptables and the IP stack can see them. The only part of
    the kernel which could drop the packets before tcpdump can see them is
    the ethernet driver.
    There is nothing difficult to setup : just IP settings on each interface
    and enable IP forwarding.
    You do not need any filtering rules. Just leave everything open.
    My guess is the problem lies in the firewall or IP setup on the hosts,
    not on the router box.
     
    Pascal Hambourg, Apr 9, 2012
    #18
  19. lincy

    Tauno Voipio Guest

    There is a strong smell of Windows 'firewalling' off networks
    that are not own. Is there such a 'security feature' in Windows
    that it will respond to pings from local network only?

    Which version/flavour or Windows?
     
    Tauno Voipio, Apr 10, 2012
    #19
  20. lincy

    lincy Guest

    Tauno Voipioæ–¼ 2012å¹´4月10日星期二UTC+8下åˆ1時55分06秒寫é“:
    Windows 7
     
    lincy, Apr 10, 2012
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.