«

On Wed, Apr 20, 2011 at 10:09 AM, Grant Edwards <> wrote:
> The management program can then send another broadcast packet to
> configure the IP address of a device. After that, the management
> program switches over to normal unicast TCP and UDP protocols (HTTP,
> TFTP, etc.) to set up the device.
>

Wonder if it would be possible to (ab)use DHCP for this. If every
device registers itself with a central DHCP server, you could query
that to find out what's around, and configuring of IP addresses would
then be out of your hands.

Or can you simply use a stupid netmask like /1 that picks up all the
IP ranges? That way, the source-IP check wouldn't fail.

Chris Angelico

On 2011-04-20, Chris Angelico <> wrote:
> On Wed, Apr 20, 2011 at 10:09 AM, Grant Edwards <> wrote:
>> The management program can then send another broadcast packet to
>> configure the IP address of a device. After that, the management
>> program switches over to normal unicast TCP and UDP protocols (HTTP,
>> TFTP, etc.) to set up the device.
>>
>
> Wonder if it would be possible to (ab)use DHCP for this. If every
> device registers itself with a central DHCP server, you could query
> that to find out what's around, and configuring of IP addresses would
> then be out of your hands.

I can't require the presense of a DHCP server, and many installations
won't have one.

> Or can you simply use a stupid netmask like /1 that picks up all the
> IP ranges? That way, the source-IP check wouldn't fail.

That would require that the device somehow knows that it's not
configured correctly and should change the netmask to /1. The device
doesn't have any way to know that, and it must respond to the
discovery commands both before and after it's properly configured.

I've reread the protocol documentation and noticed that the device has
to respond not only to broadcasts to 255.255.255.255 but also to
subnet broadcasts send to subnets it's not on. That pretty much
clinches the requirement to use a raw socket. :/

--
Grant

On Wed, Apr 20, 2011 at 11:15 AM, Grant Edwards <> wrote:
>> Or can you simply use a stupid netmask like /1 that picks up all the
>> IP ranges? That way, the source-IP check wouldn't fail.
>
> That would require that the device somehow knows that it's not
> configured correctly and should change the netmask to /1. *The device
> doesn't have any way to know that, and it must respond to the
> discovery commands both before and after it's properly configured.

Was hoping that you could make such a change _only_ on the computer
that's receiving the data - that way it's only one change, the devices
don't need any tweaking. But if it can't be, it can't be.

> I've reread the protocol documentation and noticed that the device has
> to respond not only to broadcasts to 255.255.255.255 but also to
> subnet broadcasts send to subnets it's not on. *That pretty much
> clinches the requirement to use a raw socket. :/

Sounds to me like someone majorly abused IP to do weird things. Looks
like you're stuck doing the same weirdness, in whatever way you can
manage Sorry.

Chris Angelico

On 2011-04-20, Chris Angelico <> wrote:
> On Wed, Apr 20, 2011 at 11:15 AM, Grant Edwards <> wrote:
>>> Or can you simply use a stupid netmask like /1 that picks up all the
>>> IP ranges? That way, the source-IP check wouldn't fail.
>>
>> That would require that the device somehow knows that it's not
>> configured correctly and should change the netmask to /1. ?The device
>> doesn't have any way to know that, and it must respond to the
>> discovery commands both before and after it's properly configured.
>
> Was hoping that you could make such a change _only_ on the computer
> that's receiving the data - that way it's only one change, the devices
> don't need any tweaking. But if it can't be, it can't be.

There can by any number of devices that have to receive the broadcast
packet, and any of them can be on different subnets (or have no IP
address at all).

>> I've reread the protocol documentation and noticed that the device
>> has to respond not only to broadcasts to 255.255.255.255 but also to
>> subnet broadcasts send to subnets it's not on.

It turns out that some OSes (BSD and some flavors of Windows) can't
broadcast to 255.255.255.255, only to the subnet broadcast address.
Hence the requirement for the devices to be able to receive a subnet
broadcast regardless of their IP address.

>> That pretty much clinches the requirement to use a raw socket. :/
>
> Sounds to me like someone majorly abused IP to do weird things.

Indeed. The other option is to do something that's not based on IP
but done completely at the Ethernet layer. Implementing management
programs that can do that can be very nasty on Windows, which
unfortunately matters to most customers.

So you bite the bullet on the device end and implement all sorts of
weirdness in order to allow the management program to use standard
UDP.

> Looks like you're stuck doing the same weirdness, in whatever way you
> can manage

Yup. It doesn't even appear that it can be done with a raw UDP
socket. According to all of the documentation I can find, such a
socket is supposed to receive copies of all UDP packets seen by the
kernel, but it doesn't. Even if I use a raw UDP socket, the kernel is
still dropping broadcast packets whose source address don't match any
interface's subnet.

AFAICT, I'm going to have to go completely raw and process in
user-space every single IP packet recieved. That's going to be
brutal on the CPU...

--
Grant

On Tue, Apr 19, 2011 at 6:15 PM, Grant Edwards <> wrote:

>> Or can you simply use a stupid netmask like /1 that picks up all the
>> IP ranges? That way, the source-IP check wouldn't fail.
>
> That would require that the device somehow knows that it's not
> configured correctly and should change the netmask to /1. *The device
> doesn't have any way to know that, and it must respond to the
> discovery commands both before and after it's properly configured.

- Actually, you Might be able to configure your device to have a
netmask of 0.0.0.0, IP address of 255.255.255.255 and broadcast of
255.255.255.255.
- I've seen something a bit similar used for detecting IP address
conflicts automatically.
- A network guru I used to work with told me that you could configure
a machine with a broadcast of 255.255.255.255 more simply than messing
around with the netmask, while still achieving the same result for
general purpose networking.

> I've reread the protocol documentation and noticed that the device has
> to respond not only to broadcasts to 255.255.255.255 but also to
> subnet broadcasts send to subnets it's not on. *That pretty much
> clinches the requirement to use a raw socket. :/

With a netmask of 0.0.0.0, I suspect you will receive all broadcasts
on the wire, given appropriate listening code.

You could probably also modify a copy of tshark or tcpdump to flush
after every line of output (or run an unmodified one on a pty to avoid
maintaining your own copy of the binary), and parse that output in
Python. That should make the Python code pretty simple.

There's an old program called "pty" that makes it easy to run
something on a pty, to get unbuffered output - it's in
comp.sources.unix volumes 22, 23 and 25; it's written in C. You'd
just open a subprocess with no buffering on the python side, that runs
tcpdump or tshark under pty. Beware though - the pty program predates
GNU autoconf, so might be a little involved to compile.

I agree though that you're kind of pushing IP in a direction it wasn't
intended to go.

On Tue, Apr 19, 2011 at 8:12 PM, Dan Stromberg <> wrote:
> I agree though that you're kind of pushing IP in a direction it wasn't
> intended to go.

It just occurred to me: You might get some additional mileage out of
popping the network adapter into promiscuous mode. In fact, it Might
be necessary irrespective of the rest of your approach.

In article <iol875$ah2$>,
Grant Edwards <> wrote:

> I'm trying to implement a device discovery/configuration protocol that
> uses UDP broadcast packets to discover specific types of devices on
> the local Ethernet segment. The management program broadcasts a
> discovery command to a particular UDP port. All devices who get that
> packet are expected to answer regardless of thier current IP address.

Have you considered what will happen if you have, say, 1000 such
devices, and they all respond at the same time?

Am 20.04.2011 01:54, schrieb Grant Edwards:
> I guess the problem is that I expected to receive a packet on an
> interface anytime a packet was received with a destination IP address
> that matched that of the the interface. Apprently there's some
> filtering in the network stack based on the _source_ address as well
> (that seems very counter-intuitive to me).

Just to pitch in here (because nobody's mentioned it yet AFAICT): yes,
there's a filtering done (at least under Linux, and I'd guess something
similar on xBSD too) to packets based on the source address coming in on
an interface, and it's called the reverse path filter and is on by
default (the tunable on Linux is /proc/sys/net/ipv4/conf/*/rp_filter).

The idea behind the reverse path filter is that your machine won't
accept packets coming in over an interface when a return packet (i.e.,
the presumed response) won't be routed over the same interface, and from
what I gather, this is what makes the TCP/IP stack drop the packets
because your machine will not route packets to 192.168.x.x over the same
interface it sees the packet coming in. This is a _security_ feature,
because it makes address spoofing harder.

If you need to see the packets regardless, either use a promiscuous mode
sniffer (i.e., tcpdump, but that's relatively easy to mirror in Python
using SOCK_RAW, capturing packets at the ethernet level), or add a route
on your system for the 192.168.x.x network on the same interface.

HTH!

--
--- Heiko.

Am 20.04.2011 00:21, schrieb Grant Edwards:
> I'm have problems figuring out how to receive UDP broadcast packets on
> Linux.
[...]
>
> On the receiving machine, I've used tcpdump to verify that broadcast
> packets are being seen and have a destination IP of 255.255.255.255 and
> destination MAC of ff:ff:ff:ff:ff:ff
[...]
> But, the receiving Python program never sees any packets unless the
> _source_ IP address in the packets is on the same subnet as the
> receiving machine. In this test case, the receiving machine has an IP
> address of 172.16.12.34/16. If I change the receiving machine's IP
> address to 10.0.0.123, then the receiving program sees the packets.
>
> Even though the destination address is 255.255.255.255, the receiving
> machine appears to discard the packets based on the _source_ IP. Can
> anybody provide example Python code for Linux that receives UDP
> broadcast packets regardless of their source IP address?
>
> This probably is more of a Linux networking question than a Python
> question, but I'm hoping somebody has solved this problem in Python.
>

You must set the network interface to promiscous mode on the receiving side:

os.system("ifconfig eth0 promisc")

Thomas

On Wed, 2011-04-20 at 06:07 -0400, Sherm Pendley wrote:
> Grant Edwards <> writes:
> > I'm trying to implement a device discovery/configuration protocol that
> > uses UDP broadcast packets to discover specific types of devices on
> > the local Ethernet segment. The management program broadcasts a
> > discovery command to a particular UDP port. All devices who get that
> > packet are expected to answer regardless of thier current IP address.
> Have you looked at the source for Apple's Bonjour?

On LINUX this is called "avahi", which has Python bindings. Avahi
auto-configuration / discovery works very well.

<http://avahi.org/>
<http://freshmeat.net/projects/avahi>
<http://pypi.python.org/pypi/pybonjour/1.1.1>
<http://avahi.org/wiki/PythonBrowseExample>

See also:
<http://pypi.python.org/pypi/pybonjour/1.1.1>
at least you may be able to lift code from them (License is non-viral
MIT)

> Might be interesting to see how it does announcement/discovery. Or maybe
> just use it directly, if this happens to be a case of "gee, I didn't
> know that wheel had already been invented."

--
Adam Tauno Williams <> LPIC-1, Novell CLA
<http://www.whitemiceconsulting.com>
OpenGroupware, Cyrus IMAPd, Postfix, OpenLDAP, Samba