«

I've been working on this for more hours than I'm willing to admit,
perhaps someone here can help me make it happen.


This us using Python 2.3.3
- I do have access to a SunOS 5.8 machine, and the script at the end
of this email works.
- I need it to work on linux kernel 2.4.x.


I'm trying to write the equivalent of what the author calls "ringd"
described in the below article, and use it with python 2.3.x on linux 2.4:
http://www.remote.org/jochen/work/pub/zero-downtime.pdf


The script that I provide at the end of this post is a variation of one
posted in this thread:
http://groups.google.com/groups?hl=e...mers.se&rnum=8

There is a C version listed later in that article, but I've not yet
tried it out.

Certainly I need a two things:
1. Unix domain socket, local socket (standard socket connected locally),
or pipe
2. sendmsg/recvmsg, fcntl.ioctl, or equivalent file descriptor manipulation

In the script listed at the end of this post, I use a file descriptor
pair returned by os.pipe(), which should be sufficient. I also use
fcntl.ioctl().


As stated previously, this works properly on SunOS 5.8:
jcarlson@synergistic-envision% python2.3 fdpass.py
Parent ioctl() returned 0
#!/usr/pd/bin/python
jcarlson@synergistic-envision%

It does not work on the linux machine I'm testing it on:
[jcarlson@dev jcarlson]$ python fdpass.py
[Errno 22] Invalid argument
Traceback (most recent call last):
File "fdpass.py", line 58, in ?
ret = fcntl.ioctl(pRead, fcntl.I_RECVFD, s)
IOError: [Errno 22] Invalid argument
[jcarlson@dev jcarlson]$

Seemingly this is because I_SENDFD/I_RECVFD is not properly implemented
on linux 2.4, but maybe I'm doing something wrong.


I've also tried using SCM_RIGHTS as per this thread:
http://groups.google.com/groups?hl=e...ing.google.com

It is not defined in python's fcntl module, but I did find the C
definition in the linux /usr/include/bits/socket.h...
SCM_RIGHTS = 0x01, /* Transfer file descriptors. */

So I passed the integer 1 manually, on both linux and SunOS 5.8 and got
exceptions like I normally do on linux.

There is another C-based option that wraps sendmsg and recvmsg in the
twistedmatrix sandbox:
http://cvs.twistedmatrix.com/cvs/tru...0&root=Twisted

Does anyone have an idea of how to get it working on linux? I would
prefer to not have to break into C, if only because I don't want to
accidentally leak memory (once bitten, twice shy they say). Certainly
Pyrex and SWIG are options, but first I'd like to try a pure Python version.


Thanks
- Josiah

I forgot to post the script...


- Josiah

-----clip here-----

#!/usr/pd/bin/python
#
# fdpass.py

#
# Example of passing an open filedescriptor with Python. Will only work
# on UNIX dialects that support the I_SENDFD and I_RECVFD ioctl() calls.
#

import fcntl, os, sys, struct, socket

#
# fork() off!
#
(pRead, pWrite) = os.pipe()

pid = os.fork()


if pid != 0:

# We're in the parent.

# Open a file for passing along to child. Use own source code,
# which is guaranteed to exist.

fileObj = open('./fdpass.py', 'r')

# ioctl() will only pass raw filedescriptors. Find fd of fileObj.
fd = fileObj.fileno()

# Send to the child using ioctl().
try:

retval = fcntl.ioctl(pWrite, 1, fd)

# Should probably check retval rather than just printing it.
print "Parent ioctl() returned %d" % retval
except Exception, e:
print e

# Wait for child to terminate, then exit.
os.waitpid(pid, 0)
sys.exit(0)

else:
import time
time.sleep(1)
# We're in the child.

# Create a string representing the strrecvfd structure that ioctl()
# will return.
s = struct.pack('iii', 0, 0, 0)

# Receive filedescriptor. Will block until descriptor is sent.
ret = fcntl.ioctl(pRead, 1, s)

# Unpack the strrecvfd-structure that ioctl() should return.
# fd is the filedescriptor, uid/gid the user/group id of the
# sending stream.
(fd, uid, gid) = struct.unpack('iii', ret)

# Reopen the filedescriptor as a Python File-object.
fileObj = os.fdopen(fd, 'r')

# Example usage: Read file, print the first line.
lines = fileObj.readlines()
print lines[0],
sys.exit(0)

Josiah Carlson wrote:
> I've been working on this for more hours than I'm willing to admit,
> perhaps someone here can help me make it happen.

When I first saw the subject, I was about to point out that you had
the wrong newsgroup: you want comp.lang.perl.misc

I see you were talking about something else though.


> This us using Python 2.3.3
> - I do have access to a SunOS 5.8 machine, and the script at the end
> of this email works.
> - I need it to work on linux kernel 2.4.x.
>
>
> I'm trying to write the equivalent of what the author calls "ringd"
> described in the below article, and use it with python 2.3.x on linux 2.4:
> http://www.remote.org/jochen/work/pub/zero-downtime.pdf
>
>
> The script that I provide at the end of this post is a variation of one
> posted in this thread:
> http://groups.google.com/groups?hl=e...mers.se&rnum=8
>
> There is a C version listed later in that article, but I've not yet
> tried it out.
>
> Certainly I need a two things:
> 1. Unix domain socket, local socket (standard socket connected locally),
> or pipe
> 2. sendmsg/recvmsg, fcntl.ioctl, or equivalent file descriptor manipulation
>
> In the script listed at the end of this post, I use a file descriptor
> pair returned by os.pipe(), which should be sufficient. I also use
> fcntl.ioctl().
>
>
> As stated previously, this works properly on SunOS 5.8:
> jcarlson@synergistic-envision% python2.3 fdpass.py
> Parent ioctl() returned 0
> #!/usr/pd/bin/python
> jcarlson@synergistic-envision%
>
> It does not work on the linux machine I'm testing it on:
> [jcarlson@dev jcarlson]$ python fdpass.py
> [Errno 22] Invalid argument
> Traceback (most recent call last):
> File "fdpass.py", line 58, in ?
> ret = fcntl.ioctl(pRead, fcntl.I_RECVFD, s)
> IOError: [Errno 22] Invalid argument
> [jcarlson@dev jcarlson]$
>
> Seemingly this is because I_SENDFD/I_RECVFD is not properly implemented
> on linux 2.4, but maybe I'm doing something wrong.
>
> I've also tried using SCM_RIGHTS as per this thread:
> http://groups.google.com/groups?hl=e...ing.google.com
>
> It is not defined in python's fcntl module, but I did find the C
> definition in the linux /usr/include/bits/socket.h...
> SCM_RIGHTS = 0x01, /* Transfer file descriptors. */
>
> So I passed the integer 1 manually, on both linux and SunOS 5.8 and got
> exceptions like I normally do on linux.

Last I checked, you have to use a Unix-domain socket to do this in
Linux.


--
CARL BANKS http://www.aerojockey.com/software
"If you believe in yourself, drink your school, stay on drugs, and
don't do milk, you can get work."
-- Parody of Mr. T from a Robert Smigel Cartoon

> When I first saw the subject, I was about to point out that you had
> the wrong newsgroup: you want comp.lang.perl.misc
>
> I see you were talking about something else though.

No perl here.

> Last I checked, you have to use a Unix-domain socket to do this in
> Linux.

No dice...

[jcarlson@dev jcarlson]$ python fdpass.py
Traceback (most recent call last):
File "fdpass.py", line 63, in ?
ret = fcntl.ioctl(pRead, 1, s)
IOError: [Errno 22] Invalid argument
[Errno 14] Bad address
[jcarlson@dev jcarlson]$

I've tested the data transfer and it works fine.

Using the below modified code, unix domain sockets do not work. Any
other ideas?

- Josiah


-----cut here-----

#!/usr/pd/bin/python
#
# fdpass.py

#
# Example of passing an open filedescriptor with Python. Will only work
# on UNIX dialects that support the I_SENDFD and I_RECVFD ioctl() calls.
#

import fcntl, os, sys, struct, socket

#
# fork() off!
#

pid = os.fork()

port = '10001'

if pid != 0:
# We're in the parent.

s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.bind(port)
s.listen(1)
pWrite, addr = s.accept()

# Open a file for passing along to child. Use own source code,
# which is guaranteed to exist.

fileObj = open('./fdpass.py', 'r')

# ioctl() will only pass raw filedescriptors. Find fd of fileObj.
fd = fileObj.fileno()

# Send to the child using ioctl().
try:

retval = fcntl.ioctl(pWrite, fcntl.I_SENDFD, fd)

# Should probably check retval rather than just printing it.
print "Parent ioctl() returned %d" % retval
except Exception, e:
print e

# Wait for child to terminate, then exit.
os.waitpid(pid, 0)
sys.exit(0)

else:
import time
time.sleep(1)
# We're in the child.

pRead = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
pRead.connect(port)

# Create a string representing the strrecvfd structure that ioctl()
# will return.
s = struct.pack('iii', 0, 0, 0)

# Receive filedescriptor. Will block until descriptor is sent.
ret = fcntl.ioctl(pRead, fcntl.I_RECVFD, s)

# Unpack the strrecvfd-structure that ioctl() should return.
# fd is the filedescriptor, uid/gid the user/group id of the
# sending stream.
(fd, uid, gid) = struct.unpack('iii', ret)

# Reopen the filedescriptor as a Python File-object.
fileObj = os.fdopen(fd, 'r')

# Example usage: Read file, print the first line.
lines = fileObj.readlines()
print lines[0],
sys.exit(0)

> I've tested the data transfer and it works fine.
>
> Using the below modified code, unix domain sockets do not work. Any
> other ideas?

Oh, and on SunOS 5.8, using unix domain sockets gets the following error:
jcarlson@synergistic-envision% python2.3 fdpass.py
[Errno 122] Operation not supported on transport endpoint


I needed to interrupt it because the read blocks in the child process.

- Josiah

In article <ca8rlq$c8$>,
Josiah Carlson <> wrote:

[... evidently wishing to pass a file descriptor over a
local socket connection ...]

> Certainly I need a two things:
> 1. Unix domain socket, local socket (standard socket connected locally),
> or pipe
> 2. sendmsg/recvmsg, fcntl.ioctl, or equivalent file descriptor manipulation
>
> In the script listed at the end of this post, I use a file descriptor
> pair returned by os.pipe(), which should be sufficient. I also use
> fcntl.ioctl().
>
>
> As stated previously, this works properly on SunOS 5.8:
....
> It does not work on the linux machine I'm testing it on:
....
> IOError: [Errno 22] Invalid argument

> Seemingly this is because I_SENDFD/I_RECVFD is not properly implemented
> on linux 2.4, but maybe I'm doing something wrong.

I'd say it's a fair bet that I_SENDFD is not implemented on Linux,
properly or otherwise. It looks to me like an AT&T STREAMS function,
as opposed to Berkeley socket. Casual look around the include files
doesn't suggest any support on Linux for any STREAMS stuff.

As usual, there's a parallel Berkeley way to do this, using as
already mentioned a UNIX domain socket, and sendmsg, and SCM_RIGHTS.

If Python's socketmodule.c doesn't directly support sendmsg and
the attendant data structures, you'll have to add that in C, either
in socketmodule.c or your own module. That means mainly getting
the msghdr struct together (I think the control object you want
to pass, with SCM_RIGHTS and the fds can be packed up in Python.)

Donn Cave,

At some point, Josiah Carlson <> wrote:

> I'm trying to write the equivalent of what the author calls "ringd"
> described in the below article, and use it with python 2.3.x on linux
> 2.4:
> http://www.remote.org/jochen/work/pub/zero-downtime.pdf
>
> The script that I provide at the end of this post is a variation of
> one posted in this thread:
> http://groups.google.com/groups?hl=e...mers.se&rnum=8
[...]
> Certainly I need a two things:
> 1. Unix domain socket, local socket (standard socket connected
> locally), or pipe
> 2. sendmsg/recvmsg, fcntl.ioctl, or equivalent file descriptor manipulation
>
> In the script listed at the end of this post, I use a file descriptor
> pair returned by os.pipe(), which should be sufficient. I also use
> fcntl.ioctl().
[...]
> Does anyone have an idea of how to get it working on linux? I would
> prefer to not have to break into C, if only because I don't want to
> accidentally leak memory (once bitten, twice shy they say). Certainly
> Pyrex and SWIG are options, but first I'd like to try a pure Python
> version.

Have a look at passfd.c in Neil Schemenauer's SCGI protocol implementation:
http://www.mems-exchange.org/software/scgi/
It wraps sendmsg/recvmsg to send and receive file descriptors.

It's a C module, but's it's very lightweight. I think it does what you
want to do (the test_passfd.py is almost exactly like the script you
posted; showing their common ancestors...) It's supposed to work under
Linux, FreeBSD and Solaris.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca

>>Seemingly this is because I_SENDFD/I_RECVFD is not properly implemented
>>on linux 2.4, but maybe I'm doing something wrong.
>
>
> I'd say it's a fair bet that I_SENDFD is not implemented on Linux,
> properly or otherwise. It looks to me like an AT&T STREAMS function,
> as opposed to Berkeley socket. Casual look around the include files
> doesn't suggest any support on Linux for any STREAMS stuff.
>
> As usual, there's a parallel Berkeley way to do this, using as
> already mentioned a UNIX domain socket, and sendmsg, and SCM_RIGHTS.
>
> If Python's socketmodule.c doesn't directly support sendmsg and
> the attendant data structures, you'll have to add that in C, either
> in socketmodule.c or your own module. That means mainly getting
> the msghdr struct together (I think the control object you want
> to pass, with SCM_RIGHTS and the fds can be packed up in Python.)

Since I'm going to have to write a wrapper anyways, I may as well spend
some time learning Pyrex and gain python garbage collection.

Thank you for the information,
- Josiah