Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Embedded Python interpreter and sockets

Reply
Thread Tools

Embedded Python interpreter and sockets

 
 
wahn@acm.org
Guest
Posts: n/a
 
      11-19-2005
Hi,

Here is a problem I came across and need some help with. I developed a
little Python script with some classes which runs standalone and
communicates with a database via sockets. So far everything works fine.
I also successfully embedded the Python interpreter into a plugin for a
commercial 3D package (in this case Houdini - http://www.sidefx.com/ -
using the HDK). I can use the interpreter to run Python commands and
get values back and forth between Python and C++. Anyway, I can also
successfully import SOME modules, but unfortunately not ALL. One
example would be the socket module which works fine in the standalone
version of my Python script but NOT from the embedded interpreter:

Traceback (most recent call last):
File "<string>", line 40, in ?
File "/swdevl/jwalter/Houdini/ROPS/HoudiniToMayaBake/damn.py", line
10, in ?
import socket
File
"/job/SYSTEMS/3PS/python/2.4.1/linux_intel//lib/python2.4/socket.py",
line 45, in ?
import _socket
ImportError:
/job/SYSTEMS/3PS/python/2.4.1/linux_intel/lib/python2.4/lib-dynload/_socket.so:
undefined symbol: PyObject_GenericGetAttr

Other modules which get imported before just work fine (e.g. re and
string) ...

The Python library gets linked into the plugin DSO as a static library
.... Any ideas what's going wrong and why?

Cheers,

Jan

 
Reply With Quote
 
 
 
 
chris.atlee@gmail.com
Guest
Posts: n/a
 
      11-19-2005
Hi Jan,

I believe the problem lies with how Houdini uses dlopen() to open your
plugin. It uses RTLD_LOCAL to load your plugin, which means that all
your plugin's symbols (including the python symbols) are private to
that library. Subsequent dlopen() calls, including those made by the
python library, won't be able to see any of those symbols. This
problem isn't unique to Python, or Houdini, it's just a matter of how
dlopen() is being used.

One thing that may work (I just tried here, and it looks like it
does...but it's late on a Friday night, so I may have overlooked
something) is creating a wrapper plugin. Say you have your plugin
called myplugin.so which links aginst libpython. Create a new plugin
called myplugin_wrapper.so which exports the same required symbols.
myplugin_wrapper will need to dlopen("myplugin.so", RTLD_GLOBAL), and
then call the real methods inside myplugin.so. Since myplugin.so is
the one linked against libpython, and it is being loaded with
RTLD_GLOBAL, then the python symbols should be available to other
shared libraries loaded at a future time.

Let me know if this works out! If you need any help, head on over to
http://www.sidefx.com/forum.

Cheers,
Chris

 
Reply With Quote
 
 
 
 
wahn@acm.org
Guest
Posts: n/a
 
      11-22-2005
Hi Chris,

Thanks for your help. I'll try that ...

Cheers,

Jan

 
Reply With Quote
 
wahn@acm.org
Guest
Posts: n/a
 
      11-22-2005
Hey Chris,

I fixed the problem in another way (don't ask me why that works). One
detail I didn't talk about is that I use the Boost.Python library. So I
just made sure that I load the socket module before I import my own
Python script (using that socket module):

....
object
main_module((handle<>(borrowed(PyImport_AddModule( "__main__")))));
object
socket_module((handle<>(borrowed(PyImport_AddModul e("socket")))));
object main_namespace = main_module.attr("__dict__");
handle<>
result((allow_null(PyRun_String("...",
Py_file_input,
main_namespace.ptr(),
main_namespace.ptr()))));
....

Maybe someone can shed some light on what's going on here but for the
moment I'm happy that it works at all !!!

Cheers,

Jan

 
Reply With Quote
 
wahn@acm.org
Guest
Posts: n/a
 
      11-23-2005
Hi,

actually that didn't solve the problem. As soon as you do something
with the socket module it fails. Well, the solution I came up with is
simply link the ../_socket.so into my Houdini plugin DSO which is ugly
but solves the problem for the moment ...

Happy hacking,

Jan

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Python embedded interpreter: how to initialize the interpreter ? ycollet@freesurf.fr Python 3 01-03-2007 01:00 AM
Scoping bugs in an embedded Python interpreter - Wizards please morris.slutsky@gmail.com Python 3 10-01-2006 09:50 AM
Debugging of Python code in embedded interpreter TK Python 1 11-30-2005 04:05 PM
Embedded Python interpreter, runtime libs, and crashing Peter Newman Python 1 10-26-2005 03:38 AM
embedded python and interpreter threads Charlie DeTar Python 2 12-07-2004 04:52 PM



Advertisments