Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Right way to initialize python embedded in a multi-threaded application

Reply
Thread Tools

Right way to initialize python embedded in a multi-threaded application

 
 
shooshx
Guest
Posts: n/a
 
      05-17-2012
I'm embedding python in a multi-threaded C application.
I've taken care to wrap every call to the Python C API with

gstate = PyGILState_Ensure();
// call python code
PyGILState_Release(gstate);

But I'm stumped with what to do in the initialization.
Right after the call to Py_IsInitialized() I've added a call:

PyEval_InitThreads();

The docs say that this function leaves the GIL locked when it returns.
I do some more initializations like importing modules and then I call

PyEval_ReleaseLock();

This seems to cause a problem since not long after a call to
PyGILState_Release(gstate) that's made in a different thread crashes.
with

"Fatal Python error: This thread state must be current when releasing"

If I don't do the call to PyEval_ReleaseLock() in the main thread
right after initialization, the GIL seems to be released
after the first PyGILState_Ensure() - PyGILState_Release() pair.

So what am I doing wrong here?
What is the correct way of initializing a multi-threaded application?


 
Reply With Quote
 
 
 
 
francis.brosnan@gmail.com
Guest
Posts: n/a
 
      05-11-2013
Hi,

Maybe you already fixed the issue, but for the record, I've got the same
problem and finally it turned out that I was calling PyEval_InitThreads twice
and also after fixing that, I also had to move the call to PyEval_ReleaseLock(); at the end of the entire initialization (not just after PyEval_initThreads).

The key thing there is to follow:

<<at initialization thread>>
Py_Initialize ();
PyEval_InitThreads();
/* now call here to initialize all python code by loading external files
or internal module loading (i.e. Py_InitModule3) */
/* ..and now, once no more Python C/API call is required, release
the GIL so other threads can come into play */
PyEval_ReleaseLock ();

<<and now, from other threads, use>>
/* wait til gil acquired */
state = PyGILState_Ensure();
/* your code */
/* release GIL */
PyGILState_Release (state);

Hope it helps, Cheers!


> I'm embedding python in a multi-threaded C application.
>
> I've taken care to wrap every call to the Python C API with
>
>
>
> gstate = PyGILState_Ensure();
>
> // call python code
>
> PyGILState_Release(gstate);
>
>
>
> But I'm stumped with what to do in the initialization.
>
> Right after the call to Py_IsInitialized() I've added a call:
>
>
>
> PyEval_InitThreads();
>
>
>
> The docs say that this function leaves the GIL locked when it returns.
>
> I do some more initializations like importing modules and then I call
>
>
>
> PyEval_ReleaseLock();
>
>
>
> This seems to cause a problem since not long after a call to
>
> PyGILState_Release(gstate) that's made in a different thread crashes.
>
> with
>
>
>
> "Fatal Python error: This thread state must be current when releasing"
>
>
>
> If I don't do the call to PyEval_ReleaseLock() in the main thread
>
> right after initialization, the GIL seems to be released
>
> after the first PyGILState_Ensure() - PyGILState_Release() pair.
>
>
>
> So what am I doing wrong here?
>
> What is the correct way of initializing a multi-threaded application?

 
Reply With Quote
 
 
 
 
francis.brosnan@gmail.com
Guest
Posts: n/a
 
      05-11-2013
Just clarify there's no problem about calling twice to PyEval_InitThreads ()
as indicated by Python's doc.

> Hi,
>
>
>
> Maybe you already fixed the issue, but for the record, I've got the same
>
> problem and finally it turned out that I was calling PyEval_InitThreads twice
>
> and also after fixing that, I also had to move the call to PyEval_ReleaseLock(); at the end of the entire initialization (not just after PyEval_initThreads).
>
>
>
> The key thing there is to follow:
>
>
>
> <<at initialization thread>>
>
> Py_Initialize ();
>
> PyEval_InitThreads();
>
> /* now call here to initialize all python code by loading external files
>
> or internal module loading (i.e. Py_InitModule3) */
>
> /* ..and now, once no more Python C/API call is required, release
>
> the GIL so other threads can come into play */
>
> PyEval_ReleaseLock ();
>
>
>
> <<and now, from other threads, use>>
>
> /* wait til gil acquired */
>
> state = PyGILState_Ensure();
>
> /* your code */
>
> /* release GIL */
>
> PyGILState_Release (state);
>
>
>
> Hope it helps, Cheers!
>
>
>
>
>
> > I'm embedding python in a multi-threaded C application.

>
> >

>
> > I've taken care to wrap every call to the Python C API with

>
> >

>
> >

>
> >

>
> > gstate = PyGILState_Ensure();

>
> >

>
> > // call python code

>
> >

>
> > PyGILState_Release(gstate);

>
> >

>
> >

>
> >

>
> > But I'm stumped with what to do in the initialization.

>
> >

>
> > Right after the call to Py_IsInitialized() I've added a call:

>
> >

>
> >

>
> >

>
> > PyEval_InitThreads();

>
> >

>
> >

>
> >

>
> > The docs say that this function leaves the GIL locked when it returns.

>
> >

>
> > I do some more initializations like importing modules and then I call

>
> >

>
> >

>
> >

>
> > PyEval_ReleaseLock();

>
> >

>
> >

>
> >

>
> > This seems to cause a problem since not long after a call to

>
> >

>
> > PyGILState_Release(gstate) that's made in a different thread crashes.

>
> >

>
> > with

>
> >

>
> >

>
> >

>
> > "Fatal Python error: This thread state must be current when releasing"

>
> >

>
> >

>
> >

>
> > If I don't do the call to PyEval_ReleaseLock() in the main thread

>
> >

>
> > right after initialization, the GIL seems to be released

>
> >

>
> > after the first PyGILState_Ensure() - PyGILState_Release() pair.

>
> >

>
> >

>
> >

>
> > So what am I doing wrong here?

>
> >

>
> > What is the correct way of initializing a multi-threaded application?

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
"The application failed to initialize properly (0xc000007b). Click Ok to terminate the application." error massage in vista 64 Richard Windows 64bit 2 10-09-2009 12:04 AM
How to define multiple initialize methods in embedded mode? Chirag Mistry Ruby 1 03-10-2008 11:31 AM
Python embedded interpreter: how to initialize the interpreter ? ycollet@freesurf.fr Python 3 01-03-2007 01:00 AM
if instance variable get initialize after assigning some values or after constructor then when does static variable get initialize Tony Morris Java 3 02-04-2006 08:39 AM
Embedded Python application don't work if Python is installed Miki Tebeka Python 2 07-07-2004 07:04 AM



Advertisments