Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > threads, XSUB allocated memory, destructors, destruction

Reply
Thread Tools

threads, XSUB allocated memory, destructors, destruction

 
 
Andrew Torda
Guest
Posts: n/a
 
      09-30-2005
I have a perl module built out of XSUBs.
The functions malloc() space, build structures and return
pointers to perl. Perl calls their destructor routines with no
problems. The C structures go back to the perl interpreter, as
T_PTROBJ things, specified in a typemap file.

Now, if I use threads, I
make lots of new data
break my arrays into 2 pieces
threads->new( myfunc, @array_to_read_from);
threads->new( myfunc, @array_to_read_from);
thread[0]->join
thread[1]->join

Unfortunately, at the join() stage, each thread decides to
cleanup, and call the destructors for my @array_to_read_from.
Obviously free() gets called multiple times, terrible things
happen to the heap and everything dies.

What should I be doing in my XSUBs when I create objects (really
just pointers to malloc()'d space) to ask perl not to do this ?
I am sure I am missing something obvious, but I cannot find it in
perlxs or perlxstut man pages.
Many thanks for any advice.

 
Reply With Quote
 
 
 
 
Sisyphus
Guest
Posts: n/a
 
      10-02-2005

"Andrew Torda" <(E-Mail Removed)-hamburg.de> wrote in message
news:dhjp8d$a4l$(E-Mail Removed)-hamburg.de...
> I have a perl module built out of XSUBs.
> The functions malloc() space, build structures and return
> pointers to perl. Perl calls their destructor routines with no
> problems. The C structures go back to the perl interpreter, as
> T_PTROBJ things, specified in a typemap file.
>
> Now, if I use threads, I
> make lots of new data
> break my arrays into 2 pieces
> threads->new( myfunc, @array_to_read_from);
> threads->new( myfunc, @array_to_read_from);
> thread[0]->join
> thread[1]->join
>
> Unfortunately, at the join() stage, each thread decides to
> cleanup, and call the destructors for my @array_to_read_from.
> Obviously free() gets called multiple times, terrible things
> happen to the heap and everything dies.
>


Can't quite reproduce the exact problem. I gather that '@array_to_read_from'
is an array of these "pointer" objects. Below my sig is an Inline::C script
that (I think) does pretty much as described (and not much else). It doesn't
use a typemap, but I don't think there's any relevance in that.

I find that there's no problem at the join() stage. The cleanup of the
pointer objects does not take place (afaict) until the script is about to
exit (which is as it should be). At that time, however, a problem usually
(but not always) does arise - after the majority of the pointer objects have
been freed, I get (on Win32) the "Free to wrong pool...during global
destruction" error. (That's a problem - but it may be a Win32-specific
issue, so I won't go delving into that just yet. Which OS are you on ? I was
going to give the script a run on Linux .... but my perl on Linux wasn't
built with threads support.)

Anyway ... the output I get looks like this:

Thread started
Thread started
Thread started
200 400 650
All joined
destroying int object
int object destroyed
destroying int object
int object destroyed
..
..
..
destroying int object
int object destroyed
destroying int object
Free to wrong pool b39078 not c5d040 during global destruction.

If you already have (or can be bothered installing) Inline::C then you might
run the script and check that the problem really does occur when you think
it does. Otoh, I might have been way off-beam with my interpretation of what
you've said - and/or my script may be irrelevant to the problem you're
facing - in which case feel free to modify it to better demonstrate the
issue at hand.

(Btw, with that script, I've established that the cleanup problem *is*
associated with the using of threads. If I remove the threads stuff , then
the cleanup proceeds smoothly every time. I also tried replacing malloc/free
with New/Safefree, but it made no difference - which is not surprising.)

Hth - but don't stress too much if it doesn't

Cheers,
Rob

use warnings;
use threads;

package Experimental;

use Inline C => Config =>
BUILD_NOISY => 1;

use Inline C => <<'EOC';

SV * create_int_obj(SV * x) {
int * int_obj, i, s;
SV * obj_ref, * obj;

s = (int)SvUV(x);

/* Allocate space for s ints */
/* New(1, int_obj, s, int); */
int_obj = malloc(sizeof(int) * s);
if(int_obj == NULL) croak("Failed to allocate memory in create_int_obj
function");
obj_ref = newSViv(0);
obj = newSVrv(obj_ref, "Experimental");

sv_setiv(obj, (IV)int_obj);
SvREADONLY_on(obj);
return obj_ref;
}

void DESTROY(SV * m) {
printf("destroying int object\n");
/* Safefree((int *) SvIV(SvRV(m))); */
free((int *) SvIV(SvRV(m)));
printf("int object destroyed\n");
}


EOC

# Create an array of 200 pointer objects
@array_to_read_from_1 = create_em(200);

# Create an array of 400 pointer objects.
@array_to_read_from_2 = create_em(400);

# Create an array of 650 pointers.
@array_to_read_from_3 = create_em(650);

$thread1 = threads->new("start_thread", @array_to_read_from_1);
$thread2 = threads->new("start_thread", @array_to_read_from_2);
$thread3 = threads->new("start_thread", @array_to_read_from_3);

$s1 = $thread1->join;
$s2 = $thread2->join;
$s3 = $thread3->join;

print "$s1 $s2 $s3\n";
print "All joined\n";

sleep(2);

# Then usually crashes at some time during the destruction of
# the pointer objects with the error "Free to wrong pool...during global
destruction"

sub start_thread {
print "Thread started\n";
return scalar(@_);
}

sub create_em {
# Return an array of pointer objects.
my @ret = ();
for(1..$_[0]) { push(@ret, create_int_obj(10 + int(rand(10))))}
return @ret;
}


 
Reply With Quote
 
 
 
 
Andrew Torda
Guest
Posts: n/a
 
      10-02-2005
"Sisyphus" <(E-Mail Removed)> writes:

> "Andrew Torda" <(E-Mail Removed)-hamburg.de> wrote in message

[.... original code deleted ...]
> > Unfortunately, at the join() stage, each thread decides to
> > cleanup, and call the destructors for my @array_to_read_from.
> > Obviously free() gets called multiple times, terrible things
> > happen to the heap and everything dies.
> >

>
> Can't quite reproduce the exact problem. I gather that '@array_to_read_from'


But your example does reproduce the problem. The exact point of
cleanup is not specified (different under windows and linux).

> is an array of these "pointer" objects. Below my sig is an Inline::C script
> that (I think) does pretty much as described (and not much else). It doesn't
> use a typemap, but I don't think there's any relevance in that.


In order to see the problem with threads cleaning up malloc'd
memory, I think you should take your nice example, and put a loop
like
for (my $i = 0; $i < 10; $i++) {
around your code below. Under linux, at least, I think you will
find you will not reach the second iteration of the loop.
> $thread1 = threads->new("start_thread", @array_to_read_from_1);
> $thread2 = threads->new("start_thread", @array_to_read_from_2);
> $thread3 = threads->new("start_thread", @array_to_read_from_3);
>
> $s1 = $thread1->join;
> $s2 = $thread2->join;
> $s3 = $thread3->join;
>
> print "$s1 $s2 $s3\n";
> print "All joined\n";


Thanks for the example.
 
Reply With Quote
 
Sisyphus
Guest
Posts: n/a
 
      10-03-2005

"Andrew Torda" <(E-Mail Removed)> wrote in message
..
..
> In order to see the problem with threads cleaning up malloc'd
> memory, I think you should take your nice example, and put a loop
> like
> for (my $i = 0; $i < 10; $i++) {
> around your code below. Under linux, at least, I think you will
> find you will not reach the second iteration of the loop.
> > $thread1 = threads->new("start_thread", @array_to_read_from_1);
> > $thread2 = threads->new("start_thread", @array_to_read_from_2);
> > $thread3 = threads->new("start_thread", @array_to_read_from_3);
> >
> > $s1 = $thread1->join;
> > $s2 = $thread2->join;
> > $s3 = $thread3->join;
> >
> > print "$s1 $s2 $s3\n";
> > print "All joined\n";

>


So I did :

for($i = 0; $i < 10; $i++){

$thread1 = threads->new("start_thread", @array_to_read_from_1);
$thread2 = threads->new("start_thread", @array_to_read_from_2);
$thread3 = threads->new("start_thread", @array_to_read_from_3);

$s1 = $thread1->join;
$s2 = $thread2->join;
$s3 = $thread3->join;

print "$s1 $s2 $s3\n";
print "All joined\n";

sleep(2);
}

And there was no problem on Win32 with that .... until the global
destruction phase after the 10 loops had run (just prior to exit). ie, I
got:

Thread started
Thread started
Thread started
200 400 650
All joined
Thread started
Thread started
Thread started
200 400 650
All joined
..
..
Thread started
Thread started
Thread started
200 400 650
All joined
Thread started
Thread started
Thread started
200 400 650
All joined
destroying int object
int object destroyed
destroying int object
int object destroyed
destroying int object
int object destroyed
..
..
destroying int object
int object destroyed
destroying int object
Free to wrong pool 336c878 not abababab during global destruction.

Looks like the error is different on linux, then. Now wishing I'd built my
perl on linux with threads support ... just so I can see for myself.

Afaik, if you built your pointer objects as I built mine, then you've done
it right - and what you're up against is simply a bug. I guess that would
call for a bug report (see 'perldoc perlbug').

If you haven't already, you could check that everything functions ok when
threads are removed from the scene (just to prove that it's threads that are
stuffing things up, and not something else).

You could also try replacing malloc/free with New/Safefree (though I'm not
expecting that to fix anything) ..... plus any other straws you care to
clutch at

Cheers,
Rob


 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      10-03-2005
"Sisyphus" <(E-Mail Removed)> wrote:
>
> Anyway ... the output I get looks like this:
>
> Thread started
> Thread started
> Thread started
> 200 400 650
> All joined
> destroying int object
> int object destroyed
> destroying int object
> int object destroyed
> .
> .
> .
> destroying int object
> int object destroyed
> destroying int object
> Free to wrong pool b39078 not c5d040 during global destruction.


On linux, I just get a segfault.

....
> void DESTROY(SV * m) {
> printf("destroying int object\n");


printf("destroying int object %d\n", SvIV(SvRV(m)));

> /* Safefree((int *) SvIV(SvRV(m))); */
> free((int *) SvIV(SvRV(m)));
> printf("int object destroyed\n");
> }


For reasons I don't understand, it is trying to call DESTROY twice for each
object. It goes through calling DESTROY once for each object, then starts
running through again, seg faulting at some random point on the second
pass.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
Sisyphus
Guest
Posts: n/a
 
      10-04-2005

<(E-Mail Removed)> wrote in message

>
> On linux, I just get a segfault.
>


On Win32 I get a segfault, too - produced, I presume, by freeing "to wrong
pool".

...

>
> For reasons I don't understand, it is trying to call DESTROY twice for

each
> object. It goes through calling DESTROY once for each object, then starts
> running through again, seg faulting at some random point on the second
> pass.
>


On Win32, it looks like it's only being called once for each object - and it
seems to be the freeing of the very last object that needs to be freed that
produces the error (now that I look more closely).

On further investigation, I found that if the pointer objects are not
blessed into a package, then there seems to be no problem at all - see the
(modified) script below my sig. Does it also work ok on linux ? If so then
perhaps the op can get around his problem by using unblessed objects - which
means that it's then his responsibility to call _destroy() explicitly at the
appropriate time(s).

Cheers,
Rob

use warnings;
use threads;

#package Experimental;

use Inline C => Config =>
BUILD_NOISY => 1;

use Inline C => <<'EOC';

SV * create_int_obj(SV * x) {
int * int_obj, i, s;
SV * obj_ref, * obj;

s = (int)SvUV(x);

/* Allocate space for s ints */
/* New(1, int_obj, s, int); */
int_obj = malloc(sizeof(int) * s);
if(int_obj == NULL) croak("Failed to allocate memory in create_int_obj
function");
obj_ref = newSViv(0);
obj = newSVrv(obj_ref, NULL);

sv_setiv(obj, (IV)int_obj);
SvREADONLY_on(obj);
return obj_ref;
}

void _destroy(SV * m) {
printf("destroying int object\n");
/* Safefree((int *) SvIV(SvRV(m))); */
free((int *) SvIV(SvRV(m)));
printf("int object destroyed\n");
}


EOC

# Create an array of 200 pointer objects
@array_to_read_from_1 = create_em(200);

# Create an array of 400 pointer objects.
@array_to_read_from_2 = create_em(400);

# Create an array of 650 pointers.
@array_to_read_from_3 = create_em(650);

$thread1 = threads->new("start_thread", @array_to_read_from_1);
$thread2 = threads->new("start_thread", @array_to_read_from_2);
$thread3 = threads->new("start_thread", @array_to_read_from_3);

$s1 = $thread1->join;
$s2 = $thread2->join;
$s3 = $thread3->join;

print "$s1 $s2 $s3\n";
print "All joined\n";

sleep(2);

# Let's now re-assign to @array_to_read_from_1.
# First we free up the existing pointer objects:
for(@array_to_read_from_1) {_destroy($_)}

# Then assign some new pointer objects:
@array_to_read_from_1 = create_em(310);


$thread4 = threads->new("start_thread", @array_to_read_from_1);
$s4 = $thread4->join;

print "$s4\nStill ok\n";
sleep(2);

for(@array_to_read_from_1) {_destroy($_)}
for(@array_to_read_from_2) {_destroy($_)}
for(@array_to_read_from_3) {_destroy($_)}

sub start_thread {
print "Thread started\n";
return scalar(@_);
}

sub create_em {
# Return an array of pointer objects.
my @ret = ();
for(1..$_[0]) { push(@ret, create_int_obj(10 + int(rand(10))))}
return @ret;
}



 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      10-04-2005
"Sisyphus" <(E-Mail Removed)> wrote:
> <(E-Mail Removed)> wrote in message
>
> >
> > On linux, I just get a segfault.
> >

>
> On Win32 I get a segfault, too - produced, I presume, by freeing "to
> wrong pool".
>
> ...
>
> >
> > For reasons I don't understand, it is trying to call DESTROY twice for

> each
> > object. It goes through calling DESTROY once for each object, then
> > starts running through again, seg faulting at some random point on the
> > second pass.
> >

>
> On Win32, it looks like it's only being called once for each object - and
> it seems to be the freeing of the very last object that needs to be freed
> that produces the error (now that I look more closely).


Could it be that Win32 is also trying to destroy everything twice, and it
is the destroying of the very first object, but on the second pass, which
produces the error? (doing so before the message on that one can be
printed)


> On further investigation, I found that if the pointer objects are not
> blessed into a package, then there seems to be no problem at all - see
> the (modified) script below my sig. Does it also work ok on linux ?


Yes, it seems to work on Linux.

> If so
> then perhaps the op can get around his problem by using unblessed objects
> - which means that it's then his responsibility to call _destroy()
> explicitly at the appropriate time(s).


I tried moving the DESTROY from XSUB to Perl, and have the Perl DESTROY
call _destroy. But now DESTROY is getting called twice again, and we are
back to seg-faulting.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
Sisyphus
Guest
Posts: n/a
 
      10-04-2005

<(E-Mail Removed)> wrote in message
>
> Could it be that Win32 is also trying to destroy everything twice, and it
> is the destroying of the very first object, but on the second pass, which
> produces the error? (doing so before the message on that one can be
> printed)
>


Perhaps something along those lines is occurring. I've now found that
whenever the script runs without error, the number of times that DESTROY()
gets called is exactly double the number of times it ought to get called. So
the doubling up is definitely happening.

It's just that whenever it segfaults, the printout on the screen indicates
that it's happening on the last object of the array. ie - if there's only
one array involved (simplest scenario), containing x pointer objects, the
segfault apparently occurs the xth time that DESTROY() is called. If it
doesn't segfault at that point, then DESTROY() will be successfully called
2x times.

> I tried moving the DESTROY from XSUB to Perl, and have the Perl DESTROY
> call _destroy. But now DESTROY is getting called twice again, and we are
> back to seg-faulting.
>


Heh - I'd tried that, too. Didn't help *me*, either

Cheers,
Rob


 
Reply With Quote
 
Tassilo v. Parseval
Guest
Posts: n/a
 
      10-05-2005
Also sprach Sisyphus:

><(E-Mail Removed)> wrote in message
>>
>> Could it be that Win32 is also trying to destroy everything twice, and it
>> is the destroying of the very first object, but on the second pass, which
>> produces the error? (doing so before the message on that one can be
>> printed)
>>

>
> Perhaps something along those lines is occurring. I've now found that
> whenever the script runs without error, the number of times that DESTROY()
> gets called is exactly double the number of times it ought to get called. So
> the doubling up is definitely happening.
>
> It's just that whenever it segfaults, the printout on the screen indicates
> that it's happening on the last object of the array. ie - if there's only
> one array involved (simplest scenario), containing x pointer objects, the
> segfault apparently occurs the xth time that DESTROY() is called. If it
> doesn't segfault at that point, then DESTROY() will be successfully called
> 2x times.
>
>> I tried moving the DESTROY from XSUB to Perl, and have the Perl DESTROY
>> call _destroy. But now DESTROY is getting called twice again, and we are
>> back to seg-faulting.
>>

>
> Heh - I'd tried that, too. Didn't help *me*, either


Admiring all your attempts to understand and fix the problem, the
explanation for that behaviour is a different one.

When creating a new thread, data aren't shared by default. Instead, each
thread gets a clone. Perl can easily create clones of pure-Perl types
however it's out of its wits when a Perl type actually refers to an
outside resource (such as those allocated integers you were dealing
with).

So what happens is that the pure-Perl part is still cloned, but not the
underlying integer objects. Each of these cloned objects refer to the
same memory allocations. So for each clone DESTROY is called, trying to
deallocate the integer-objects. No wonder this results in double-,
triple-, etc.-frees, depending on how many threads you have.

So in essence: the XS portion of your code isn't thread-safe. In order
to make it thread-safe, you have to take control over the cloning
process. See 'perldoc perlmod' and search for "CLONE". For recent perls
(>= 5.8.7) all you have to do is provide a CLONE_SKIP method for the
package 'Experimental' (in your case). If it returns a true value, no
cloning of these object happens and therefore no multiple frees either.
Older perls only have a CLONE method that is called once for each
thread. It's not clear to me how it can be used to prevent cloning. I
think it can't and that's why CLONE_SKIP was introduced.

Tassilo
--
use bigint;
$n=71423350343770280161397026330337371139054411854 220053437565440;
$m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($ m+=<=200);
 
Reply With Quote
 
Sisyphus
Guest
Posts: n/a
 
      10-07-2005

"Tassilo v. Parseval" <(E-Mail Removed)> wrote in message
..
..
>
> So in essence: the XS portion of your code isn't thread-safe. In order
> to make it thread-safe, you have to take control over the cloning
> process. See 'perldoc perlmod' and search for "CLONE". For recent perls
> (>= 5.8.7) all you have to do is provide a CLONE_SKIP method for the
> package 'Experimental' (in your case).


Yep - CLONE_SKIP seems to be the missing ingredient - works for me, anyway.

I don't understand why the problem arises only with *blessed* objects. Does
cloning not occur wrt unblessed objects ? Or is it just that problems are
avoided simply because perl doesn't (automatically) clean up unblessed
objects ?

Cheers,
Rob


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
XSUB detecting if it was called via Package->method() or Package::method()? Chris Perl Misc 7 11-18-2010 03:33 AM
xsub and gcc 4.0.2, static variables thorsten kracht Perl Misc 1 03-06-2006 09:45 AM
Dynamically Allocated Memory vs. Statically allocated Memory csnerd@gmail.com C++ 5 12-09-2004 01:44 AM
XS/XSUB FAQs? Tutorials? Jeff Perl Misc 2 09-26-2003 11:29 AM
mod_perl errors: prototype mismatch ... during global destruction ian douglas Perl 0 08-18-2003 11:17 PM



Advertisments