Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > parallel child processes not working properly

Reply
Thread Tools

parallel child processes not working properly

 
 
vkinger@hotmail.com
Guest
Posts: n/a
 
      06-11-2007
Hi,

I have a problem where I am creating multiple parallel child processes
to connect to multiple servers. Each child process has a different
filehandle to read starttime from a text file at the time of fork and
then write into this text file each time it runs. Each child process
has a forever loop so that it never exits unless killed or reboot
occured at server and then it tries to reconnect. My problem is it
works for couple of times and logs data from different servers into
syslog but on 2nd or 3rd attempt(after being stopped by kill -9
<process id>) it only works with the one server( it can be 1st or 2nd
or 3rd as I am testing with only three servers at present) I am not
sure what is wrong, may be I can use some help here from the experts.
Here is what I am doing.

sub fork_pids
{
my @pids;
my $i = 0;
my $length = scalar(@lines);
my $pm = new Parallel::ForkManager($length);
print " Connecting to " , $length , " servers\n ";
for $i ( 1 .. $length )
{
print "spawning child processes $i\n";
$pm->start and next; #do the fork
print "Inside the child process\n";
&GetEventData($i-1);
$pm->finish;
}
$pm->wait_all_children;
}

Here I am checking the length that how many servers are configured in
the config.txt file and GetEventData is my function that takes index
into the number of server and calls the forever loop function. Same
index is used to create another text file inside this function to
record the timestamp associated with the data. Each server here
essentially does the same thing and logs into the syslog. It works
once or twice and then works with only one server.

 
Reply With Quote
 
 
 
 
xhoster@gmail.com
Guest
Posts: n/a
 
      06-11-2007
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> Hi,
>
> I have a problem where I am creating multiple parallel child processes
> to connect to multiple servers. Each child process has a different
> filehandle to read starttime from a text file at the time of fork and
> then write into this text file each time it runs. Each child process
> has a forever loop so that it never exits unless killed or reboot
> occured at server and then it tries to reconnect.


How does the no longer existent child process try to reconnect?


> My problem is it
> works for couple of times and logs data from different servers into
> syslog but on 2nd or 3rd attempt(after being stopped by kill -9
> <process id>)


What is killed by kill -9, the parent or the child or something else? And
why -9?

> it only works with the one server( it can be 1st or 2nd
> or 3rd as I am testing with only three servers at present)


What do you mean by "works" in this case?

> I am not
> sure what is wrong, may be I can use some help here from the experts.
> Here is what I am doing.
>
> sub fork_pids
> {
> my @pids;


This variable doesn't seem to be used.

> my $i = 0;


premature declaration, no?

> my $length = scalar(@lines);
> my $pm = new Parallel::ForkManager($length);
> print " Connecting to " , $length , " servers\n ";
> for $i ( 1 .. $length )
> {
> print "spawning child processes $i\n";


How many of these get printed?

> $pm->start and next; #do the fork
> print "Inside the child process\n";


How many of these get printed?

> &GetEventData($i-1);
> $pm->finish;
> }
> $pm->wait_all_children;
> }


Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
 
 
 
J. Gleixner
Guest
Posts: n/a
 
      06-11-2007
(E-Mail Removed) wrote:
> Hi,
>
> I have a problem where I am creating multiple parallel child processes
> to connect to multiple servers. [...] My problem is it
> works for couple of times and logs data from different servers into
> syslog but on 2nd or 3rd attempt(after being stopped by kill -9
> <process id>) it only works with the one server( it can be 1st or 2nd
> or 3rd as I am testing with only three servers at present) I am not
> sure what is wrong, may be I can use some help here from the experts.
> Here is what I am doing.
>
> sub fork_pids
> {
> my @pids;

@pids isn't used..

> my $i = 0;

No need for that.
> my $length = scalar(@lines);

No need for 'scalar'.

> my $pm = new Parallel::ForkManager($length);
> print " Connecting to " , $length , " servers\n ";
> for $i ( 1 .. $length )

for my $i ( 1 .. $length )
> {
> print "spawning child processes $i\n";
> $pm->start and next; #do the fork
> print "Inside the child process\n";
> &GetEventData($i-1);


Probably want to call GetEventData( $i - 1 ) there.

> $pm->finish;
> }
> $pm->wait_all_children;
> }


Nothing wrong with that, so it must be in another part of your code.
 
Reply With Quote
 
vkinger@hotmail.com
Guest
Posts: n/a
 
      06-11-2007
On Jun 11, 12:42 pm, (E-Mail Removed) wrote:
> (E-Mail Removed) wrote:
> > Hi,

>
> > I have a problem where I am creating multiple parallel child processes
> > to connect to multiple servers. Each child process has a different
> > filehandle to read starttime from a text file at the time of fork and
> > then write into this text file each time it runs. Each child process
> > has a forever loop so that it never exits unless killed or reboot
> > occured at server and then it tries to reconnect.

>
> How does the no longer existent child process try to reconnect?
>
> > My problem is it
> > works for couple of times and logs data from different servers into
> > syslog but on 2nd or 3rd attempt(after being stopped by kill -9
> > <process id>)

>
> What is killed by kill -9, the parent or the child or something else? And
> why -9?
>From cygwin I do ps -ef and then use kill -9 <process id>, -9 is flag

to force it to die, the result is same even if I press ctrl-c.
> > it only works with the one server( it can be 1st or 2nd
> > or 3rd as I am testing with only three servers at present)

>
> What do you mean by "works" in this case?

it works once or twice, that means I can see the events coming from
the both servers but most of the time it show events coming from only
one server and the other server just shuts up:

>
> > I am not
> > sure what is wrong, may be I can use some help here from the experts.
> > Here is what I am doing.

>
> > sub fork_pids
> > {
> > my @pids;

>
> This variable doesn't seem to be used.
>
> > my $i = 0;

>
> premature declaration, no?
>
> > my $length = scalar(@lines);
> > my $pm = new Parallel::ForkManager($length);
> > print " Connecting to " , $length , " servers\n ";
> > for $i ( 1 .. $length )
> > {
> > print "spawning child processes $i\n";

>
> How many of these get printed?

I am working one server and opening two connections with two child
processes and third server configured in the text file has a wrong
address, to handle a condition where the server address is wrong, get
a open failed condition and exit that child process. here is what I
get:
Connecting to 3 servers
spawning child processes 1
spawning child processes 2
Inside the child process
<Here it calls GetEventData(0) function and stops just before entering
the forever loop>
Inside the child process
spawning child processes 3
<Here it calls the GetEventData(1) function and stops just before
entering the forever loop>
Inside the child process
<Here it calls the GetEventData(2) function and stops just before
entering the forever loop, it also tell me here that it can not
connect to third server as the IP address is wrong, for testing I gave
it 1.2.3.4>
>
> > $pm->start and next; #do the fork
> > print "Inside the child process\n";

>
> How many of these get printed?


>
> > &GetEventData($i-1);
> > $pm->finish;
> > }
> > $pm->wait_all_children;
> > }

>

Here is the layout of my program:

sub GetEventData
{
my $i = shift; #index into the config.text file that was passed
from the fork below
$s = Net::SDEE->new(
returnXML=> 1,
debug => 1,
callback => \&xmlCallback , #it is huge function I created to log
events into syslog and creat text file to take timestamp of the event
came from server so next time server starts from that time
debug_callback => \&local_debug
);
$s->Server($items[0]);
$s->Port($items[1]);
$s->Username($items[2]);
$s->Password($items[3]);
my $subs = Net::SDEE::Subscription->new();
$subs->maxNbrOfEvents(30);
$subs->startTime($lasttimearray[$i]);
$s->open($subs);
for(;
{
print "Getting events $$ and line $i\n";
$s->get($varsubid); #varsubid is a global variable
}
}
sub fork_pids
{
my $length = @lines;
my $pm = new Parallel::ForkManager($length);
print " Connecting to " , $length , " servers\n ";
for $i ( 1 .. $length )
{
print "spawning child processes $i\n";
$pm->start and next; #do the fork
print "Inside the child process\n";
GetEventData($i-1);
$pm->finish;
}
$pm->wait_all_children;



}

> Xho
>
> --
> --------------------http://NewsReader.Com/--------------------
> Usenet Newsgroup Service $9.95/Month 30GB



 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      06-11-2007
(E-Mail Removed) wrote:
> On Jun 11, 12:42 pm, (E-Mail Removed) wrote:
> > (E-Mail Removed) wrote:
> >
> > > My problem is it
> > > works for couple of times and logs data from different servers into
> > > syslog but on 2nd or 3rd attempt(after being stopped by kill -9
> > > <process id>)

> >
> > What is killed by kill -9, the parent or the child or something else?
> > And why -9?


> From cygwin I do ps -ef and then use kill -9 <process id>, -9 is flag
> to force it to die, the result is same even if I press ctrl-c.


So once the program has been running once you kill it with ctrl-c, and
then it starts acting up the next time you run it. What does it take to
"clear" this problem so that you can run it again? A machine reboot? Or
if you just leave it for a few minutes, will it start working? Also,
aren't the children still running? At this point, the parent wasn't doing
anything but waiting on the children anyway, so the death of the parent
should have no effect--the children should still be running the same as
before. But that is from my experience on Linux, since you are using
cygwin then that very well may not apply.


> > > it only works with the one server( it can be 1st or 2nd
> > > or 3rd as I am testing with only three servers at present)

> >
> > What do you mean by "works" in this case?


> it works once or twice, that means I can see the events coming from
> the both servers but most of the time it show events coming from only
> one server and the other server just shuts up:


What do you see in the way of internal debugging information?

> > > I am not
> > > sure what is wrong, may be I can use some help here from the experts.
> > > Here is what I am doing.

> >
> > > sub fork_pids
> > > {
> > > my @pids;

> >
> > This variable doesn't seem to be used.
> >
> > > my $i = 0;

> >
> > premature declaration, no?
> >
> > > my $length = scalar(@lines);
> > > my $pm = new Parallel::ForkManager($length);
> > > print " Connecting to " , $length , " servers\n ";
> > > for $i ( 1 .. $length )
> > > {
> > > print "spawning child processes $i\n";

> >
> > How many of these get printed?

> I am working one server and opening two connections with two child
> processes and third server configured in the text file has a wrong
> address, to handle a condition where the server address is wrong, get
> a open failed condition and exit that child process. here is what I
> get:


(To clean it up, I snipped some editorial comments from the below)

> Connecting to 3 servers
> spawning child processes 1
> spawning child processes 2
> Inside the child process
> Inside the child process
> spawning child processes 3
> Inside the child process


Do you see this both when your code "works" and when your code "doesn't
work"? If so, then the problem probably lies in the part of your code that
uses Net::SDEE, rather than here. If not, then what does it print when the
code doesn't work?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
vkinger@hotmail.com
Guest
Posts: n/a
 
      06-12-2007
I got it to work, it seems my code was fine. The problem I found was
in converting the .pl file into .exe file using perl2exe utility. .pl
did exactly what I was looking for where as .exe choked!!
I do need exe so I am looking into fixing it.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Parallel in, Parallel out shift register Vivek Menon VHDL 0 06-10-2011 10:15 PM
Parallel in, Parallel out shift register Vivek Menon VHDL 5 06-08-2011 03:56 PM
Parallel port control with USB->Parallel converter Soren Python 4 02-14-2008 03:18 PM
Parallel processes Tommy VHDL 4 12-29-2004 11:26 AM
How do I: Main thread spawn child threads, which child processes...control those child processes? Jeff Rodriguez C Programming 23 12-09-2003 11:06 PM



Advertisments