Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Long script "just stops" sometime

Reply
Thread Tools

Long script "just stops" sometime

 
 
Jerry Krinock
Guest
Posts: n/a
 
      09-24-2010
I've written a 1500-line script which processes several dozen files of
source text written in Markdown to html. It takes several minutes to
run, indicating progress by printf statements. However, about 20% of
the time, in the middle of processing a Markdown file, it just stops
progressing, as though it is in an infinite loop. If I kill the
process and restart, it always completes successfully.

My script is, of course, being a script, not particularly efficient.
I was thinking that maybe Perl was running out of memory or something,
although that's not supposed to happen nowadays (Perl 5.10.0, Mac OS X
10.6). And when I check it in Apple's Activity Monitor during normal
operation, I find that its CPU and memory usage are hardly noticeable,
maybe 3% and a few tens of megabytes.

Are there any conditions under which Perl would "just stop"?

Any suggestions to troubleshoot this would be appreciated.

Thanks!

Jerry Krinock
 
Reply With Quote
 
 
 
 
Ilya Zakharevich
Guest
Posts: n/a
 
      09-25-2010
On 2010-09-24, Jerry Krinock <(E-Mail Removed)> wrote:
> operation, I find that its CPU and memory usage are hardly noticeable,
> maybe 3% and a few tens of megabytes.
>
> Are there any conditions under which Perl would "just stop"?


With no CPU usage? I would say it reads from STDIN. Did you try to
press Enter or C-d?

> Any suggestions to troubleshoot this would be appreciated.


Have not you heard about debugger? If this happens often, you can
just wait for it to happen in the debugger. If worse comes to worst,
and interactive debugging does not help, you can always start in
NonStop mode with `tracing', and look for the last several thousands
of lines when the `stop' happens.

Hope this helps,
Ilya
 
Reply With Quote
 
 
 
 
C.DeRykus
Guest
Posts: n/a
 
      09-26-2010
On Sep 24, 3:11*pm, Jerry Krinock <(E-Mail Removed)> wrote:
> I've written a 1500-line script which processes several dozen files of
> source text written in Markdown to html. *It takes several minutes to
> run, indicating progress by printf statements. *However, about 20% of
> the time, in the middle of processing a Markdown file, it just stops
> progressing, as though it is in an infinite loop. *If I kill the
> process and restart, it always completes successfully.
>
> My script is, of course, being a script, not particularly efficient.
> I was thinking that maybe Perl was running out of memory or something,
> although that's not supposed to happen nowadays (Perl 5.10.0, Mac OS X
> 10.6). *And when I check it in Apple's Activity Monitor during normal
> operation, I find that its CPU and memory usage are hardly noticeable,
> maybe 3% and a few tens of megabytes.
>
> Are there any conditions under which Perl would "just stop"?
>
> Any suggestions to troubleshoot this would be appreciated.
>


You might try just setting a timeout around
whatever code section turns up in a stack
trace. (perldoc -f alarm).

Then exec the program again (perldoc -f exec)
if there's a timeout.. Of course if memory's
the problem, you may be able to find some way
to reduce memory usage and eliminate the timeout
workaround.

--
Charles DeRykus
 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      09-26-2010
On 2010-09-25 20:50, Ilya Zakharevich <(E-Mail Removed)> wrote:
> On 2010-09-24, Jerry Krinock <(E-Mail Removed)> wrote:
>> operation, I find that its CPU and memory usage are hardly noticeable,
>> maybe 3% and a few tens of megabytes.
>>
>> Are there any conditions under which Perl would "just stop"?

>
> With no CPU usage? I would say it reads from STDIN. Did you try to
> press Enter or C-d?


Two tools I find indispensable when trying to figure out "strange"
behaviour of programs are lsof and strace.

lsof lists the open files of a process. It has been ported to lots of
unixoid systems (I first encountered it on HP-UX) and should be
available on MacOS.

strace prints out the system calls a process invokes. Unfortunately,
while most unixoid systems have a program which does this, it seems to
have a different name on each ("truss" on Solaris, "tusc" on HP-UX, ...)
so the OP will have to find out himself how its called on MacOS.

In this case, if your guess is right, strace would show that the process
is currently waiting for a read on fd 0 to complete, and then lsof could
be used to find out which file fd 0 is (ok, so for fd 0 you may know
that it's the tty, for for (say) fd 43 you want a tool to look it up).

hp
 
Reply With Quote
 
Xho Jingleheimerschmidt
Guest
Posts: n/a
 
      09-26-2010
Peter J. Holzer wrote:
> On 2010-09-25 20:50, Ilya Zakharevich <(E-Mail Removed)> wrote:
>> On 2010-09-24, Jerry Krinock <(E-Mail Removed)> wrote:
>>> operation, I find that its CPU and memory usage are hardly noticeable,
>>> maybe 3% and a few tens of megabytes.
>>>
>>> Are there any conditions under which Perl would "just stop"?

>> With no CPU usage? I would say it reads from STDIN. Did you try to
>> press Enter or C-d?

>
> Two tools I find indispensable when trying to figure out "strange"
> behaviour of programs are lsof and strace.
>
> lsof lists the open files of a process. It has been ported to lots of
> unixoid systems (I first encountered it on HP-UX) and should be
> available on MacOS.
>
> strace prints out the system calls a process invokes. Unfortunately,
> while most unixoid systems have a program which does this, it seems to
> have a different name on each ("truss" on Solaris, "tusc" on HP-UX, ...)
> so the OP will have to find out himself how its called on MacOS.
>
> In this case, if your guess is right, strace would show that the process
> is currently waiting for a read on fd 0 to complete, and then lsof could
> be used to find out which file fd 0 is (ok, so for fd 0 you may know
> that it's the tty, for for (say) fd 43 you want a tool to look it up).


Unfortunately, if you use strace "-p" option to attach to an
already-running process, if often doesn't show you what call the process
was waiting on at the time of the attachment. You would have to start
stracing the process from the beginning, which is inconvenient if the
situation at question only happens occasionally.

Xho
 
Reply With Quote
 
Eric Pozharski
Guest
Posts: n/a
 
      09-27-2010
with <(E-Mail Removed)> Ben Morrow wrote:
> Quoth Xho Jingleheimerschmidt <(E-Mail Removed)>:
>>
>> Unfortunately, if you use strace "-p" option to attach to an
>> already-running process, if often doesn't show you what call the process
>> was waiting on at the time of the attachment. You would have to start
>> stracing the process from the beginning, which is inconvenient if the
>> situation at question only happens occasionally.

>
> You can find out what call the process is currently blocking in with ps.


I've just checked (linux, 2.6.30, Debian). Neither '-n
/boot/System.map-2.6.30*' nor '-n /proc/*/wchan' helps 'ps' to find
syscall. 'ps' fails with its default routine either. Thus the output
for 'WCHAN' column is always either '-' or '?'. I remember, it was
there. Now it's missing.

--
Torvalds' goal for Linux is very simple: World Domination
Stallman's goal for GNU is even simpler: Freedom
 
Reply With Quote
 
Xho Jingleheimerschmidt
Guest
Posts: n/a
 
      09-28-2010
Ben Morrow wrote:
> Quoth Xho Jingleheimerschmidt <(E-Mail Removed)>:
>> Peter J. Holzer wrote:
>>
>> Unfortunately, if you use strace "-p" option to attach to an
>> already-running process, if often doesn't show you what call the process
>> was waiting on at the time of the attachment. You would have to start
>> stracing the process from the beginning, which is inconvenient if the
>> situation at question only happens occasionally.

>
> You can find out what call the process is currently blocking in with ps.


Well, maybe *you* can. I've never been able to.

Xho
 
Reply With Quote
 
Eric Pozharski
Guest
Posts: n/a
 
      09-28-2010
with <(E-Mail Removed)> Eric Pozharski wrote:
*SKIP*
> I've just checked (linux, 2.6.30, Debian). Neither '-n
> /boot/System.map-2.6.30*' nor '-n /proc/*/wchan' helps 'ps' to find
> syscall. 'ps' fails with its default routine either. Thus the output
> for 'WCHAN' column is always either '-' or '?'. I remember, it was
> there. Now it's missing.


I've checked more thoroughly, all 'wchan' files (there're
'/proc/*/task/*/wchan' too) are always '0'. BTW, they don't have
trailing newline either. R.I.P.



--
Torvalds' goal for Linux is very simple: World Domination
Stallman's goal for GNU is even simpler: Freedom
 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      09-28-2010
On 2010-09-26 23:06, Xho Jingleheimerschmidt <(E-Mail Removed)> wrote:
> Peter J. Holzer wrote:
>> In this case, if your guess is right, strace would show that the process
>> is currently waiting for a read on fd 0 to complete, and then lsof could
>> be used to find out which file fd 0 is (ok, so for fd 0 you may know
>> that it's the tty, for for (say) fd 43 you want a tool to look it up).

>
> Unfortunately, if you use strace "-p" option to attach to an
> already-running process, if often doesn't show you what call the process
> was waiting on at the time of the attachment.


I don't think that ever happened to me in many years of using strace.
Except with multithreaded processes, but for those strace doesn't work
reliably anyway (I don't understand why).

hp
 
Reply With Quote
 
Xho Jingleheimerschmidt
Guest
Posts: n/a
 
      09-29-2010
Peter J. Holzer wrote:
> On 2010-09-26 23:06, Xho Jingleheimerschmidt <(E-Mail Removed)> wrote:
>> Peter J. Holzer wrote:
>>> In this case, if your guess is right, strace would show that the process
>>> is currently waiting for a read on fd 0 to complete, and then lsof could
>>> be used to find out which file fd 0 is (ok, so for fd 0 you may know
>>> that it's the tty, for for (say) fd 43 you want a tool to look it up).

>> Unfortunately, if you use strace "-p" option to attach to an
>> already-running process, if often doesn't show you what call the process
>> was waiting on at the time of the attachment.

>
> I don't think that ever happened to me in many years of using strace.
> Except with multithreaded processes, but for those strace doesn't work
> reliably anyway (I don't understand why).


This is what I get:

$ perl -le '<>' &
[1] 17657
$ strace -p 17657
Process 17657 attached - interrupt to quit

And then no output until a break out of strace.

Xho
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Having compilation error: no match for call to (const __gnu_cxx::hash<long long int>) (const long long int&) veryhotsausage C++ 1 07-04-2008 05:41 PM
createImage sometime returns null and sometime returns non-null. vizlab Java 3 10-17-2007 11:21 AM
I don't undestand why sometime python (or other) add "tmpWvk7zy"(tmp + random string) after "buildout-eggs" in path of python script KLEIN Stephane Python 0 09-24-2007 08:47 PM
msg filters only work sometime on newsgroups in ThunderB Fuddzy Firefox 2 02-19-2005 05:11 AM
CGI.pm form query sometime return null value YesBalala Perl 1 02-13-2004 06:23 PM



Advertisments