«

I've written a 1500-line script which processes several dozen files of
source text written in Markdown to html. It takes several minutes to
run, indicating progress by printf statements. However, about 20% of
the time, in the middle of processing a Markdown file, it just stops
progressing, as though it is in an infinite loop. If I kill the
process and restart, it always completes successfully.

My script is, of course, being a script, not particularly efficient.
I was thinking that maybe Perl was running out of memory or something,
although that's not supposed to happen nowadays (Perl 5.10.0, Mac OS X
10.6). And when I check it in Apple's Activity Monitor during normal
operation, I find that its CPU and memory usage are hardly noticeable,
maybe 3% and a few tens of megabytes.

Are there any conditions under which Perl would "just stop"?

Any suggestions to troubleshoot this would be appreciated.

Thanks!

Jerry Krinock

On 2010-09-24, Jerry Krinock <> wrote:
> operation, I find that its CPU and memory usage are hardly noticeable,
> maybe 3% and a few tens of megabytes.
>
> Are there any conditions under which Perl would "just stop"?

With no CPU usage? I would say it reads from STDIN. Did you try to
press Enter or C-d?

> Any suggestions to troubleshoot this would be appreciated.

Have not you heard about debugger? If this happens often, you can
just wait for it to happen in the debugger. If worse comes to worst,
and interactive debugging does not help, you can always start in
NonStop mode with `tracing', and look for the last several thousands
of lines when the `stop' happens.

Hope this helps,
Ilya

On Sep 24, 3:11*pm, Jerry Krinock <jerrykrin...@gmail.com> wrote:
> I've written a 1500-line script which processes several dozen files of
> source text written in Markdown to html. *It takes several minutes to
> run, indicating progress by printf statements. *However, about 20% of
> the time, in the middle of processing a Markdown file, it just stops
> progressing, as though it is in an infinite loop. *If I kill the
> process and restart, it always completes successfully.
>
> My script is, of course, being a script, not particularly efficient.
> I was thinking that maybe Perl was running out of memory or something,
> although that's not supposed to happen nowadays (Perl 5.10.0, Mac OS X
> 10.6). *And when I check it in Apple's Activity Monitor during normal
> operation, I find that its CPU and memory usage are hardly noticeable,
> maybe 3% and a few tens of megabytes.
>
> Are there any conditions under which Perl would "just stop"?
>
> Any suggestions to troubleshoot this would be appreciated.
>

You might try just setting a timeout around
whatever code section turns up in a stack
trace. (perldoc -f alarm).

Then exec the program again (perldoc -f exec)
if there's a timeout.. Of course if memory's
the problem, you may be able to find some way
to reduce memory usage and eliminate the timeout
workaround.

--
Charles DeRykus

On 2010-09-25 20:50, Ilya Zakharevich <nospam-> wrote:
> On 2010-09-24, Jerry Krinock <> wrote:
>> operation, I find that its CPU and memory usage are hardly noticeable,
>> maybe 3% and a few tens of megabytes.
>>
>> Are there any conditions under which Perl would "just stop"?
>
> With no CPU usage? I would say it reads from STDIN. Did you try to
> press Enter or C-d?

Two tools I find indispensable when trying to figure out "strange"
behaviour of programs are lsof and strace.

lsof lists the open files of a process. It has been ported to lots of
unixoid systems (I first encountered it on HP-UX) and should be
available on MacOS.

strace prints out the system calls a process invokes. Unfortunately,
while most unixoid systems have a program which does this, it seems to
have a different name on each ("truss" on Solaris, "tusc" on HP-UX, ...)
so the OP will have to find out himself how its called on MacOS.

In this case, if your guess is right, strace would show that the process
is currently waiting for a read on fd 0 to complete, and then lsof could
be used to find out which file fd 0 is (ok, so for fd 0 you may know
that it's the tty, for for (say) fd 43 you want a tool to look it up).

hp

Peter J. Holzer wrote:
> On 2010-09-25 20:50, Ilya Zakharevich <nospam-> wrote:
>> On 2010-09-24, Jerry Krinock <> wrote:
>>> operation, I find that its CPU and memory usage are hardly noticeable,
>>> maybe 3% and a few tens of megabytes.
>>>
>>> Are there any conditions under which Perl would "just stop"?
>> With no CPU usage? I would say it reads from STDIN. Did you try to
>> press Enter or C-d?
>
> Two tools I find indispensable when trying to figure out "strange"
> behaviour of programs are lsof and strace.
>
> lsof lists the open files of a process. It has been ported to lots of
> unixoid systems (I first encountered it on HP-UX) and should be
> available on MacOS.
>
> strace prints out the system calls a process invokes. Unfortunately,
> while most unixoid systems have a program which does this, it seems to
> have a different name on each ("truss" on Solaris, "tusc" on HP-UX, ...)
> so the OP will have to find out himself how its called on MacOS.
>
> In this case, if your guess is right, strace would show that the process
> is currently waiting for a read on fd 0 to complete, and then lsof could
> be used to find out which file fd 0 is (ok, so for fd 0 you may know
> that it's the tty, for for (say) fd 43 you want a tool to look it up).

Unfortunately, if you use strace "-p" option to attach to an
already-running process, if often doesn't show you what call the process
was waiting on at the time of the attachment. You would have to start
stracing the process from the beginning, which is inconvenient if the
situation at question only happens occasionally.

Xho

with <m635n7-> Ben Morrow wrote:
> Quoth Xho Jingleheimerschmidt <>:
>>
>> Unfortunately, if you use strace "-p" option to attach to an
>> already-running process, if often doesn't show you what call the process
>> was waiting on at the time of the attachment. You would have to start
>> stracing the process from the beginning, which is inconvenient if the
>> situation at question only happens occasionally.
>
> You can find out what call the process is currently blocking in with ps.

I've just checked (linux, 2.6.30, Debian). Neither '-n
/boot/System.map-2.6.30*' nor '-n /proc/*/wchan' helps 'ps' to find
syscall. 'ps' fails with its default routine either. Thus the output
for 'WCHAN' column is always either '-' or '?'. I remember, it was
there. Now it's missing.

--
Torvalds' goal for Linux is very simple: World Domination
Stallman's goal for GNU is even simpler: Freedom

Ben Morrow wrote:
> Quoth Xho Jingleheimerschmidt <>:
>> Peter J. Holzer wrote:
>>
>> Unfortunately, if you use strace "-p" option to attach to an
>> already-running process, if often doesn't show you what call the process
>> was waiting on at the time of the attachment. You would have to start
>> stracing the process from the beginning, which is inconvenient if the
>> situation at question only happens occasionally.
>
> You can find out what call the process is currently blocking in with ps.

Well, maybe *you* can. I've never been able to.

Xho

with <> Eric Pozharski wrote:
*SKIP*
> I've just checked (linux, 2.6.30, Debian). Neither '-n
> /boot/System.map-2.6.30*' nor '-n /proc/*/wchan' helps 'ps' to find
> syscall. 'ps' fails with its default routine either. Thus the output
> for 'WCHAN' column is always either '-' or '?'. I remember, it was
> there. Now it's missing.

I've checked more thoroughly, all 'wchan' files (there're
'/proc/*/task/*/wchan' too) are always '0'. BTW, they don't have
trailing newline either. R.I.P.



--
Torvalds' goal for Linux is very simple: World Domination
Stallman's goal for GNU is even simpler: Freedom

On 2010-09-26 23:06, Xho Jingleheimerschmidt <> wrote:
> Peter J. Holzer wrote:
>> In this case, if your guess is right, strace would show that the process
>> is currently waiting for a read on fd 0 to complete, and then lsof could
>> be used to find out which file fd 0 is (ok, so for fd 0 you may know
>> that it's the tty, for for (say) fd 43 you want a tool to look it up).
>
> Unfortunately, if you use strace "-p" option to attach to an
> already-running process, if often doesn't show you what call the process
> was waiting on at the time of the attachment.

I don't think that ever happened to me in many years of using strace.
Except with multithreaded processes, but for those strace doesn't work
reliably anyway (I don't understand why).

hp

Peter J. Holzer wrote:
> On 2010-09-26 23:06, Xho Jingleheimerschmidt <> wrote:
>> Peter J. Holzer wrote:
>>> In this case, if your guess is right, strace would show that the process
>>> is currently waiting for a read on fd 0 to complete, and then lsof could
>>> be used to find out which file fd 0 is (ok, so for fd 0 you may know
>>> that it's the tty, for for (say) fd 43 you want a tool to look it up).
>> Unfortunately, if you use strace "-p" option to attach to an
>> already-running process, if often doesn't show you what call the process
>> was waiting on at the time of the attachment.
>
> I don't think that ever happened to me in many years of using strace.
> Except with multithreaded processes, but for those strace doesn't work
> reliably anyway (I don't understand why).

This is what I get:

$ perl -le '<>' &
[1] 17657
$ strace -p 17657
Process 17657 attached - interrupt to quit

And then no output until a break out of strace.

Xho