Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > sysopen failures

Reply
Thread Tools

sysopen failures

 
 
Marc Girod
Guest
Posts: n/a
 
      08-06-2010
Hello,

A script saves mails sent by a crontab--so, there may be bursts...
It uses sysopen, I assume to make sure it doesn't overwrite existing
files.
At times, we get bursts of errors (File exists), which I trace to the
sysopen call.
However, I cannot find that all the corresponding files would have
existed.
I read the doc and get to:

In many systems the "O_EXCL" flag is available for opening
files
in exclusive mode. This is not locking: exclusiveness
means here
that if the file already exists, sysopen() fails. "O_EXCL"
may
not work on network filesystems, and has no effect unless
the
"O_CREAT" flag is set as well.

The script does write to a network filesystems (home directory on a
remote filer, 4 ms round-trip).

Shoud I look for a replacement for sysopen?
Or for an other theory to explain the problem?

The bit of code doing the open:

if(defined($mode)? sysopen(FILE, $file, O_EXCL | O_CREAT | O_WRONLY,
$mode):
sysopen(FILE, $file, O_EXCL | O_CREAT | O_WRONLY))
{
_dump(*FILE, @$r_lines);
return close(FILE);
}
return(0);

Thanks,
Marc

 
Reply With Quote
 
 
 
 
John W. Krahn
Guest
Posts: n/a
 
      08-07-2010
Marc Girod wrote:
>
> A script saves mails sent by a crontab--so, there may be bursts...
> It uses sysopen, I assume to make sure it doesn't overwrite existing
> files.
> At times, we get bursts of errors (File exists), which I trace to the
> sysopen call.
> However, I cannot find that all the corresponding files would have
> existed.
> I read the doc and get to:
>
> In many systems the "O_EXCL" flag is available for opening
> files
> in exclusive mode. This is not locking: exclusiveness
> means here
> that if the file already exists, sysopen() fails. "O_EXCL"
> may
> not work on network filesystems, and has no effect unless
> the
> "O_CREAT" flag is set as well.
>
> The script does write to a network filesystems (home directory on a
> remote filer, 4 ms round-trip).
>
> Shoud I look for a replacement for sysopen?
> Or for an other theory to explain the problem?
>
> The bit of code doing the open:
>
> if(defined($mode)? sysopen(FILE, $file, O_EXCL | O_CREAT | O_WRONLY,
> $mode):
> sysopen(FILE, $file, O_EXCL | O_CREAT | O_WRONLY))
> {


Your code does not check the return value from sysopen() so the error
message you are receiving is not related to sysopen().


> _dump(*FILE, @$r_lines);
> return close(FILE);
> }
> return(0);



John
--
Any intelligent fool can make things bigger and
more complex... It takes a touch of genius -
and a lot of courage to move in the opposite
direction. -- Albert Einstein
 
Reply With Quote
 
 
 
 
Marc Girod
Guest
Posts: n/a
 
      08-07-2010
On Aug 7, 7:40*am, "John W. Krahn" <(E-Mail Removed)> wrote:

> Your code does not check the return value from sysopen() so the error
> message you are receiving is not related to sysopen().


Doesn't it?
The block is an if block.
If the condition (the return from sysopen) is false, the function
returns 0 unconditionally.

Besides, the error I get from $!, at the time of reporting in the
calling function is, as I wrote it, 'File exists'.

And last: it is not *my* code.
Marc
 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      08-07-2010
On 2010-08-07 08:55, Marc Girod <(E-Mail Removed)> wrote:
> On Aug 7, 7:40*am, "John W. Krahn" <(E-Mail Removed)> wrote:
>> Your code does not check the return value from sysopen() so the error
>> message you are receiving is not related to sysopen().

>
> Doesn't it?
> The block is an if block.
> If the condition (the return from sysopen) is false, the function
> returns 0 unconditionally.


But you haven't shown where the error is reported. We could only guess
that it's the next thing after “return 0”.

> Besides, the error I get from $!, at the time of reporting in the
> calling function is, as I wrote it, 'File exists'.


There may be something between the sysopen and the printing of $! which
causes that error. Since we haven't seen that code we can't tell.
I do agree that it's very likely that “File exists” is set by sysopen in
this case.

> And last: it is not *my* code.


You posted it, so in the context of this thread it's your code.

hp

 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      08-07-2010
On 2010-08-06 17:06, Marc Girod <(E-Mail Removed)> wrote:
> A script saves mails sent by a crontab--so, there may be bursts...
> It uses sysopen, I assume to make sure it doesn't overwrite existing
> files.
> At times, we get bursts of errors (File exists), which I trace to the
> sysopen call.
> However, I cannot find that all the corresponding files would have
> existed.
> I read the doc and get to:
>
> In many systems the "O_EXCL" flag is available for opening
> files
> in exclusive mode. This is not locking: exclusiveness
> means here
> that if the file already exists, sysopen() fails. "O_EXCL"
> may
> not work on network filesystems, and has no effect unless
> the
> "O_CREAT" flag is set as well.


If the network filesystem doesn't support O_EXCL, then the open will
succeed even though it shouldn't (or it will fail every time with
EINVAL), it won't fail when the file doesn't exist.


> The script does write to a network filesystems (home directory on a
> remote filer, 4 ms round-trip).
>
> Shoud I look for a replacement for sysopen?


No. sysopen is the closest you can get to the OS. If sysopen reporte
EEXIST, then the OS really thinks the file exists at the time.

> Or for an other theory to explain the problem?


Yes. Most likely causes are IMHO:

* The file really exists at the time. You have to find out why
(maybe your script is supposed to remove the file before it
terminates and it either doesn't do it or a previous invocation
hasn't finished yet). If you know why the solution is probably
obvious.
* The file did exist and has already been removed at the time the
script runs, but the information about the file's existence is still
cached by the OS. In this case you should check the configuration of
the file system (on both the client and the server).

hp
 
Reply With Quote
 
John W. Krahn
Guest
Posts: n/a
 
      08-07-2010
Marc Girod wrote:
> On Aug 7, 7:40 am, "John W. Krahn"<(E-Mail Removed)> wrote:
>
>> Your code does not check the return value from sysopen() so the error
>> message you are receiving is not related to sysopen().

>
> Doesn't it?
> The block is an if block.
> If the condition (the return from sysopen) is false,


Or true.

> the function returns 0 unconditionally.


Correct.

> Besides, the error I get from $!, at the time of reporting in the
> calling function is, as I wrote it, 'File exists'.


By that time $! could have been changed by some other system function.

You need to capture or print the value of $! immediately after the
system function that sets it, for example:

sysopen my $FILE, $file, O_EXCL | O_CREAT | O_WRONLY, $mode or do {
warn "Cannot open '$file' $!";
return 0;
};


*Note that "$mode" is really the permissions field, and the MODE field
is actually "O_EXCL | O_CREAT | O_WRONLY".




John
--
Any intelligent fool can make things bigger and
more complex... It takes a touch of genius -
and a lot of courage to move in the opposite
direction. -- Albert Einstein
 
Reply With Quote
 
Marc Girod
Guest
Posts: n/a
 
      08-09-2010
On Aug 7, 1:10*pm, "Peter J. Holzer" <(E-Mail Removed)> wrote:

> If the network filesystem doesn't support O_EXCL, then the open will
> succeed even though it shouldn't (or it will fail every time with
> EINVAL), it won't fail when the file doesn't exist.


Thanks.

> No. sysopen is the closest you can get to the OS.


OK.

> Yes. Most likely causes are IMHO:
>
> ** The file really exists at the time. You have to find out why
> * *(maybe your script is supposed to remove the file before it
> * *terminates and it either doesn't do it or a previous invocation
> * *hasn't finished yet). If you know why the solution is probably
> * *obvious.
> ** The file did exist and has already been removed at the time the
> * *script runs, but the information about the file's existence is still
> * *cached by the OS. In this case you should check the configuration of
> * *the file system (on both the client and the server).


Well, I have to figure out how either could be possible.
I cannot see what in the code could remove anything...
Those files are left to be accessible from a web page.
There is a loop which increments a digit in the file name and tries
100 times.
I don't think this is very smart, but I cannot see how the 100
different files could exist.

The error returned in the end is of course only the one for the last
try.
So that I don't *know* why the 99 first attempts failed.
But in the general case, I cannot find the 100th file either.

I may leave this for a while: I have more urgent, and the symptom has
stopped for now.
But I don't doubt it will come back.

Thanks.
Marc
 
Reply With Quote
 
Marc Girod
Guest
Posts: n/a
 
      08-09-2010
On Aug 7, 6:40*pm, "John W. Krahn" <(E-Mail Removed)> wrote:

> Or true.


Indeed.

> By that time $! could have been changed by some other system function.


Yes.

> You need to capture or print the value of $! immediately after the
> system function that sets it, for example:


Yes...

> *Note that "$mode" is really the permissions field, and the MODE field
> is actually "O_EXCL | O_CREAT | O_WRONLY".


I saw that. It was set to 0444.
Thanks. I'll save a pointer to this and come back later.

Marc
 
Reply With Quote
 
Peter J. Holzer
Guest
Posts: n/a
 
      08-09-2010
On 2010-08-09 11:50, Marc Girod <(E-Mail Removed)> wrote:

[sysopen(... O_EXCL ...) fails with EEXIST although the file shouldn't
exist]

> Well, I have to figure out how either could be possible.
> I cannot see what in the code could remove anything...
> Those files are left to be accessible from a web page.
> There is a loop which increments a digit in the file name and tries
> 100 times.
> I don't think this is very smart, but I cannot see how the 100
> different files could exist.
>
> The error returned in the end is of course only the one for the last
> try.
> So that I don't *know* why the 99 first attempts failed.
> But in the general case, I cannot find the 100th file either.


I would log each failure with all the details I can think of. In this
case:

* exact name of the file to be created
* $! immediately after the failure
* cwd at the time of the failure
* try to lstat the file just after the failure (but after logging $!,
or you will change it!) and log all relevant information if the
file exists - this will help you to determine where the spurious
files come from


> I may leave this for a while: I have more urgent, and the symptom has
> stopped for now.
> But I don't doubt it will come back.


If you add the diagnostics now you will have the information next time
the problem occurs.

hp
 
Reply With Quote
 
Martijn Lievaart
Guest
Posts: n/a
 
      08-09-2010
On Mon, 09 Aug 2010 04:50:35 -0700, Marc Girod wrote:

> Well, I have to figure out how either could be possible. I cannot see
> what in the code could remove anything... Those files are left to be
> accessible from a web page. There is a loop which increments a digit in
> the file name and tries 100 times.
> I don't think this is very smart, but I cannot see how the 100 different
> files could exist.
>
> The error returned in the end is of course only the one for the last
> try.
> So that I don't *know* why the 99 first attempts failed. But in the
> general case, I cannot find the 100th file either.


Stupid suggestion and does not match exactly what you wrote above, but:
Are there maybe two instances of your program running and interfering
with each other?

M4
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
sysopen - die only if EBUSY? Tomasz Chmielewski Perl Misc 3 10-02-2008 04:03 PM
WLAN driver failures after re-install =?Utf-8?B?Sm9uIEdyZWVu?= Wireless Networking 0 08-24-2005 10:08 AM
Q re sysopen() and more... Michele Dondi Perl Misc 6 10-15-2004 02:49 PM
problem with sysopen() on nfs Michele Dondi Perl Misc 2 09-19-2004 03:42 PM
sysopen problem Hon Guin Lee - Web Producer - SMI Marketing Perl Misc 1 10-07-2003 09:57 PM



Advertisments