Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > File::Slurp/IO::String/wantarray interaction bug

Reply
Thread Tools

File::Slurp/IO::String/wantarray interaction bug

 
 
kj
Guest
Posts: n/a
 
      06-11-2010


Consider the following demo script:

# first line
# second line

use File::Slurp 'slurp';
use IO::String;

{
my $string = slurp( $0 );
my $io_string = IO::String->new( $string );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

{
my $io_string = IO::String->new( slurp( $0 ) );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

__END__


The only difference between the two blocks is that the first one
assigns the value returned by slurp( $0 ) to the intermediate
lexical variable $io_string, while the second one uses this value
directly.

As far as I can tell, the output of both blocks should be identical,
but they are not even close. The output I get for the script above
is:

>>># first line

<<<
>>># second line

<<<


>>># first line

<<<
>>><<<



Note that in the second block, the second line never gets printed;
it is treated as an empty string.

Stepping through the code with the debugger, I narrowed down the
problem to the first line in the definition of IO::String::READLINE:

sub READLINE
{
goto &getlines if wantarray;
goto &getline;
}

Somehow, in the failing cases, the wantarray in the first line of
READLINE evaluates to true, even though the original calling
environment clearly specifies the scalar keyword. Therefore getlines
gets inappropriately called, rather than the correct getline.

How can the *really* force a scalar environment? (Clearly, the
scalar keyword is not doing the job here.)

FWIW, I'm using perl v5.10.0 on Ubuntu Linux.

TIA!

~K

 
Reply With Quote
 
 
 
 
kj
Guest
Posts: n/a
 
      06-11-2010
In <huuf1m$9kt$(E-Mail Removed)> kj <(E-Mail Removed)> writes:

>Consider the following demo script:


># first line
># second line


>use File::Slurp 'slurp';
>use IO::String;


>{
> my $string = slurp( $0 );
> my $io_string = IO::String->new( $string );
> printf ">>>%s<<<\n", scalar $io_string->READLINE;
> printf ">>>%s<<<\n", scalar $io_string->READLINE;
>}


>{
> my $io_string = IO::String->new( slurp( $0 ) );
> printf ">>>%s<<<\n", scalar $io_string->READLINE;
> printf ">>>%s<<<\n", scalar $io_string->READLINE;
>}


>__END__



>The only difference between the two blocks is that the first one
>assigns the value returned by slurp( $0 ) to the intermediate
>lexical variable $io_string, while the second one uses this value
>directly.


>As far as I can tell, the output of both blocks should be identical,
>but they are not even close. The output I get for the script above
>is:


>>>># first line

><<<
>>>># second line

><<<



>>>># first line

><<<
>>>><<<



>Note that in the second block, the second line never gets printed;
>it is treated as an empty string.


>Stepping through the code with the debugger, I narrowed down the
>problem to the first line in the definition of IO::String::READLINE:


>sub READLINE
>{
> goto &getlines if wantarray;
> goto &getline;
>}


>Somehow, in the failing cases, the wantarray in the first line of
>READLINE evaluates to true, even though the original calling
>environment clearly specifies the scalar keyword. Therefore getlines
>gets inappropriately called, rather than the correct getline.


>How can the *really* force a scalar environment? (Clearly, the
>scalar keyword is not doing the job here.)



OK, I found out one more detail. If I replace the original second block

{
my $io_string = IO::String->new( slurp( $0 ) );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

with

{
my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

now the output is identical for both blocks.

Still, this is quite a curveball. I see that in the original
version, the call to slurp was happenning in a list context, and
this probably messes up the string that actually gets used to define
the IO::String object. But still, this does not explain why READLINE
believes its in a list context. (I'm sorry that I can't show this
easily; one needs to step there with the debugger).

~K
 
Reply With Quote
 
 
 
 
Uri Guttman
Guest
Posts: n/a
 
      06-12-2010
>>>>> "k" == kj <(E-Mail Removed)> writes:

k> In <huuf1m$9kt$(E-Mail Removed)> kj <(E-Mail Removed)> writes:
>> Consider the following demo script:


>> # first line
>> # second line


>> use File::Slurp 'slurp';


most people use the read_file sub. slurp is just an alias to it for
backwards compatability.

>> use IO::String;


why do you play with io::string? perl's open can do that builtin these
days. it does require a scalar var and can't do it on data but that
isn't much of an issue.

and a general question, what are you trying to do here??

>> {
>> my $string = slurp( $0 );
>> my $io_string = IO::String->new( $string );
>> printf ">>>%s<<<\n", scalar $io_string->READLINE;
>> printf ">>>%s<<<\n", scalar $io_string->READLINE;
>> }


>> {
>> my $io_string = IO::String->new( slurp( $0 ) );


all sub/method calls pass list context to their arguments. remember,
those args get set into the @_ array so that makes sense. otherwise you
couldn't pass in an array and it would be converted to its count which
is usually not wanted in @_.

>> printf ">>>%s<<<\n", scalar $io_string->READLINE;
>> printf ">>>%s<<<\n", scalar $io_string->READLINE;
>> }


printf also like other funcs which can take a list of args, has list
context for everything past the format arg.

k> OK, I found out one more detail. If I replace the original second block

k> {
k> my $io_string = IO::String->new( slurp( $0 ) );
k> printf ">>>%s<<<\n", scalar $io_string->READLINE;
k> printf ">>>%s<<<\n", scalar $io_string->READLINE;
k> }

k> with

k> {
k> my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed

well, now it slurps the file to a scalar and not a list of lines.

and the IO::Scalar docs disagree with your call:

new [ARGS...]
Class method. Return a new, unattached scalar handle. If any
arguments are given, they're sent to open().

open [SCALARREF]
Instance method. Open the scalar handle on a new scalar, pointed
to by SCALARREF. If no SCALARREF is given, a "private" scalar is
created to hold the file data.

Returns the self object on success, undefined on error.

so you aren't even calling it correctly. you need to pass it a scalar
ref and you are passing it a scalar value or a scalar. maybe it can
handle that but it doesn't say that in the docs (at least what i read).

k> Still, this is quite a curveball. I see that in the original
k> version, the call to slurp was happenning in a list context, and
k> this probably messes up the string that actually gets used to define
k> the IO::String object. But still, this does not explain why READLINE
k> believes its in a list context. (I'm sorry that I can't show this
k> easily; one needs to step there with the debugger).

as i said, see the docs for printf. after the format it expects a list
so that call will be in list context.

uri

--
Uri Guttman ------ http://www.velocityreviews.com/forums/(E-Mail Removed) -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
 
Reply With Quote
 
Willem
Guest
Posts: n/a
 
      06-12-2010
kj wrote:
) OK, I found out one more detail. If I replace the original second block
)
) {
) my $io_string = IO::String->new( slurp( $0 ) );
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) }
)
) with
)
) {
) my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) }
)
) now the output is identical for both blocks.
)
) Still, this is quite a curveball. I see that in the original
) version, the call to slurp was happenning in a list context, and
) this probably messes up the string that actually gets used to define
) the IO::String object. But still, this does not explain why READLINE
) believes its in a list context. (I'm sorry that I can't show this
) easily; one needs to step there with the debugger).

When I saw your first post, it was immediately apparent to me that the
call to slurp was being evaluated in list context, and that caused it
to pass only the first line to the IO::String object.

The READLINE context is a red herring. Maybe you misinterpreted the
debugger output, maybe the debugger did something strange, maybe something
else happened. In any case, it's quite clear that it *is* being called in
scalar context, because that final fix works like it should.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
 
Reply With Quote
 
kj
Guest
Posts: n/a
 
      06-14-2010
In <(E-Mail Removed)> Willem <(E-Mail Removed)> writes:

>...Maybe you misinterpreted the
>debugger output, maybe the debugger did something strange, maybe something
>else happened.


Well, it looks that it's one of those. To determine the calling
context I was halting the execution within the READLINE method,
and then printing the value of wantarray from the debugger prompt
like this:

DB<1> p wantarray
1

Apparentely, in the debugger "p wantarray" always produces this
result, irrespective of the value of wantarray in the executing
program. (That's quite the "banana peel", IMHO.)

Poking around in the perldebug man page I discovered that the right
way to determine the calling context from within the debugger is
to use the T command:

DB<4> T
$ = IO::String::READLINE(ref(IO::String)) called from file `t/wantarraybug.pl' line 28

The leading '$' is shorthand for "called in scalar context".

Thanks for your post!

~K
 
Reply With Quote
 
kj
Guest
Posts: n/a
 
      06-14-2010
In <(E-Mail Removed)> "Uri Guttman" <(E-Mail Removed)> writes:

>and the IO::Scalar docs disagree with your call:


> new [ARGS...]
> Class method. Return a new, unattached scalar handle. If any
> arguments are given, they're sent to open().


> open [SCALARREF]
> Instance method. Open the scalar handle on a new scalar, pointed
> to by SCALARREF. If no SCALARREF is given, a "private" scalar is
> created to hold the file data.


> Returns the self object on success, undefined on error.


>so you aren't even calling it correctly.


I'm using IO::String, not IO::Scalar. My IO::String docs says:

$io = IO::String->new
$io = IO::String->new( $string )
The constructor returns a newly-created "IO::String"
object. It takes an optional argument, which is the
string to read from or write into. ...

As I described in the post before this one, what led me down the
wrong path was how I was determining the value of wantarray within
the debugger.

Thanks for your comments!

~K
 
Reply With Quote
 
Ilya Zakharevich
Guest
Posts: n/a
 
      06-15-2010
On 2010-06-14, kj <(E-Mail Removed)> wrote:
> Well, it looks that it's one of those. To determine the calling
> context I was halting the execution within the READLINE method,
> and then printing the value of wantarray from the debugger prompt
> like this:
>
> DB<1> p wantarray
> 1
>
> Apparentely, in the debugger "p wantarray" always produces this
> result, irrespective of the value of wantarray in the executing
> program.


Sure, how else would it be? `p' evaluates its expression in list context...

Yours,
Ilya
 
Reply With Quote
 
kj
Guest
Posts: n/a
 
      06-15-2010
In <(E-Mail Removed)> Ilya Zakharevich <(E-Mail Removed)> writes:

>On 2010-06-14, kj <(E-Mail Removed)> wrote:
>> Well, it looks that it's one of those. To determine the calling
>> context I was halting the execution within the READLINE method,
>> and then printing the value of wantarray from the debugger prompt
>> like this:
>>
>> DB<1> p wantarray
>> 1
>>
>> Apparentely, in the debugger "p wantarray" always produces this
>> result, irrespective of the value of wantarray in the executing
>> program.


>Sure, how else would it be? `p' evaluates its expression in list context...


Yes, in retrospect I see that what I was doing is pretty dumb.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Interaction btw unittest.assertRaises and __getattr__. Bug? Inyeol Python 3 10-27-2010 07:15 AM
interaction of mode 'r+', file.write(), and file.tell(): a bug orundefined behavior? Lie Ryan Python 3 02-01-2010 11:29 PM
*bug* *bug* *bug* David Raleigh Arnold Firefox 12 04-02-2007 03:13 AM
sys.settrace closure interaction bug Scott_Marks Python 0 10-01-2006 04:25 PM
user interaction disabling? Dan Perl 0 07-20-2003 06:50 AM



Advertisments