Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > bug in require?

Reply
Thread Tools

bug in require?

 
 
Ara.T.Howard
Guest
Posts: n/a
 
      09-16-2005


[ahoward@localhost ~]$ strace ruby -e' 1.times{ require "digest/md5.so" }' 2>&1 |grep open|grep md5
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3

[ahoward@localhost ~]$ strace ruby -e' 2.times{ require "digest/md5.so" }' 2>&1 |grep open|grep md5
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3

[ahoward@localhost ~]$ strace ruby -e' 3.times{ require "digest/md5.so" }' 2>&1 |grep open|grep md5
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3

[ahoward@localhost ~]$ strace ruby -e' 4.times{ require "digest/md5.so" }' 2>&1 |grep open|grep md5
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3

[ahoward@localhost ~]$ cat a.rb
4.times{ require 'digest/md5' }

[ahoward@localhost ~]$ strace ruby a.rb 2>&1 |grep open|grep md5
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3

why the difference?

-a
--
================================================== =============================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
================================================== =============================



 
Reply With Quote
 
 
 
 
Han Holl
Guest
Posts: n/a
 
      09-16-2005
------=_Part_8219_4949088.1126865657887
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

[ahoward@localhost ~]$ cat a.rb
> 4.times{ require 'digest/md5' }
>=20
> [ahoward@localhost ~]$ strace ruby a.rb 2>&1 |grep open|grep md5
> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",=20
> O_RDONLY|O_LARGEFILE) =3D 3
> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",=20
> O_RDONLY|O_LARGEFILE) =3D 3
> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) =3D 3
> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",=20
> O_RDONLY|O_LARGEFILE) =3D 3
> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",=20
> O_RDONLY|O_LARGEFILE) =3D 3
> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",=20
> O_RDONLY|O_LARGEFILE) =3D 3
>=20
> why the difference?



If you look at the stracefile, you'll notice that the last three times it=
=20
just open and closes the file, without reading. I'm very interested in this=
,=20
because I've noticed the same behaviour with my Rails application, and ther=
e=20
the search path is _huge_.

Han Holl

-a
> --
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D=3D=3D=3D=3D=3D=3D=3D=

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
> | email :: ara [dot] t [dot] howard [at] noaa [dot] gov
> | phone :: 303.497.6469
> | Your life dwells amoung the causes of death
> | Like a lamp standing in a strong breeze. --Nagarjuna
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D=3D=3D=3D=3D=3D=3D=3D=

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
>=20
>=20
>


------=_Part_8219_4949088.1126865657887--


 
Reply With Quote
 
 
 
 
Mauricio Fernández
Guest
Posts: n/a
 
      09-16-2005
On Fri, Sep 16, 2005 at 07:14:21PM +0900, Han Holl wrote:
> If you look at the stracefile, you'll notice that the last three times it
> just open and closes the file, without reading. I'm very interested in this,
> because I've noticed the same behaviour with my Rails application, and there
> the search path is _huge_.


static int
file_load_ok(const char *file)
{
FILE *f;

if (!file) return 0;
f = fopen(file, "r");
if (f == NULL) return 0;
fclose(f);
return 1;
}

(file.c)

--
Mauricio Fernandez


 
Reply With Quote
 
Han Holl
Guest
Posts: n/a
 
      09-16-2005
------=_Part_8319_2271832.1126871795722
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Yes, that's answering where it happens.
Remains the question why.

Han

On 9/16/05, Mauricio Fern=E1ndez <(E-Mail Removed)> wrote:
>=20
> On Fri, Sep 16, 2005 at 07:14:21PM +0900, Han Holl wrote:
> > If you look at the stracefile, you'll notice that the last three times=

=20
> it
> > just open and closes the file, without reading. I'm very interested in=

=20
> this,
> > because I've noticed the same behaviour with my Rails application, and=

=20
> there
> > the search path is _huge_.

>=20
> static int
> file_load_ok(const char *file)
> {
> FILE *f;
>=20
> if (!file) return 0;
> f =3D fopen(file, "r");
> if (f =3D=3D NULL) return 0;
> fclose(f);
> return 1;
> }
>=20
> (file.c)
>=20
> --
> Mauricio Fernandez
>=20
>


------=_Part_8319_2271832.1126871795722--


 
Reply With Quote
 
Yukihiro Matsumoto
Guest
Posts: n/a
 
      09-16-2005
Hi,

In message "Re: bug in require?"
on Fri, 16 Sep 2005 20:56:38 +0900, Han Holl <(E-Mail Removed)> writes:

|Yes, that's answering where it happens.
|Remains the question why.

It was the easiest way to check if a file is loadable (i.e. a file
exists and has proper permissions). Maybe it's possible to replace it
with access(3).

matz.


 
Reply With Quote
 
Mauricio Fernández
Guest
Posts: n/a
 
      09-16-2005
On Fri, Sep 16, 2005 at 12:18:33PM +0900, Ara.T.Howard wrote:
> [ahoward@localhost ~]$ strace ruby -e' 4.times{ require "digest/md5.so"

===
[...]
> [ahoward@localhost ~]$ cat a.rb
> 4.times{ require 'digest/md5' }

===
> why the difference?


batsman@tux-chan:/tmp$ strace ruby -e' 4.times{ require "digest/md5.so" }' 2>&1 |grep open|grep md5
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3
batsman@tux-chan:/tmp$ strace ruby -e' 4.times{ require "digest/md5" }' 2>&1 |grep open|grep md5
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3

If no extension is given, ruby verifies if any new 'feature' can be loaded:

batsman@tux-chan:/tmp$ strace ruby -rfileutils -e'require "digest/md5"; FileUtils.mkdir("digest"); File.open("digest/md5.rb","w"){|f| f.puts %{puts "==== opened & loaded md5.rb"} }; $:.unshift "."; require "digest/md5" ' 2>&1 |grep open|grep md5
execve("/home/batsman/usr/bin/ruby", ["ruby", "-rfileutils", "-erequire \"digest/md5\"; FileUtils.mkdir(\"digest\"); File.open(\"digest/md5.rb\",\"w\"){|f| f.puts %{puts \"==== opened & loaded md5.rb\"} }; $:.unshift \".\"; require \"digest/md5\" "], [/* 31 vars */]) = 0
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY|O_LARGEFILE) = 3
open("/home/batsman/usr//lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3
open("digest/md5.rb", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
write(3, "puts \"==== opened & loaded md5.r"..., 35) = 35
open("./digest/md5.rb", O_RDONLY|O_LARGEFILE) = 3
open("./digest/md5.rb", O_RDONLY|O_LARGEFILE) = 3
open("./digest/md5.rb", O_RDONLY|O_LARGEFILE) = 3
open("./digest/md5.rb", O_RDONLY|O_LARGEFILE) = 3
open("./digest/md5.rb", O_RDONLY|O_LARGEFILE) = 3
read(3, "puts \"==== opened & loaded md5.r"..., 4096) = 35
write(1, "==== opened & loaded md5.rb\n", 28==== opened & loaded md5.rb

--
Mauricio Fernandez


 
Reply With Quote
 
Ara.T.Howard
Guest
Posts: n/a
 
      09-16-2005
On Fri, 16 Sep 2005, Han Holl wrote:

> [ahoward@localhost ~]$ cat a.rb
>> 4.times{ require 'digest/md5' }
>>
>> [ahoward@localhost ~]$ strace ruby a.rb 2>&1 |grep open|grep md5
>> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",
>> O_RDONLY|O_LARGEFILE) = 3
>> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",
>> O_RDONLY|O_LARGEFILE) = 3
>> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so", O_RDONLY) = 3
>> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",
>> O_RDONLY|O_LARGEFILE) = 3
>> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",
>> O_RDONLY|O_LARGEFILE) = 3
>> open("/usr/local/lib/ruby/1.8/i686-linux/digest/md5.so",
>> O_RDONLY|O_LARGEFILE) = 3
>>
>> why the difference?

>
>
> If you look at the stracefile, you'll notice that the last three times it
> just open and closes the file, without reading. I'm very interested in this,
> because I've noticed the same behaviour with my Rails application, and there
> the search path is _huge_.


yeah exactly - i'm trying to speed up acgi running strace's and was seeing md5
get opened like 20 times. this sure could slow something like rails down
alright.

guess this is the point where it's be smart to look at the source for
rb_require...

cheers.

-a
--
================================================== =============================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
================================================== =============================



 
Reply With Quote
 
Han Holl
Guest
Posts: n/a
 
      09-16-2005
------=_Part_8576_8914796.1126880653560
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On 9/16/05, Yukihiro Matsumoto <(E-Mail Removed)> wrote:
>=20
> Hi,
>=20
> In message "Re: bug in require?"
> on Fri, 16 Sep 2005 20:56:38 +0900, Han Holl <(E-Mail Removed)> writes:
>=20
> |Yes, that's answering where it happens.
> |Remains the question why.
>=20
> It was the easiest way to check if a file is loadable (i.e. a file
> exists and has proper permissions). Maybe it's possible to replace it
> with access(3).
>=20
> matz.
>=20
> Ara's question was what the difference was between execution from the=20

commandline and from a script.
Only now I see that the difference is that from the commandline he requires=
=20
"digest/md5.so", and from the file
'digest/md5', without the extension.
Which means that you get a considerable perfomance penalty by omitting the=
=20
extension. In a gem-installed rails environment the seachpath can easily be=
=20
25 directories. Multiple requires withou extension can be costly there.

Cheers,

Han

------=_Part_8576_8914796.1126880653560--


 
Reply With Quote
 
Ara.T.Howard
Guest
Posts: n/a
 
      09-16-2005
On Fri, 16 Sep 2005, Han Holl wrote:

> On 9/16/05, Yukihiro Matsumoto <(E-Mail Removed)> wrote:
>>
>> Hi,
>>
>> In message "Re: bug in require?"
>> on Fri, 16 Sep 2005 20:56:38 +0900, Han Holl <(E-Mail Removed)> writes:
>>
>> |Yes, that's answering where it happens.
>> |Remains the question why.
>>
>> It was the easiest way to check if a file is loadable (i.e. a file
>> exists and has proper permissions). Maybe it's possible to replace it
>> with access(3).
>>
>> matz.
>>
>> Ara's question was what the difference was between execution from the

> commandline and from a script. Only now I see that the difference is that
> from the commandline he requires "digest/md5.so", and from the file
> 'digest/md5', without the extension. Which means that you get a
> considerable perfomance penalty by omitting the extension. In a
> gem-installed rails environment the seachpath can easily be 25 directories.
> Multiple requires withou extension can be costly there.



right. try this fix for your rails app:

harp:~ > strace ruby -e'4.times{ require "digest/md5" }' 2>&1 |egrep 'open.*md5'
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3


harp:~ > strace ruby -r ./require.rb -e'4.times{ require "digest/md5" }' 2>&1 |egrep 'open.*md5'
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3
open("/usr/lib/ruby/1.6/i386-linux-gnu/digest/md5.so", O_RDONLY) = 3


harp:~ > cat ./require.rb
require 'rbconfig.rb'
module Kernel
alias_method '__require__', 'require'
def require lib
if((ext = lib[ %r/\.[^\.]+$/ ]))
__require__ lib
else
dlext, dlext2 = ::Config::CONFIG['DLEXT'], ::Config::CONFIG['DLEXT2']
__require__ "#{ lib }.#{ dlext }" rescue
__require__ "#{ lib }.#{ dlext2 }" rescue
__require__ "#{ lib }.rb"
end
end
end


cheers.

-a
--
================================================== =============================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
================================================== =============================



 
Reply With Quote
 
nobu.nokada@softhome.net
Guest
Posts: n/a
 
      09-16-2005
Hi,

At Fri, 16 Sep 2005 23:24:16 +0900,
Han Holl wrote in [ruby-talk:156435]:
> Ara's question was what the difference was between execution from the
> commandline and from a script.
> Only now I see that the difference is that from the commandline he requires
> "digest/md5.so", and from the file
> 'digest/md5', without the extension.
> Which means that you get a considerable perfomance penalty by omitting the
> extension. In a gem-installed rails environment the seachpath can easily be
> 25 directories. Multiple requires withou extension can be costly there.


If md5.rb exists in loadpath, it should be loaded even if
md5.so has been loaded.

--
Nobu Nakada


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
*bug* *bug* *bug* David Raleigh Arnold Firefox 12 04-02-2007 03:13 AM
ASP.NET Login control bug or SQL 2005 bug? RedEye ASP .Net 2 12-13-2005 10:57 AM
Re: BUG? OR NOT A BUG? John ASP .Net 2 09-21-2005 10:31 AM
Bug Parade Bug 4953793 Michel Joly de Lotbiniere Java 4 12-02-2003 05:05 AM
how to report bug to g++ ? got a bug and fixed up source code DarkSpy C++ 4 06-27-2003 09:05 AM



Advertisments