Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Analyzer for errors in code ?

Reply
Thread Tools

Analyzer for errors in code ?

 
 
David Unric
Guest
Posts: n/a
 
      11-09-2010
Ryan Davis> Yep, I really understand it's impossible to do static
(lexical) code analysis in case of a programming language with dynamic
type system and selfmodifying abilities at the runtime you can 100% rely
on.
I would just appreciate some helper tool to warn me about _possible_
dumb typing errors I will make at writing my code and I would decide if
it's false or positive warning. Most of time I'm fixing typos rather
then a mistake in an algorithm. Better then nothing.

Unfortunately none of above mentioned or linked tools detected the use
of unassigned variable. For an illustration what does pylint for the
equivalent code in python:

_____ snip ________________________________________
import sys

if __name__ == "__main__":
my_msg = 'Hello'
if sys.argv[-1] == 'doit':
print mymsg
else:
print 'Nope'

~$ pylint -E test.py
No config file found, using default configuration
************* Module test
E: 6: Undefined variable 'mymsg'
_____ snip ________________________________________

I'm using the new generation of Ruby, version 1.9.2 concretely. It
produduces bytecode for YARV interpreter. I'm sure catching an
intermediate output between parsed source and bytecode would allow
static analysis I'm calling for. Is this possible with current versions
Ruby ?

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
Ryan Davis
Guest
Posts: n/a
 
      11-10-2010

On Nov 9, 2010, at 14:56 , David Unric wrote:

> Unfortunately none of above mentioned or linked tools detected the use=20=


> of unassigned variable. For an illustration what does pylint for the=20=


> equivalent code in python:
>=20
> _____ snip ________________________________________
> import sys
>=20
> if __name__ =3D=3D "__main__":
> my_msg =3D 'Hello'
> if sys.argv[-1] =3D=3D 'doit':
> print mymsg
> else:
> print 'Nope'
>=20
> ~$ pylint -E test.py
> No config file found, using default configuration
> ************* Module test
> E: 6: Undefined variable 'mymsg'


In python, 'mymsg' _has_ to be a variable. This isn't true in ruby. It =
can be a variable OR a method call. In the case of the latter, we don't =
know if it is valid or not without evaluating.

in ruby, 'def x; mymsg; end' (the most boiled down version of your =
example) looks like this internally:

% echo 'def x; mymsg; end' | parse_tree_show=20
s(:defn,
,
s(:args),
s(:scope, s(:block, s(:call, nil, :mymsg, s(:arglist)))))

Ruby parsed the code and decided that mymsg must be a method call. It is =
determined at runtime (since everything is late bound) who (if anyone) =
implemented the method and if not, it goes to method_missing.

Python simply doesn't have this flexibility. 'x' is a variable and 'x()' =
is a call. AFAIK, there is no "__" hook equivalent to method_missing in =
python... But it has been a while for me.


 
Reply With Quote
 
 
 
 
Ryan Davis
Guest
Posts: n/a
 
      11-10-2010

On Nov 9, 2010, at 14:56 , David Unric wrote:

> Most of time I'm fixing typos rather=20
> then a mistake in an algorithm. Better then nothing.


I also suggest that you deal with the problem instead of the symptom. I =
_don't_ typo that much. Much of that is attributed to my use of "M-/" =
(dabbrev-expand -- Expand previous word "dynamically") in emacs. Vim has =
C-n (keyword completion). I'm sure whatever text editor you're using can =
help... If not, then I have another suggestion for you.


 
Reply With Quote
 
David Unric
Guest
Posts: n/a
 
      11-10-2010
Ryan Davis wrote in post #960436:
>
> In python, 'mymsg' _has_ to be a variable. This isn't true in ruby. It
> can be a variable OR a method call. In the case of the latter, we don't
> know if it is valid or not without evaluating.
>
> in ruby, 'def x; mymsg; end' (the most boiled down version of your
> example) looks like this internally:
>
> % echo 'def x; mymsg; end' | parse_tree_show
> s(:defn,
> ,
> s(:args),
> s(:scope, s(:block, s(:call, nil, :mymsg, s(:arglist)))))
>
> Ruby parsed the code and decided that mymsg must be a method call. It is
> determined at runtime (since everything is late bound) who (if anyone)
> implemented the method and if not, it goes to method_missing.
>
> Python simply doesn't have this flexibility. 'x' is a variable and 'x()'
> is a call. AFAIK, there is no "__" hook equivalent to method_missing in
> python... But it has been a while for me.


Great info Ryan, thank you.

My appologies I did not realized sooner the deeper conceptual
differencies between these languages.
Despite of that, if Ruby cann't decide if mymsg is a variable or a
method call in advance, how does it rule out the possibility for a
static parser to notify the 'mymsg' symbol has no assignment or method
definiton in the source code ? I.e. if 'x' in your example is a method
call, how can arise without some form of an assignment ?

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Ryan Davis
Guest
Posts: n/a
 
      11-10-2010

On Nov 9, 2010, at 17:50 , David Unric wrote:

> Despite of that, if Ruby cann't decide if mymsg is a variable or a=20
> method call in advance, how does it rule out the possibility for a=20
> static parser to notify the 'mymsg' symbol has no assignment or method=20=


> definiton in the source code ? I.e. if 'x' in your example is a method=20=


> call, how can arise without some form of an assignment ?


I don't entirely understand your question, as "x" in my example is the =
wrapping method and "mymsg" was the message send from your example.

So let's try again, with the code sample reformatted better:

> % echo 'def x; mymsg; end' | parse_tree_show


begets:

> s(:defn, , s(:args),
> s(:scope,
> s(:block,
> s(:call, nil, :mymsg, s(:arglist)))))


read in English: "method definition with no args. calls mymsg w/ no =
args"

So, ruby parsed "def x;" and realized it was a 0 arg method. The body =
refers to "mymsg;" and since there was no previous variable assignment =
of the same name, it must be a method call (with no args). That's it. At =
that point, the method is "compiled" down to a single method invocation =
for the message "mymsg". Notice I said "message"... that's an important =
distinction between static and dynamic languages. In static languages =
(or in dynamic/hybrid languages where the method is pre-calculated), =
methods/functions are called by having a pointer to the function body =
and simply jumping to that address.

In a dynamic invocation, a receiver is sent "a message" at runtime. This =
roughly translates to asking the hierarchy of classes if they implement =
that message, and if so, to please execute it. If not, then the runtime =
does the same thing, but this time around the message is =
"method_missing" with the previous message pushed onto the front of the =
args.=

 
Reply With Quote
 
David Unric
Guest
Posts: n/a
 
      11-10-2010
Ryan Davis wrote in post #960452:
> So let's try again, with the code sample reformatted better:
>
>> % echo 'def x; mymsg; end' | parse_tree_show

>
> begets:
>
>> s(:defn, , s(:args),
>> s(:scope,
>> s(:block,
>> s(:call, nil, :mymsg, s(:arglist)))))

>
> read in English: "method definition with no args. calls mymsg w/ no
> args"
>
> So, ruby parsed "def x;" and realized it was a 0 arg method. The body
> refers to "mymsg;" and since there was no previous variable assignment
> of the same name, it must be a method call (with no args). That's it. At
> that point, the method is "compiled" down to a single method invocation
> for the message "mymsg". Notice I said "message"... that's an important
> distinction between static and dynamic languages. In static languages
> (or in dynamic/hybrid languages where the method is pre-calculated),
> methods/functions are called by having a pointer to the function body
> and simply jumping to that address.
>
> In a dynamic invocation, a receiver is sent "a message" at runtime. This
> roughly translates to asking the hierarchy of classes if they implement
> that message, and if so, to please execute it. If not, then the runtime
> does the same thing, but this time around the message is
> "method_missing" with the previous message pushed onto the front of the
> args.


First sorry for the confusion, I had 'mymsg'in your x method example in
mind, of course.

Second, thank you for the more detailed explanation. It closely
describes what is done during code execution by interpreter's point of
view. From the static/naive human code reader's point of view, I would
search for symbol 'mymsg' in current/parent/global scope including
imported modules 'if there is any assignment to this symbol': lvalue,
method definition, method's argument etc. If it's not found, thus
'mymsg' symbol is suspicious and I would notice about it. Similar
procedure I would expect from possibly naive but still handy code
checker.

Btw. I'm using (g)Vim editor and know about C-n shortcut but use it
sparsely because it offers too much symbols at times and it only slows
me down. Code completion C-x C-o makes much more sense but it doesn't
work in all cases. One of reasons of my typing errors are caused by
switching between various keyboards (MS Natural KB at PC and
significantly different at corporate notebook).

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      11-10-2010
On Wed, Nov 10, 2010 at 8:52 AM, David Unric <(E-Mail Removed)> wrote:
> Ryan Davis wrote in post #960452:
>> So let's try again, with the code sample reformatted better:
>>
>>> % echo 'def x; mymsg; end' | parse_tree_show

>>
>> begets:
>>
>>> s(:defn, , s(:args),
>>> =A0s(:scope,
>>> =A0 =A0s(:block,
>>> =A0 =A0 =A0s(:call, nil, :mymsg, s(:arglist)))))

>>
>> read in English: "method definition with no args. calls mymsg w/ no
>> args"
>>
>> So, ruby parsed "def x;" and realized it was a 0 arg method. The body
>> refers to "mymsg;" and since there was no previous variable assignment
>> of the same name, it must be a method call (with no args). That's it. At
>> that point, the method is "compiled" down to a single method invocation
>> for the message "mymsg". Notice I said "message"... that's an important
>> distinction between static and dynamic languages. In static languages
>> (or in dynamic/hybrid languages where the method is pre-calculated),
>> methods/functions are called by having a pointer to the function body
>> and simply jumping to that address.
>>
>> In a dynamic invocation, a receiver is sent "a message" at runtime. This
>> roughly translates to asking the hierarchy of classes if they implement
>> that message, and if so, to please execute it. If not, then the runtime
>> does the same thing, but this time around the message is
>> "method_missing" with the previous message pushed onto the front of the
>> args.

>
> First sorry for the confusion, I had 'mymsg'in your x method example =A0i=

n
> mind, of course.
>
> Second, thank you for the more detailed explanation. It closely
> describes what is done during code execution by interpreter's point of
> view. From the static/naive human code reader's point of view, I would
> search for symbol 'mymsg' in current/parent/global scope including
> imported modules 'if there is any assignment to this symbol': lvalue,
> method definition, method's argument etc. If it's not found, thus
> 'mymsg' symbol is suspicious and I would notice about it. Similar
> procedure I would expect from possibly naive but still handy code
> checker.


As we tried to explain, you cannot do any static lookups - not even
for constants:

09:35:26 Temp$ ruby19 -e 'class X;end;X.const_set("Foo"+"Bar",123);p X::Foo=
Bar'
123

Any tool trying to statically find X::FooBar would be lost yet the
code is perfectly legal and error free.

It's worse with methods since the you additionally get the inheritance
chain as an additional, orthogonal dimension for lookups.

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 
Reply With Quote
 
David Unric
Guest
Posts: n/a
 
      11-10-2010
Robert Klemme wrote in post #960467:
> On Wed, Nov 10, 2010 at 8:52 AM, David Unric <(E-Mail Removed)>
> wrote:
> 09:35:26 Temp$ ruby19 -e 'class X;end;X.const_set("Foo"+"Bar",123);p
> X::FooBar'
> 123
>
> Any tool trying to statically find X::FooBar would be lost yet the
> code is perfectly legal and error free.
>
> It's worse with methods since the you additionally get the inheritance
> chain as an additional, orthogonal dimension for lookups.
>
> Kind regards
>
> robert


I'm saying it from the start I don't expect the static checker would
throw positive warnings in 100% of cases. You demonstrated one example.
I know it depends on the programming style but I dare to claim such kind
of generic constant/variable creation is less frequent and so the static
checker would cover larger subset of cases. After a brief lookup to a
sample of standard Ruby libraries (written in Ruby, not in C) I didn't
find a class where a constant/variable/method name is generated "on the
fly" as you've pointed out. So false warnnings needn't to be too much.

Take care

David

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
Brian Candler
Guest
Posts: n/a
 
      11-11-2010
David Unric wrote in post #960444:
> My appologies I did not realized sooner the deeper conceptual
> differencies between these languages.
> Despite of that, if Ruby cann't decide if mymsg is a variable or a
> method call in advance, how does it rule out the possibility for a
> static parser to notify the 'mymsg' symbol has no assignment or method
> definiton in the source code ?


On the contrary: ruby *does* always decide (statically, at parse time,
not at run-time) whether a bareword is a local variable or a method
call, based on whether a previous assignment statement exists previously
in the same scope.

e.g.

def foo
if false
x = 123
end
puts x # prints nil - x is a local variable
puts y # NameError
puts z() # NoMethodError
end
foo

Although the assignment to x was never executed, it exists in the parse
tree before the point where it is referenced. Therefore at that point
bareword x is known to be a local variable. At runtime the value is nil
because no assignment was actually executed.

y is decided statically to be a method call, as the parsetree shows,
because no assignment was seen. Note however that the error message says
"undefined local variable or method `y'". The programmer might have
meant to assign to a local variable called y, or might have meant to
define a method called y. Neither was done, but Ruby has no way to tell
which you intended.

z() is known from the syntax definitely to be a method call, and so the
error is "undefined method `z'". The same would be true for self.z. It
doesn't matter whether a statement z=... exists earlier.

Unfortunately, because Ruby doesn't require you to annotate your code,
there is no additional information which could be used to highlight an
anomaly.

For example, in perl, scalar variables are flagged by $. If you do

$myarg = 123;
...
print $my_arg;

it can see that $my_arg is used only once, and is therefore likely an
error. With 'use strict' you are forced to declare your variables, so
every variable must have a matching declaration.

The non-verbose nature of Ruby, combined with the dynamic definition of
methods at runtime means that these tests simply can't be done. Dynamic
method definitions are very widely used - look at code which uses
ActiveRecord for instance. The methods in an object are based on the
columns in the database which the program connects to.

And what this means is, you need tests. Your test suite needs to execute
each line of code in your source at least once. A code coverage tool
like 'rcov' can help you achieve that, or if you are paranoid look at
'heckle'.

Is this harder or more expensive than static code checking? Only if you
take the point of view that you don't need unit tests.

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
JSP Code Review Tool or Syntax Analyzer? scottleff@hotmail.com Java 1 08-05-2011 06:36 AM
What was that Java code analyzer link someone posted a few days ago? Alex Hunsley Java 6 03-28-2006 03:20 PM
Source code analyzer tools for J2SE / J2EE code base Manfred Schneider Java 6 12-17-2005 12:39 PM
Errors, errors, errors Mark Goldin ASP .Net 2 01-17-2004 08:05 PM
Java code analyzer recommendations? Harpstein Java 7 11-18-2003 09:43 PM



Advertisments