Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > c++ compiler

Reply
Thread Tools

c++ compiler

 
 
BGB
Guest
Posts: n/a
 
      08-06-2011
On 8/6/2011 6:38 AM, Dombo wrote:
> Op 06-Aug-11 11:42, Ian Collins schreef:
>> On 08/ 6/11 08:42 PM, Dombo wrote:
>>> Op 06-Aug-11 5:25, Ian Collins schreef:
>>>> On 08/ 6/11 01:30 PM, BGB wrote:
>>>>> On 8/5/2011 3:20 PM, Ian Collins wrote:
>>>>>> On 08/ 6/11 10:10 AM, BGB wrote:
>>>>>>> On 8/5/2011 2:53 PM, Ian Collins wrote:
>>>>>>>> On 08/ 5/11 11:58 PM, Victor Bazarov wrote:
>>>>>>>>> On 8/5/2011 5:41 AM, Miles Bader wrote:
>>>>>>>>>> [..]
>>>>>>>>>> My general issue with the VS debugger was that very often there
>>>>>>>>>> would
>>>>>>>>>> be cases where you could _see_ interesting data, but not be
>>>>>>>>>> able to
>>>>>>>>>> manipulate it, or be able to manipulate it, but only rather
>>>>>>>>>> awkwardly
>>>>>>>>>> (o-n-e-s-t-e-p-a-t-a-t-i-m-e... argh!) For complex debugging
>>>>>>>>>> tasks,
>>>>>>>>>> this quickly became absolutely miserable
>>>>>>>>>>
>>>>>>>>>> With gdb, on the other hand, which is largely based on expression
>>>>>>>>>> evaluation, while it was harder to visualize data, manipulation
>>>>>>>>>> of it
>>>>>>>>>> was vastly faster and easier (and more easily repeatable; often
>>>>>>>>>> one
>>>>>>>>>> wants to do the weird manipulation several times).
>>>>>>>>>
>>>>>>>>> In other words, with VC++'s debugger you're actually deBUGging,
>>>>>>>>> finding
>>>>>>>>> out what's wrong, one step at a time, stopping to think,
>>>>>>>>> exiting to
>>>>>>>>> change the code, compile, run again, etc.. With gdb you're
>>>>>>>>> actually
>>>>>>>>> developing by tweaking the data, tweaking the program, altering
>>>>>>>>> execution, etc.. Yes, no, maybe? It's a style debate, not the
>>>>>>>>> quality
>>>>>>>>> of tools debate.
>>>>>>>>
>>>>>>>> If you need a debugger, your unit tests aren't good enough!
>>>
>>> Unit tests never are. Even when the code coverage is 100% there is still
>>> no guarantee that you have covered all possible execution flows.

>>
>> There should be if you wrote the tests first.

>
> If you believe that than those tests give you a false sense of security.
> If you write the test first (as I do) you still have no guarantee it
> covers every possible scenario (even when the code coverage is 100%).
> Usually code fails on scenarios that weren't anticipated, rather than
> the scenario's that were anticipated (and tested). As long as tests (and
> specifications for that matter) are created by humans chances are that
> they are flawed as well.
>
>>>>>>>> Dives for cover
>>>>>>>
>>>>>>> unit tests wont help finding out where and why one is experiencing a
>>>>>>> segfault or similar...
>>>
>>> Unless you can reproduce the problem with your unit test (extend it if
>>> needed).
>>>
>>>>>> If a change causes a crash, back it out and try again...
>>>>>
>>>>> or, just invoke the debugger and see why it has crashed...
>>>>
>>>> If that's quicker than redoing the change, you are changing too much
>>>> between test runs.
>>>
>>> Invoking a debugger to see the point where a process has crashed, see
>>> the call stack and see the variables takes less than a second. How many
>>> changes can you make in that time, recompile and running your unit
>>> tests?

>>
>> That all depends whether the platform (and language) you are using has a
>> decent debugger.

>
> If it doesn't I consider that a handicap, no matter how good the unit
> tests are. Sometimes things you depend on just don't work as advertised,
> a decent debugger can help analyzing what is really going a lot.
> Debuggers don't make unit tests obsolete, nor vise versa.
>


yep.

unit tests are good at testing that the thing does what it is defined as
doing, so one can write out a basic spec, and tests to make sure that
behavior is what it is supposed to be (most often, some things are
harder to run automated tests on than others).

debuggers are better at finding random errors and edge cases which may
not have been adequately tested for, or which may fall outside the scope
of its defined behavior, for example:
what happens when the API is called prior to initialization?
what happens if the API's Init functions are called during operation or
multiple times?
what happens if NULL is passed in unexpected places?
....

these things are not normally tested in unit tests, which typically
handle defined behavior and use, and not anomalous behavior and
use-patterns (for which there may be far more anomalous patterns to test
for than to test for defined behavior, but programming errors are often
much better at stepping on these edge cases).

or such...
 
Reply With Quote
 
 
 
 
Ian Collins
Guest
Posts: n/a
 
      08-06-2011
On 08/ 7/11 01:38 AM, Dombo wrote:
> Op 06-Aug-11 11:42, Ian Collins schreef:
>>>
>>> Unit tests never are. Even when the code coverage is 100% there is still
>>> no guarantee that you have covered all possible execution flows.

>>
>> There should be if you wrote the tests first.

>
> If you believe that than those tests give you a false sense of security.


Hence my use of "should". One of these days I will write the perfect
set of tests. Until then I still end up resorting to a debugger from
time to time. I still consider this to be a failure in my process.

> If you write the test first (as I do) you still have no guarantee it
> covers every possible scenario (even when the code coverage is 100%).
> Usually code fails on scenarios that weren't anticipated, rather than
> the scenario's that were anticipated (and tested). As long as tests (and
> specifications for that matter) are created by humans chances are that
> they are flawed as well.


Very true. Just make sure to add a test to show the new scenario fails
before fixing it.

--
Ian Collins
 
Reply With Quote
 
 
 
 
none
Guest
Posts: n/a
 
      08-09-2011
In article <j1k35l$uhv$(E-Mail Removed)>, BGB <(E-Mail Removed)> wrote:
>
>yep.
>
>unit tests are good at testing that the thing does what it is defined as
>doing, so one can write out a basic spec, and tests to make sure that
>behavior is what it is supposed to be (most often, some things are
>harder to run automated tests on than others).
>
>debuggers are better at finding random errors and edge cases which may
>not have been adequately tested for, or which may fall outside the scope
>of its defined behavior, for example:
>what happens when the API is called prior to initialization?
>what happens if the API's Init functions are called during operation or
>multiple times?
>what happens if NULL is passed in unexpected places?
>...
>
>these things are not normally tested in unit tests, which typically
>handle defined behavior and use, and not anomalous behavior and
>use-patterns (for which there may be far more anomalous patterns to test
>for than to test for defined behavior, but programming errors are often
>much better at stepping on these edge cases).


Wow! Ian was a bit tongue-in-cheek with his unit test comments but
you are way off the mark. Unit tests are absolutely not meant to only
cover normal execution path. In fact, you will never reach high code
coverage stats if you only unit test the normal success path. The
proportion of code that exist to handle error is huge. If you write
it, test it.

You should have a unit test for calling the API prior to
initialisation (assuming your API requires initialisation)
You should have unit tests to verify that calling init more that
once.
If your interface takes pointers, you should test what happens with
NULL pointers.
Etc.

You probably never reach 100% coverage but really, at least try to
cover the basic failure scenarios you listed above. Plus you should
run your unit test in a memory analyser so that leaks and boundary
violation are detected.

That said, I have nothing against debuggers. I've even been know to
run through some unit tests in a debuggers

Yannick


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Compiler Error Message: The compiler failed with error code -1073741819 Ram ASP .Net 0 09-13-2005 09:52 AM
Why is a JIT compiler faster than a byte-compiler RickMuller Python 4 03-26-2005 04:30 PM
Compiler compiler with C++ as output Andrey Batyuck C++ 3 05-17-2004 08:17 PM
Can we use <compiler> tag to avoid RunTime Compiler error? Jack Wright ASP .Net 5 01-19-2004 04:36 PM
Compiler Error Message: The compiler failed with error code 128. Yan ASP .Net 0 07-21-2003 10:49 PM



Advertisments