Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > VHDL > Is there a way to make testbenches assume simulation failure as default?

Reply
Thread Tools

Is there a way to make testbenches assume simulation failure as default?

 
 
py
Guest
Posts: n/a
 
      11-05-2012
Hi,

I'm trying to make all my testbench such that when the test finished, they would all indicated the simulation failured (assert, severity error or failure), unless the simulation explicitly arrived at the test vector comparison routine (and passed).

Proof-positive method would allow me to catch cases where the buses hung, or the sim logic is stuck in a loop.

What VHDL construct in a testbench would accomplish this? I can't think of anything that would kicks in and check a status flag at the end of simulation (since the testbench itself has no knowledge of the user-defined simulation time). Post-processing would work, but I want to know if there is a wayto keep everything inside the testbench.


Thanks
 
Reply With Quote
 
 
 
 
Rob Gaddi
Guest
Posts: n/a
 
      11-05-2012
What about making the last line of your testbench report out "*** SIMULATION COMPLETE ***"? If it gets to that line, then it got to that line.

Or am I missing something?

-- Rob


On Mon, 5 Nov 2012 10:34:07 -0800 (PST)
py <(E-Mail Removed)> wrote:

> Hi,
>
> I'm trying to make all my testbench such that when the test finished, they would all indicated the simulation failured (assert, severity error or failure), unless the simulation explicitly arrived at the test vector comparison routine (and passed).
>
> Proof-positive method would allow me to catch cases where the buses hung, or the sim logic is stuck in a loop.
>
> What VHDL construct in a testbench would accomplish this? I can't think of anything that would kicks in and check a status flag at the end of simulation (since the testbench itself has no knowledge of the user-defined simulation time). Post-processing would work, but I want to know if there is a way to keep everything inside the testbench.
>
>
> Thanks



--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.
 
Reply With Quote
 
 
 
 
py
Guest
Posts: n/a
 
      11-05-2012
quick Google Group test, just post a reply and it didn't show up
 
Reply With Quote
 
py
Guest
Posts: n/a
 
      11-05-2012
Hi Rob,

A bit more context to my original question. The goal here is to make the simulator itself awares of the failure (simualtor logs fatal asserts), so that an outer layer sim environment can track overall testcase pass and fail.

Currently, a test passed is expressed as a lack of fatal assert at the end of the sim time. But this methology has a loophole where since it will not pick up tests that actually hung. So I'm looking for a VHDL construct that operates opposite to fatal assert.

If such construct doesn't exist nor standardized, then I would have to parse out "SIMULATION COMPLETE" from the sim console log.


Thanks,


On Monday, 5 November 2012 11:30:47 UTC-8, Rob Gaddi wrote:
> What about making the last line of your testbench report out "*** SIMULATION COMPLETE ***"? If it gets to that line, then it got to that line. Or amI missing something? -- Rob On Mon, 5 Nov 2012 10:34:07 -0800 (PST) py <(E-Mail Removed)> wrote: > Hi, > > I'm trying to make all my testbench such that when the test finished, they would all indicated the simulation failured (assert, severity error or failure), unless the simulation explicitly arrived at the test vector comparison routine (and passed). > > Proof-positive method would allow me to catch cases where the buses hung, or the sim logic is stuck in a loop. > > What VHDL construct in a testbench would accomplish this? I can't think of anything that would kicks in and check a status flag at the end of simulation (since the testbench itself has no knowledge of the user-defined simulation time). Post-processing would work, but I want to know if there is a way to keep everything inside the testbench. > > > Thanks -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix.


 
Reply With Quote
 
Andy
Guest
Posts: n/a
 
      11-05-2012
On Monday, November 5, 2012 1:52:50 PM UTC-6, py wrote:
> Hi Rob, A bit more context to my original question. The goal here is to make the simulator itself awares of the failure (simualtor logs fatal asserts), so that an outer layer sim environment can track overall testcase pass and fail. Currently, a test passed is expressed as a lack of fatal assert at the end of the sim time. But this methology has a loophole where since itwill not pick up tests that actually hung. So I'm looking for a VHDL construct that operates opposite to fatal assert. If such construct doesn't exist nor standardized, then I would have to parse out "SIMULATION COMPLETE" from the sim console log.


OK, so your testbench is capable of knowing whether all tests actually ran and passed (some sort of scoreboard)? If that is the case, can you then (and only then) report success (severity note), and shut down all clocks, using event starvation to stop the simulation?

Event starvation occurs when there are no more events scheduled on any signals for which there is a sensitive process. There are also events from "wait for..." statements and from waveforms using "after...". that are essentially events scheduled at some simulation time in the future.

Note, some simulators report different conditions (note, warning, error, failure) upon event starvation, so this may not be any different from what you are getting now. Also event starvation can be difficult to cause if you have various clocks created in different parts of the test bench, and no wayto communicate to them to stop the clock(s).

Andy
 
Reply With Quote
 
Rob Gaddi
Guest
Posts: n/a
 
      11-06-2012
On Mon, 5 Nov 2012 11:52:50 -0800 (PST)
py <(E-Mail Removed)> wrote:

> Hi Rob,
>
> A bit more context to my original question. The goal here is to make the simulator itself awares of the failure (simualtor logs fatal asserts), so that an outer layer sim environment can track overall testcase pass and fail.
>
> Currently, a test passed is expressed as a lack of fatal assert at the end of the sim time. But this methology has a loophole where since it will not pick up tests that actually hung. So I'm looking for a VHDL construct that operates opposite to fatal assert.
>
> If such construct doesn't exist nor standardized, then I would have to parse out "SIMULATION COMPLETE" from the sim console log.
>
>
> Thanks,
>


HANGUP_MONITOR: process
begin
wait for 1 ms;
report "Simulation took too damn long, something's broken." severity failure;
std.env.stop(0)
end process HANGUP_MONITOR;

--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.
 
Reply With Quote
 
HT-Lab
Guest
Posts: n/a
 
      11-06-2012
On 06/11/2012 01:38, Rob Gaddi wrote:
> On Mon, 5 Nov 2012 11:52:50 -0800 (PST)
> py <(E-Mail Removed)> wrote:
>
>> Hi Rob,
>>
>> A bit more context to my original question. The goal here is to make the simulator itself awares of the failure (simualtor logs fatal asserts), so that an outer layer sim environment can track overall testcase pass and fail.
>>
>> Currently, a test passed is expressed as a lack of fatal assert at the end of the sim time. But this methology has a loophole where since it will not pick up tests that actually hung. So I'm looking for a VHDL construct that operates opposite to fatal assert.
>>
>> If such construct doesn't exist nor standardized, then I would have to parse out "SIMULATION COMPLETE" from the sim console log.
>>
>>
>> Thanks,
>>

>
> HANGUP_MONITOR: process
> begin
> wait for 1 ms;
> report "Simulation took too damn long, something's broken." severity failure;
> std.env.stop(0)
> end process HANGUP_MONITOR;
>


Hi Py,

If you are using Modelsim then you can pass this failure to your "outer
layers" using the "exit -code xx" command and then take appropriate action.

For example, if you want a good old DOS batch file to detect you have an
error then you can use the Modelsim "when" command to trigger a
breakpoint followed by the "exit" command to pass an error code to your
batch file.

Here is some pseudo code:

VHDL source:

process
begin
end_of_sim <= '1';
end process;

Modelsim do/tcl file:

set dosexit 0
when -label end_of_simulation {end_of_sim == '1'} {stop ; set dosexit 1}
run -all

if { $dosexit } {
puts "Regression Failed"
exit -code $dosexit
} else {
puts "Regression Passed"
}
quit -f

DOS batch file:

vsim -c xxx_tb -do "source run.do"
IF %ERRORLEVEL% EQU 0 GOTO NOERROR:
ECHO "**** VSIM returned an error ***"
:NOERROR


Hans
www.ht-lab.com
 
Reply With Quote
 
rickman
Guest
Posts: n/a
 
      11-06-2012
On 11/5/2012 2:52 PM, py wrote:
> Hi Rob,
>
> A bit more context to my original question. The goal here is to make the simulator itself awares of the failure (simualtor logs fatal asserts), so that an outer layer sim environment can track overall testcase pass and fail.
>
> Currently, a test passed is expressed as a lack of fatal assert at the end of the sim time. But this methology has a loophole where since it will not pick up tests that actually hung. So I'm looking for a VHDL construct that operates opposite to fatal assert.
>
> If such construct doesn't exist nor standardized, then I would have to parse out "SIMULATION COMPLETE" from the sim console log.


As others indicate, this will depend on your program. I don't know of
any way to change the default exit from the simulator.

But you should be able to find a place in your code, a "safety monitor"
if you will, that "knows" all of the internal tests have passed and
flags a NOTE about the "good" exit. Absent the "good" exit the
simulation monitor could assume the simulation was hung.

To make sure the safety monitor "knows" about the simulation, it would
require that every important process be designed for communication with
the safety monitor.

Rick
 
Reply With Quote
 
KJ
Guest
Posts: n/a
 
      11-07-2012
On Monday, November 5, 2012 2:52:50 PM UTC-5, py wrote:
> Hi Rob, A bit more context to my original question. The goal
> here is to make the simulator itself awares of the failure
> (simualtor logs fatal asserts), so that an outer layer sim
> environment can track overall testcase pass and fail. Currently,
> a test passed is expressed as a lack of fatal assert at the end
> of the sim time. But this methology has a loophole where since
> it will not pick up tests that actually hung. So I'm looking for
> a VHDL construct that operates opposite to fatal assert. If such
> construct doesn't exist nor standardized, then I would have to
> parse out "SIMULATION COMPLETE" from the sim console log.


One method is to inspect your testbench and make sure that each wait statement has a timeout. Rather than this...

wait until (This = That);
....continue on

Instead you do this...

wait until (This = That) for 1 us;
assert (This = That) report "Timeout waiting for this to equal that" severity ERROR;

This will prevent the problem you describe. The downside can be having to know how long is 'too long' to use for the timeout and having to search through the testbench for all of the wait statements and modify them appropriately. The big plus though is that when the bus hangs, the simulator not only stops but the process that is hung has just asserted so you know exactlywhat condition you were waiting for.

The other approach is to have some form of external monitoring. This can take the form of monitoring activity on a bus. When activity goes away for 'too long' then the simulator asserts. This can be problematic though if there are busses that are naturally idle during certain tests so figuring out an appropriate timeout value can be a problem. It's maybe less upfront work then the previous approach of adding timeouts/asserts to each wait statement but it is also of less value too. All you know when the hang causes an assertion is that the bus stopped, you don't know which process is actually hung up waiting.

Kevin Jennings
 
Reply With Quote
 
py
Guest
Posts: n/a
 
      11-09-2012
On Monday, 5 November 2012 10:34:08 UTC-8, py wrote:
> Hi,
>
>
>
> I'm trying to make all my testbench such that when the test finished, they would all indicated the simulation failured (assert, severity error or failure), unless the simulation explicitly arrived at the test vector comparison routine (and passed).
>
>
>
> Proof-positive method would allow me to catch cases where the buses hung,or the sim logic is stuck in a loop.
>
>
>
> What VHDL construct in a testbench would accomplish this? I can't think of anything that would kicks in and check a status flag at the end of simulation (since the testbench itself has no knowledge of the user-defined simulation time). Post-processing would work, but I want to know if there is a way to keep everything inside the testbench.
>
>
>
>
>
> Thanks


Thank you all for your helpful suggestions. In the end we found a way in our outer sim environment to detect the difference between exit on end of simtime and exit on user-triggered stop. So that served our usage.

Rob's HANGUP_MONITOR would also work well.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Date#parse assume a EU format - can it assume a US format? Josh Sharpe Ruby 1 09-21-2010 09:16 AM
Is it safe to assume default value from dynamic memory allocation? howa C++ 7 12-03-2006 07:44 AM
Question: Writing text file based TestBenches vs. Waveform file based simulation. BLF VHDL 4 08-07-2004 12:44 AM
Developing testbenches with ISE & Modelsim AndyAtHome VHDL 0 06-02-2004 08:36 PM
a newbie question about modelsim and testbenches Yttrium VHDL 1 10-31-2003 02:41 PM



Advertisments