Survey: simulator usage

Apologies for the (slightly) commercial nature of this post.

Those of you with long memories may recall coding devices in ABEL. One
nice thing about ABEL was that you could write very simple vector
files to simulate your device, where a vector was something like

[C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]

ie. setup some inputs, apply a clock pulse, and check some outputs.
I've written some software over the years that lets me do the same
thing in VHDL, with various extensions, and I use it to test most of
my RTL code. It's simple, you don't need to write or know *any* VHDL
to use it, and it gives you a pass/fail very quickly, for a module or
a whole device.

I'm thinking about brushing this up a bit, adding Verilog support, and
flogging it for maybe 100 - 300 USD a go. To use it, you obviously
still need a simulator - the software currently produces VHDL-only
output, and uses your simulator to simulate your chip using the
auto-generated verification code.

This brings me to my problem. I can make the software a lot more
sophisticated if I can generate C code, as well as (or instead of) the
VHDL or Verilog. There are some testbenchy things which are just very
difficult to do in pure VHDL or Verilog. *But*, most of the potential
users of this software will be FPGA coders with a cheap simulator that
doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
for example, doesn't do this, and presumably the FPGA-specific
simulators don't do this either).

What I'd really like to find out, if you can spare the time and this
might be of interest to you, is:

* What simulator do you use?

* Is your RTL code in Verilog/VHDL/both?

* Does your simulator have a C-language interface? From Verilog, or
VHDL, or both?

* If your simulator doesn't support C, would you be willing to upgrade
it to use a product of this sort? Or would you prefer to get pure VHDL
or Verilog out of this software, even if it means reduced vector file
functionality?

As a bonus, if you add the line "this is a great idea and I claim my
50% discount", then you can have 50% off the (initial) purchase price,
if I ever get around to doing this.

You can reply here or directly to me at 'unet+50' 'at'
'riverside-machines' 'dot' 'com'.

Thanks -

Evan

Evan Lavelle

"Evan Lavelle" <> wrote in message
news:...
...

> This brings me to my problem. I can make the software a lot more
> sophisticated if I can generate C code, as well as (or instead of) the
> VHDL or Verilog.

What about supporting Tcl, most if not all simulators support this language.
You can force and examine signal using Modelsim's Tcl.

> There are some testbenchy things which are just very
> difficult to do in pure VHDL or Verilog.

Such as?

> *But*, most of the potential
> users of this software will be FPGA coders with a cheap simulator that
> doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
> for example, doesn't do this, and presumably the FPGA-specific
> simulators don't do this either).

You can get a SystemC license for Modelsim PE which IMHO is much much easier
to use than the FLI/PLI.

>
> What I'd really like to find out, if you can spare the time and this
> might be of interest to you, is:
>
> * What simulator do you use?

Modelsim SE.

>
> * Is your RTL code in Verilog/VHDL/both?

VHDL.

>
> * Does your simulator have a C-language interface? From Verilog, or
> VHDL, or both?

Yes, SE comes with the FLI which I have used in the past until I discovered
SystemC

>
> * If your simulator doesn't support C, would you be willing to upgrade
> it to use a product of this sort?

No since I don't understand the advantages of your product, also changing PE
to SE or adding a SystemC license to PE is not particular cheap and it might
be more cost effective to spend some more time on your testbench

Hans
www.ht-lab.com

Hans

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Evan Lavelle wrote:

> This brings me to my problem. I can make the software a lot more
> sophisticated if I can generate C code, as well as (or instead of) the
> VHDL or Verilog. There are some testbenchy things which are just very
> difficult to do in pure VHDL or Verilog. *But*, most of the potential
> users of this software will be FPGA coders with a cheap simulator that
> doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
> for example, doesn't do this, and presumably the FPGA-specific
> simulators don't do this either).

Cheap/free simulators *should* support some sort of C interface.
Icarus Verilog supports VPI and I'd be astonished if the bundled
Verilog compilers w/ FPGA vendor tools doesn't also support VPI.

- --
Steve Williams "The woods are lovely, dark and deep.
steve at icarus.com But I have promises to keep,
http://www.icarus.com and lines to code before I sleep,
http://www.picturel.com And lines to code before I sleep."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFFQjburPt1Sc2b3ikRAlWSAJ4z5s8Hn9mQ4B0U8IevKZ zmOsCkDQCg500z
EziQS9sPImvMVTE3FP0T3f0=
=tv0x
-----END PGP SIGNATURE-----

Stephen Williams

Evan Lavelle wrote:
> Apologies for the (slightly) commercial nature of this post.
>
> Those of you with long memories may recall coding devices in ABEL. One
> nice thing about ABEL was that you could write very simple vector
> files to simulate your device, where a vector was something like
>
> [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]
>
> ie. setup some inputs, apply a clock pulse, and check some outputs.
> I've written some software over the years that lets me do the same
> thing in VHDL, with various extensions, and I use it to test most of
> my RTL code. It's simple, you don't need to write or know *any* VHDL
> to use it, and it gives you a pass/fail very quickly, for a module or
> a whole device.

So this requires you generate all outputs ?
If the sim does fail, what format is the failure report, and
how does the user relate that back to a given line of test source code ?

>
> I'm thinking about brushing this up a bit, adding Verilog support, and
> flogging it for maybe 100 - 300 USD a go. To use it, you obviously
> still need a simulator - the software currently produces VHDL-only
> output, and uses your simulator to simulate your chip using the
> auto-generated verification code.
>
> This brings me to my problem. I can make the software a lot more
> sophisticated if I can generate C code, as well as (or instead of) the
> VHDL or Verilog. There are some testbenchy things which are just very
> difficult to do in pure VHDL or Verilog. *But*, most of the potential
> users of this software will be FPGA coders with a cheap simulator that
> doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
> for example, doesn't do this, and presumably the FPGA-specific
> simulators don't do this either).
>
> What I'd really like to find out, if you can spare the time and this
> might be of interest to you, is:

Interesting idea.
What about doing a simple VHDL/Verilog/Demo-C version that is free, and
a smarter version that is $$ ?

A problem with this type of tool, is explaining to each user how
its feature set can offer more to a given task, so the free/simple
versions ( and good examples) do that.

For those who want it, I think most? FPGA vendors have (free) waveform
entry for simulation entry, so that is one competition point.

There is also this work by Jan Decaluwe, that uses Python for
both verilog generation, and testbench generation.
http://myhdl.jandecaluwe.com/doku.php/start
http://myhdl.jandecaluwe.com/doku.ph...book:stopwatch

-jg

Jim Granville

Evan Lavelle <> writes:

> Apologies for the (slightly) commercial nature of this post.
>
> Those of you with long memories may recall coding devices in ABEL. One
> nice thing about ABEL was that you could write very simple vector
> files to simulate your device, where a vector was something like
>
> [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]
>
> ie. setup some inputs, apply a clock pulse, and check some outputs.
> I've written some software over the years that lets me do the same
> thing in VHDL, with various extensions, and I use it to test most of
> my RTL code. It's simple, you don't need to write or know *any* VHDL
> to use it, and it gives you a pass/fail very quickly, for a module or
> a whole device.

If you really mean such a simple format, my answer is: don't bother,
that way is obsolete for anything but the most trivial designs. Given
the amount of testing and the complexity of many interfaces/busses
today, I need to write my test vectors at a higher level than applying
a bitvector per clock cycle.

In order to do that, I normally create a library of high-level
functions in a scripting language like Perl or Tcl, which would do
"the dirty work" for me. Normally I would make the script library
interface directly to a VHDL/Verilog testbench through TEXTIO files.
With your extention, I would still have to make the script library,
but just have a "backend" to that emits files compatible with your
format. All I'd save is to write my VHDL testbench for the interface
(which may could have serious performance impacts, since you make the
simulator stop for each clock cycle and is a real no-no for
performance), and frankly, writing the VHDL testbench isn't the
problem.

> doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
> for example, doesn't do this, and presumably the FPGA-specific
> simulators don't do this either).

Affirmative (at least for the XE version).

> * What simulator do you use?

Modelsim PE / XE

> * Is your RTL code in Verilog/VHDL/both?

VHDL, but we have to deal with Verilog as well due to backannotated GL
sims.

> * If your simulator doesn't support C, would you be willing to upgrade
> it to use a product of this sort?

Considering that the list price of Modelsim SE is 170K DKK vs 70K DKK
for a PE license (both floating licenses), not bloody likely.

> Or would you prefer to get pure VHDL
> or Verilog out of this software, even if it means reduced vector file
> functionality?

With a little imagination, you can do a surprising amount intelligent
work with "dumb" TEXTIO vector files, by carefully dividing the work
between script library (vector generation) text files, and the
testbench itself.


Kai
--
Kai Harrekilde-Petersen <khp(at)harrekilde(dot)dk>

Kai Harrekilde-Petersen

Evan Lavelle wrote:
> Those of you with long memories may recall coding devices in ABEL. One
> nice thing about ABEL was that you could write very simple vector
> files to simulate your device, where a vector was something like
>
> [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]
>
> ie. setup some inputs, apply a clock pulse, and check some outputs.
> I've written some software over the years that lets me do the same
> thing in VHDL, with various extensions, and I use it to test most of
> my RTL code. It's simple, you don't need to write or know *any* VHDL
> to use it, and it gives you a pass/fail very quickly, for a module or
> a whole device.

This sort of test-vector entry is basically unmaintainable, and to be
honest, I haven't written this type of go/no-go test in ages (maybe
since the last time I used ABEL).

I prefer to write test benches at a higher level. I like to have
models of the things to which the DUT connects drive the DUT. For
example, if my FPGA is a NAND flash controller that sits between a
microprocessor bus and the NAND flash devices, I'll use a
bus-functional model of the micro and the flash, and my test bench will
mostly consist of MicroWriteBus() and MicroReadBus() "commands."

I can't go back to the simple drive-a-vector test.

-a

Andy Peters

>
> What I'd really like to find out, if you can spare the time and this
> might be of interest to you, is:
>
> * What simulator do you use?

NC-Verilog

>
> * Is your RTL code in Verilog/VHDL/both?

Verilog

>
> * Does your simulator have a C-language interface? From Verilog, or
> VHDL, or both?

I'm sure it does

>
> * If your simulator doesn't support C, would you be willing to upgrade
> it to use a product of this sort? Or would you prefer to get pure VHDL
> or Verilog out of this software, even if it means reduced vector file
> functionality?

Frankly, as an ASIC designer. I probably wouldn't use this language. In
the future, we plan to use System Verilog.

>
> As a bonus, if you add the line "this is a great idea and I claim my
> 50% discount", then you can have 50% off the (initial) purchase price,
> if I ever get around to doing this.
>
> You can reply here or directly to me at 'unet+50' 'at'
> 'riverside-machines' 'dot' 'com'.
>
> Thanks -
>
> Evan

bronzefury

On Sat, 28 Oct 2006 08:18:51 +1300, Jim Granville
<> wrote:

>So this requires you generate all outputs ?

Only if you actually need to test them. If a given vector file wants
to check a specific port, but that port is a don't care for most of
the vectors, then you just enter a '-' for that port in the expected
outputs. You could also specify the expected outputs algorithmically,
if possible: you can declare variables and do arithmetic on them, and
use them as module inputs or test values.

>If the sim does fail, what format is the failure report, and
>how does the user relate that back to a given line of test source code ?

Here's an example from a chip I've just done. Line 392 of one of the
vector files looks like (note that I've put in a line break to try to
avoid wrapping problems on your newsreader):

[11C 1 0011 0 5A5A5A5A xxxxxxx xxxxxxx zzzzzzzz zzzzzzzz] ->
[-0 -------- ---- EE FF 011 5A5A5A5A 011 5A5A5A5A ------- -- --]

If I change the last '5A5A5A5A' on the expected outputs to '5B5A5A5A',
then ModelSim reports:

# ** Note: L2HDB error: line 392 (expected 0x5b5a5a5a, got 0x5a5a5a5a)
# Time: 4381 ns Iteration: 0 Instance: /testbench
# ** Note: simulation complete: 305 out of 306 vectors passed.
# Time: 7007400 ps Iteration: 0 Instance: /testbench

The pin's identified, the line of the vector file, and the time.

>Interesting idea.
> What about doing a simple VHDL/Verilog/Demo-C version that is free, and
>a smarter version that is $$ ?
>
> A problem with this type of tool, is explaining to each user how
>its feature set can offer more to a given task, so the free/simple
>versions ( and good examples) do that.

Hmmm... maybe a 'linear' version (ie. no smarts) for free, and
looping/variables/procedures/macros etc. for $$? My recollection is
that ABEL just gave you the dumb version, but it seemed useful at the
time.

Yes, explaining that this is (hopefully) useful is difficult. I went
through this with a friend recently, who never writes testbenches, and
who spent years on ABEL. He wasn't impressed...

> For those who want it, I think most? FPGA vendors have (free) waveform
>entry for simulation entry, so that is one competition point.

I've never got this. Does anyone actually do that?

> There is also this work by Jan Decaluwe, that uses Python for
>both verilog generation, and testbench generation.
>http://myhdl.jandecaluwe.com/doku.php/start
>http://myhdl.jandecaluwe.com/doku.ph...book:stopwatch

Yes, I know of Jan's stuff, but hadn't realised he also does TB
generation - I'll have a look at that.

Evan

Evan Lavelle

On 27 Oct 2006 15:53:42 -0700, "Andy Peters" <>
wrote:

>This sort of test-vector entry is basically unmaintainable, and to be
>honest, I haven't written this type of go/no-go test in ages (maybe
>since the last time I used ABEL).
>
>I prefer to write test benches at a higher level. I like to have
>models of the things to which the DUT connects drive the DUT. For
>example, if my FPGA is a NAND flash controller that sits between a
>microprocessor bus and the NAND flash devices, I'll use a
>bus-functional model of the micro and the flash, and my test bench will
>mostly consist of MicroWriteBus() and MicroReadBus() "commands."
>
>I can't go back to the simple drive-a-vector test.

Yes, makes sense; I've got no problem with high-level testbenches. If
a device or module has lots of buried state, and it's not obvious when
any outputs or observable points should change as a result of an input
change, or even *what* that change should be, then a test vector
approach is pretty useless. If your problem is, for example, apply an
opcode to a CPU and check that a memory location changes at some
arbitrary later time, then you need to be a lot smarter. But, horses
for courses. Consider:

- Your complex DUT is probably built of simpler modules. If a module
was just an FSM, a register block, a decoder, or a bus interface, for
example, and you could test the module simply by writing
20/50/100/whatever vectors, would it be worthwhile? Or do you just
stitch the whole thing together and test the top level?

- Many complete chips are surprisingly simple (seen from the outside,
anyway). I first used this code for a (large) DSP chain for baseband
processing. I got MATLAB data for the inputs, with expected data for
the outputs. Once I'd automated the test, verification was trivial.
Seven years on, and I've just finished part of a Spartan-3. Nothing
fancy - 25K marketing-gates - it just translates accesses between two
different buses connected to 15 external chips, handles bursts, has a
register block with control and status ops, does some address mapping
between the various chips, and so on. I did a basic test in a day by
writing 200-odd vectors, and I had a complete-ish test of 752 vectors
with a bit more work. Whatever way I'd tested it, these 752 vectors
would have existed somewhere, if only implicitly in the VHDL code, or
even in the documentation. Why not make the vectors themselves the
test? The rest of it is just unnecessary verbiage that you have to
repeat again, and again, for each new module and device you do.

- What are MicroWriteBus() and MicroReadBus()? I can do macros and
pass parameters to the macros; you can call the macros from wherever
you want in the vector file. I can also do basic C-like control
structures - looping, branching based on tested values, and so on.

Ok, is it worth any more than $0 now?

Evan

Evan Lavelle

On Fri, 27 Oct 2006 15:39:39 GMT, "Hans" <> wrote:

>No since I don't understand the advantages of your product, also changing PE
>to SE or adding a SystemC license to PE is not particular cheap and it might
>be more cost effective to spend some more time on your testbench

Ok, here's a really trivial example. You've just coded a D-type F/F -
how do you test it?

- Option 1: code a testbench module, instantiate your D-type, provide
a clock and some way to apply inputs (in this case, probably just
explicitly driving the pins sequentially as required), check that the
output is as required, inform the user that the test passed or failed.
If it failed, tell the user where. Not difficult, but tedious. Also
much more tedious if you need to do a timing sim, and you have to
ensure that the inputs are driven only for the required setup and hold
times, and the outputs only change between tCOmin and tCOmax. It's
also not productive; this is all code you've written a thousand times
before. And it may have a bug in it, and the testbench itself must be
tested before you can use it to test your DUT.

- Option 2:

([CLK, RST, D] -> [Q, QB])
[x, 0, x] -> [0, 1]; async reset
[C, 1, 1] -> [1, 0]; clock 1
[C, 1, 0] -> [0, 1]; clock 0

As I said, it's trivial. You don't need to know any VHDL or Verilog,
and you don't have to write a test program. All that tedious stuff has
been done for you. Of course, You're not going to test a D-type, but
there are certainly far more complex cases where this is useful. And
there are more complex cases where this isn't useful; see my reply to
Andy.

Evan

Evan Lavelle