RFC: VHDL testbench enhancements

Hi,
Let me try this again.

The VHDL standards community has been considering whether
to enhance VHDL to add advanced testbench features.

If you are a VHDL user,
Do you want these features added to VHDL?
Would you rather adopt a verification language that
already supports these (SystemVerilog, SystemC, E, Vera).

I think VHDL needs these features to stay competitive.

The current plan in the VHDL working group is to make
these features similar to other verification languages
while at the same time keeping the nature of VHDL.

We need your to voice your support. You can post here,
send your reply to me (let me know if I can use either
your name and/or company name when I tally the results
for the Accellera VHDL TSC), or join the Accellera VHDL
TSC (which you can do as a non-Accellera member by
registering) and post your reply there.

More on the politics of the situation are below.

Thanks,
Jim Lewis
VHDL Evangelist
SynthWorks VHDL Training



P.S.
One of the simulator vendors has indicated that will
only implement new features if the user community
demands them. Business wise, this makes sense - you
don't build something unless someone will buy it.

If you fail to voice your support, they may be able to
successfully block these proposals from going forward.

If we let this opportunity to add these features to the
standard pass, I do not think we will have to opportunity
to add them later - hence you will be stuck with using
some other language for advanced verification features.

This work is work in progress and below is the current status.
Keep in mind too that your interest/support of this work will
help raise the focus and inspiration of those doing the work.

Classes / OO:
Classes are useful for creating verification data structures,
transaction communication, and grouping for transaction based
randomization (building relationships between separate data
items). Many of the data structures (such as scoreboards,
memories, and linked structures) can already be created, however,
classes give you the ability to hide all pointer manipulations.
For example, using a memory would require a declaration with
initialization of the memory data structure, then MemWrite and
MemRead would allow a user to store and retrieve items from the
data structure. Pointers and allocation of the sparse data
structure are handled by MemWrite and MemRead and the
user would not need to be aware of it.
Status:
Peter Ashenden submitted a class proposal last year and
provided updates to it this year at DVCon. Currently
he plans on finishing an updated draft soon.

Randomization:
Randomization is useful for designs that have numerous configurable
features. Testing features individually in an isolated manner is
typically straightforward. However, testing how these features
interact can be a large verification space – one that may not be able
to be simulated completely. It is also may be difficult to predict
all of the corner cases. Randomization has been used to sequence a
test in a non-deterministic way to get reasonably good coverage of
this verification space.
While I do not share the thought that randomization should be
adapted to work for all verification problems, I do believe it
to be a valuable technique for some problems.

I wrote a draft of the randomization proposal and it is ready
for review.

Functional Coverage:
Tool/structural coverage can tell you that you did a FIFO
read or that the the FIFO went empty, but it can't tell
you that you did a read while the FIFO was empty.

Functional coverage constructs allow you to track this.
Some functional coverage capability will come from assertions
(since PSL has been integrated into VHDL). Additional constructs
will be added to allow data binning (coverage groups) and
correlation between different coverage items (cross coverage).

I have started working on this - anyone else who is interested
is welcome to contribute as much as they would like.


With a focused effort, like the one to finish the Accellera 3.0
draft of the standard, I think we can be done with these by
September.

Although some have expressed doubt, it is clear that vendors
will do what their user community asks them to do - otherwise,
someone else will and, as a result, will earn your business.


Jim

Jim Lewis

On Apr 3, 11:10 am, Jim Lewis <j...@synthworks.com> wrote:
> Hi,
> Let me try this again.
>
> The VHDL standards community has been considering whether
> to enhance VHDL to add advanced testbench features.
>
> If you are a VHDL user,
> Do you want these features added to VHDL?
> Would you rather adopt a verification language that
> already supports these (SystemVerilog, SystemC, E, Vera).
>
> I think VHDL needs these features to stay competitive.
>
> The current plan in the VHDL working group is to make
> these features similar to other verification languages
> while at the same time keeping the nature of VHDL.
>
> We need your to voice your support. You can post here,
> send your reply to me (let me know if I can use either
> your name and/or company name when I tally the results
> for the Accellera VHDL TSC), or join the Accellera VHDL
> TSC (which you can do as a non-Accellera member by
> registering) and post your reply there.
>
> More on the politics of the situation are below.
>
> Thanks,
> Jim Lewis
> VHDL Evangelist
> SynthWorks VHDL Training
>
> P.S.
> One of the simulator vendors has indicated that will
> only implement new features if the user community
> demands them. Business wise, this makes sense - you
> don't build something unless someone will buy it.
>
> If you fail to voice your support, they may be able to
> successfully block these proposals from going forward.
>
> If we let this opportunity to add these features to the
> standard pass, I do not think we will have to opportunity
> to add them later - hence you will be stuck with using
> some other language for advanced verification features.
>
> This work is work in progress and below is the current status.
> Keep in mind too that your interest/support of this work will
> help raise the focus and inspiration of those doing the work.
>
> Classes / OO:
> Classes are useful for creating verification data structures,
> transaction communication, and grouping for transaction based
> randomization (building relationships between separate data
> items). Many of the data structures (such as scoreboards,
> memories, and linked structures) can already be created, however,
> classes give you the ability to hide all pointer manipulations.
> For example, using a memory would require a declaration with
> initialization of the memory data structure, then MemWrite and
> MemRead would allow a user to store and retrieve items from the
> data structure. Pointers and allocation of the sparse data
> structure are handled by MemWrite and MemRead and the
> user would not need to be aware of it.
> Status:
> Peter Ashenden submitted a class proposal last year and
> provided updates to it this year at DVCon. Currently
> he plans on finishing an updated draft soon.
>
> Randomization:
> Randomization is useful for designs that have numerous configurable
> features. Testing features individually in an isolated manner is
> typically straightforward. However, testing how these features
> interact can be a large verification space - one that may not be able
> to be simulated completely. It is also may be difficult to predict
> all of the corner cases. Randomization has been used to sequence a
> test in a non-deterministic way to get reasonably good coverage of
> this verification space.
> While I do not share the thought that randomization should be
> adapted to work for all verification problems, I do believe it
> to be a valuable technique for some problems.
>
> I wrote a draft of the randomization proposal and it is ready
> for review.
>
> Functional Coverage:
> Tool/structural coverage can tell you that you did a FIFO
> read or that the the FIFO went empty, but it can't tell
> you that you did a read while the FIFO was empty.
>
> Functional coverage constructs allow you to track this.
> Some functional coverage capability will come from assertions
> (since PSL has been integrated into VHDL). Additional constructs
> will be added to allow data binning (coverage groups) and
> correlation between different coverage items (cross coverage).
>
> I have started working on this - anyone else who is interested
> is welcome to contribute as much as they would like.
>
> With a focused effort, like the one to finish the Accellera 3.0
> draft of the standard, I think we can be done with these by
> September.
>
> Although some have expressed doubt, it is clear that vendors
> will do what their user community asks them to do - otherwise,
> someone else will and, as a result, will earn your business.
>
> Jim

Yes! I whole-heartedly support expanding the "testbench" features of
VHDL. The main problem with using separate languages to create the
testbench are two-fold. 1) multi-language simulators are more
expensive, and 2) there are always some gotcha's at the interface when
it comes to connecting the testbench to the UUT, especially if you use
other than SL or SLV port types for your UUT.

That being said, I would caution against creating too much variance in
VHDL between what can be done in a simulation only environment, vs a
synthesis environment. OO principles come to mind. The eventual
application of OO to synthesis should be carefully considered when
developing the OO structure and syntax, even though it is now just for
simulation.

Also, there are plenty of areas in synthesizable vhdl that need work
too, like user definable modes for record ports, expanded integer
synthesis (boolean operator definition, expanded range), as well as
fixed point representations along the lines of ada.

Andy

Andy

Andy,
Thanks for your post. We are in the position where we need to
hear all users speak up (privately to me is ok), even when they
agree with what has already been stated.

> Yes! I whole-heartedly support expanding the "testbench" features of
> VHDL. The main problem with using separate languages to create the
> testbench are two-fold. 1) multi-language simulators are more
> expensive, and 2) there are always some gotcha's at the interface when
> it comes to connecting the testbench to the UUT, especially if you use
> other than SL or SLV port types for your UUT.
>
> That being said, I would caution against creating too much variance in
> VHDL between what can be done in a simulation only environment, vs a
> synthesis environment. OO principles come to mind. The eventual
> application of OO to synthesis should be carefully considered when
> developing the OO structure and syntax, even though it is now just for
> simulation.
>
> Also, there are plenty of areas in synthesizable vhdl that need work
> too, like user definable modes for record ports, expanded integer
> synthesis (boolean operator definition, expanded range), as well as
> fixed point representations along the lines of ada.

Record ports is high on my list too. Have to finish the verification
features first. The other items you mention also sound interesting.

Cheers,
Jim

Jim Lewis

On Apr 3, 2:25 pm, Jim Lewis <j...@synthworks.com> wrote:
> Andy,
> Thanks for your post. We are in the position where we need to
> hear all users speak up (privately to me is ok), even when they
> agree with what has already been stated.
>
>
>
> > Yes! I whole-heartedly support expanding the "testbench" features of
> > VHDL. The main problem with using separate languages to create the
> > testbench are two-fold. 1) multi-language simulators are more
> > expensive, and 2) there are always some gotcha's at the interface when
> > it comes to connecting the testbench to the UUT, especially if you use
> > other than SL or SLV port types for your UUT.
>
> > That being said, I would caution against creating too much variance in
> > VHDL between what can be done in a simulation only environment, vs a
> > synthesis environment. OO principles come to mind. The eventual
> > application of OO to synthesis should be carefully considered when
> > developing the OO structure and syntax, even though it is now just for
> > simulation.
>
> > Also, there are plenty of areas in synthesizable vhdl that need work
> > too, like user definable modes for record ports, expanded integer
> > synthesis (boolean operator definition, expanded range), as well as
> > fixed point representations along the lines of ada.
>
> Record ports is high on my list too. Have to finish the verification
> features first. The other items you mention also sound interesting.
>
> Cheers,
> Jim

These new features are greatly needed and appreciated.
-- Amal

Amal

On Apr 3, 6:10 pm, Jim Lewis <j...@synthworks.com> wrote:
> Hi,
> Let me try this again.
>
> The VHDL standards community has been considering whether
> to enhance VHDL to add advanced testbench features.
>
> If you are a VHDL user,
> Do you want these features added to VHDL?
> Would you rather adopt a verification language that
> already supports these (SystemVerilog, SystemC, E, Vera).
>
> I think VHDL needs these features to stay competitive.
>
> The current plan in the VHDL working group is to make
> these features similar to other verification languages
> while at the same time keeping the nature of VHDL.
>

This would be excellent, and I will be happy to use such features.
VHDL already supports a large (compared, say, to Verilog) non
synthesizable subset for the sake of testbenches. Then why not make it
more useful.

Eli Bendersky

Hi Jim,

Jim Lewis <> writes:

> Classes / OO:
> Classes are useful for creating verification data structures,
> transaction communication, and grouping for transaction based
> randomization (building relationships between separate data
> items). Many of the data structures (such as scoreboards,
> memories, and linked structures) can already be created, however,
> classes give you the ability to hide all pointer manipulations.
> For example, using a memory would require a declaration with
> initialization of the memory data structure, then MemWrite and
> MemRead would allow a user to store and retrieve items from the
> data structure. Pointers and allocation of the sparse data
> structure are handled by MemWrite and MemRead and the
> user would not need to be aware of it.
> Status:
> Peter Ashenden submitted a class proposal last year and
> provided updates to it this year at DVCon. Currently
> he plans on finishing an updated draft soon.
>

Is there anyway us non-members can see this? I'm on the reflector,
and see the links to files, but of course, I can't see them

I'd like to see more OO, and I agree with Andy, I'd like synthesis of
these extensions to be taken very seriously early on.

> Randomization:
> Randomization is useful for designs that have numerous configurable
> features. Testing features individually in an isolated manner is
> typically straightforward. However, testing how these features
> interact can be a large verification space – one that may not be able
> to be simulated completely. It is also may be difficult to predict
> all of the corner cases. Randomization has been used to sequence a
> test in a non-deterministic way to get reasonably good coverage of
> this verification space.
> While I do not share the thought that randomization should be
> adapted to work for all verification problems, I do believe it
> to be a valuable technique for some problems.
>
> I wrote a draft of the randomization proposal and it is ready
> for review.
>

I have to admit, I've never had a need for random testing... yet.
However, I can see where it would be useful!

> Functional Coverage:
> Tool/structural coverage can tell you that you did a FIFO
> read or that the the FIFO went empty, but it can't tell
> you that you did a read while the FIFO was empty.
>
> Functional coverage constructs allow you to track this.
> Some functional coverage capability will come from assertions
> (since PSL has been integrated into VHDL). Additional constructs
> will be added to allow data binning (coverage groups) and
> correlation between different coverage items (cross coverage).
>
> I have started working on this - anyone else who is interested
> is welcome to contribute as much as they would like.
>

Yes, this would be great!

Thanks for all your efforts!

Cheers,
Martin

--

TRW Conekt - Consultancy in Engineering, Knowledge and Technology
http://www.conekt.net/electronics.html

Martin Thompson

On Apr 4, 7:02 am, Martin Thompson <martin.j.thomp...@trw.com> wrote:
> Hi Jim,
>
>
>
> Jim Lewis <j...@synthworks.com> writes:
> > Classes / OO:
> > Classes are useful for creating verification data structures,
> > transaction communication, and grouping for transaction based
> > randomization (building relationships between separate data
> > items). Many of the data structures (such as scoreboards,
> > memories, and linked structures) can already be created, however,
> > classes give you the ability to hide all pointer manipulations.
> > For example, using a memory would require a declaration with
> > initialization of the memory data structure, then MemWrite and
> > MemRead would allow a user to store and retrieve items from the
> > data structure. Pointers and allocation of the sparse data
> > structure are handled by MemWrite and MemRead and the
> > user would not need to be aware of it.
> > Status:
> > Peter Ashenden submitted a class proposal last year and
> > provided updates to it this year at DVCon. Currently
> > he plans on finishing an updated draft soon.
>
> Is there anyway us non-members can see this? I'm on the reflector,
> and see the links to files, but of course, I can't see them
>
> I'd like to see more OO, and I agree with Andy, I'd like synthesis of
> these extensions to be taken very seriously early on.
>
>
>
> > Randomization:
> > Randomization is useful for designs that have numerous configurable
> > features. Testing features individually in an isolated manner is
> > typically straightforward. However, testing how these features
> > interact can be a large verification space - one that may not be able
> > to be simulated completely. It is also may be difficult to predict
> > all of the corner cases. Randomization has been used to sequence a
> > test in a non-deterministic way to get reasonably good coverage of
> > this verification space.
> > While I do not share the thought that randomization should be
> > adapted to work for all verification problems, I do believe it
> > to be a valuable technique for some problems.
>
> > I wrote a draft of the randomization proposal and it is ready
> > for review.
>
> I have to admit, I've never had a need for random testing... yet.
> However, I can see where it would be useful!
>
> > Functional Coverage:
> > Tool/structural coverage can tell you that you did a FIFO
> > read or that the the FIFO went empty, but it can't tell
> > you that you did a read while the FIFO was empty.
>
> > Functional coverage constructs allow you to track this.
> > Some functional coverage capability will come from assertions
> > (since PSL has been integrated into VHDL). Additional constructs
> > will be added to allow data binning (coverage groups) and
> > correlation between different coverage items (cross coverage).
>
> > I have started working on this - anyone else who is interested
> > is welcome to contribute as much as they would like.
>
> Yes, this would be great!
>
> Thanks for all your efforts!
>
> Cheers,
> Martin
>
> --
> martin.j.thomp...@trw.com
> TRW Conekt - Consultancy in Engineering, Knowledge and Technologyhttp://www.conekt.net/electronics.html

Where can one find the proposed changes documents and files to get an
idea of the the syntax and features? Should you be a member to have
access to them? Are they at Accellera?

-- Amal

Amal