Tuesday, April 25, 2006, 04:07 PMThis last week, the Jewish holiday of Passover was celebrated. The Passover holiday starts with the “Seder”, a special meal that is accompanied by the reading of the “Hagada”. The “Hagada” is a small book that personally I find rather boring and incomprehensible. It tells the story of the liberation of the people of Israel from their slavery in Egypt by the almighty god. I wouldn’t have bothered you with all these details, but for a famous part in the “Hagada”, that speaks of a son whose most salient characteristic is his inability to ask questions. I must admit I never understood this part, but this time I finally got it: It is an exact description of those R&D senior engineers who insist on interviewing candidates, and yet have absolutely no idea about the relevant questions that they should pose to these candidates. I guess there is not a single Hi-Tech employee, who hasn’t had the annoying experience of being interviewed by someone suffering from severe “asking disability”…In the majority of cases such an interview is made out of a bunch of annoying “logical” riddles, that have no relation whatsoever with the actual work to be done.
The purpose of this entry is, therefore, (A) to provide those people who conduct interviews for verification positions (i.e. R&D team leaders, project managers or senior engineers) with some relevant questions, and (B) to give candidates who apply for these positions a general idea of the things they will be required to know, if the interview is in fact conducted by someone who knows what he is talking about.
So, here is my “dream work interview” for a verification position, along with the answers that I expect to hear from the anxious applicant in front of me. As you will soon see, the questions are not language dependent and will do for any HVL because they deal with methodology. I might add further questions or some language specific sections later on.
Q1: What are the differences between directed testbench and random testbench?
A1: This is a warm-up question…There are plenty of differences, the most important of which can be found here.
Q2: Draw the structure of a typical random testbench, describing each of the parts in detail.
A2: Answer should include random generators, BFMs (also known as drivers or agents), scoreboards and coverage collectors arranged in a way similar to this one.
Q3: Why is it important to keep code for generators/scoreboards and code for BFMs separated?
A3: This is just a specific example of “layer separation”, a general programming principle that makes code reuse a lot easier. The data that is sent over a physical interface has nothing to do with the physical interface, so there is no reason at all to mix the code for these two. A single protocol can be used to transfer many types of data, and similar data might be sent over different interfaces and protocols.
Q4: Is it a must to have an automatic checker (a scoreboard) in a directed testbench? Is it a must in a random testbench?
A4: In a directed testbench you have a limited set of tests, each of which written in order to check a single feature. Since the effort of writing an automatic checker is extremely high, it may not pay off, if you are running only 50 or even 100 tests. The tests will always run in the exact same way, so you might be able to check them manually. An automatic checker will become a viable option, if the number of tests grows or changes to DUT code get too frequent.
In a random testbench an automatic checker is a must because you will be running each test hundreds and thousands of times with different seeds, and each time the results will be totally different. Checking all of these runs manually will drive you mad in a single day. In contrast with directed testing where each test is “to the point”, random testing generates a lot of garbage that must be screened.
Q5: What is the difference between “protocol checking” and “data checking”? In which part of the testbench should each be done?
A5: Protocol checking means checking that the DUT interfaces comply with protocol laws. For example, that the valid signal is asserted with valid data, or that an acknowledge is always given after a request. Protocol checking refers to the BFM layer, or the physical layer. The data that is actually passed over the interface is not checked.
Data checking, on the other hand, refers to the data that is passed over the interface. So, for example, you will check that all the fields of a packet get the values you expected them to. Accordingly, protocol checks are usually located in the BFM, while data checks belong in the scoreboard. Protocol checks are often done using assertion.
Q6: Which of the parts in the testbench should add data to the scoreboard?
A6: That’s a million dollar question…there are two main approaches and the decision between the two should be done according to the project you have at hand. The first approach, which is faster and simpler is to take scoreboard data straight from the generator. Its major drawback is that the scoreboard won’t function if you have to move from the block level to the system level. The second one is to take scoreboard data from the physical interface (sub question: what is lost if you get the data from the physical interface in the form of bits and bytes, instead of taking it from the generator? Hint: all the logical fields), and then reconstruct it. This will allow the scoreboard to work at system level as well.
Therefore the correct answer would be: it depends. If our DUT has a chance of becoming an internal block then go for the second option. If, on the other hand, it is not and data reconstruction is not trivial, go for the first one. You can read more about this subject here.
Q7: What are the advantages and disadvantages of taking scoreboard data from the generator? What are the advantages and disadvantages of taking scoreboard data from the BFM?
A7: Oups, I’ve already answered that question above.
Q8: Suppose you have a scoreboard for a specific block inside the DUT. Will that scoreboard be useful during end to end (or full chip) tests as well? What for?
A8: Yes it will and that’s way the (hard) effort of reconstructing logical data from physical data might pay off. Scoreboards take a long time to write and it would be a stupid thing to throw them away when you move from the block level to the chip level. They could run in a chip level simulation as well, and help you discover bugs earlier and neared to their immediate source, hence saving you valuable simulation time and human debugging effort.
Q9: What is the use of the test file? Is it supposed to make the data that is generated by the testbench more or less random?
A9: Of course the test file constrains the data further and therefore makes it less random.
Q10: What is the difference between the constraints that are added through the test file, and those constraints that are placed in the environment files themselves?
A10: The constraints that are an integral part of the environment are those found in the specification document. They limit the generated data to the DUT’s “specified behavior”, i.e. to correct or erroneous data that the DUT should know how to handle according to the specification. The constraints in the test file, on the other hand, are intended to direct the test further (towards a specific potential bug in the design, or a coverage hole, for example) and are often not related at all to the specification.
It is important to remember that constraints placed inside the actual environment file should not limit the data any further then the specification does. Test constraints are meant to do just that.
Q11: Please explain the difference between declarative code (also known as static code), and sequential code, and give an example of each. What is the major advantage of controlling generation via declarative code? Which of the two is harder to debug?
A11: Simply put, sequential code is a piece of code that you can run step by step in a debugger. For example, a function is a block of sequential code. On the other hand, a lot of the code you write is declarative and can not be stepped through…common examples are function and variable declarations, or generation constraints in HVLs.
The major advantage of controlling generation via declarative code is that it makes the generation extendable. You can add further constraints (or cancel existing ones in SystemVerilog and with some effort in E as well), without modifications to the original code. For example, an external test file can be used to change the generated data. This advantage is the one that made all HVLs opt for this mode of writing.
The major disadvantage is that the code is that declarative code is a lot harder to debug, but that’s a small price to pay compared with the first advantage.
Q12: What are test scenarios (also known as sequences)? What are they used for? Do the use of test scenarios makes the testbench more or less random?
A12: Test scenarios are another way to direct tests and therefore (as their name implies), make data less random. They are used in order to constrain the generated data in a way that is dependent on the simulation stage or on the DUT state – “At the beginning we would like data of this type, then data of that type, and finally data of the third type”. They are used to fill in coverage holes, to direct tests towards a specific potential bug, or to reconstruct bugs that had been discovered somewhere else in the verification environment.
Q13: Please give an example of when a test scenario should be used.
A13: To cite Britney, “Oups I did it again”, see above.
Q14: (Similar to question 9) At which phase the use of sequences is more common – at the beginning, at the middle, or at the end of the verification process?
A14: Since the major role of sequences is to fill in coverage halls, they are more likely to be used at the end of the verification process. At the middle of the verification process you just let the environment run free, while collecting coverage and hoping for the best. At the beginning (bring up), regular constraints that are not time dependent will often do…you will just limit data to its simplest type, check manually that everything’s fine and move on to the next data type.
more on the issue:
Check out a "recording" of a phone interview I lately had with Verilab.
Good luck :-)