[Libre-soc-isa] [Bug 686] create Power ISA test API

[bugzilla-daemon at libre-soc.org]

https://bugs.libre-soc.org/show_bug.cgi?id=686

--- Comment #44 from Luke Kenneth Casson Leighton <lkcl at lkcl.net> ---
(In reply to Jacob Lifshay from comment #43)
> (In reply to Luke Kenneth Casson Leighton from comment #42)
> > (In reply to Jacob Lifshay from comment #41)
> > > one thing to think about, if there is an expected state, where do we get the
> > > expected results? 
> > 
> > either by manual inspection or by running an equivalent
> > function in python that is known-good (and also verified).
> > 
> > there are about.... 100 such tests already, and many more
> > are needed.
> > 
> > in the case of the Power ISA Conformance Documents we *have*
> > to assume the values listed are "correct".
> > 
> > and be able to enter them.  an ExpectedState allows that to
> > be formally specified, where at the moment it is nonstandard
> > and a bit of a mess.
> > 
> > > I think it might be better to just run multiple
> > > simulators/VMs and compare their results instead of isolating one of them as
> > > the "canonical expected result".
> > 
> > the idea is to do both.
> > 
> > 1) compare simulators against simulators.
> > 2) compare simulators against "expected results".
> 
> I'm saying expected results are a kind of simulator and should be treated as
> such.

none of these can in any way be called "a simulator"

https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/isa/test_caller.py;hb=HEAD

even the DCT / FFT ones cannot in any way be called "a simulator".

https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/isa/test_caller_svp64_fft.py;hb=HEAD

i used an external (excellent, well-written) library from the Nayuki
Project which *itself* had unit tests that trace back to the original
(very slow) DCT algorithm.



> easy (adjust API to match real tester API):
> # tester library class
> class ExplicitCaseSimulator(Simulator):

that's exactly what the ExpectedState class is designed
to hold.

expected results.


> # in unit test we have the list of explicit test cases:
> cases = { # xor for demo purposes
>     (1, 2): {"r1": 3},
>     (0x123, 0x456): {"r1": 0x123 ^ 0x456}
> }

yes.

that is exactly and precisely what i said.

calculate or type the expected results, put them in an
object, hand them to the check_regs function.

done.

calling it a "Simulator" is not useful or helpful when the
API is in no way intended for simulating.

it gets data *FROM* simulators but is not *itself* a simulator.

ExpectedState is intended to be STATIC. you call it ONCE
at the *END* of a *COMPLETED* simulation run.

what you propose is massively more complex and requires actually
understanding instructions, decoding them, maintaining a PC,
updating of memory, it's huge and counterproductive.

the purpose of ExpectedState is to be abundantly clear, brutally
minimalist, and in no way complex in any way.

-- 
You are receiving this mail because:
You are on the CC list for the bug.