[Libre-soc-isa] [Bug 686] create Power ISA test API

[bugzilla-daemon at libre-soc.org]

https://bugs.libre-soc.org/show_bug.cgi?id=686

--- Comment #43 from Jacob Lifshay <programmerjake at gmail.com> ---
(In reply to Luke Kenneth Casson Leighton from comment #42)
> (In reply to Jacob Lifshay from comment #41)
> > one thing to think about, if there is an expected state, where do we get the
> > expected results? 
> 
> either by manual inspection or by running an equivalent
> function in python that is known-good (and also verified).
> 
> there are about.... 100 such tests already, and many more
> are needed.
> 
> in the case of the Power ISA Conformance Documents we *have*
> to assume the values listed are "correct".
> 
> and be able to enter them.  an ExpectedState allows that to
> be formally specified, where at the moment it is nonstandard
> and a bit of a mess.
> 
> > I think it might be better to just run multiple
> > simulators/VMs and compare their results instead of isolating one of them as
> > the "canonical expected result".
> 
> the idea is to do both.
> 
> 1) compare simulators against simulators.
> 2) compare simulators against "expected results".

I'm saying expected results are a kind of simulator and should be treated as
such.

> 
> > building a custom result generator just to get the "expected result" can be
> > alternatively seen as building a custom simulator to be used just for that
> > specific test case, so I think it would be better to acknowledge that that
> > is the case and that it is just like the other simulators, so should also be
> > a simulator.
> 
> one immediate counterexample which shows this line of reasoning to be
> invalid is the
> OpenPOWER Foundation Compliance Test documents.
> 
> we are *required* to extract the listed values and *required* to demonstrate
> that they have been run.

easy (adjust API to match real tester API):
# tester library class
class ExplicitCaseSimulator(Simulator):
    def __init__(self, cases, make_key):
        self.cases = cases
        self.make_key = make_key

    def simulate(self, regs):
        key = self.make_key(regs)
        if key in self.cases:
            for k, v in self.cases[key].items():
                setattr(regs, k, v)
            return regs
        raise NoResults() # causes tester to skip this simulator

# in unit test we have the list of explicit test cases:
cases = { # xor for demo purposes
    (1, 2): {"r1": 3},
    (0x123, 0x456): {"r1": 0x123 ^ 0x456}
}
tester.addSimulator(ExplicitCaseSimulator(cases,
    lambda regs: (regs.r2, regs.r3)))

-- 
You are receiving this mail because:
You are on the CC list for the bug.