[Libre-soc-bugs] [Bug 980] Implement C-based Power ISA pseudocode compiler

[bugzilla-daemon at libre-soc.org]

https://bugs.libre-soc.org/show_bug.cgi?id=980

--- Comment #15 from Luke Kenneth Casson Leighton <lkcl at lkcl.net> ---
(In reply to Dmitry Selyutin from comment #12)
> I've spent a LOT of hours debugging and checking the opportunities. Here are
> some conclusions I came to so far.
> 
> 
> 1. I might appear to be a devil's advocate, but I actually think that
> python-ply is a good choice. Regardless of our implementation pros and
> contras, for me it already advocates python-ply as a sufficiently good
> instrument, and I'm going to stick with it.
> 2. What is bad from C code generation perspective is that we produce Python
> AST. It's not a suitable candidate for C code generation. What's worse, we
> generate multiple nodes for a single entity, and it gets too complicated
> even in Python, not even speaking of C.
> 3. It gets especially hairy when we produce Python calls and modify the code
> after it has been obtained with AST. That is, we get the AST result and
> patch it in place.
> 4. We cannot avoid modifying the nodes produced, but there are several
> issues. First, all our nodes are Python AST. Second, some nodes are built in
> place, e.g. concat and selectassign and many others (basically all which
> lead to ast.Call). Third, whenever I'll update this code, this would
> potentially bring a disaster.
> 
> All in all, I don't believe that the current code is a good basis for C code
> generator. It works for Python, and I'd keep it so that the code WORKS.
> 
> 
> Here's what I suggest instead.
> 1. Keep the python-ply, but recreate the overall code so that it produces
> CUSTOM nodes instead. That is, where we have <u, there will be a `class
> LTUnsigned(Node)` construction, similarly for all other stuff. Basically the
> real language-agnostic AST. Language-agnostic is an absolutely critical item!
> 2. Once we have AST collected and working, we can have walkers and visitors
> for it.
> 3. This AST can be used for C code generation. Ideally, had we not time
> constraints, we'd build ANOTHER AST atop of that: PseudocodeAST => CAst. But
> for now likely we'll have to follow the simpler approach.

i'm good with that.  if this was a "full" language (scopes exceptions
globl/local variabes classes gotos) i would say NO hell NO.

(btw please drop functions. they were not added and should not have beem
added by jacob. please drop "def" from the AST and modify pyfnwriter
and appendix pseudocode to not support functions.  this means
separating each "function" into an explicit separate pseudocode file)


> 4. If we have time and opportunities, we'd just eventually follow the same
> approach for Python code. That is, just convert PseudocodeAST to ast.Module.
> 
> 
> I don't think it's doable in terms of the current approach mostly due to the
> fact that it produces a pretty complicated Python AST.

yes. if you have seen the notes i started from dabeaz's GardenSnake.py
as *literally* the first commit, then fixed the lexer problem (with
an additional lexer phase that inline-inserted the required TOKENs):

https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/pseudo/lexer.py;h=c42f8330;hb=HEAD#l62

and then spent about FOUR MONTHS over the next 2 years performing incremental
changes and fixes that got it to do the job.

the only reason i could start from GardenSnake.py was because i spotted that
fundamentally the underlying syntax (not the grammar) of python and
Section 1.3 POWER Book I ISA are identical.

*over time* the use of python-astor, which i used because some dickhead
in the python community ripped out ast.py of python2.7 and replaced it
with fucking c code *with no introspection or source* "bcuz faster",
has become a serious impediment.

i can barely edit parser.py without also creating a prerequisite
code-fragment in power_pseuo.py and using astor.dump()

https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/power_pseudo.py;hb=HEAD

> I could try changing
> the current code, but I'm absolutely sure my changes will make all the code
> around completely nuts.

yyep. it's reached the "mature complex but stable" phase that all software
developed incrementally will get to.


> I wish I had returned with a more encouraging news, but these are
> conclusions I've been able to reach so far. Basically my suggestion is to
> use python-ply again and output a completely different nodes. So the code
> eventually will look familiar but still build something completely different.

seriously and honestly, i was expecting that anyway. an astor-to-c converter
(basically a replication of cython) was a "hope" rather than a requirement.

> 
> I'd like to know your opinion and will meanwhile move on with checking other
> projects using ply to better understand how it can be used. Before we
> proceed, two important items: 1) python-ply is to stay, we don't have time
> for brand new frameworks,

no we do not.

> plus I'm mostly satisfied with it;

it's pretty damn good. it is sad that dabeaz abandoned it.

> 2) no there's
> no time (and there's a strong lack of desire) to teach the whole Python ast
> to produce Python code.

it *used* to do that.  go right back to the original very first commit.

(context: GardenSnake.py was - is - an interactive python interpreter
written *in* python using python-ply.  it worked perfectly for *very
early* versions of python and i am going back 20+ years here but at
some point the syntax changed and dabeaz never got round to fixing
GardenSnake.py... that was my very first task)

-- 
You are receiving this mail because:
You are on the CC list for the bug.