[Libre-soc-dev] nmigen tutorials etc.

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sun, Oct 10, 2021 at 6:59 PM Andrey Miroshnikov
<andrey at technepisteme.xyz> wrote:

> > actually you can still get many of them by using archive.org
> Great idea! I'll check to see what I replaced and find the equivalent
> archive.org links.

i think i did that for lambdaconcept tutorials already, somewhere.
http://libre-soc.org/resources/ ?

> You mean fun right? And many sleepless nights? XD
> I've been hand-writing Robert's tutorial (not word for word though), to
> make sure I get the concepts ingrained. Better take out all the stops
> when it comes to learning nMigen.

yeah i remember doing that

> > jacob just wrote this which helps, there
> > http://bugs.libre-soc.org/show_bug.cgi?id=722 but hoo-boy are the
> > memory requirements massive.
> I get the gist (remove redundant logic instantiations?), but there's a
> lot of python I'm seeing for the first time. At least now I know what
> function decorator is haha:
> https://docs.python.org/3.7/glossary.html#term-decorator

:)

> At least with ASTs there are pruning techniques to remove redundancies
> (as with other tree types), right?

exactly.

> > the alternatives - which i describe in detail here - are absolutely
> > flat-out bat-s*** insane.
> > https://libre-soc.org/3d_gpu/architecture/dynamic_simd/
> Gave it a read. Now I finally have an idea of what PartionedSignal is
> meant to do. With a signal example it made much more sense.

you see why it's completely bast-hit sinane to try to do a batch
if elif-elif-elif-elif cases in the actual ALU?

imagine you have 600 lines of ALU code (something really complex).
now duplicate that 4 times.

or, worse, try to do explicit gate-level optimisation, where in the
ALU you explictly design-in the low-level SIMD "partitions"
actually right there, in the ALU.


> Parallel bit groups that can be operated on separately, or combined
> together if no partitions are enabled.

yes.  now it turns out (and it was Jacob who came up with this
concept, first) that you can do tricks for e.g. "add" by having
partition bits actually *in* the underlying add: a 64-bit add becomes
a 64+7 bit add, and you use carry roll-over to get it to *look*
like they're "real" partitions.

once you've absorbed that "hey, neat!" trick, it turns out you
can then do the exact same thing to all other operations:
break them down into small chunks that get joined together
to create bigger ALU operations.

greater-than, less-than, equal, shift, etc. etc. they can all
be broken down into small bits that create big bits with some
routing.

don't get me wrong: it took _months_ to implement, but it's
quite straightforward.

> For nMigen to automatically
> expand those if-statements is a no-brainer.

yeah. duh.  which makes it really puzzling that whitequark has been
fighting and resisting such a natural and obvious logical / clear
concept.

> Other than the surface level, I'll need to continue reading other
> material, and re-read it a few times. There's a lot of computer science
> theory I'll be catching up on :)

:)

> What I can't think of is an example use-case? Can you give an example of
> where one might want to take a dataword, split it into partitions, and
> operate in tandem?

anything involving large parallel computations.

> Pixels for a frame buffer?

yep, good one.  lots of things there.  colour-balance, normalisation,

stereo audio processing is another obvious one.

> Velocity calculations?
> Hashing/encryption algos?

3D xyz vectors, 3D quaternions, you name it.

> This is still unclear to me, but it'll become clearer as I continue
> reading and studying. (a lot of bookmarked material/videos to go through)
> If anything, better that I started clueless, hopefully won't have any
> mistaken lines of thought.

good philosophy.

> > https://git.libre-soc.org/?p=openpower-isa.git;a=blob;f=src/openpower/decoder/power_decoder.py;hb=HEAD
> >
> Had a brief look at the code, nMigen did start pretty far down, most of
> the start is data structure setup.

ridiculous, isn't it? 99% of it is python.  you actually have to look
*really hard*
to find actual nmigen AST.

if this were a "normal" industry-standard project, the ISA decoder would
have been written out explicitly "by hand", wouldn't it?
in the actual "language" of the project, yeh?

like this:

https://github.com/antonblanchard/microwatt/blob/master/decode1.vhdl

i went, "oo that looks like a CSV file, let's *make* it a CSV file and
then it can be machine-read elsewhere"

> > you'll love this: i recovered the GardenSnake.py example from python-ply,
> > added a 3rd lexer pass which allowed it to properly support full python
> > syntax, then used it as the basis for the openpower ISA pseudo-code
> > to python compiler.
> Great use of public domain code ;)

:)

> One of my friends in recent years has been a bigger proponent of Python,
> while I was moving more towards low-level. However now that I've been
> exposed to its flexibility, it's starting to grow on me. At least for
> applications not constrained by time, making very complex behaviour is
> almost trivial (like extending a compiler).

yehyeh.

> 90s?...hahaha most of these concepts are at least 100 years old!

:)

> > ... oh except for proprietary companies creating tools that you can only
> > use under NDA.
> I understand paying for a tool when it saves you time and money. Most
> tools I've seen however, intend to keep the user busy, instead of using
> software eng. techniques for automation, as you have described. Digital
> domain (of all things) should be the most automatable.

i cannot count the number of Dilbert cartoons i've read on this theme...

> > and it takes so long that they can't possibly do design-iteration.
> I must admit, I've been having a paradigm shift now after watching the
> conversations here. Before my bias was mainly toward lower-level design.
> However when abstractions are well-known and predictable, I see no issue
> with utilising them as much as possible.

it's kinda a no-brainer but it isn't? as in, you have to *really* trust and
also know the "levels" if you know what i mean.

with the SIMD thing we'll reaaaallly have to do a walk-through of the
resultant HDL because it risks expanding out of control.

i remember someone doing operator-overloads in c++ (pointers,
casts), and wondering why their code wasn't running very fast,
and a more senior engineer taking them through it in a debugger,
single-stepping down the chain of overloads, dozens of levels...

> C++ still terrifies me a little. Almost like Javascript and Java, C++'s
> abstract concepts have almost nothing to do with C, and with each
> release they seem to pile up more and more.

sigh. i loved c++ when it was Bjarne Strousup's 2nd Edition book.
the templating libraries have gotten waaay out of hand since then.

> Didn't the first C++
> compilers take a really long time before they became viable?

my first exposure to c++ was i think borland's compiler.  1992.


> On 10/10/2021 14:26, lkcl wrote:
>  > Andrey can i suggest dropping the list links into the page as well,
> for context.
>  > at least this one:
>  >
> http://lists.libre-soc.org/pipermail/libre-soc-dev/2021-October/003858.html
> Not sure what you meant here.
>
> Do you mean add the mail archive link to the tutorial page?

yes.  usually we do a "links" section bullet-points at the top.

>  > and one about the reason why to use "show top"
> And as for "show top" do you want an explanation ("block diagram is
> useful for visualising the internal architecture and compare with
> code...etc.")?

if *you* think it would be helpful - to you - and to others - then yes.

l.