[Libre-soc-dev] clamping/saturation semantics

[Luke Kenneth Casson Leighton]

On 12/13/20, Jacob Lifshay <programmerjake at gmail.com> wrote:
> On Sat, Dec 12, 2020, 19:37 Luke Kenneth Casson Leighton <lkcl at lkcl.net>
> wrote:
>
>> On 12/13/20, Jacob Lifshay <programmerjake at gmail.com> wrote:
>>
>> the one that is particularly odd is avg.  that one: how the heck do we
>> SIMD-i-fy and saturate that one??
>>
>
> Well, assuming the input and output types are the same, it never overflows,
> so saturation is a no-op.

.. average.  of course, duh :)

that leaves when dest elwidth is smaller.

oh wait... avg(255, 255) but with signed 8bit would be truncated to 127

but still no overflow.  yep.

> All of the above (and me previous email in this thread) is assuming
> signed/unsigned integers, FP may behave differently.

deep breath: we'll need to go patiently through them all, but first
estimating time involved.  then prioritise.

l.