[Libre-soc-dev] clamping/saturation semantics

Luke Kenneth Casson Leighton lkcl at lkcl.net
Sun Dec 13 04:01:01 GMT 2020

On 12/13/20, Jacob Lifshay <programmerjake at gmail.com> wrote:
> On Sat, Dec 12, 2020, 19:37 Luke Kenneth Casson Leighton <lkcl at lkcl.net>
> wrote:
>> On 12/13/20, Jacob Lifshay <programmerjake at gmail.com> wrote:
>> the one that is particularly odd is avg.  that one: how the heck do we
>> SIMD-i-fy and saturate that one??
> Well, assuming the input and output types are the same, it never overflows,
> so saturation is a no-op.

.. average.  of course, duh :)

that leaves when dest elwidth is smaller.

oh wait... avg(255, 255) but with signed 8bit would be truncated to 127

but still no overflow.  yep.

> All of the above (and me previous email in this thread) is assuming
> signed/unsigned integers, FP may behave differently.

deep breath: we'll need to go patiently through them all, but first
estimating time involved.  then prioritise.


More information about the Libre-soc-dev mailing list