[Libre-soc-dev] clamping/saturation semantics
Luke Kenneth Casson Leighton
lkcl at lkcl.net
Sun Dec 13 04:01:01 GMT 2020
On 12/13/20, Jacob Lifshay <programmerjake at gmail.com> wrote:
> On Sat, Dec 12, 2020, 19:37 Luke Kenneth Casson Leighton <lkcl at lkcl.net>
>> On 12/13/20, Jacob Lifshay <programmerjake at gmail.com> wrote:
>> the one that is particularly odd is avg. that one: how the heck do we
>> SIMD-i-fy and saturate that one??
> Well, assuming the input and output types are the same, it never overflows,
> so saturation is a no-op.
.. average. of course, duh :)
that leaves when dest elwidth is smaller.
oh wait... avg(255, 255) but with signed 8bit would be truncated to 127
but still no overflow. yep.
> All of the above (and me previous email in this thread) is assuming
> signed/unsigned integers, FP may behave differently.
deep breath: we'll need to go patiently through them all, but first
estimating time involved. then prioritise.
More information about the Libre-soc-dev