[R] tried half-precision but "size 2 is unknown on this machine"

[Mike Miller]

Thanks!  So it looks like I can say "R writeBin/readBin does not support 
half-precision floats" even though the error message "size 2 is unknown on 
this machine" seems to contradict that (for some machine).  I tried to 
figure out from the source code (src/main/connections.c) how it decides 
what is possible, but that was a little beyond me.  That was really just 
to satisfy my curiosity.  The unsigned 16-bit integer approach works 
well-enough for me and it has the advantage that I know it will always 
work on anyone's system.  I'm working with numbers from 0 to 2 with no 
more than 4 significant digits, so a 16-bit float with 11 digits of 
precision was appealing.  It's not hard to work with uint16, though, and 
od also reads it easily.  I've been working on a message about this 
application which I will share soon, probably later tonight.

I'm also experimenting with a lossy storage using a single byte per 
integer (uint8).  That might be a good strategy because the numbers I'm 
working with are inherently imprecise.  It seems to work fine in R, but it 
doesn't seem to work with GNU od (Linux/UNIX program) and that makes me 
wonder what else can handle it.  uint16 seems the safer bet, and there is 
no loss of precision.  Of course, the downside is that the uint16 file is 
twice as big as the uint8 file, and these files may be several hundred GB 
in size.

Mike


On Sun, 4 Jan 2015, Uwe Ligges wrote:

> Following the posting guide and hence reading the help page first helps:
>
> "Possible sizes are 1, 2, 4 and possibly 8 for integer or logical vectors, 
> and 4, 8 and possibly 12/16 for numeric vectors."


On Sun, 4 Jan 2015, Duncan Murdoch wrote:

> On 04/01/2015 12:31 AM, Mike Miller wrote:
>> It's an IEEE standard format:
>>
>> http://en.wikipedia.org/wiki/Half-precision_floating-point_format#IEEE_754_half-precision_binary_floating-point_format:_binary16
>>
>> This is what I see:
>>
>>> writeBin(vec , con, size=2 )
>> Error in writeBin(vec, con, size = 2) : size 2 is unknown on this machine
>>
>> I'm not sure what the machine has to do with it.  It's really up to the
>> software, isn't it?
>
> Yes, but R relies on the underlying C run-time library for a lot of 
> things like this.  On your platform, is there a C type corresponding to 
> half precision?  If so, let us know the details, and we'll possibly add 
> it to writeBin.
>
>
>>
>> Is there a way to get R to read/write half-precision numbers 
>> (binary16)?
>
> If it's not supported by the C run-time library and has to be done 
> entirely using other types, that's the sort of thing that belongs in a 
> user-contributed package.  I'm not aware of one that already has it, so 
> you may have to write this yourself.