A couple of days ago I found out that gzread from the zlib library can be used for fast reading of binary files in Maple from the disk to memory - about 100 times faster than readbytes - something like in the following simplified example, 

A:=Array(1..2^26,99,datatype=integer[1]):

time(writebytes("A",A));close("A");

                                9.360

B:=Array(1..2^26,1,datatype=integer[1]):
time(readbytes("A",B));close("A");

                                8.065
B[1],B[-1];

                                99, 99

myreadbytes:=proc(f)
local gzopen, gzread, gzclose, n, p, A;
gzopen:=define_external('gzopen',
    'path'::string,
    'mode'::string,
    'RETURN'::integer[4],
    'LIB'="zlibwapi.dll");
gzread:=define_external('gzread',
    'file'::integer[4],
    'buf'::REF(ARRAY(datatype=integer[1])),    
    'len'::integer[4],
    'RETURN'::integer[4],
    'LIB'="zlibwapi.dll");
gzclose:=define_external('gzclose',
    'file'::integer[4],
    'RETURN'::integer[4],
    'LIB'="zlibwapi.dll");
n:=FileTools:-Size(f);
A:=Array(1..n,datatype=integer[1]);
try p:=gzopen(f,"rb");
if gzread(p,A,n)=n
then return A end if
finally gzclose(p)
end try
end proc:
time(assign(C=myreadbytes("A")));

                                0.062

C[1],C[-1];

                                99, 99

'time(myreadbytes("A"))'$5;

                  0.078, 0.062, 0.046, 0.046, 0.046

E:=Array(1..2^26,2,datatype=integer[1]):
time(ArrayTools:-Copy(A,E));

                                0.093

That needs some tweaking, because that works only on uncompressed files. If a file ("A" in this example) was gzipped, then the gzread would ungzip n (uncompressed) bytes in it in this example, instead of copying it into the memory - but it is not a big deal, in general.

Does anybody know about a similar replacement for writebytes? gzwrite doesn't work for copying (it compresses the array.)

I used the zlibwapi.dll library from http://www.winimage.com/zLibDll/index.html, it is a version of zlib 1.2.5 (written by Jean-Loup Gailly and Mark Adler) built by Gilles Vollant. The code is for a 32-bit system (Windows). That should work in 32-bit Linux after replacing that dll with standard libz.so.1, as well as on 64-bit systems after replacing integer[4] with integer[8] in most places.

_______________
Alec Mihailovs, PhD
Maplesoft Member