Re: making RandomAccessStream use seek(long)

Posted by Mario Emmenlauer-3 on Aug 21, 2007; 7:02pm
URL: http://imagej.273.s1.nabble.com/making-RandomAccessStream-use-seek-long-tp3698555p3698558.html

Hi,

I might have put this a little confusing, since I didn't think anyone
would actually care :-)

Curtis Rueden wrote:
> Hi Mario,
>
> I am a little confused. I downloaded the LSM Toolbox but was unable to
> find a class called RandomAccessStream inside the JAR file. However,
> the Bio-Formats library has a RandomAccessStream class very similar to
> what you describe.

The LSM Toolbox uses ij.io.RandomAccessStream, which AFAIK extends
javas InputStream but uses an RandomAccessFile to do the loading and storing.
The weired part is that javas RandomAccessFile fully supports long seek
offsets at least since java 1.4.2 (see
http://java.sun.com/j2se/1.4.2/docs/api/java/io/RandomAccessFile.html),
but ij.io.RandomAccessStream explicitly masks the offset to 32 bit.

I didn't even bother thinking about good style and just fixed
ij.io.RandomAccessStream (removed the masking, changed the pointers to
long) and put it with the LSM Toolbox sources.

> Patrick: did you adapt our RandomAccessStream class for use in LSM
> Toolbox? If so, I suggest updating the class to the latest version in
> Bio-Formats, since many bugs have been fixed and we now support >2GB
> files (meaning seek uses longs, etc.).

He didn't, I did. You can find my version (for testing) here:
   http://lmb.informatik.uni-freiburg.de/people/emmenlau/LSM_Toolbox.jar
but it's only temporary. once it's tested I hope the patch will be
part of the official Toolbox.

> Mario, you could also try Bio-Formats for reading very large LSMs. It
> does not have as many nifty LSM-related features as LSM Toolbox does,
> but could be a stopgap measure until LSM Toolbox has been updated to
> handle larger files.

Would I have known I wouldn't have gone through all the work :-(
Now the LSM Toolbox seems to do the job, but I might try Bio-Formats
anyways.

Thanks!

    Mario

> -Curtis
>
> On 8/16/07, Mario Emmenlauer <[hidden email]> wrote:
>> Hi all,
>>
>> I was working on extending the LSM Toolbox
>> (http://imagejdocu.tudor.lu/Members/ppirrotte/lsmtoolbox)
>> to be able to read Zeiss LSM files of more than 4GB filesize. For
>> that it would be good if the RandomAccessStream class would use
>> long (64bit) pointers for file offsets, especially for seek().
>> Below patch solved the issue for me, however I didn't check
>> if it's completely sane.
>>
>> BTW: I was surprised to see seek(int) anyways, especially since
>> java nowadays uses long. Is it a remainder from old times or did
>> I overlook anything?
>>
>> Cheers,
>>
>>     Mario
>>
>>
>>
>> --- RandomAccessStream.java     2007-08-16 23:45:02.000000000 +0200
>> +++ myRandomAccessStream.java   2007-08-16 23:48:06.000000000 +0200
>> @@ -1,3 +1,4 @@
>> +package ij.io;
>>   import java.io.*;
>>   import java.util.Vector;
>>
>> @@ -15,7 +16,7 @@
>>
>>       private InputStream src;
>>       private RandomAccessFile ras;
>> -    private int pointer;
>> +    private long pointer;
>>       private Vector data;
>>       private int length;
>>       private boolean foundEOS;
>> @@ -35,7 +36,7 @@
>>           this.ras = ras;
>>       }
>>
>> -    public int getFilePointer() throws IOException {
>> +    public long getFilePointer() throws IOException {
>>           if (ras!=null)
>>               return (int)ras.getFilePointer();
>>           else
>> @@ -45,8 +46,8 @@
>>       public int read() throws IOException {
>>           if (ras!=null)
>>               return ras.read();
>> -        int l = pointer + 1;
>> -        int l1 = readUntil(l);
>> +        long l = pointer + 1;
>> +        long l1 = readUntil(l);
>>           if(l1 >= l) {
>>               byte abyte0[] = (byte[])data.elementAt((int)(pointer>>BLOCK_SHIFT));
>>               return abyte0[(int)(pointer++ & BLOCK_MASK)] & 0xff;
>> @@ -63,7 +64,7 @@
>>               throw new IndexOutOfBoundsException();
>>           if(len == 0)
>>               return 0;
>> -        int l = readUntil(pointer+len);
>> +        long l = readUntil(pointer+len);
>>           if(l<=pointer)
>>               return -1;
>>           else {
>> @@ -88,7 +89,7 @@
>>           } while (read<len);
>>       }
>>
>> -    private int readUntil(int l) throws IOException {
>> +    private long readUntil(long l) throws IOException {
>>           if(l<length)
>>               return l;
>>           if(foundEOS)
>> @@ -116,10 +115,10 @@
>>           return length;
>>       }
>>
>> -    public void seek(int loc) throws IOException {
>> +    public void seek(long loc) throws IOException {
>>           //if (ij.IJ.debugMode) ij.IJ.log("seek: "+loc);
>>           if (ras!=null)
>> -            {ras.seek(loc&0xffffffff); return;}
>> +            {ras.seek(loc); return;}
>>           if(loc < 0)
>>               pointer = 0;
>>           else
>>
>>
>> --
>> Mario Emmenlauer                             Phone: [phone]
>> Institute for Computer Science               Fax:   [phone]
>> Chair of Pattern Recognition and Image Processing
>> Albert-Ludwigs-University
>> Georges-Koehler-Allee 052, room 01-022
>> 79110 Freiburg             mailto:emmenlau * informatik.uni-freiburg.de
>> Germany          http://lmb.informatik.uni-freiburg.de/people/emmenlau/
>>
>

--
Mario Emmenlauer                             Phone: [phone]
Institute for Computer Science               Fax:   [phone]
Chair of Pattern Recognition and Image Processing
Albert-Ludwigs-University
Georges-Koehler-Allee 052, room 01-022
79110 Freiburg             mailto:emmenlau * informatik.uni-freiburg.de
Germany          http://lmb.informatik.uni-freiburg.de/people/emmenlau/