Login  Register

Re: Reading in Image Files (Hopefully a "re-inventing the wheel" sort of question)

Posted by Unruh, Jay on Apr 22, 2008; 5:21pm
URL: http://imagej.273.s1.nabble.com/Reading-in-Image-Files-Hopefully-a-re-inventing-the-wheel-sort-of-question-tp3696471p3696472.html

ImageJ community,

Below is a csv_reader.java file that is just a copy of the text_reader plugin that is built in in ImageJ with a few edits for comma delimiters in addition to tab delimiters.  The key is the definition of word chars and white space chars in the read method.  This may or may not be a good edit to make permanent considering the different uses of commas throughout the world.

Jay Unruh

import ij.*;
import ij.process.*;
import ij.gui.*;
import java.awt.*;
import ij.plugin.*;
import java.awt.image.*;
import java.io.*;
import ij.io.*;
import ij.util.Tools;
//here is a csv reader.  This is just a copy of text_reader.java with the tokenizer changed to accept commas as delimiters
//changes made 4/22/08 by Jay Unruh, Research Specialist, Stowers Institute for Medical Research


public class csv_reader implements PlugIn {
    int words = 0, chars = 0, lines = 0, width=1;;
    String directory, name, path;
    boolean hideErrorMessages;

    public void run(String arg) {
        if (showDialog()) {
            IJ.showStatus("Opening: " + path);
            ImageProcessor ip = open(path);
            if (ip!=null)
                new ImagePlus(name, ip).show();
        }
    }

    boolean showDialog() {
        OpenDialog od = new OpenDialog("Open Text Image...", null);
        directory = od.getDirectory();
        name = od.getFileName();
        if (name!=null)
            path = directory + name;
        return name!=null;
    }

    /** Displays a file open dialog and opens the specified text file as a float image. */
    public ImageProcessor open(){
        if (showDialog())
            return open(path);
        else
            return null;
    }

    /** Opens the specified text file as a float image. */
    public ImageProcessor open(String path){
        ImageProcessor ip = null;
        try {
            words = chars = lines = 0;
            Reader r = new BufferedReader(new FileReader(path));
            countLines(r);
            r.close();
            r = new BufferedReader(new FileReader(path));
            //int width = words/lines;
            float[] pixels = new float[width*lines];
            ip = new FloatProcessor(width, lines, pixels, null);
            read(r, width*lines, pixels);
            r.close();
            ip.resetMinAndMax();
        }
        catch (IOException e) {
            String msg = e.getMessage();
            if (msg==null || msg.equals(""))
                msg = ""+e;
            if (!hideErrorMessages)
                IJ.error("TextReader", msg);
            ip = null;
        }
        return ip;
    }

    public void hideErrorMessages() {
        hideErrorMessages = true;
    }

    /** Returns the file name. */
    public String getName() {
        return name;
    }

    void countLines(Reader r) throws IOException {
        StreamTokenizer tok = new StreamTokenizer(r);
        int wordsPerLine=0, wordsInPreviousLine=0;

        tok.resetSyntax();
        tok.wordChars(46, 127);
        tok.whitespaceChars(0, 45);
        tok.whitespaceChars(128, 255);
        tok.eolIsSignificant(true);

        while (tok.nextToken() != StreamTokenizer.TT_EOF) {
            switch (tok.ttype) {
                case StreamTokenizer.TT_EOL:
                    lines++;
                    if (wordsPerLine==0)
                        lines--;  // ignore empty lines
                    if (lines==1)
                        width = wordsPerLine;
                    else if (wordsPerLine!=0 && wordsPerLine!=wordsInPreviousLine)
                         throw new IOException("Line "+lines+ " is not the same length as the first line.");
                    if (wordsPerLine!=0)
                        wordsInPreviousLine = wordsPerLine;
                    wordsPerLine = 0;
                    if (lines%20==0 && width>1 && lines<=width)
                        IJ.showProgress(((double)lines/width)/2.0);
                    break;
                case StreamTokenizer.TT_WORD:
                    words++;
                    wordsPerLine++;
                    break;
            }
        }
        if (wordsPerLine==width)
            lines++; // last line does not end with EOL
   }

    void read(Reader r, int size, float[] pixels) throws IOException {
        StreamTokenizer tok = new StreamTokenizer(r);
        tok.resetSyntax();
        //the next two lines were changed by JU to allow for comma delimiters
        tok.wordChars(46, 127);
        tok.whitespaceChars(0, 45);
        tok.whitespaceChars(128, 255);
        //tok.parseNumbers();

        int i = 0;
        int inc = size/20;
        if (inc<1)
            inc = 1;
        while (tok.nextToken() != StreamTokenizer.TT_EOF) {
            if (tok.ttype==StreamTokenizer.TT_WORD) {
                pixels[i++] = (float)Tools.parseDouble(tok.sval, 0.0);
                 if (i==size)
                     break;
                 if (i%inc==0)
                     IJ.showProgress(0.5+((double)i/size)/2.0);
            }
        }
        IJ.showProgress(1.0);
    }

}

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Dwight Bartholomew
Sent: Tuesday, April 22, 2008 11:13 AM
To: [hidden email]
Subject: Reading in Image Files (Hopefully a "re-inventing the wheel" sort of question)

There are a couple of types of image files I'm having trouble reading.



1) The comma-deliminated floating-point file:

         Nrows of lines with Ncols of floating-point number separated by commas

         Example:

0.359534,0.36763,0.365069,0.377957,0.369861, <etc>

0.385474,0.396379,0.397619,0.399188,0.39291, <etc>

0.398032,0.403897,0.406541,0.411498,0.403815, <etc>

<etc>



   When I try to read the file (import as a "text image"), I get an image that is Nrows x 1.

   I tried writing a PlugIn that reads in each line, splits the string using the split() function and put each number into an array but WOW that takes a really long time!  There MUST be a faster way and I've got a feeling this has been done before.



2) The comma-deliminated integer file with 4 lines of header
information:

         Example:

This is the first line of header info

This is the second line of header info

This is the third line of header info

This is the fourth line of header info

23828,23768,24002,23808,23904,23970,<etc>

23861,23569,23673,23714,23988,23671, <etc>

<etc>



   This is a similar problem.  I can read in the file, drop the first 4 lines, and start splitting the lines but it takes TOO LONG.  There must be a faster way.



Sincerely,

Dwight Bartholomew