Reading in Image Files (Hopefully a "re-inventing the wheel" sort of question)

[DwightBart]

There are a couple of types of image files I'm having trouble reading.

 

1) The comma-deliminated floating-point file:

         Nrows of lines with Ncols of floating-point number separated by
commas

         Example:

0.359534,0.36763,0.365069,0.377957,0.369861, <etc>

0.385474,0.396379,0.397619,0.399188,0.39291, <etc>

0.398032,0.403897,0.406541,0.411498,0.403815, <etc>

<etc>

   

   When I try to read the file (import as a "text image"), I get an
image that is Nrows x 1.

   I tried writing a PlugIn that reads in each line, splits the string
using the split() function and put each number into an array but WOW
that takes a really long time!  There MUST be a faster way and I've got
a feeling this has been done before.

 

2) The comma-deliminated integer file with 4 lines of header
information:

         Example:

This is the first line of header info

This is the second line of header info

This is the third line of header info

This is the fourth line of header info

23828,23768,24002,23808,23904,23970,<etc>

23861,23569,23673,23714,23988,23671, <etc>

<etc>

   

   This is a similar problem.  I can read in the file, drop the first 4
lines, and start splitting the lines but it takes TOO LONG.  There must
be a faster way.

 

Sincerely,

Dwight Bartholomew

 

[Harry Parker]

Hello Dwight,

I tried opening your sample images.
I was able to get ImageJ to read them by simply adding a space after each comma and deleting the headers with a text editor.
You can do add the spaces easily with any programmer's text editor's Search and Replace feature.

If you have many images like that could write a little script to do that to all your files. (Or get a programmer friend to do it.)
It would be about 5 to 10 lines in Python or Perl, for example.
 
--  
Harry Parker  
Senior Systems Engineer  
Digital Imaging Systems, Inc.

----- Original Message ----
From: Dwight Bartholomew <[hidden email]>
To: [hidden email]
Sent: Tuesday, April 22, 2008 12:13:22 PM
Subject: Reading in Image Files (Hopefully a "re-inventing the wheel" sort of question)

There are a couple of types of image files I'm having trouble reading.

 

1) The comma-deliminated floating-point file:

         Nrows of lines with Ncols of floating-point number separated by
commas

         Example:

0.359534,0.36763,0.365069,0.377957,0.369861, <etc>

0.385474,0.396379,0.397619,0.399188,0.39291, <etc>

0.398032,0.403897,0.406541,0.411498,0.403815, <etc>

<etc>

   

   When I try to read the file (import as a "text image"), I get an
image that is Nrows x 1.

   I tried writing a PlugIn that reads in each line, splits the string
using the split() function and put each number into an array but WOW
that takes a really long time!  There MUST be a faster way and I've got
a feeling this has been done before.

 

2) The comma-deliminated integer file with 4 lines of header
information:

         Example:

This is the first line of header info

This is the second line of header info

This is the third line of header info

This is the fourth line of header info

23828,23768,24002,23808,23904,23970,<etc>

23861,23569,23673,23714,23988,23671, <etc>

<etc>

   

   This is a similar problem.  I can read in the file, drop the first 4
lines, and start splitting the lines but it takes TOO LONG.  There must
be a faster way.

 

Sincerely,

Dwight Bartholomew

 





      ____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

[Gluender]

>There are a couple of types of image files I'm having trouble reading.
>
>1) The comma-deliminated floating-point file:
>
>          Nrows of lines with Ncols of floating-point number separated by
>commas
>
>          Example:
>
>0.359534,0.36763,0.365069,0.377957,0.369861, <etc>
>
>0.385474,0.396379,0.397619,0.399188,0.39291, <etc>
>
>0.398032,0.403897,0.406541,0.411498,0.403815, <etc>
>
><etc>
>
>    When I try to read the file (import as a "text image"), I get an
>image that is Nrows x 1.
>
>    I tried writing a PlugIn that reads in each line, splits the string
>using the split() function and put each number into an array but WOW
>that takes a really long time!  There MUST be a faster way and I've got
>a feeling this has been done before.
>
>2) The comma-deliminated integer file with 4 lines of header
>information:
>
>          Example:
>
>This is the first line of header info
>
>This is the second line of header info
>
>This is the third line of header info
>
>This is the fourth line of header info
>
>23828,23768,24002,23808,23904,23970,<etc>
>
>23861,23569,23673,23714,23988,23671, <etc>
>
><etc>
>
>    This is a similar problem.  I can read in the file, drop the first 4
>lines, and start splitting the lines but it takes TOO LONG.  There must
>be a faster way.
>
>Sincerely,
>Dwight Bartholomew

If you know N then try "Import>Raw..." and please read the IJ-docs.

HTH

herbie

[Unruh, Jay]

ImageJ community,

Below is a csv_reader.java file that is just a copy of the text_reader plugin that is built in in ImageJ with a few edits for comma delimiters in addition to tab delimiters.  The key is the definition of word chars and white space chars in the read method.  This may or may not be a good edit to make permanent considering the different uses of commas throughout the world.

Jay Unruh

import ij.*;
import ij.process.*;
import ij.gui.*;
import java.awt.*;
import ij.plugin.*;
import java.awt.image.*;
import java.io.*;
import ij.io.*;
import ij.util.Tools;
//here is a csv reader.  This is just a copy of text_reader.java with the tokenizer changed to accept commas as delimiters
//changes made 4/22/08 by Jay Unruh, Research Specialist, Stowers Institute for Medical Research


public class csv_reader implements PlugIn {
    int words = 0, chars = 0, lines = 0, width=1;;
    String directory, name, path;
    boolean hideErrorMessages;

    public void run(String arg) {
        if (showDialog()) {
            IJ.showStatus("Opening: " + path);
            ImageProcessor ip = open(path);
            if (ip!=null)
                new ImagePlus(name, ip).show();
        }
    }

    boolean showDialog() {
        OpenDialog od = new OpenDialog("Open Text Image...", null);
        directory = od.getDirectory();
        name = od.getFileName();
        if (name!=null)
            path = directory + name;
        return name!=null;
    }

    /** Displays a file open dialog and opens the specified text file as a float image. */
    public ImageProcessor open(){
        if (showDialog())
            return open(path);
        else
            return null;
    }

    /** Opens the specified text file as a float image. */
    public ImageProcessor open(String path){
        ImageProcessor ip = null;
        try {
            words = chars = lines = 0;
            Reader r = new BufferedReader(new FileReader(path));
            countLines(r);
            r.close();
            r = new BufferedReader(new FileReader(path));
            //int width = words/lines;
            float[] pixels = new float[width*lines];
            ip = new FloatProcessor(width, lines, pixels, null);
            read(r, width*lines, pixels);
            r.close();
            ip.resetMinAndMax();
        }
        catch (IOException e) {
            String msg = e.getMessage();
            if (msg==null || msg.equals(""))
                msg = ""+e;
            if (!hideErrorMessages)
                IJ.error("TextReader", msg);
            ip = null;
        }
        return ip;
    }

    public void hideErrorMessages() {
        hideErrorMessages = true;
    }

    /** Returns the file name. */
    public String getName() {
        return name;
    }

    void countLines(Reader r) throws IOException {
        StreamTokenizer tok = new StreamTokenizer(r);
        int wordsPerLine=0, wordsInPreviousLine=0;

        tok.resetSyntax();
        tok.wordChars(46, 127);
        tok.whitespaceChars(0, 45);
        tok.whitespaceChars(128, 255);
        tok.eolIsSignificant(true);

        while (tok.nextToken() != StreamTokenizer.TT_EOF) {
            switch (tok.ttype) {
                case StreamTokenizer.TT_EOL:
                    lines++;
                    if (wordsPerLine==0)
                        lines--;  // ignore empty lines
                    if (lines==1)
                        width = wordsPerLine;
                    else if (wordsPerLine!=0 && wordsPerLine!=wordsInPreviousLine)
                         throw new IOException("Line "+lines+ " is not the same length as the first line.");
                    if (wordsPerLine!=0)
                        wordsInPreviousLine = wordsPerLine;
                    wordsPerLine = 0;
                    if (lines%20==0 && width>1 && lines<=width)
                        IJ.showProgress(((double)lines/width)/2.0);
                    break;
                case StreamTokenizer.TT_WORD:
                    words++;
                    wordsPerLine++;
                    break;
            }
        }
        if (wordsPerLine==width)
            lines++; // last line does not end with EOL
   }

    void read(Reader r, int size, float[] pixels) throws IOException {
        StreamTokenizer tok = new StreamTokenizer(r);
        tok.resetSyntax();
        //the next two lines were changed by JU to allow for comma delimiters
        tok.wordChars(46, 127);
        tok.whitespaceChars(0, 45);
        tok.whitespaceChars(128, 255);
        //tok.parseNumbers();

        int i = 0;
        int inc = size/20;
        if (inc<1)
            inc = 1;
        while (tok.nextToken() != StreamTokenizer.TT_EOF) {
            if (tok.ttype==StreamTokenizer.TT_WORD) {
                pixels[i++] = (float)Tools.parseDouble(tok.sval, 0.0);
                 if (i==size)
                     break;
                 if (i%inc==0)
                     IJ.showProgress(0.5+((double)i/size)/2.0);
            }
        }
        IJ.showProgress(1.0);
    }

}

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Dwight Bartholomew
Sent: Tuesday, April 22, 2008 11:13 AM
To: [hidden email]
Subject: Reading in Image Files (Hopefully a "re-inventing the wheel" sort of question)

There are a couple of types of image files I'm having trouble reading.



1) The comma-deliminated floating-point file:

         Nrows of lines with Ncols of floating-point number separated by commas

         Example:

0.359534,0.36763,0.365069,0.377957,0.369861, <etc>

0.385474,0.396379,0.397619,0.399188,0.39291, <etc>

0.398032,0.403897,0.406541,0.411498,0.403815, <etc>

<etc>



   When I try to read the file (import as a "text image"), I get an image that is Nrows x 1.

   I tried writing a PlugIn that reads in each line, splits the string using the split() function and put each number into an array but WOW that takes a really long time!  There MUST be a faster way and I've got a feeling this has been done before.



2) The comma-deliminated integer file with 4 lines of header
information:

         Example:

This is the first line of header info

This is the second line of header info

This is the third line of header info

This is the fourth line of header info

23828,23768,24002,23808,23904,23970,<etc>

23861,23569,23673,23714,23988,23671, <etc>

<etc>



   This is a similar problem.  I can read in the file, drop the first 4 lines, and start splitting the lines but it takes TOO LONG.  There must be a faster way.



Sincerely,

Dwight Bartholomew

[DwightBart]

Thanks!

That works great on my first problem.  I'm sure I can modify it for my
second problem.

Dwight

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
Unruh, Jay
Sent: Tuesday, April 22, 2008 12:21 PM
To: [hidden email]
Subject: Re: Reading in Image Files (Hopefully a "re-inventing the
wheel" sort of question)

ImageJ community,

Below is a csv_reader.java file that is just a copy of the text_reader
plugin that is built in in ImageJ with a few edits for comma delimiters
in addition to tab delimiters.  The key is the definition of word chars
and white space chars in the read method.  This may or may not be a good
edit to make permanent considering the different uses of commas
throughout the world.

Jay Unruh

import ij.*;
import ij.process.*;
import ij.gui.*;
import java.awt.*;
import ij.plugin.*;
import java.awt.image.*;
import java.io.*;
import ij.io.*;
import ij.util.Tools;
//here is a csv reader.  This is just a copy of text_reader.java with
the tokenizer changed to accept commas as delimiters
//changes made 4/22/08 by Jay Unruh, Research Specialist, Stowers
Institute for Medical Research


public class csv_reader implements PlugIn {
    int words = 0, chars = 0, lines = 0, width=1;;
    String directory, name, path;
    boolean hideErrorMessages;

    public void run(String arg) {
        if (showDialog()) {
            IJ.showStatus("Opening: " + path);
            ImageProcessor ip = open(path);
            if (ip!=null)
                new ImagePlus(name, ip).show();
        }
    }

    boolean showDialog() {
        OpenDialog od = new OpenDialog("Open Text Image...", null);
        directory = od.getDirectory();
        name = od.getFileName();
        if (name!=null)
            path = directory + name;
        return name!=null;
    }

    /** Displays a file open dialog and opens the specified text file as
a float image. */
    public ImageProcessor open(){
        if (showDialog())
            return open(path);
        else
            return null;
    }

    /** Opens the specified text file as a float image. */
    public ImageProcessor open(String path){
        ImageProcessor ip = null;
        try {
            words = chars = lines = 0;
            Reader r = new BufferedReader(new FileReader(path));
            countLines(r);
            r.close();
            r = new BufferedReader(new FileReader(path));
            //int width = words/lines;
            float[] pixels = new float[width*lines];
            ip = new FloatProcessor(width, lines, pixels, null);
            read(r, width*lines, pixels);
            r.close();
            ip.resetMinAndMax();
        }
        catch (IOException e) {
            String msg = e.getMessage();
            if (msg==null || msg.equals(""))
                msg = ""+e;
            if (!hideErrorMessages)
                IJ.error("TextReader", msg);
            ip = null;
        }
        return ip;
    }

    public void hideErrorMessages() {
        hideErrorMessages = true;
    }

    /** Returns the file name. */
    public String getName() {
        return name;
    }

    void countLines(Reader r) throws IOException {
        StreamTokenizer tok = new StreamTokenizer(r);
        int wordsPerLine=0, wordsInPreviousLine=0;

        tok.resetSyntax();
        tok.wordChars(46, 127);
        tok.whitespaceChars(0, 45);
        tok.whitespaceChars(128, 255);
        tok.eolIsSignificant(true);

        while (tok.nextToken() != StreamTokenizer.TT_EOF) {
            switch (tok.ttype) {
                case StreamTokenizer.TT_EOL:
                    lines++;
                    if (wordsPerLine==0)
                        lines--;  // ignore empty lines
                    if (lines==1)
                        width = wordsPerLine;
                    else if (wordsPerLine!=0 &&
wordsPerLine!=wordsInPreviousLine)
                         throw new IOException("Line "+lines+ " is not
the same length as the first line.");
                    if (wordsPerLine!=0)
                        wordsInPreviousLine = wordsPerLine;
                    wordsPerLine = 0;
                    if (lines%20==0 && width>1 && lines<=width)
                        IJ.showProgress(((double)lines/width)/2.0);
                    break;
                case StreamTokenizer.TT_WORD:
                    words++;
                    wordsPerLine++;
                    break;
            }
        }
        if (wordsPerLine==width)
            lines++; // last line does not end with EOL
   }

    void read(Reader r, int size, float[] pixels) throws IOException {
        StreamTokenizer tok = new StreamTokenizer(r);
        tok.resetSyntax();
        //the next two lines were changed by JU to allow for comma
delimiters
        tok.wordChars(46, 127);
        tok.whitespaceChars(0, 45);
        tok.whitespaceChars(128, 255);
        //tok.parseNumbers();

        int i = 0;
        int inc = size/20;
        if (inc<1)
            inc = 1;
        while (tok.nextToken() != StreamTokenizer.TT_EOF) {
            if (tok.ttype==StreamTokenizer.TT_WORD) {
                pixels[i++] = (float)Tools.parseDouble(tok.sval, 0.0);
                 if (i==size)
                     break;
                 if (i%inc==0)
                     IJ.showProgress(0.5+((double)i/size)/2.0);
            }
        }
        IJ.showProgress(1.0);
    }

}

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
Dwight Bartholomew
Sent: Tuesday, April 22, 2008 11:13 AM
To: [hidden email]
Subject: Reading in Image Files (Hopefully a "re-inventing the wheel"
sort of question)

There are a couple of types of image files I'm having trouble reading.



1) The comma-deliminated floating-point file:

         Nrows of lines with Ncols of floating-point number separated by
commas

         Example:

0.359534,0.36763,0.365069,0.377957,0.369861, <etc>

0.385474,0.396379,0.397619,0.399188,0.39291, <etc>

0.398032,0.403897,0.406541,0.411498,0.403815, <etc>

<etc>



   When I try to read the file (import as a "text image"), I get an
image that is Nrows x 1.

   I tried writing a PlugIn that reads in each line, splits the string
using the split() function and put each number into an array but WOW
that takes a really long time!  There MUST be a faster way and I've got
a feeling this has been done before.



2) The comma-deliminated integer file with 4 lines of header
information:

         Example:

This is the first line of header info

This is the second line of header info

This is the third line of header info

This is the fourth line of header info

23828,23768,24002,23808,23904,23970,<etc>

23861,23569,23673,23714,23988,23671, <etc>

<etc>



   This is a similar problem.  I can read in the file, drop the first 4
lines, and start splitting the lines but it takes TOO LONG.  There must
be a faster way.



Sincerely,

Dwight Bartholomew