Help?

[Charles Davis]

I'm a new imagej user and am having a problem. I know that imagej will
generate the data that I want, because I [somehow] got it to do it once. But
I don't remember the sequence of operations that I went through to get that
result. I have spent 2 weeks, working several hours each day and have not
been able to study and find the answer or try everything that seems
reasonable. My pride and ego is at ZERO! I can't do this w/o help.

 

Background:

 

I'm using a Nikon D810 camera which produces large files. I need a .csv or
.txt file w/ the brightness values. I don't want a Histogram, but the data
behind the Histogram.

 

I have been able to produce files w/ all the pixel data [X, Y, R, G, B] in
one row. That is too large to load into Excel [36,200,000 is too many rows].
Yes, I have successfully attempted to break it up into several smaller
files, but that takes hours to do one file! Then I have to extract the
summary data from each of these 2-dozen+ files and create a global summary
file.

 

As I said, I was able to [somehow] create a .txt file that was organized
differently. I had all the Y's in the rows and all the X's in the columns.
The data in each cell was the gray value [I think]. This had only 4913 rows
x 7359 columns and loaded into Excel [slowly]; it's still big, but
manageable.

 

Sorry for the long intro, getting to my question: How do I recreate another
of those Measurements.txt files, w/ only gray brightness data in cells and
in a 2D array?

 

TIA!


--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Gary Sellani]

If you can exact the data, have you considered sqlite3 for analysis? SQL programs don't choke on large files. Sqlite3 is probably the easiest one to learn.

  Original Message  
From: Charles Davis
Sent: Tuesday, June 14, 2016 12:26 PM
To: [hidden email]
Reply To: [hidden email]
Subject: Help?

I'm a new imagej user and am having a problem. I know that imagej will
generate the data that I want, because I [somehow] got it to do it once. But
I don't remember the sequence of operations that I went through to get that
result. I have spent 2 weeks, working several hours each day and have not
been able to study and find the answer or try everything that seems
reasonable. My pride and ego is at ZERO! I can't do this w/o help.



Background:



I'm using a Nikon D810 camera which produces large files. I need a .csv or
.txt file w/ the brightness values. I don't want a Histogram, but the data
behind the Histogram.



I have been able to produce files w/ all the pixel data [X, Y, R, G, B] in
one row. That is too large to load into Excel [36,200,000 is too many rows].
Yes, I have successfully attempted to break it up into several smaller
files, but that takes hours to do one file! Then I have to extract the
summary data from each of these 2-dozen+ files and create a global summary
file.



As I said, I was able to [somehow] create a .txt file that was organized
differently. I had all the Y's in the rows and all the X's in the columns.
The data in each cell was the gray value [I think]. This had only 4913 rows
x 7359 columns and loaded into Excel [slowly]; it's still big, but
manageable.



Sorry for the long intro, getting to my question: How do I recreate another
of those Measurements.txt files, w/ only gray brightness data in cells and
in a 2D array?



TIA!


--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Melissa Krueger]

On Jun 14, 2016, at 09:10 , Charles Davis <[hidden email]> wrote:

>
> I'm using a Nikon D810 camera which produces large files. I need a .csv or
> .txt file w/ the brightness values. I don't want a Histogram, but the data
> behind the Histogram.


I assume that you are starting with a Nikon RAW file (.nef). What you need to use is the DCRAW plugin <https://sourceforge.net/projects/ij-plugins/files/ij-dcraw/> that I think is automatically installed with Fiji. Run the plugin either from the menu Plugins->Input-Output->DCRAW or from the Command Finder. Make sure the following two options are checked: a) Do not automatically brighten and b) Document mode without scaling. Also make the Read As popup say 16-bit linear. This should read the raw sensor data in without changing the grayscale values.

> Sorry for the long intro, getting to my question: How do I recreate another
> of those Measurements.txt files, w/ only gray brightness data in cells and
> in a 2D array?
>

From the menu, click File->Save As->Text Image... You will end up with a tab-delimited file with the grayscale values.

Melissa

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Charles,

it seems you need a grayscale image first. Assuming you are starting
with RGB, use Image>Type>8 bits.
Make sure you have the correct choice of "Weighted RGB Conversions" in
Edit>Options>Conversions
 
https://imagej.nih.gov/ij/docs/guide/146-27.html#toc-Subsubsection-27.13.11

Then save as text image.

Anyhow, what kind of analysis with all pixels do you want to do in Excel?
I guess that ImageJ could do essentially everything that can be done in
Excel, but much faster.

Michael
________________________________________________________________
On 2016-06-14 18:10, Charles Davis wrote:

> I'm a new imagej user and am having a problem. I know that imagej will
> generate the data that I want, because I [somehow] got it to do it once. But
> I don't remember the sequence of operations that I went through to get that
> result. I have spent 2 weeks, working several hours each day and have not
> been able to study and find the answer or try everything that seems
> reasonable. My pride and ego is at ZERO! I can't do this w/o help.
>
>
>
> Background:
>
>
>
> I'm using a Nikon D810 camera which produces large files. I need a .csv or
> .txt file w/ the brightness values. I don't want a Histogram, but the data
> behind the Histogram.
>
>
>
> I have been able to produce files w/ all the pixel data [X, Y, R, G, B] in
> one row. That is too large to load into Excel [36,200,000 is too many rows].
> Yes, I have successfully attempted to break it up into several smaller
> files, but that takes hours to do one file! Then I have to extract the
> summary data from each of these 2-dozen+ files and create a global summary
> file.
>
>
>
> As I said, I was able to [somehow] create a .txt file that was organized
> differently. I had all the Y's in the rows and all the X's in the columns.
> The data in each cell was the gray value [I think]. This had only 4913 rows
> x 7359 columns and loaded into Excel [slowly]; it's still big, but
> manageable.
>
>
>
> Sorry for the long intro, getting to my question: How do I recreate another
> of those Measurements.txt files, w/ only gray brightness data in cells and
> in a 2D array?
>
>
>
> TIA!
>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Mauro Maiorca]

On Tue, Jun 14, 2016 at 5:10 PM, Charles Davis <[hidden email]> wrote:

>
> I'm using a Nikon D810 camera which produces large files. I need a .csv or
> .txt file w/ the brightness values. I don't want a Histogram, but the data
> behind the Histogram.
>

Dear Charles,

In my understanding you are looking for the histogram values of an image.
If this is the case, you just go to "Analyze->Hitogram" in the menu
(alternatively use CTRL+H) and it will pop up a window with the image
histogram. On that window click on "list" and you have the list of values
(of the histogram) in another window, you can save the values by clicking
"file->save as.." in that window menu. For RGB image in the histogram
window there is a button RGB that gives the values for each single channel
(again, if you want the numbers just click "List").
If you are interested in doing specific analysis for each channel of the
image (i.e. Red, Green or Blue), you can split the channels of the original
RGB image into single grayscale images (Image->Color->Split Channels), and
analyse each channel image independently.


> As I said, I was able to [somehow] create a .txt file that was organized
> differently. I had all the Y's in the rows and all the X's in the columns.
>

you can get this sort of file by saving an image as a text file
("file->Save As->Text Image..."). For RGB images, you might want to split
the channels first (Image->Color->Split Channels) and save each single
channel image as a text file.

Hope it helps,
Mauro

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Charles Davis]

I hope I'm doing this reply correctly [this is my 1st reply].

To Melissa: No I'm loading a JPEG image, not a NEF.

YES! The File>Save As>Text Image creates a text file organized the way I
want. Thanks. But it has no headings on either axis. I may not need them,
but in the Measurements.txt file that I created 2 weeks ago, there was a Yn
column and an Xn row to ID the data. Any clue how to get those?

Thanks.

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
Melissa Krueger
Sent: Tuesday, June 14, 2016 1:49 PM
To: [hidden email]
Subject: Re: Help?

On Jun 14, 2016, at 09:10 , Charles Davis <[hidden email]> wrote:

>
> I'm using a Nikon D810 camera which produces large files. I need a .csv or
> .txt file w/ the brightness values. I don't want a Histogram, but the data
> behind the Histogram.


I assume that you are starting with a Nikon RAW file (.nef). What you need
to use is the DCRAW plugin
<https://sourceforge.net/projects/ij-plugins/files/ij-dcraw/> that I think
is automatically installed with Fiji. Run the plugin either from the menu
Plugins->Input-Output->DCRAW or from the Command Finder. Make sure the
following two options are checked: a) Do not automatically brighten and b)
Document mode without scaling. Also make the Read As popup say 16-bit
linear. This should read the raw sensor data in without changing the
grayscale values.

> Sorry for the long intro, getting to my question: How do I recreate
another
> of those Measurements.txt files, w/ only gray brightness data in cells and
> in a 2D array?
>

From the menu, click File->Save As->Text Image... You will end up with a
tab-delimited file with the grayscale values.

Melissa

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Charles Davis]

Michael, I had figured out that I needed to convert to grayscale. Otherwise
I assumed I would get RGB values. I also dug deeply enough to have found the
choice of grey conversion. As I'm not doing perceptual analysis, I kept the
non-weighted conversion.

The answer was to simply File>Save As>Text Image. DUH!

I am comparing two images that are very similar. I have been subtracting
them w/ Photoshop, but have also found that imagej can do that subtraction.
:-)

I am wanting to count pixel values and use a custom algorithm to quantify
the difference. I probably could use imagej, but the learning curve is very
steep and I am comfortable w/ Excel.

Thanks...

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
Michael Schmid
Sent: Tuesday, June 14, 2016 1:52 PM
To: [hidden email]
Subject: Re: Help?

Hi Charles,

it seems you need a grayscale image first. Assuming you are starting
with RGB, use Image>Type>8 bits.
Make sure you have the correct choice of "Weighted RGB Conversions" in
Edit>Options>Conversions
 
https://imagej.nih.gov/ij/docs/guide/146-27.html#toc-Subsubsection-27.13.11

Then save as text image.

Anyhow, what kind of analysis with all pixels do you want to do in Excel?
I guess that ImageJ could do essentially everything that can be done in
Excel, but much faster.

Michael
________________________________________________________________
On 2016-06-14 18:10, Charles Davis wrote:
> I'm a new imagej user and am having a problem. I know that imagej will
> generate the data that I want, because I [somehow] got it to do it once.
But
> I don't remember the sequence of operations that I went through to get
that

> result. I have spent 2 weeks, working several hours each day and have not
> been able to study and find the answer or try everything that seems
> reasonable. My pride and ego is at ZERO! I can't do this w/o help.
>
>
>
> Background:
>
>
>
> I'm using a Nikon D810 camera which produces large files. I need a .csv or
> .txt file w/ the brightness values. I don't want a Histogram, but the data
> behind the Histogram.
>
>
>
> I have been able to produce files w/ all the pixel data [X, Y, R, G, B] in
> one row. That is too large to load into Excel [36,200,000 is too many

rows].

> Yes, I have successfully attempted to break it up into several smaller
> files, but that takes hours to do one file! Then I have to extract the
> summary data from each of these 2-dozen+ files and create a global summary
> file.
>
>
>
> As I said, I was able to [somehow] create a .txt file that was organized
> differently. I had all the Y's in the rows and all the X's in the columns.
> The data in each cell was the gray value [I think]. This had only 4913

rows
> x 7359 columns and loaded into Excel [slowly]; it's still big, but
> manageable.
>
>
>
> Sorry for the long intro, getting to my question: How do I recreate
another

> of those Measurements.txt files, w/ only gray brightness data in cells and
> in a 2D array?
>
>
>
> TIA!
>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Charles,

if you use JPG images, please don't expect any quantitative results!

(1) JPG is a lossy compression, with lots of compression artifacts.
See
http://imagej.net/Principles#Why_.28lossy.29_JPEGs_should_not_be_used_in_imaging

Especially the color information (chroma) has low resolution and many
artifacts. If you want to compare gray level, make sure you use weighted
conversion, otherwise you have to reduce the resolution by at least a
factor of 2 to get rid of at least the worst compression artifacts.

(2) JPG has a nonlinear relation between pixel value and intensity. If
you calculate differences, the result will NOT be the difference of the
intensity values.


Michael
________________________________________________________________
On 2016-06-14 22:20, Charles Davis wrote:
 > I hope I'm doing this reply correctly [this is my 1st reply].
 >
 > To Melissa: No I'm loading a JPEG image, not a NEF.

On 2016-06-14 22:30, Charles Davis wrote:

> Michael, I had figured out that I needed to convert to grayscale. Otherwise
> I assumed I would get RGB values. I also dug deeply enough to have found the
> choice of grey conversion. As I'm not doing perceptual analysis, I kept the
> non-weighted conversion.
>
> The answer was to simply File>Save As>Text Image. DUH!
>
> I am comparing two images that are very similar. I have been subtracting
> them w/ Photoshop, but have also found that imagej can do that subtraction.
> :-)
>
> I am wanting to count pixel values and use a custom algorithm to quantify
> the difference. I probably could use imagej, but the learning curve is very
> steep and I am comfortable w/ Excel.
>
> Thanks...
>
> -----Original Message-----
> From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
> Michael Schmid
> Sent: Tuesday, June 14, 2016 1:52 PM
> To: [hidden email]
> Subject: Re: Help?
>
> Hi Charles,
>
> it seems you need a grayscale image first. Assuming you are starting
> with RGB, use Image>Type>8 bits.
> Make sure you have the correct choice of "Weighted RGB Conversions" in
> Edit>Options>Conversions
>
> https://imagej.nih.gov/ij/docs/guide/146-27.html#toc-Subsubsection-27.13.11
>
> Then save as text image.
>
> Anyhow, what kind of analysis with all pixels do you want to do in Excel?
> I guess that ImageJ could do essentially everything that can be done in
> Excel, but much faster.
>
> Michael
> ________________________________________________________________
> On 2016-06-14 18:10, Charles Davis wrote:
>> I'm a new imagej user and am having a problem. I know that imagej will
>> generate the data that I want, because I [somehow] got it to do it once.
> But
>> I don't remember the sequence of operations that I went through to get
> that
>> result. I have spent 2 weeks, working several hours each day and have not
>> been able to study and find the answer or try everything that seems
>> reasonable. My pride and ego is at ZERO! I can't do this w/o help.
>>
>>
>>
>> Background:
>>
>>
>>
>> I'm using a Nikon D810 camera which produces large files. I need a .csv or
>> .txt file w/ the brightness values. I don't want a Histogram, but the data
>> behind the Histogram.
>>
>>
>>
>> I have been able to produce files w/ all the pixel data [X, Y, R, G, B] in
>> one row. That is too large to load into Excel [36,200,000 is too many
> rows].
>> Yes, I have successfully attempted to break it up into several smaller
>> files, but that takes hours to do one file! Then I have to extract the
>> summary data from each of these 2-dozen+ files and create a global summary
>> file.
>>
>>
>>
>> As I said, I was able to [somehow] create a .txt file that was organized
>> differently. I had all the Y's in the rows and all the X's in the columns.
>> The data in each cell was the gray value [I think]. This had only 4913
> rows
>> x 7359 columns and loaded into Excel [slowly]; it's still big, but
>> manageable.
>>
>>
>>
>> Sorry for the long intro, getting to my question: How do I recreate
> another
>> of those Measurements.txt files, w/ only gray brightness data in cells and
>> in a 2D array?
>>
>>
>>
>> TIA!
>>
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Charles Davis]

Hi, again Michael,

I agree. BUT what I'm doing is quantifying how different RAW editors render
JPEG images.
Thus, I HAVE to use JPEGs. :-)

Thanks for your help.

-----Original Message-----
From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
Michael Schmid
Sent: Wednesday, June 15, 2016 3:04 AM
To: [hidden email]
Subject: Re: Help?

Hi Charles,

if you use JPG images, please don't expect any quantitative results!

(1) JPG is a lossy compression, with lots of compression artifacts.
See
http://imagej.net/Principles#Why_.28lossy.29_JPEGs_should_not_be_used_in_ima
ging

Especially the color information (chroma) has low resolution and many
artifacts. If you want to compare gray level, make sure you use weighted
conversion, otherwise you have to reduce the resolution by at least a
factor of 2 to get rid of at least the worst compression artifacts.

(2) JPG has a nonlinear relation between pixel value and intensity. If
you calculate differences, the result will NOT be the difference of the
intensity values.


Michael
________________________________________________________________
On 2016-06-14 22:20, Charles Davis wrote:
 > I hope I'm doing this reply correctly [this is my 1st reply].
 >
 > To Melissa: No I'm loading a JPEG image, not a NEF.

On 2016-06-14 22:30, Charles Davis wrote:
> Michael, I had figured out that I needed to convert to grayscale.
Otherwise
> I assumed I would get RGB values. I also dug deeply enough to have found
the
> choice of grey conversion. As I'm not doing perceptual analysis, I kept
the
> non-weighted conversion.
>
> The answer was to simply File>Save As>Text Image. DUH!
>
> I am comparing two images that are very similar. I have been subtracting
> them w/ Photoshop, but have also found that imagej can do that
subtraction.
> :-)
>
> I am wanting to count pixel values and use a custom algorithm to quantify
> the difference. I probably could use imagej, but the learning curve is
very

> steep and I am comfortable w/ Excel.
>
> Thanks...
>
> -----Original Message-----
> From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
> Michael Schmid
> Sent: Tuesday, June 14, 2016 1:52 PM
> To: [hidden email]
> Subject: Re: Help?
>
> Hi Charles,
>
> it seems you need a grayscale image first. Assuming you are starting
> with RGB, use Image>Type>8 bits.
> Make sure you have the correct choice of "Weighted RGB Conversions" in
> Edit>Options>Conversions
>
>

https://imagej.nih.gov/ij/docs/guide/146-27.html#toc-Subsubsection-27.13.11

>
> Then save as text image.
>
> Anyhow, what kind of analysis with all pixels do you want to do in Excel?
> I guess that ImageJ could do essentially everything that can be done in
> Excel, but much faster.
>
> Michael
> ________________________________________________________________
> On 2016-06-14 18:10, Charles Davis wrote:
>> I'm a new imagej user and am having a problem. I know that imagej will
>> generate the data that I want, because I [somehow] got it to do it once.
> But
>> I don't remember the sequence of operations that I went through to get
> that
>> result. I have spent 2 weeks, working several hours each day and have not
>> been able to study and find the answer or try everything that seems
>> reasonable. My pride and ego is at ZERO! I can't do this w/o help.
>>
>>
>>
>> Background:
>>
>>
>>
>> I'm using a Nikon D810 camera which produces large files. I need a .csv

or
>> .txt file w/ the brightness values. I don't want a Histogram, but the
data
>> behind the Histogram.
>>
>>
>>
>> I have been able to produce files w/ all the pixel data [X, Y, R, G, B]
in
>> one row. That is too large to load into Excel [36,200,000 is too many
> rows].
>> Yes, I have successfully attempted to break it up into several smaller
>> files, but that takes hours to do one file! Then I have to extract the
>> summary data from each of these 2-dozen+ files and create a global
summary
>> file.
>>
>>
>>
>> As I said, I was able to [somehow] create a .txt file that was organized
>> differently. I had all the Y's in the rows and all the X's in the
columns.

>> The data in each cell was the gray value [I think]. This had only 4913
> rows
>> x 7359 columns and loaded into Excel [slowly]; it's still big, but
>> manageable.
>>
>>
>>
>> Sorry for the long intro, getting to my question: How do I recreate
> another
>> of those Measurements.txt files, w/ only gray brightness data in cells

and

>> in a 2D array?
>>
>>
>>
>> TIA!
>>
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html