Problem saving PNG files in particular order

[Robert Lockwood]

I have been writing software that controls a one-of-a-kind research grade
aerial infrared camera running the Java code from the Eclipse editor
(Ubuntu 14.04). When deployed scripts initialize all the code and start the
various pieces running.

I import the data, 16 bit unsigned, into IJ and save it as 16 bit unsigned
PNG ('raw' data).  Then I use the data already in IJ, assign a LUT, etc.
and save this now displayable PMG file to different folder and tell my
webpage GUI to get and display the file for the operator to view.

This works fine when I run the code from Eclipse but fails when I use the
scripts to automatically start the code when the (headless) computer boots.

What I think is happening is that IJ tells the OS to save the 16 bit data
file and the OS says, OK it's a write - that's not a priority.  Then the
displayable file is saved and now there may be two files in the queue.  The
message is sent to the webpage, the webpage requests the displayable file
but it's not yet saved.  The notebook, iPad, etc. that is the webpage
client is either connected directly to the server or to the router, there's
no traffic going across the internet so it's quite fast.

There are a couple code snippets following.

I'd like to try to force the OS to save the files quickly, but don't know
if that's possible in IJ.  I'd also like to save the displayable file with
its LUT first but don't know how to do that without making a second copy of
the raw data.

Is it possible to use the IJ objects after the LUT has been applied and
'recover' (so to speak) the 16 bit data and save it?

I'd appreciate any suggestions ...

  // ----- save 'raw' data as 16 bit unsigned short as PNG -----
  ImagePlus imp = IJ.createImage("Untitled", imageWidth, imageHeight, 1
,imageBitsPerPix );
 ShortProcessor ip = (ShortProcessor) imp.getProcessor();
 short[] data = (short[]) ip.getPixels();
// copy pixel data to IJ ip
  imageByteBuffer.position(0);
  int index = 0;
  while(imageByteBuffer.hasRemaining())
  data[index++] = imageByteBuffer.getShort();
  if(State.isRecording())  {
      IJ.saveAs(imp, "PNG", imageFullPathName);

 // ---- apply LUT to data already in IJ  and save it ----
 if (useLUT) {
 ImageStatistics stats = ip.getStatistics();
  ip.setMinAndMax(stats.histMin, stats.histMax);
  ip.setLut(Shared.getLUT());
 imp.setProcessor("LUT", ip);
  final String imageName = getDisplayFileName();
  final String imageFullPathname = imageDisplyFolderPath + imageName;
  IJ.saveAs(imp, "PNG", imageFullPathname);
   }

Thanks in advance
Nate

One's first step in wisdom is to question everything - and one's last is to
come to terms with everything.

Georg C. Lichtenberg

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Robert,

in case the problem occurs because the file has not been written yet while the next command is executed:
Maybe have a loop, waiting until the file exists?

In ImageJ macro language, something like the following:

for (i=0; i<100; i++) {
  if (File.exists(path) && File.length(path)>100) i=100; //break the loop
  if (i==99) exit("timeout, did not save so far");
  wait(50);
}

It won't tell you whether the file is has been completely written. Some operating systems might report zero length for a file that is unfinished, but I would not rely on that.

Michael
________________________________________________________________
On Mar 16, 2015, at 00:33, Robert Lockwood wrote:

> I have been writing software that controls a one-of-a-kind research grade
> aerial infrared camera running the Java code from the Eclipse editor
> (Ubuntu 14.04). When deployed scripts initialize all the code and start the
> various pieces running.
>
> I import the data, 16 bit unsigned, into IJ and save it as 16 bit unsigned
> PNG ('raw' data).  Then I use the data already in IJ, assign a LUT, etc.
> and save this now displayable PMG file to different folder and tell my
> webpage GUI to get and display the file for the operator to view.
>
> This works fine when I run the code from Eclipse but fails when I use the
> scripts to automatically start the code when the (headless) computer boots.
>
> What I think is happening is that IJ tells the OS to save the 16 bit data
> file and the OS says, OK it's a write - that's not a priority.  Then the
> displayable file is saved and now there may be two files in the queue.  The
> message is sent to the webpage, the webpage requests the displayable file
> but it's not yet saved.  The notebook, iPad, etc. that is the webpage
> client is either connected directly to the server or to the router, there's
> no traffic going across the internet so it's quite fast.
>
> There are a couple code snippets following.
>
> I'd like to try to force the OS to save the files quickly, but don't know
> if that's possible in IJ.  I'd also like to save the displayable file with
> its LUT first but don't know how to do that without making a second copy of
> the raw data.
>
> Is it possible to use the IJ objects after the LUT has been applied and
> 'recover' (so to speak) the 16 bit data and save it?
>
> I'd appreciate any suggestions ...
>
>  // ----- save 'raw' data as 16 bit unsigned short as PNG -----
>  ImagePlus imp = IJ.createImage("Untitled", imageWidth, imageHeight, 1
> ,imageBitsPerPix );
> ShortProcessor ip = (ShortProcessor) imp.getProcessor();
> short[] data = (short[]) ip.getPixels();
> // copy pixel data to IJ ip
>  imageByteBuffer.position(0);
>  int index = 0;
>  while(imageByteBuffer.hasRemaining())
>  data[index++] = imageByteBuffer.getShort();
>  if(State.isRecording())  {
>      IJ.saveAs(imp, "PNG", imageFullPathName);
>
> // ---- apply LUT to data already in IJ  and save it ----
> if (useLUT) {
> ImageStatistics stats = ip.getStatistics();
>  ip.setMinAndMax(stats.histMin, stats.histMax);
>  ip.setLut(Shared.getLUT());
> imp.setProcessor("LUT", ip);
>  final String imageName = getDisplayFileName();
>  final String imageFullPathname = imageDisplyFolderPath + imageName;
>  IJ.saveAs(imp, "PNG", imageFullPathname);
>   }
>
> Thanks in advance
> Nate
>
> One's first step in wisdom is to question everything - and one's last is to
> come to terms with everything.
>
> Georg C. Lichtenberg
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Robert Lockwood]

Michael, that worked - in development mode at least.  Sunday I tried to
detect the file without checking that it had a non-zero length but that
failed.  It takes between 380 and 400 milliseconds for the 640,000 file to
be non-zero. That's transferring from memory to a RAM disk!

I'd like to save this displayable file, with the LUT first and then the
'raw data' second since saving the raw data is not critical.

Does setting the LUT actually modify the original data?
I'd like to, in a manner of speaking, roll back the maximum, minimum, and
LUT that were set to save the original data last:

  ip.setMinAndMax(stats.histMin, stats.histMax); // 0, 2^16 -1
  ip.setLut(Shared.getLUT()); // set to null?
  imp.setProcessor("LUT", ip);  //

I could just start over, of course, but I'm frugal.




One's first step in wisdom is to question everything - and one's last is to
come to terms with everything.

Georg C. Lichtenberg

On Mon, Mar 16, 2015 at 9:20 AM, Michael Schmid <[hidden email]>
wrote:

> Hi Robert,
>
> in case the problem occurs because the file has not been written yet while
> the next command is executed:
> Maybe have a loop, waiting until the file exists?
>
> In ImageJ macro language, something like the following:
>
> for (i=0; i<100; i++) {
>   if (File.exists(path) && File.length(path)>100) i=100; //break the loop
>   if (i==99) exit("timeout, did not save so far");
>   wait(50);
> }
>
> It won't tell you whether the file is has been completely written. Some
> operating systems might report zero length for a file that is unfinished,
> but I would not rely on that.
>
> Michael
> ________________________________________________________________
> On Mar 16, 2015, at 00:33, Robert Lockwood wrote:
>
> > I have been writing software that controls a one-of-a-kind research grade
> > aerial infrared camera running the Java code from the Eclipse editor
> > (Ubuntu 14.04). When deployed scripts initialize all the code and start
> the
> > various pieces running.
> >
> > I import the data, 16 bit unsigned, into IJ and save it as 16 bit
> unsigned
> > PNG ('raw' data).  Then I use the data already in IJ, assign a LUT, etc.
> > and save this now displayable PMG file to different folder and tell my
> > webpage GUI to get and display the file for the operator to view.
> >
> > This works fine when I run the code from Eclipse but fails when I use the
> > scripts to automatically start the code when the (headless) computer
> boots.
> >
> > What I think is happening is that IJ tells the OS to save the 16 bit data
> > file and the OS says, OK it's a write - that's not a priority.  Then the
> > displayable file is saved and now there may be two files in the queue.
> The
> > message is sent to the webpage, the webpage requests the displayable file
> > but it's not yet saved.  The notebook, iPad, etc. that is the webpage
> > client is either connected directly to the server or to the router,
> there's
> > no traffic going across the internet so it's quite fast.
> >
> > There are a couple code snippets following.
> >
> > I'd like to try to force the OS to save the files quickly, but don't know
> > if that's possible in IJ.  I'd also like to save the displayable file
> with
> > its LUT first but don't know how to do that without making a second copy
> of
> > the raw data.
> >
> > Is it possible to use the IJ objects after the LUT has been applied and
> > 'recover' (so to speak) the 16 bit data and save it?
> >
> > I'd appreciate any suggestions ...
> >
> >  // ----- save 'raw' data as 16 bit unsigned short as PNG -----
> >  ImagePlus imp = IJ.createImage("Untitled", imageWidth, imageHeight, 1
> > ,imageBitsPerPix );
> > ShortProcessor ip = (ShortProcessor) imp.getProcessor();
> > short[] data = (short[]) ip.getPixels();
> > // copy pixel data to IJ ip
> >  imageByteBuffer.position(0);
> >  int index = 0;
> >  while(imageByteBuffer.hasRemaining())
> >  data[index++] = imageByteBuffer.getShort();
> >  if(State.isRecording())  {
> >      IJ.saveAs(imp, "PNG", imageFullPathName);
> >
> > // ---- apply LUT to data already in IJ  and save it ----
> > if (useLUT) {
> > ImageStatistics stats = ip.getStatistics();
> >  ip.setMinAndMax(stats.histMin, stats.histMax);
> >  ip.setLut(Shared.getLUT());
> > imp.setProcessor("LUT", ip);
> >  final String imageName = getDisplayFileName();
> >  final String imageFullPathname = imageDisplyFolderPath + imageName;
> >  IJ.saveAs(imp, "PNG", imageFullPathname);
> >   }
> >
> > Thanks in advance
> > Nate
> >
> > One's first step in wisdom is to question everything - and one's last is
> to
> > come to terms with everything.
> >
> > Georg C. Lichtenberg
> >
> > --
> > ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Robert,

the LUT does not affect the image data.
I guess that ip.setLut(null) should be fine to set the LUT to grayscale.
The more explicit way of getting the standard grayscale LUT would be
  ip.setColorModel(ip.getDefaultColorModel())

Michael
________________________________________________________________
On Mar 16, 2015, at 19:58, Robert Lockwood wrote:

> Michael, that worked - in development mode at least.  Sunday I tried to
> detect the file without checking that it had a non-zero length but that
> failed.  It takes between 380 and 400 milliseconds for the 640,000 file to
> be non-zero. That's transferring from memory to a RAM disk!
>
> I'd like to save this displayable file, with the LUT first and then the
> 'raw data' second since saving the raw data is not critical.
>
> Does setting the LUT actually modify the original data?
> I'd like to, in a manner of speaking, roll back the maximum, minimum, and
> LUT that were set to save the original data last:
>
>  ip.setMinAndMax(stats.histMin, stats.histMax); // 0, 2^16 -1
>  ip.setLut(Shared.getLUT()); // set to null?
>  imp.setProcessor("LUT", ip);  //
>
> I could just start over, of course, but I'm frugal.
>
>
>
>
> One's first step in wisdom is to question everything - and one's last is to
> come to terms with everything.
>
> Georg C. Lichtenberg
>
> On Mon, Mar 16, 2015 at 9:20 AM, Michael Schmid <[hidden email]>
> wrote:
>
>> Hi Robert,
>>
>> in case the problem occurs because the file has not been written yet while
>> the next command is executed:
>> Maybe have a loop, waiting until the file exists?
>>
>> In ImageJ macro language, something like the following:
>>
>> for (i=0; i<100; i++) {
>>  if (File.exists(path) && File.length(path)>100) i=100; //break the loop
>>  if (i==99) exit("timeout, did not save so far");
>>  wait(50);
>> }
>>
>> It won't tell you whether the file is has been completely written. Some
>> operating systems might report zero length for a file that is unfinished,
>> but I would not rely on that.
>>
>> Michael
>> ________________________________________________________________
>> On Mar 16, 2015, at 00:33, Robert Lockwood wrote:
>>
>>> I have been writing software that controls a one-of-a-kind research grade
>>> aerial infrared camera running the Java code from the Eclipse editor
>>> (Ubuntu 14.04). When deployed scripts initialize all the code and start
>> the
>>> various pieces running.
>>>
>>> I import the data, 16 bit unsigned, into IJ and save it as 16 bit
>> unsigned
>>> PNG ('raw' data).  Then I use the data already in IJ, assign a LUT, etc.
>>> and save this now displayable PMG file to different folder and tell my
>>> webpage GUI to get and display the file for the operator to view.
>>>
>>> This works fine when I run the code from Eclipse but fails when I use the
>>> scripts to automatically start the code when the (headless) computer
>> boots.
>>>
>>> What I think is happening is that IJ tells the OS to save the 16 bit data
>>> file and the OS says, OK it's a write - that's not a priority.  Then the
>>> displayable file is saved and now there may be two files in the queue.
>> The
>>> message is sent to the webpage, the webpage requests the displayable file
>>> but it's not yet saved.  The notebook, iPad, etc. that is the webpage
>>> client is either connected directly to the server or to the router,
>> there's
>>> no traffic going across the internet so it's quite fast.
>>>
>>> There are a couple code snippets following.
>>>
>>> I'd like to try to force the OS to save the files quickly, but don't know
>>> if that's possible in IJ.  I'd also like to save the displayable file
>> with
>>> its LUT first but don't know how to do that without making a second copy
>> of
>>> the raw data.
>>>
>>> Is it possible to use the IJ objects after the LUT has been applied and
>>> 'recover' (so to speak) the 16 bit data and save it?
>>>
>>> I'd appreciate any suggestions ...
>>>
>>> // ----- save 'raw' data as 16 bit unsigned short as PNG -----
>>> ImagePlus imp = IJ.createImage("Untitled", imageWidth, imageHeight, 1
>>> ,imageBitsPerPix );
>>> ShortProcessor ip = (ShortProcessor) imp.getProcessor();
>>> short[] data = (short[]) ip.getPixels();
>>> // copy pixel data to IJ ip
>>> imageByteBuffer.position(0);
>>> int index = 0;
>>> while(imageByteBuffer.hasRemaining())
>>> data[index++] = imageByteBuffer.getShort();
>>> if(State.isRecording())  {
>>>     IJ.saveAs(imp, "PNG", imageFullPathName);
>>>
>>> // ---- apply LUT to data already in IJ  and save it ----
>>> if (useLUT) {
>>> ImageStatistics stats = ip.getStatistics();
>>> ip.setMinAndMax(stats.histMin, stats.histMax);
>>> ip.setLut(Shared.getLUT());
>>> imp.setProcessor("LUT", ip);
>>> final String imageName = getDisplayFileName();
>>> final String imageFullPathname = imageDisplyFolderPath + imageName;
>>> IJ.saveAs(imp, "PNG", imageFullPathname);
>>>  }
>>>
>>> Thanks in advance
>>> Nate
>>>
>>> One's first step in wisdom is to question everything - and one's last is
>> to
>>> come to terms with everything.
>>>
>>> Georg C. Lichtenberg
>>>
>>> --
>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Robert Lockwood]

Thanks, Michael, I'm going to try that as soon as I solve a related problem
that I will post in a few minutes.

One's first step in wisdom is to question everything - and one's last is to
come to terms with everything.

Georg C. Lichtenberg

On Tue, Mar 17, 2015 at 1:52 AM, Michael Schmid <[hidden email]>
wrote:

> Hi Robert,
>
> the LUT does not affect the image data.
> I guess that ip.setLut(null) should be fine to set the LUT to grayscale.
> The more explicit way of getting the standard grayscale LUT would be
>   ip.setColorModel(ip.getDefaultColorModel())
>
> Michael
> ________________________________________________________________
> On Mar 16, 2015, at 19:58, Robert Lockwood wrote:
>
> > Michael, that worked - in development mode at least.  Sunday I tried to
> > detect the file without checking that it had a non-zero length but that
> > failed.  It takes between 380 and 400 milliseconds for the 640,000 file
> to
> > be non-zero. That's transferring from memory to a RAM disk!
> >
> > I'd like to save this displayable file, with the LUT first and then the
> > 'raw data' second since saving the raw data is not critical.
> >
> > Does setting the LUT actually modify the original data?
> > I'd like to, in a manner of speaking, roll back the maximum, minimum, and
> > LUT that were set to save the original data last:
> >
> >  ip.setMinAndMax(stats.histMin, stats.histMax); // 0, 2^16 -1
> >  ip.setLut(Shared.getLUT()); // set to null?
> >  imp.setProcessor("LUT", ip);  //
> >
> > I could just start over, of course, but I'm frugal.
> >
> >
> >
> >
> > One's first step in wisdom is to question everything - and one's last is
> to
> > come to terms with everything.
> >
> > Georg C. Lichtenberg
> >
> > On Mon, Mar 16, 2015 at 9:20 AM, Michael Schmid <[hidden email]
> >
> > wrote:
> >
> >> Hi Robert,
> >>
> >> in case the problem occurs because the file has not been written yet
> while
> >> the next command is executed:
> >> Maybe have a loop, waiting until the file exists?
> >>
> >> In ImageJ macro language, something like the following:
> >>
> >> for (i=0; i<100; i++) {
> >>  if (File.exists(path) && File.length(path)>100) i=100; //break the loop
> >>  if (i==99) exit("timeout, did not save so far");
> >>  wait(50);
> >> }
> >>
> >> It won't tell you whether the file is has been completely written. Some
> >> operating systems might report zero length for a file that is
> unfinished,
> >> but I would not rely on that.
> >>
> >> Michael
> >> ________________________________________________________________
> >> On Mar 16, 2015, at 00:33, Robert Lockwood wrote:
> >>
> >>> I have been writing software that controls a one-of-a-kind research
> grade
> >>> aerial infrared camera running the Java code from the Eclipse editor
> >>> (Ubuntu 14.04). When deployed scripts initialize all the code and start
> >> the
> >>> various pieces running.
> >>>
> >>> I import the data, 16 bit unsigned, into IJ and save it as 16 bit
> >> unsigned
> >>> PNG ('raw' data).  Then I use the data already in IJ, assign a LUT,
> etc.
> >>> and save this now displayable PMG file to different folder and tell my
> >>> webpage GUI to get and display the file for the operator to view.
> >>>
> >>> This works fine when I run the code from Eclipse but fails when I use
> the
> >>> scripts to automatically start the code when the (headless) computer
> >> boots.
> >>>
> >>> What I think is happening is that IJ tells the OS to save the 16 bit
> data
> >>> file and the OS says, OK it's a write - that's not a priority.  Then
> the
> >>> displayable file is saved and now there may be two files in the queue.
> >> The
> >>> message is sent to the webpage, the webpage requests the displayable
> file
> >>> but it's not yet saved.  The notebook, iPad, etc. that is the webpage
> >>> client is either connected directly to the server or to the router,
> >> there's
> >>> no traffic going across the internet so it's quite fast.
> >>>
> >>> There are a couple code snippets following.
> >>>
> >>> I'd like to try to force the OS to save the files quickly, but don't
> know
> >>> if that's possible in IJ.  I'd also like to save the displayable file
> >> with
> >>> its LUT first but don't know how to do that without making a second
> copy
> >> of
> >>> the raw data.
> >>>
> >>> Is it possible to use the IJ objects after the LUT has been applied and
> >>> 'recover' (so to speak) the 16 bit data and save it?
> >>>
> >>> I'd appreciate any suggestions ...
> >>>
> >>> // ----- save 'raw' data as 16 bit unsigned short as PNG -----
> >>> ImagePlus imp = IJ.createImage("Untitled", imageWidth, imageHeight, 1
> >>> ,imageBitsPerPix );
> >>> ShortProcessor ip = (ShortProcessor) imp.getProcessor();
> >>> short[] data = (short[]) ip.getPixels();
> >>> // copy pixel data to IJ ip
> >>> imageByteBuffer.position(0);
> >>> int index = 0;
> >>> while(imageByteBuffer.hasRemaining())
> >>> data[index++] = imageByteBuffer.getShort();
> >>> if(State.isRecording())  {
> >>>     IJ.saveAs(imp, "PNG", imageFullPathName);
> >>>
> >>> // ---- apply LUT to data already in IJ  and save it ----
> >>> if (useLUT) {
> >>> ImageStatistics stats = ip.getStatistics();
> >>> ip.setMinAndMax(stats.histMin, stats.histMax);
> >>> ip.setLut(Shared.getLUT());
> >>> imp.setProcessor("LUT", ip);
> >>> final String imageName = getDisplayFileName();
> >>> final String imageFullPathname = imageDisplyFolderPath + imageName;
> >>> IJ.saveAs(imp, "PNG", imageFullPathname);
> >>>  }
> >>>
> >>> Thanks in advance
> >>> Nate
> >>>
> >>> One's first step in wisdom is to question everything - and one's last
> is
> >> to
> >>> come to terms with everything.
> >>>
> >>> Georg C. Lichtenberg
> >>>
> >>> --
> >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
> >>
> >> --
> >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
> >>
> >
> > --
> > ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html