Issues with real-time display in ImageJ

[MRivers]

Folks,



I develop real-time detector software called areaDetector for the EPICS control system.



https://epics.anl.gov/

https://areadetector.github.io/master/index.html



One of the components of areaDetector is a real-time image display plugin in areaDetector that uses ImageJ.



https://areadetector.github.io/master/ADViewers/ad_viewers.html#imagej-viewers

https://github.com/areaDetector/ADViewers/tree/master/ImageJ/EPICS_areaDetector



It receives images over the network using the EPICS Channel Access or pvAccess protocols and displays them in ImageJ.  This plugin is used worldwide at a large number of large physics facilities such as synchrotrons, neutron sources, and telescopes.  The ImageJ performance is excellent.  Here is a screen shot of ImageJ receiving and displaying 180 frames/s 1024x1024 pixels, 8-bit.





[cid:image001.png@01D5B57D.253F69F0]



[cid:image002.png@01D5B57D.253F69F0]



There are just a couple of things that don’t work quite right when using ImageJ in this mode.



-          When displaying monochrome images (ByteProcessor, ShortProcessor, FloatProcessor) the Image/Adjust/Brightness/Contrast works fine.  I can set the brightness and contrast or high and low levels and they are “permanent” until I change them again, and all subsequent images are displayed using these values.  However when displaying RGB images (ColorProcessor) when I change the brightness and contrast it only applies to the current image, they reset automatically when the next image is displayed.  Is there a way to prevent this behavior, or could this be fixed?

-          When images are arriving quickly the cursor x, y, and value numbers in the main ImageJ window do not track the cursor in the image window correctly.  They don’t just have a lag, they often never update to the current cursor position, even if the cursor is left on the same pixel for a long time.  This means that in order to make a measurement of a pixel position or value I need to  stop the real-time updates, make the measurement and then turn the real-time update back on.  Is there a way to avoid this behavior, or could this be fixed?



Thanks,

Mark



--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[MRivers]

I have learned more about my second problem.  

In my code that displays the new image I am calling ImagePlus:: img.updateStatusbarValue().  If I call this function then the intensity value updates, but the mouse does not track accurately.  If I move the mouse quickly it tracks somewhat, but if I move it slowly it does not track at all.  

If I don’t call updateStatusbarValue() then the mouse tracks perfectly, but the intensity value does not update when a new image is displayed.

Thanks,
Mark


--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[MRivers]

>  When displaying monochrome images (ByteProcessor, ShortProcessor, FloatProcessor) the Image/Adjust/Brightness/Contrast works fine. 
> I can set the brightness and contrast or high and low levels and they are “permanent” until I change them again, and all subsequent images are displayed using these values. 
> However when displaying RGB images (ColorProcessor) when I change the brightness and contrast it only applies to the current image, they reset automatically when the next image is displayed. 
> Is there a way to prevent this behavior, or could this be fixed?

I tried adding this line to my code that displays the RGB images, where img is the ImagePlus object.

 img.setDisplayRange(img.getDisplayRangeMin(), img.getDisplayRangeMax(), 7);

However, with this line present as soon as I adjust the brightness and contrast the images stop updating with no error message.

Mark



--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Mark,

probably you are calling the ImagePlus.updateStatusbarValue() too often
per second.

With a simple test case and on a rather fast computer, it works well
when calling it 10 times a second. If I try to call it e.g. 100 times a
second, I hardly get any MouseEvents with mouseMoved() any more. Calling
ImagePlus.mouseMoved() updates the coordinates used by
updateStatusbarValue().

To me, it is somewhat surprising how slow it seems to be; it gets
sluggish even when calling updateStatusbarValue 20 times per second,
which is not much for today's computers. I don't think that redrawing
the status line would occupy the event queue too much, but maybe there
are some built-in limits in Java.AWT to avoid too many events per second?
I have the impression that the maximum rate of MouseEvents with
mouseMoved() is higher than what one can do with updateStatusbarValue.

Anyhow, you might try calling updateStatusbarValue() only if something
has changed and/or check that the time since the last call has been more
than some minimum.

Michael
________________________________________________________________
On 18.12.19 18:56, Mark Rivers wrote:

> I have learned more about my second problem.
>
> In my code that displays the new image I am calling ImagePlus:: img.updateStatusbarValue().  If I call this function then the intensity value updates, but the mouse does not track accurately.  If I move the mouse quickly it tracks somewhat, but if I move it slowly it does not track at all.
>
> If I don’t call updateStatusbarValue() then the mouse tracks perfectly, but the intensity value does not update when a new image is displayed.
>
> Thanks,
> Mark
>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[MRivers]

Hi Christian,

Thanks for the comment.  I have changed my code to do the following when displaying RGB images:

- Call ImagePlus::getDisplayRangeMin() and ImagePlus::getDisplayRangeMax().
- That returns the current setting of the range in the Image/Contrast control
- If those are 0 and 255 it does nothing and leaves the image pixels unchanged
- If these are not 0 and 255 then it does the following:
   - Constructs a 256 entry lookup table (LUT) that goes from the min to the max
   - Translates every pixel in the image through this LUT
 
For efficiency it only creates a new LUT if the min and max have changed from the previous time it created the LUT.

Contrary to your expectation this has no measureable impact on the frame rate.  Without the LUT my plugin displays 90 frames/s of 1024x1024 RGB images.  With the LUT it also displays 90 frames/s.  This rate means I can run GigE cameras at their full frame rate (that saturates the GigE network) and ImageJ can keep up displaying every image.

Thanks for your hint about the lookup table, it allowed me to solve that problem.

Mark

-----Original Message-----
From: Phase GmbH <[hidden email]>
Sent: Thursday, December 19, 2019 10:51 AM
To: Mark Rivers <[hidden email]>
Subject: Issues with real-time display in ImageJ

Dear Mark,

Image/Ajustment ... of monochrome images is performed in ImageJ by adjusting the look-up table associated with the image. The pixel values remain unchanged, only the display is changed. The look-up table is an array consisting of 256 elements for each RGB channel. You may also use look-up tables to psycho-colourize a monochrome image or image stream.
There are lot of predefined look-up table in ImageJ.

RGB images do not have a  look-up table so any pixel is recalculated according to the actual adjustment setting. Any new image would over-write the previous one. If you want to preserve some settings you have to apply it to any new frame. This is a time-consuming operation which will drop your frame rate seriously.


Regards

Christian Kreutzfeldt

--

+++++++++++++++++++++++++++++
PHASE GmbH
Bluecherstr. 2 , 23564 Luebeck, Germany
Tel. [phone], Fax 792644
www.phase-hl.com
[hidden email]
--------------------------------------------------
Amtsgericht Lübeck, HRB 2484
Geschäftsführer Dr. Christian Kreutzfeldt

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Wayne Rasband-2]

> On Dec 18, 2019, at 9:42 AM, Mark Rivers <[hidden email]> wrote:
>  
> - When images are arriving quickly the cursor x, y, and value numbers in the main ImageJ window do not track the cursor in the image window correctly.

In an attempt to reproduce this problem, I updated the Help>Examples>JavaScript>Terabyte VirtualStack example in the ImageJ 1.52t14 daily build so that, when the stack is animating, it displays the cursor location and pixel value in the status bar. I found that the cursor location is tracked correctly at up to 1000 fps on my Mac laptop. Set the frame rate by typing alt+\ (Image>Stacks>Animation>Animation Options).

This is the code the Animator class uses to update the status bar after each frame is displayed:

   ImageCanvas ic = imp.getCanvas();
   Point loc = ic!=null?ic.getCursorLoc():null;
   if (loc!=null)
      imp.mouseMoved(loc.x,loc.y);

-wayne

> Folks,
>  
> I develop real-time detector software called areaDetector for the EPICS control system.
>  
> https://epics.anl.gov/
> https://areadetector.github.io/master/index.html
>  
> One of the components of areaDetector is a real-time image display plugin in areaDetector that uses ImageJ.
>  
> https://areadetector.github.io/master/ADViewers/ad_viewers.html#imagej-viewers
> https://github.com/areaDetector/ADViewers/tree/master/ImageJ/EPICS_areaDetector
>  
> It receives images over the network using the EPICS Channel Access or pvAccess protocols and displays them in ImageJ.  This plugin is used worldwide at a large number of large physics facilities such as synchrotrons, neutron sources, and telescopes.  The ImageJ performance is excellent.  Here is a screen shot of ImageJ receiving and displaying 180 frames/s 1024x1024 pixels, 8-bit.
>  
>  
> <image001.png>
>  
> <image002.png>
>  
> There are just a couple of things that don’t work quite right when using ImageJ in this mode.
>  
> -          When displaying monochrome images (ByteProcessor, ShortProcessor, FloatProcessor) the Image/Adjust/Brightness/Contrast works fine.  I can set the brightness and contrast or high and low levels and they are “permanent” until I change them again, and all subsequent images are displayed using these values.  However when displaying RGB images (ColorProcessor) when I change the brightness and contrast it only applies to the current image, they reset automatically when the next image is displayed.  Is there a way to prevent this behavior, or could this be fixed?
> -          When images are arriving quickly the cursor x, y, and value numbers in the main ImageJ window do not track the cursor in the image window correctly.  They don’t just have a lag, they often never update to the current cursor position, even if the cursor is left on the same pixel for a long time.  This means that in order to make a measurement of a pixel position or value I need to  stop the real-time updates, make the measurement and then turn the real-time update back on.  Is there a way to avoid this behavior, or could this be fixed?
>  
> Thanks,
> Mark

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[MRivers]

Wayne Rasband wrote:


Ø  This is the code the Animator class uses to update the status bar after each frame is displayed:

        ImageCanvas ic = img.getCanvas();
        Point loc = ic!=null ? ic.getCursorLoc() : null;
        if (loc!=null)
            img.mouseMoved(loc.x,loc.y);

Thanks, that’s exactly what I needed.  When I add that code to my image display function it now works perfectly!

I was thrown off by the documentation for ImagePlus.mouseMoved().  It says:

Ø  public void mouseMoved(int x,

                        int y)

Ø  Displays the cursor coordinates and pixel value in the status bar. Called by ImageCanvas when the mouse moves. Can be overridden by ImagePlus subclasses.

That led me to think that this was a function that was called by the display system on mouse events, and which I needed to subclass ImagePlus to implement.  I didn’t understand that I could get the mouse coordinates via the ImageCanvas and call this function to have the status bar update the coordinates.

Thanks again, both of my problems are now solved.

Happy Holidays!

Mark


From: Mark Rivers
Sent: Wednesday, December 18, 2019 8:43 AM
To: [hidden email]; Wayne Rasband <[hidden email]>
Subject: Issues with real-time display in ImageJ


Folks,



I develop real-time detector software called areaDetector for the EPICS control system.



https://epics.anl.gov/

https://areadetector.github.io/master/index.html



One of the components of areaDetector is a real-time image display plugin in areaDetector that uses ImageJ.



https://areadetector.github.io/master/ADViewers/ad_viewers.html#imagej-viewers

https://github.com/areaDetector/ADViewers/tree/master/ImageJ/EPICS_areaDetector



It receives images over the network using the EPICS Channel Access or pvAccess protocols and displays them in ImageJ.  This plugin is used worldwide at a large number of large physics facilities such as synchrotrons, neutron sources, and telescopes.  The ImageJ performance is excellent.  Here is a screen shot of ImageJ receiving and displaying 180 frames/s 1024x1024 pixels, 8-bit.





[cid:image001.png@01D5B8EB.45774F50]



[cid:image002.png@01D5B8EB.45774F50]



There are just a couple of things that don’t work quite right when using ImageJ in this mode.



-          When displaying monochrome images (ByteProcessor, ShortProcessor, FloatProcessor) the Image/Adjust/Brightness/Contrast works fine.  I can set the brightness and contrast or high and low levels and they are “permanent” until I change them again, and all subsequent images are displayed using these values.  However when displaying RGB images (ColorProcessor) when I change the brightness and contrast it only applies to the current image, they reset automatically when the next image is displayed.  Is there a way to prevent this behavior, or could this be fixed?

-          When images are arriving quickly the cursor x, y, and value numbers in the main ImageJ window do not track the cursor in the image window correctly.  They don’t just have a lag, they often never update to the current cursor position, even if the cursor is left on the same pixel for a long time.  This means that in order to make a measurement of a pixel position or value I need to  stop the real-time updates, make the measurement and then turn the real-time update back on.  Is there a way to avoid this behavior, or could this be fixed?



Thanks,

Mark



--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html