Low res -> High Res Calculations

[Jon Harman]

Hi,

I have an application for ImageJ that calculates a new value for each
pixel in an image, based only on the single input pixel.  The
calculation is complex and can take many seconds for a large digital
camera image.  I want to give the user a quick way to see if the new
image is worthwhile and stop the calculation if not.  I wonder if I can
get some advice on how to do this.  Some imaging applications I have do
this sort of thing by creating a low resolution image quickly then
filling in at higher resolution in the background.  Is this possible in
ImageJ?  Could this sort of functionality be made into a routine in
ImageJ that takes my per pixel routine and handles the low res/high res
parts automatically?

Thanks for any advice,
Jon

[Michael Schmid]

Hi Jon,

at the moment some code for "preview" in ImageJ (for PlugInFilters)
is under construction - I hope it will be implemented in the
not-too far future in a way that all PlugInFilters can use it
without too many modifications.

Downscaling for preview will not work with this preview code.

What I could think of:

The easy way - write your code in a way that only the selection
(ROI) is processed.

Then you could select a small part of the image before preview
(manually or in the code), and run the filter again on the full
image (or the original selection) afterwards.

Would this solve your problem?


If you want to, I have some code parts that I could share with
you so you can try to assemble your own downscale&preview code.

---
What I do not understand: what kind of code that processes only
single pixels, irrespective of their neighborhood, can take that
long?

Michael
________________________________________________________________
On 15 Mar 2007, at 17:34, Jon Harman wrote:

> Hi,
>
> I have an application for ImageJ that calculates a new value for  
> each pixel in an image, based only on the single input pixel.  The  
> calculation is complex and can take many seconds for a large  
> digital camera image.  I want to give the user a quick way to see  
> if the new image is worthwhile and stop the calculation if not.  I  
> wonder if I can get some advice on how to do this.  Some imaging  
> applications I have do this sort of thing by creating a low  
> resolution image quickly then filling in at higher resolution in  
> the background.  Is this possible in ImageJ?  Could this sort of  
> functionality be made into a routine in ImageJ that takes my per  
> pixel routine and handles the low res/high res parts automatically?
>
> Thanks for any advice,
> Jon

[Jon Harman]

Hi,

I was thinking of some sort of subsampling scheme. Say calculate one out
of every n pixels in x and y and copy that value to the other pixels
initially.  Then go through the rest of the pixels in the background,
slowly filling in the replicated pixels with calculated values.  My
plugin does a color transformation on RGB images.  The slow part is that
it has to convert a pixel to LAB colorspace, a non-linear
transformation, then multiply by a 3x3 matrix, then convert back to RGB.

Jon

Michael Schmid wrote:

> Hi Jon,
>
> at the moment some code for "preview" in ImageJ (for PlugInFilters)
> is under construction - I hope it will be implemented in the
> not-too far future in a way that all PlugInFilters can use it
> without too many modifications.
>
> Downscaling for preview will not work with this preview code.
>
> What I could think of:
>
> The easy way - write your code in a way that only the selection
> (ROI) is processed.
>
> Then you could select a small part of the image before preview
> (manually or in the code), and run the filter again on the full
> image (or the original selection) afterwards.
>
> Would this solve your problem?
>
>
> If you want to, I have some code parts that I could share with
> you so you can try to assemble your own downscale&preview code.
>
> ---
> What I do not understand: what kind of code that processes only
> single pixels, irrespective of their neighborhood, can take that
> long?
>
> Michael
> ________________________________________________________________
> On 15 Mar 2007, at 17:34, Jon Harman wrote:
>
>> Hi,
>>
>> I have an application for ImageJ that calculates a new value for each
>> pixel in an image, based only on the single input pixel.  The
>> calculation is complex and can take many seconds for a large digital
>> camera image.  I want to give the user a quick way to see if the new
>> image is worthwhile and stop the calculation if not.  I wonder if I
>> can get some advice on how to do this.  Some imaging applications I
>> have do this sort of thing by creating a low resolution image quickly
>> then filling in at higher resolution in the background.  Is this
>> possible in ImageJ?  Could this sort of functionality be made into a
>> routine in ImageJ that takes my per pixel routine and handles the low
>> res/high res parts automatically?
>>
>> Thanks for any advice,
>> Jon
>

[Michael Schmid]

Hi Jon,

ok, I see your point. As I am currently also working on a
plugin that does color transformation I tried to add a preview
function using some code that I am developing and that may
eventually get into ImageJ, if it becomes good enough.

I see that subsampling is needed for large images, but I have
implemented it a simple way: I reduce the image size by zooming
out for display, and then calculating only the pixels that are
displayed. I do not think that one can gain much by omitting
the pixels done during preview when doing the full calculation
afterwards - for an image reduced to 25% it is a matter of
recalculating 1/16th of the image. So the full calculation
simply does all the pixels, irrespective of what has been done
before.

I have put my testing environment on the web:
http://www.iap.tuwien.ac.at/www/surface/tmp/preview-tests.zip

These are quite a few files; you have to Compile&Run the file
"my_ImageJ" (named so because it contains code that parallels
code in ImageJ) - it is a dispatcher for the various Filters
that I am experimenting with. You can add your own by
duplicating "Adjust_Saturation", renaming it and entering
it into "my_ImageJ" (twice, once in String[][] commands and
once in void unused()).

Note that a filter using preview will not run as a plugin
with ImageJ - but maybe some day, something similar will become
available for plugInFilters of ImageJ.

When doing preview without the overhead of all the methods of my
experimental set, this is somewhat tricky. E.g. one needs two
objects to synchronize on, one for passing the filter parameters,
and one for synchronizing the notification to the background
thread doing processing for preview, otherwise a deadlock can
occur.

Here is a rough outline how to do it:

public class My_Filter implements PlugInFilter, TextListener,  
ItemListener, Runnable {
     /** the ImagePlus of the setup call */
     private ImagePlus imp = null;
     /** The dialog asking for the parameters */
     private GenericDialog gd;
     /** Dialog items for access from listeners & background thread */
     private Vector gdFields, gdCheckboxes, gdChoices;
     /** whether to show a preview image */
     private boolean preview = false;
     /** whether the image has been changed by preview */
     private boolean changed = false;
     /** the background thread for preview */
     private Thread bgThread = null;
     /** the filter parameters */
...

... the usual setup() method calling showDialog()

     public boolean showDialog() {
         //set up the ImageProcessor:
         ImageProcessor ip = imp.getProcessor();
         ip.snapshot();
         gd = new GenericDialog("My Filter...");
... add dialog items ...
         gdFields = gd.getNumericFields();
         ((TextField)gdFields.elementAt(0)).addTextListener(this);
         gdCheckboxes = gd.getCheckboxes();
         ((Checkbox)gdCheckboxes.elementAt(0)).addItemListener(this);
         gdChoices = gd.getChoices();
         if (gdChoices != null)
             ((Choice)gdChoices.elementAt(0)).addItemListener(this);
         gd.showDialog();  //input by the user (or macro) happens here
         stopPreview();
         if (changed) {
             ip.reset();}
         if (gd.wasCanceled()) {
             imp.updateAndDraw();
             return false;
         }
... get the parameters
         return true;
     }

     public void textValueChanged(TextEvent e) {
         newParameters();
     }

     public void itemStateChanged(ItemEvent e) {
         newParameters();
     }

     synchronized void newParameters() {
... decode TextFields, etc, to get the parameters
         preview = ((Checkbox)gdCheckboxes.elementAt(0)).getState();
         if (preview) {
             if (bgThread == null) {    //need to start a background  
thread for preview
                 bgThread = new Thread(this, "My Filter Preview");
                 bgThread.start();
                 bgThread.setPriority(bgThread.getPriority()-2);
             } else {
                 bgThread.interrupt(); //trigger background thread  
for new preview
             }
         } else {
             if (bgThread != null) stopPreview();
             imp.updateAndDraw();      //no preview any more, show  
original image
         }

     /** the background thread for preview */
     public void run() {
         ImageProcessor ip = imp.getProcessor();
         Integer waitMonitorOwner = new Object;      // get something  
we can wait on without blocking "this"
         while(preview) {
             ((Checkbox)gdCheckboxes.elementAt(0)).setForeground
(Color.BLUE); //optical feedback; does noot work on Mac OSX
             if (changed) {
                 ip.reset();
                 changed = false;
             }
             if (!Thread.currentThread().isInterrupted()) //not  
interrupted, we should process the image for preview
                 run(ip);
             if (!Thread.currentThread().isInterrupted()) //not  
interrupted, we have a preview image to show
                 if (ip.getMask() != null)
                     ip.reset(ip.getMask());             //for non-
rectangular rois, reset the pixels outside
                 imp.updateAndDraw();
             ((Checkbox)gdCheckboxes.elementAt(0)).setForeground(null);
             synchronized(waitMonitorOwner) {
                 if (preview) {;
                     try {waitMonitorOwner.wait();}      //we don't  
wait for notify but rather for interrupted
                     catch (InterruptedException e) {}
                 }
//                Thread.currentThread().interrupted();   //reset  
interrupted status
             }
         } //while(preview)
         ip.reset();
         changed = false;
     }


     /** stop the background process responsible for preview
      *  and wait until the preview thread has finished */
     synchronized void stopPreview() {
         preview = false;               //tell a possible background  
thread to terminate
         if (bgThread != null) {
             bgThread.interrupt();
             if (bgThread.isAlive()) {
                 try {bgThread.join();}  //wait until the background  
thread is done so that it can't do any harm
                 catch (InterruptedException e){}
             }
             bgThread = null;
         }
     }

     public void run(ImageProcessor ip) {
         synchronized(this) {     //get a consistent set of  
parameters to work with
             //copy the filtering parameters from class variables to  
a local set
             double localCopyOfFilterParams = classVariableFilterParams;
             changed = true;
         }
         doTheFiltering(ip, localCopyOfFilterParams);
         return;
     }

     void doTheFiltering (ip, params) {
...loop
...from time to time check whether the user has not
...changed the params, so we should abort and start again
          if (Thread.currentThread().isInterrupted()) return;
...process all pixels...
...endLoop
     }


So you see, it is not really straightforward ...

Michael
________________________________________________________________

On 15 Mar 2007, at 22:09, Jon Harman wrote:

> Hi,
>
> I was thinking of some sort of subsampling scheme. Say calculate  
> one out of every n pixels in x and y and copy that value to the  
> other pixels initially.  Then go through the rest of the pixels in  
> the background, slowly filling in the replicated pixels with  
> calculated values.  My plugin does a color transformation on RGB  
> images.  The slow part is that it has to convert a pixel to LAB  
> colorspace, a non-linear transformation, then multiply by a 3x3  
> matrix, then convert back to RGB.
>
> Jon
>
> Michael Schmid wrote:
>> Hi Jon,
>>
>> at the moment some code for "preview" in ImageJ (for PlugInFilters)
>> is under construction - I hope it will be implemented in the
>> not-too far future in a way that all PlugInFilters can use it
>> without too many modifications.
>>
>> Downscaling for preview will not work with this preview code.
>>
>> What I could think of:
>>
>> The easy way - write your code in a way that only the selection
>> (ROI) is processed.
>>
>> Then you could select a small part of the image before preview
>> (manually or in the code), and run the filter again on the full
>> image (or the original selection) afterwards.
>>
>> Would this solve your problem?
>>
>>
>> If you want to, I have some code parts that I could share with
>> you so you can try to assemble your own downscale&preview code.
>>
>> ---
>> What I do not understand: what kind of code that processes only
>> single pixels, irrespective of their neighborhood, can take that
>> long?
>>
>> Michael
>> ________________________________________________________________
>> On 15 Mar 2007, at 17:34, Jon Harman wrote:
>>
>>> Hi,
>>>
>>> I have an application for ImageJ that calculates a new value for  
>>> each pixel in an image, based only on the single input pixel.  
>>> The calculation is complex and can take many seconds for a large  
>>> digital camera image.  I want to give the user a quick way to see  
>>> if the new image is worthwhile and stop the calculation if not.  
>>> I wonder if I can get some advice on how to do this.  Some  
>>> imaging applications I have do this sort of thing by creating a  
>>> low resolution image quickly then filling in at higher resolution  
>>> in the background.  Is this possible in ImageJ?  Could this sort  
>>> of functionality be made into a routine in ImageJ that takes my  
>>> per pixel routine and handles the low res/high res parts  
>>> automatically?
>>>
>>> Thanks for any advice,
>>> Jon
>>