ImageJ

Re: race condition problem with command line ImageJ macro jobs

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

2 messages Options

Jonathan Jackson-2

Re: race condition problem with command line ImageJ macro jobs

On Sun, 23 Sep 2007 14:41:24 +0200, Gluender <[hidden email]> wrote:

>Dear Harry Parker,
>
>as far as I know from my own experience you are to test whether a
>required image actually is there and, in case it hasn't yet been
>computed or renamed, to wait until this happens.
>
>I don't think ImageJ is to blame because threading is an inherent and
>mostly welcome property of Java, although it can be suppressed or
>controlled. With ImageJ, timing problem of the reported kind mostly
>occure with macros, so it makes little to no sense to change the
>behaviour of ImageJ itself.
>
>What you've observe most likely is not a typical race condition.
>
>>Hi all,
>>
>>I hope somebody can help me debug a frustrating ImageJ macro problem
>>I'm experiencing.
>>
>>I've developed a few complex image processing macros for ImageJ that
>>always work well when started from the ImageJ menu, but only work
>>some of the time when started from the command line with the
>>"-macro" option, and never work with the "-batch" option.
>>
>>This is on Windows XP SP2 with the latest Java (1.6.0_02) and ImageJ
>>(1.39d). I first noticed the problem in Java 1.6.0_01 and ImageJ
>>1.38x before I upgraded trying to resolve it.
>>
>>At first I hoped to be able to use the much faster "-batch" mode
>>option, but this didn't work. Each run had a different error. "Image
>>not found" and other problems. When I switched to the slower
>>"-macro" option the macro sometimes worked perfectly, but often did
>>not.
>>
>>The problems seem mostly related to calls to Image Calculator or my
>>own plugins. For example:
>>
>> imageCalculator("Multiply create", title_dA, title_dA);
>> rename(title_dA2);
>> imageCalculator("Add create", title_dA2, title_dL2);
>>
>>will sometime cause an error indicating the image <title_dA2> was not found.
>>Debugging statements I added to display the image names and ID's
>>showed that the rename statement was returning AFTER the 2nd
>>imageCalculator statement, causing another image to be renamed!! The
>>result of the 1st imageCalculator was found with its default name,
>>(i.e. Result of ...")!
>>
>>The hard part is that the error occurs on different lines on
>>different runs! Looks like a race condition to me.
>>Later macro statements are running before the earlier ones have
>>returned. But only once in a while!
>>
>>What's different about the timing in macro and batch modes??
>>(Besides not displaying the images in batch mode.)
>>Looks like too much parallelism is happening, and not enough syncing.
>>
>>Any ideas on how to track this down?
>>
>>I haven't been able to reproduce this in a small macro. The smaller
>>the macro, the less likely it is to occur. This macro I'm focusing
>>on now is > 300 lines long with 11 calls to ImageCalculator(), 18
>>calls to rename(), etc.
>>
>>
>>--
>>Harry Parker
>>Senior Systems Engineer
>>Digital Imaging Systems, Inc.
>
>
>The bottom line:
>Include suitable tests and waiting loops with your macro.
>
>HTH. Best
>--

My experience with macros has been similar to Harry's in that one expects
commands in a macro to be executed sequentially. While most macros run
smoothly I encountered similar problems as Harry, which are both frustrating
and difficult to track down (test cases and waiting loops were not always
trivial). More often that not, this was with -batch mode, but in non-batch
mode also. The solution for me was to start writing plugins instead of
macros because this gives greater control over the program flow and it is
easier to keep track of the images involved. These can be called directly
from the command line if a main() method is included (I can send examples
off-list).

Regarding ImageJ, I think it would be ideal if the macro executer itself did
the job of testing threads for completion before moving on. Pperhaps it
does, I haven't checked - my macro writing is a bit out of date as I say,
because I tend to use plugins now.

regards,
Jon

Harry Parker

Re: race condition problem with command line ImageJ macro jobs

Dear Jon,

Thanks for the feedback. In ImageJ v.1.39f, Wayne fixed 2 bugs that solved my problems of the macros getting ahead of themselves and miscounting open images.

I find macros much easier to write than plugins. I only resort to plugins when a macro is too slow.

I put in a lot of wait loops as suggested, which also issued debugging messages when they found expected images missing. Now none of them get triggered.

Here is one hint I found that solved a lot of macro bugs: I switched from identifying images by title to using ID's. I just get an image's ID as soon as its created, then make it active when I need it. That eliminated problems with 2 images sometimes having the same name after some function mangled one of them (dropping everyting after a space or period or underscore, etc.).

--
Harry Parker
Senior Systems Engineer
Digital Imaging Systems, Inc.

----- Original Message ----
From: Jonathan Jackson <[hidden email]>
To: [hidden email]
Sent: Monday, October 15, 2007 6:26:01 AM
Subject: Re: race condition problem with command line ImageJ macro jobs

On Sun, 23 Sep 2007 14:41:24 +0200, Gluender <[hidden email]> wrote:

found.

>>Debugging statements I added to display the image names and ID's
>>showed that the rename statement was returning AFTER the 2nd
>>imageCalculator statement, causing another image to be renamed!! The
>>result of the 1st imageCalculator was found with its default name,
>>(i.e. Result of ...")!
>>
>>The hard part is that the error occurs on different lines on
>>different runs! Looks like a race condition to me.
>>Later macro statements are running before the earlier ones have
>>returned. But only once in a while!
>>
>>What's different about the timing in macro and batch modes??
>>(Besides not displaying the images in batch mode.)
>>Looks like too much parallelism is happening, and not enough syncing.
>>
>>Any ideas on how to track this down?
>>
>>I haven't been able to reproduce this in a small macro. The smaller
>>the macro, the less likely it is to occur. This macro I'm focusing
>>on now is > 300 lines long with 11 calls to ImageCalculator(), 18
>>calls to rename(), etc.
>>
>>
>>--
>>Harry Parker
>>Senior Systems Engineer
>>Digital Imaging Systems, Inc.
>
>
>The bottom line:
>Include suitable tests and waiting loops with your macro.
>
>HTH. Best
>--

My experience with macros has been similar to Harry's in that one
expects
commands in a macro to be executed sequentially. While most macros run
smoothly I encountered similar problems as Harry, which are both
frustrating
and difficult to track down (test cases and waiting loops were not
always
trivial). More often that not, this was with -batch mode, but in
non-batch
mode also. The solution for me was to start writing plugins instead of
macros because this gives greater control over the program flow and it
is
easier to keep track of the images involved. These can be called
directly
from the command line if a main() method is included (I can send
examples
off-list).

Regarding ImageJ, I think it would be ideal if the macro executer
itself did
the job of testing threads for completion before moving on. Pperhaps it
does, I haven't checked - my macro writing is a bit out of date as I
say,
because I tend to use plugins now.

regards,
Jon

____________________________________________________________________________________
Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7