Gaussian Blur Filter in headless mode broken ?!

[Greg]

Hi,

in a huge analysis pipeline, I finally found the specific problem which for me is of utmost irritation.
So in short, when running a macro in headless mode from the command line, the sigma parameter of
the standard Gaussian Filter seems not to be captured by the macro interpreter.

Here is a dummy example:

open('path-to-image');
name = getTitle()

run("Duplicate...", "title=Dup");
                                                   
run("Gaussian Blur...", "sigma=5");
save('gaussian1.tif');

selectImage(name)

run("Gaussian Blur...", "sigma=1");
save('gaussian2.tif');

When calling this macro from the ImageJ in Gui (so normal) mode, everything works as expected.
But when calling this macro from the commandline like 'ImageJ-macosx --headless -macro dummy.ijm'
the two produced images will be the same, meaning that sigma most likely was set to a default value.

I was somehow lucky to even find this, what I suppose is a, bug.

I am running Fiji with ImageJ version 149m on Mac OS 10.10.

It is really annoying that I am stuck now with the automatization because of this silly detail, any help appreciated !

Greets,
Greg

[Paul van Schayck]

Hi Greg,

Yes, this is an often occuring problem with running the macro language
in headless mode. It has to do with how the macro language, and
headless more are implemented. I've stumbled upon a few of these cases
myself as well where parameters are not passed with the run command
often after hours of debugging.

In the end, I've opted not to use the macro language anymore in
headless mode but switch to the Javascript interperter and talk
directly to the ImageJ API.

Kind regards,

Paul van Schayck

On Fri, Feb 6, 2015 at 5:57 PM, Greg <[hidden email]> wrote:

> Hi,
>
> in a huge analysis pipeline, I finally found the specific problem which for
> me is of utmost irritation.
> So in short, when running a macro in headless mode from the command line,
> the sigma parameter of
> the standard Gaussian Filter seems not to be captured by the macro
> interpreter.
>
> Here is a dummy example:
>
> open('path-to-image');
> name = getTitle()
>
> run("Duplicate...", "title=Dup");
>
> run("Gaussian Blur...", "sigma=5");
> save('gaussian1.tif');
>
> selectImage(name)
>
> run("Gaussian Blur...", "sigma=1");
> save('gaussian2.tif');
>
> When calling this macro from the ImageJ in Gui (so normal) mode, everything
> works as expected.
> But when calling this macro from the commandline like 'ImageJ-macosx
> --headless -macro dummy.ijm'
> the two produced images will be the same, meaning that sigma most likely was
> set to a default value.
>
> I was somehow lucky to even find this, what I suppose is a, bug.
>
> I am running Fiji with ImageJ version 149m on Mac OS 10.10.
>
> It is really annoying that I am stuck now with the automatization because of
> this silly detail, any help appreciated !
>
> Greets,
> Greg
>
>
>
> --
> View this message in context: http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Rasband, Wayne (NIH/NIMH) [E]]

On Feb 6, 2015, at 11:57 AM, Greg <[hidden email]> wrote:
>
> Hi,
>
> in a huge analysis pipeline, I finally found the specific problem which for
> me is of utmost irritation.
> So in short, when running a macro in headless mode from the command line,
> the sigma parameter of
> the standard Gaussian Filter seems not to be captured by the macro
> interpreter.

This appears to be a bug in Fiji, which you should report using the Help>Report a Bug command. The problem does not occur if you run the test macro headless in ImageJ, using a command something like

   java -jar ij.jar -batch dummy.ijm

While testing this macro, I also noticed that Fiji throws the following exception on starting.

-wayne

java.lang.IllegalArgumentException: Could not handle HTTPS in public boolean install(java.lang.String path) in ij.plugin.PluginInstaller
        at net.imagej.patcher.CodeHacker.handleHTTPS(CodeHacker.java:1221)
        at net.imagej.patcher.LegacyExtensions.injectHooks(LegacyExtensions.java:286)
        at net.imagej.patcher.LegacyInjector.inject(LegacyInjector.java:308)
        at net.imagej.patcher.LegacyInjector.injectHooks(LegacyInjector.java:109)
        at net.imagej.patcher.LegacyEnvironment.initialize(LegacyEnvironment.java:101)
        at net.imagej.patcher.LegacyEnvironment.applyPatches(LegacyEnvironment.java:495)
        at net.imagej.patcher.LegacyInjector.preinit(LegacyInjector.java:397)
        at net.imagej.patcher.LegacyInjector.preinit(LegacyInjector.java:376)
        at net.imagej.legacy.DefaultLegacyService.<clinit>(DefaultLegacyService.java:124)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:357)
        at java.lang.Class.newInstance(Class.java:310)
        at org.scijava.service.ServiceHelper.createServiceRecursively(ServiceHelper.java:302)
        at org.scijava.service.ServiceHelper.createExactService(ServiceHelper.java:269)
        at org.scijava.service.ServiceHelper.loadService(ServiceHelper.java:231)
        at org.scijava.service.ServiceHelper.loadService(ServiceHelper.java:194)
        at org.scijava.service.ServiceHelper.loadServices(ServiceHelper.java:166)
        at org.scijava.Context.<init>(Context.java:265)
        at org.scijava.Context.<init>(Context.java:222)
        at org.scijava.Context.<init>(Context.java:161)
        at org.scijava.Context.<init>(Context.java:117)
        at org.scijava.Context.<init>(Context.java:105)
        at net.imagej.ImageJ.<init>(ImageJ.java:82)
        at net.imagej.Main.launch(Main.java:61)
        at net.imagej.Main.main(Main.java:73)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at net.imagej.launcher.ClassLauncher.launch(ClassLauncher.java:258)
        at net.imagej.launcher.ClassLauncher.run(ClassLauncher.java:184)
        at net.imagej.launcher.ClassLauncher.main(ClassLauncher.java:76)


   

>
> Here is a dummy example:
>
> open('path-to-image');
> name = getTitle()
>
> run("Duplicate...", "title=Dup");
>
> run("Gaussian Blur...", "sigma=5");
> save('gaussian1.tif');
>
> selectImage(name)
>
> run("Gaussian Blur...", "sigma=1");
> save('gaussian2.tif');
>
> When calling this macro from the ImageJ in Gui (so normal) mode, everything
> works as expected.
> But when calling this macro from the commandline like 'ImageJ-macosx
> --headless -macro dummy.ijm'
> the two produced images will be the same, meaning that sigma most likely was
> set to a default value.
>
> I was somehow lucky to even find this, what I suppose is a, bug.
>
> I am running Fiji with ImageJ version 149m on Mac OS 10.10.
>
> It is really annoying that I am stuck now with the automatization because of
> this silly detail, any help appreciated !
>
> Greets,
> Greg
>
>
>
> --
> View this message in context: http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Greg]

Hi Paul,

thanks for your answer !

Can you be more specific on how to directly talk to the ImageJ API and how to use/where to get the Javascript interpreter ? Maybe pointing me to a tutorial or sth you found useful ?!

I would love to leave the macro language for good.

Greets,
Greg

Hi Greg,

Yes, this is an often occuring problem with running the macro language
in headless mode. It has to do with how the macro language, and
headless more are implemented. I've stumbled upon a few of these cases
myself as well where parameters are not passed with the run command
often after hours of debugging.

In the end, I've opted not to use the macro language anymore in
headless mode but switch to the Javascript interperter and talk
directly to the ImageJ API.

Kind regards,

Paul van Schayck

On Fri, Feb 6, 2015 at 5:57 PM, Greg <[hidden email]> wrote:

> Hi,
>
> in a huge analysis pipeline, I finally found the specific problem which for
> me is of utmost irritation.
> So in short, when running a macro in headless mode from the command line,
> the sigma parameter of
> the standard Gaussian Filter seems not to be captured by the macro
> interpreter.
>
> Here is a dummy example:
>
> open('path-to-image');
> name = getTitle()
>
> run("Duplicate...", "title=Dup");
>
> run("Gaussian Blur...", "sigma=5");
> save('gaussian1.tif');
>
> selectImage(name)
>
> run("Gaussian Blur...", "sigma=1");
> save('gaussian2.tif');
>
> When calling this macro from the ImageJ in Gui (so normal) mode, everything
> works as expected.
> But when calling this macro from the commandline like 'ImageJ-macosx
> --headless -macro dummy.ijm'
> the two produced images will be the same, meaning that sigma most likely was
> set to a default value.
>
> I was somehow lucky to even find this, what I suppose is a, bug.
>
> I am running Fiji with ImageJ version 149m on Mac OS 10.10.
>
> It is really annoying that I am stuck now with the automatization because of
> this silly detail, any help appreciated !
>
> Greets,
> Greg
>
>
>
> --
> View this message in context: http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

---

If you reply to this email, your message will be added to the discussion below:

http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489p5011498.html

To unsubscribe from Gaussian Blur Filter in headless mode broken ?!, click here.
NAML

[Greg]

Hi Wayne,

thanks a lot ! This solved my problem, and yes I will report the bug to Fiji. I wonder how this could happen with Fiji, as 'its just imageJ' ..

Greets,
Greg

[Herbie-4]

Greg,

evidently, Fiji is no longer 'just image J', at least not ImageJ-1.

Best

Herbie

::::::::::::::::::::::::::::::
On 07.02.15 20:34, Greg wrote:
> Hi Wayne,
>
> thanks a lot ! This solved my problem, and yes I will report the bug to
> Fiji. I wonder how this could happen with Fiji, as 'its just imageJ' ..
>
> Greets,
> Greg

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi Greg,

> Can you be more specific on how to directly talk to the ImageJ API and
> how to use/where to get the Javascript interpreter ? Maybe pointing me
> to a tutorial or sth you found useful ?!

I suggest checking out this documentation:

    http://imagej.net/Javascript

There are guides for many different scripting languages there, linked from
the right-hand sidebar.

> yes I will report the bug to Fiji.

Thanks, Greg! FTR, the bug report is at:

    http://fiji.sc/bugzilla/show_bug.cgi?id=1002

> I wonder how this could happen with Fiji, as 'its just imageJ' ..

This bug in ImageJ2 is unspecific to Fiji.

While ImageJ2 tries very hard to maintain backwards compatibility with
ImageJ1, there will be bugs sometimes, given the scope of ImageJ2's
additional features. It is worth noting that in this case, if you run:

    $ java -Djava.awt.headless=true -jar
/Applications/Fiji.app/jars/ij-1.49m.jar -batch gauss.ijm

Then you will see:

    Exception in thread "main" java.awt.HeadlessException

The ImageJ2-specific "--headless" flag works around this limitation of
ImageJ1 under the hood by patching ImageJ1 [1] to avoid API calls that do
not work headless [2].

Of course, such features must never come at the expense of correctness. We
are investigating a fix as soon as possible.

Regards,
Curtis

[1] There is actually an entire sub-project called ij1-patcher dedicated to
this:

    https://github.com/imagej/ij1-patcher

[2] A central part of this feature is the HeadlessGenericDialog class:


https://github.com/imagej/ij1-patcher/blob/ij1-patcher-0.11.2/src/main/java/net/imagej/patcher/HeadlessGenericDialog.java

It is likely that this bug resides there, or nearby.


On Sun, Feb 8, 2015 at 5:37 AM, Herbie <[hidden email]> wrote:

> Greg,
>
> evidently, Fiji is no longer 'just image J', at least not ImageJ-1.
>
> Best
>
> Herbie
>
> ::::::::::::::::::::::::::::::
> On 07.02.15 20:34, Greg wrote:
>
>> Hi Wayne,
>>
>> thanks a lot ! This solved my problem, and yes I will report the bug to
>> Fiji. I wonder how this could happen with Fiji, as 'its just imageJ' ..
>>
>> Greets,
>> Greg
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Greg]

Hi Curtis,

ctrueden wrote

While ImageJ2 tries very hard to maintain backwards compatibility with
ImageJ1, there will be bugs sometimes, given the scope of ImageJ2's
additional features. It is worth noting that in this case, if you run:

    $ java -Djava.awt.headless=true -jar
/Applications/Fiji.app/jars/ij-1.49m.jar -batch gauss.ijm

Then you will see:

    Exception in thread "main" java.awt.HeadlessException

The ImageJ2-specific "--headless" flag works around this limitation of
ImageJ1 under the hood by patching ImageJ1 [1] to avoid API calls that do
not work headless [2].

so according to Waynes reply I tried to run my script via pure ImageJ (ImageJ64.app) on a Mac OS server, and I got the "HeadlessException". Only when I run it with:

java -Djava.awt.headless=false -jar ij.jar -batch myscript.ijm (so false instead of true like you suggested)

I can circumvent the headless exception and happily run the analysis automated in headless mode. So this headless/ImageJ1/ImageJ2/Fiji issues are a bit confusing, at least in the beginning.
So my final question is, if I start to talk to the ImageJ API (with jython in my case, because I like python) directly, will there be less hazzle when going for automated headless analysis on a cluster ?

Greets,
Greg

[Paul van Schayck]

Hi Greg,

On Sun, Feb 8, 2015 at 5:24 PM, Greg <[hidden email]> wrote:
> so according to Waynes reply I tried to run my script via pure ImageJ
> (ImageJ64.app) on a Mac OS server, and I got the "HeadlessException". Only
> when I run it with:
>
> java -Djava.awt.headless=false -jar ij.jar -batch myscript.ijm (so false
> instead of true like you suggested)

Are you absolutely sure you can run this in your cluster? Are you for
example sure you're not running this with ssh X forwarding enabled, or
that is secretively relying on being able to start (hidden) windows?
This has tricked me before using the "-batch" mode.

> I can circumvent the headless exception and happily run the analysis
> automated in headless mode. So this headless/ImageJ1/ImageJ2/Fiji issues are
> a bit confusing, at least in the beginning.
> So my final question is, if I start to talk to the ImageJ API (with jython
> in my case, because I like python) directly, will there be less hazzle when
> going for automated headless analysis on a cluster ?

In my experience: yes! This was only way for us to reliable script for
a cluster environment. However, the macro language cannot be converted
into the ImageJ API directly in some cases. What helps a lot sometimes
is reading the Java code behind a macro command, and convert that to
api calls. What also helps if you haven't found it yet, is the API
documentation:

http://rsbweb.nih.gov/ij/developer/api/overview-summary.html

Kind regards,

Pau

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Greg]

Hi Paul,

yes you are right, I actually remotely logged in to a Desktop PC, although without X11 forwarding.
That means I am not able to run it truly headless on a cluster right now, because with

java -Djava.awt.headless=true -jar ij.jar -batch myscript.ijm

it will throw a headless exception. So the mix of broken Gaussian Blur within the macro language and this headless exception still spoils the automatization for me, and there is no workaround, right ? So when moving on to true headless via

ImageJ-macosx --headless mysuperbjythonscript.py

I hope it will work, otherwise you see me again on a new support thread :)

Thanks for your help,
Greg

[ctrueden]

Hi everyone,

> We are investigating a fix as soon as possible.

For those not following the bug on Bugzilla, it has now been fixed:
    http://fiji.sc/bugzilla/show_bug.cgi?id=1002#c2

The fix will be uploaded to the core ImageJ update site within a few days.
In the meantime, any early testing feedback is welcome.

Greg wrote:
> there is no workaround, right ?

As of this bug-fix, you can use the new version of ij1-patcher and there
should be no more issue.

> I hope it will work, otherwise you see me again on a new support
> thread :)

Great, I hope you do file more bug reports if something is not working for
you. The developers cannot fix these problems without first knowing about
them.

Regards,
Curtis

On Mon, Feb 9, 2015 at 8:01 AM, Greg <[hidden email]> wrote:

> Hi Paul,
>
> yes you are right, I actually remotely logged in to a Desktop PC, although
> without X11 forwarding.
> That means I am not able to run it truly headless on a cluster right now,
> because with
>
> java -Djava.awt.headless=true -jar ij.jar -batch myscript.ijm
>
> it will throw a headless exception. So the mix of broken Gaussian Blur
> within the macro language and this headless exception still spoils the
> automatization for me, and there is no workaround, right ? So when moving
> on
> to true headless via
>
> ImageJ-macosx --headless mysuperbjythonscript.py
>
> I hope it will work, otherwise you see me again on a new support thread :)
>
> Thanks for your help,
> Greg
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489p5011523.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Greg]

Hi Curtis,

wow, thanks for that quick fix, tests were succesful here !

Keep up the good work and many thanks,

Greg

On Tue, Feb 10, 2015 at 12:30 AM, ctrueden [via ImageJ] <[hidden email]> wrote:

Hi everyone,

> We are investigating a fix as soon as possible.

For those not following the bug on Bugzilla, it has now been fixed:
    http://fiji.sc/bugzilla/show_bug.cgi?id=1002#c2

The fix will be uploaded to the core ImageJ update site within a few days.
In the meantime, any early testing feedback is welcome.

Greg wrote:
> there is no workaround, right ?

As of this bug-fix, you can use the new version of ij1-patcher and there
should be no more issue.

> I hope it will work, otherwise you see me again on a new support
> thread :)

Great, I hope you do file more bug reports if something is not working for
you. The developers cannot fix these problems without first knowing about
them.

Regards,
Curtis

On Mon, Feb 9, 2015 at 8:01 AM, Greg <[hidden email]> wrote:

> Hi Paul,
>
> yes you are right, I actually remotely logged in to a Desktop PC, although
> without X11 forwarding.
> That means I am not able to run it truly headless on a cluster right now,
> because with
>
> java -Djava.awt.headless=true -jar ij.jar -batch myscript.ijm
>
> it will throw a headless exception. So the mix of broken Gaussian Blur
> within the macro language and this headless exception still spoils the
> automatization for me, and there is no workaround, right ? So when moving
> on
> to true headless via
>
> ImageJ-macosx --headless mysuperbjythonscript.py
>
> I hope it will work, otherwise you see me again on a new support thread :)
>
> Thanks for your help,
> Greg
>
>
>

> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489p5011523.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

---

If you reply to this email, your message will be added to the discussion below:

http://imagej.1557.x6.nabble.com/Gaussian-Blur-Filter-in-headless-mode-broken-tp5011489p5011539.html

To unsubscribe from Gaussian Blur Filter in headless mode broken ?!, click here.
NAML