Java, Linking and Portability

[Benjamin Eltzner]

Dear mailing list,

I am a novice Java programmer (~ 10 months) currently maintaining a Java
program which includes quite some image processing. I would like to use
imageJ as a backend for the image processing (and in the course of this,
transform my filters into imageJ plugins). However, when using an
external jar, I run into a portability problem:

So far my program only uses the Java class library, so I can develop on
Linux and distribute a runnable jar that will run on Windows, Linux and
Mac. What is the sweet spot between "easy for me" and "easy for the
users" to distribute my program, if it should use a local imageJ
install? (I do not want to include imageJ code in my jar, to avoid
duplication, and also because I explicitly want to make use of imageJ's
own structure, e.g. its plugin handling. Also, I am sure there must be
tons of relevant documentation on the internet, but despite the fact
that I tried hard to find a solution by myself, I have now been stuck on
this issue for weeks.)

Ideally, the user would just give the path to her ij.jar in a dialog
within my program, but this seems impossible, since the classpath must
be set before the program starts. I tried using ClassLoaders but it
seems that this will result in an extreme amount of "management code".
(If there is simple solution with ClassLoaders, I would be glad about
some pointers.)

The next best solution would be to have a platform independent
installer, that asks the user where his ij.jar is and then creates a
shell script / batch file that runs my jar with the appropriate path to
the ij.jar. Is there a convenient way to accomplish that for all
platforms when programming only on Linux?

Is there some other, clever way to accomplish what I want? I would be
very glad about any helpful comments.


Best regards,

Benjamin

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[dscho]

Hi Benjamin,

On Wed, 10 Sep 2014, Benjamin Eltzner wrote:

> So far my program only uses the Java class library, so I can develop on
> Linux and distribute a runnable jar that will run on Windows, Linux and
> Mac.

You have been avoiding what has been called "dependency hell". That is
sustainable if you develop alone, but if you want to develop together,
you need proper dependency management.

By proper dependency management, I refer to a way to declare which
dependencies (and which versions thereof) your primary project depends on.

As we had the problem you described in Fiji, ever since its beginning, I
daresay we have quite a bit of experience with it, and the best solution
we could come up with is called Maven (http://imagej.net/Maven)

Of course, this will make things a little less easy for you. But it will
make it possible for you to cooperate with other projects.

> Ideally, the user would just give the path to her ij.jar in a dialog
> within my program, but this seems impossible, since the classpath must
> be set before the program starts. I tried using ClassLoaders but it
> seems that this will result in an extreme amount of "management code".
> (If there is simple solution with ClassLoaders, I would be glad about
> some pointers.)

While this is technically feasible (I could even show you how), it seems
backwards. If you want to implement your processing as ImageJ plugins, you
should let ImageJ manage your plugins, not the other way round.

> What is the sweet spot between "easy for me" and "easy for the users" to
> distribute my program, if it should use a local imageJ install?

There is an ImageJ updater available in ImageJ2 (and consequently in Fiji,
which is just a distribution of ImageJ with plugins): Help>Update...

You can easily set up your own update site that users can simply follow:

        http://imagej.net/How_to_follow_a_3rd_party_update_site

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[dscho]

Hi Benjamin,

at LOCI we like to take conversations private only when there is a benefit
in doing so (see http://loci.wisc.edu/software/developing-loci-software
section 'Mailing Lists'). In this particular case, I actually see the
opposite of a benefit in taking the conversation private, so I re-Cc: the
list. Please let me know if you are not okay with that.

On Wed, 10 Sep 2014, Benjamin Eltzner wrote:

> >> So far my program only uses the Java class library, so I can develop
> >> on Linux and distribute a runnable jar that will run on Windows,
> >> Linux and Mac.
> >
> > You have been avoiding what has been called "dependency hell". That is
> > sustainable if you develop alone, but if you want to develop together,
> > you need proper dependency management.
> >
> > By proper dependency management, I refer to a way to declare which
> > dependencies (and which versions thereof) your primary project depends
> > on.
> >
> > As we had the problem you described in Fiji, ever since its beginning,
> > I daresay we have quite a bit of experience with it, and the best
> > solution we could come up with is called Maven
> > (http://imagej.net/Maven)
>
> thank you very much for your suggestion. If I understand the explanation
> of Maven correctly, the workflow would be,
> 1. list my dependencies in a pom.xml file
> 2. tell Maven to download them
> 3. distribute my program with all the jar files Maven fetched

Almost. You do not need to tell Maven to download dependencies. You also
do not distribute the dependencies explicitly.

Instead, the workflow is something like this:

        1) Develop your plugin in an IDE such as Netbeans

        2) When satisfied that it works as intended, i.e. when all the
           unit tests pass, you install your plugin into an existing,
           up-to-date Fiji (downloaded and unpacked from http://fiji.sc/):

                mvn -Dimagej.app.directory=/path/to/Fiji.app

           This will automatically install also the dependencies that are
           not present in that Fiji installation yet.

        3) *Test the plugin in Fiji*

        4) Start the updater with Help>Update... and upload to your update
           site (initializing your own personal update site if you have
           not done so yet).

           The updater will suggest to upload dependencies that are not
           available in Fiji yet, too.

Of course you can also distribute your own ImageJ with your plugin, in a
more manual process, but then you should not ask the person who worked so
hard to make the process more automated ;-)

> or is it more like
> 2. use Maven to make an installer, which I distribute
> 3. the user launches the installer, which looks, whether a local maven
> repository is there, which prerequisites it already has and fetches
> missing dependencies to the users local maven repository from the remote
> repository I named in the pom.xml.
> ?

No, Maven is purely for use on the developers' side. The users will never
touch Maven.

> If I misunderstood the concept of Maven, I would be glad if you could
> elaborate a little more.

Well, I hoped that the page I created at http://imagej.net/Maven was clear
enough... Maven is a dependency management system that can also build your
project.

But I probably forgot to mention the best part about Maven: the
configuration is so standardized that other software, including Netbeans
and Eclipse, can interpret the project configuration without calling the
Maven software itself. Indeed, the project configuration is so easy to
interpret that even the ImageJ updater makes use of it to extract the
project description if it finds a .jar file including the Maven
configuration.

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Benjamin Eltzner]

Hi Johannes,

thank you for re-adding this conversation to the list. My private answer
was just a mistake. Thank you for your further explanation of the Maven
workflow.

> Instead, the workflow is something like this:
>
> 1) Develop your plugin in an IDE such as Netbeans
>
> 2) When satisfied that it works as intended, i.e. when all the
>   unit tests pass, you install your plugin into an existing,
>   up-to-date Fiji (downloaded and unpacked from http://fiji.sc/):
>
> mvn -Dimagej.app.directory=/path/to/Fiji.app
>
>   This will automatically install also the dependencies that are
>   not present in that Fiji installation yet.
>
> 3) *Test the plugin in Fiji*
>
> 4) Start the updater with Help>Update... and upload to your update
>   site (initializing your own personal update site if you have
>   not done so yet).
>
>   The updater will suggest to upload dependencies that are not
>   available in Fiji yet, too.
>
> Of course you can also distribute your own ImageJ with your plugin, in a
> more manual process, but then you should not ask the person who worked so
> hard to make the process more automated ;-)

However, I think there is a missing bit of information here, namely
this: My program does not only do image processing but also data
processing and most importantly includes a basic GUI and data management
code that is geared towards its use case. I think it would not be
feasible to convert all this into ImageJ plugins (at least not in ImageJ
1). If I am mistaken here, I would be glad for any pointers to a
tutorial on how to build an alternate GUI as a plugin (set).

So with my current understanding of the scope of ImageJ and its plugin
system, I would like to use ImageJ only as a backend for image
processing (but in the course of this I would transform some of my
current image processing code into ImageJ plugins) and otherwise treat
my program as a standalone project. (Even if it is feasible to also do
the GUI and data analysis tools in terms of ImageJ plugins, I would like
to go through such an intermediary phase, as the program already has
more than 10.000 lines of code and a transition will take time which I
currently do not have abundantly.)


Best Regards,

Benjamin

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[dscho]

Hi Benjamin,

On Fri, 12 Sep 2014, Benjamin Eltzner wrote:

> I think there is a missing bit of information here, namely this: My
> program does not only do image processing but also data processing and
> most importantly includes a basic GUI and data management code that is
> geared towards its use case. I think it would not be feasible to convert
> all this into ImageJ plugins (at least not in ImageJ 1). If I am
> mistaken here, I would be glad for any pointers to a tutorial on how to
> build an alternate GUI as a plugin (set).

In ImageJ 1.x, plugins are actually very free-form: all you need to do is
to implement the ij.plugin.PlugIn interface:

        public class Benjamins_Plugin implements PlugIn {
                @Override
                public void run(final String arg) {
                        ...
                }
        }

You can easily show your own GUI inside that run() method – and that is
exactly what many plugins do.

The advantage you would gain is that your "plugin" would be able to use
the complete infrastructure provided by ImageJ, including distribution
through personal update sites.

> So with my current understanding of the scope of ImageJ and its plugin
> system, I would like to use ImageJ only as a backend for image
> processing (but in the course of this I would transform some of my
> current image processing code into ImageJ plugins) and otherwise treat
> my program as a standalone project. (Even if it is feasible to also do
> the GUI and data analysis tools in terms of ImageJ plugins, I would like
> to go through such an intermediary phase, as the program already has
> more than 10.000 lines of code and a transition will take time which I
> currently do not have abundantly.)

Of course you are free to keep your application as a standalone program,
tapping into ImageJ 1.x via IJ.run() calls. You'll have to reinvent a
couple of things you'd otherwise get for free, is all.

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html