JarUpdater: An easy way to distribute plugin updates

[Michael P Ellis]

JarUpdater  is utility class for updating an ImageJ plugin Jar file from the web.

The code is based on, and inspired by the ij.plugin.ImageJ_Updater class but uses versioning information stored in the jar manifest.

Apologies if something like this already exists! (let me know)

I'm unsure how contributions to ImageJ public domain stuff is made, but anyone interested can pickup JarUpdater here:

http://www.dsuk.biz/PublicDomain/DSUK_Public_Domain/JarUpdater.html

It's very easy to use; two lines in the manifest and one line of code to invoke JarUpdater.

It struck me that a future ImageJ could usefully provide this update functionality to any Jar that it loaded that carried the relevant manifest information. This would provide the user with a unified user experience for updates and also relieve plugin developers of the effort of rolling their own update mechanism.

I'd also be interested to hear from anyone regarding any issues or problems with the JarUpdater code, and will be happily consider improvements and suggestions.

Michael Ellis
DSUK

[dscho]

Hi,

On Sun, 27 Feb 2011, Michael Ellis wrote:

> JarUpdater is utility class for updating an ImageJ plugin Jar file from
> the web.
>
> The code is based on, and inspired by the ij.plugin.ImageJ_Updater class
> but uses versioning information stored in the jar manifest.
>
> [...]
>
> It's very easy to use; two lines in the manifest and one line of code to
> invoke JarUpdater.

For anybody interested to know why we did not do it that way in Fiji:

- we did not want to require the manifest to need a special form,

- we did not want to require a version number of a specific form,

- we did not want to hardcode the URL in the plugin's .jar file itself,

- many advanced plugins require 3rd-party libraries in addition to ImageJ.
  With the JarUpdater, this need can only be addressed by packing the
  3rd-party libraries' files into the plugin's .jar file which can lead to
  http://en.wikipedia.org/wiki/Dependency_hell when multiple plugins
  require the same 3rd-party library.

- many advanced plugins require a certain minimal version of ImageJ (and
  sometimes even a certain maximal version of ImageJ; 1.44n broke the
  Image Expression Parser, for example). If all plugin information is to
  be stored in the manifest, the manifests can get very bloated (because
  they have to store redundant information separately) and even worse: the
  manifests need to be updated _even if the plugin version does not
  change_! Therefore it is hard to tell which version of a .jar file is
  current even if the version number is known.

- The load required to inspect whether all local .jar files are up-to-date
  does not scale well, both locally and especially on the server, when for
  each and every .jar file, a connection has to be established with the
  server, quite possibly requiring 3 handshakes: getting the length of the
  file, seeking to the TOC, then seeking to the actual bytes of the
  compressed manifest.

- there are serious problems when you overwrite .jar files that are still
  in use.

Two problems would be fixable in the JarUpdater:

- .jar files can be quite large, if they bundle binary data such as
  installers or large image collections. Reading in the complete .jar
  file's contents into memory might fail quite easily.

- when there are errors during writing the file -- e.g. when the disk is
  full or the quota is reached -- the resulting .jar file is invalid.

The multi-site feature of the Fiji Updater (which is currently in beta
test phase) might be a preferable mechanism to distribute your plugins.
You just need a web server which you can access using SSH.

Hth,
Johannes

P.S.: An updater which addresses at least two previously mentioned
problems is here:
http://groups.google.com/group/fiji-devel/browse_thread/thread/422ce3a1206b6090/e3dd7a63c2fe8841#e3dd7a63c2fe8841
You might be happier with the Plugins_Updater than with your own Fiji
Updater site, although it is less convenient for users.

[Michael P Ellis]

Johannes, hi and thanks for the info. As I said in the web page footnote, I had an uneasy feeling I was reinventing the wheel with this one!

Plugin_Updater looks really neat. I shall investigate it!

Many thanks!

Michael Ellis

On 27 Feb 2011, at 13:27, Johannes Schindelin wrote:

> Hi,
>
> On Sun, 27 Feb 2011, Michael Ellis wrote:
>
>> JarUpdater is utility class for updating an ImageJ plugin Jar file from
>> the web.
>>
>> The code is based on, and inspired by the ij.plugin.ImageJ_Updater class
>> but uses versioning information stored in the jar manifest.
>>
>> [...]
>>
>> It's very easy to use; two lines in the manifest and one line of code to
>> invoke JarUpdater.
>
> For anybody interested to know why we did not do it that way in Fiji:
>
> - we did not want to require the manifest to need a special form,
>
> - we did not want to require a version number of a specific form,
>
> - we did not want to hardcode the URL in the plugin's .jar file itself,
>
> - many advanced plugins require 3rd-party libraries in addition to ImageJ.
> With the JarUpdater, this need can only be addressed by packing the
> 3rd-party libraries' files into the plugin's .jar file which can lead to
> http://en.wikipedia.org/wiki/Dependency_hell when multiple plugins
> require the same 3rd-party library.
>
> - many advanced plugins require a certain minimal version of ImageJ (and
> sometimes even a certain maximal version of ImageJ; 1.44n broke the
> Image Expression Parser, for example). If all plugin information is to
> be stored in the manifest, the manifests can get very bloated (because
> they have to store redundant information separately) and even worse: the
> manifests need to be updated _even if the plugin version does not
> change_! Therefore it is hard to tell which version of a .jar file is
> current even if the version number is known.
>
> - The load required to inspect whether all local .jar files are up-to-date
> does not scale well, both locally and especially on the server, when for
> each and every .jar file, a connection has to be established with the
> server, quite possibly requiring 3 handshakes: getting the length of the
> file, seeking to the TOC, then seeking to the actual bytes of the
> compressed manifest.
>
> - there are serious problems when you overwrite .jar files that are still
> in use.
>
> Two problems would be fixable in the JarUpdater:
>
> - .jar files can be quite large, if they bundle binary data such as
> installers or large image collections. Reading in the complete .jar
> file's contents into memory might fail quite easily.
>
> - when there are errors during writing the file -- e.g. when the disk is
> full or the quota is reached -- the resulting .jar file is invalid.
>
> The multi-site feature of the Fiji Updater (which is currently in beta
> test phase) might be a preferable mechanism to distribute your plugins.
> You just need a web server which you can access using SSH.
>
> Hth,
> Johannes
>
> P.S.: An updater which addresses at least two previously mentioned
> problems is here:
> http://groups.google.com/group/fiji-devel/browse_thread/thread/422ce3a1206b6090/e3dd7a63c2fe8841#e3dd7a63c2fe8841
> You might be happier with the Plugins_Updater than with your own Fiji
> Updater site, although it is less convenient for users.

[Jarek Sacha-2]

Johannes:

I am looking forward to a plugins updater that have the features you
mention and can run with or without Fiji. Updating plugins and their
dependencies is a pain right now. A flexible mechanism that works with
"plain" ImageJ and custom distributions is really needed.

Let me know if could help with testing or in other ways.

Jarek
http://ij-plugins.sf.net

On 2/27/2011 8:27 AM, Johannes Schindelin wrote:

> Hi,
>
> On Sun, 27 Feb 2011, Michael Ellis wrote:
>
>> JarUpdater is utility class for updating an ImageJ plugin Jar file from
>> the web.
>>
>> The code is based on, and inspired by the ij.plugin.ImageJ_Updater class
>> but uses versioning information stored in the jar manifest.
>>
>> [...]
>>
>> It's very easy to use; two lines in the manifest and one line of code to
>> invoke JarUpdater.
> For anybody interested to know why we did not do it that way in Fiji:
>
> - we did not want to require the manifest to need a special form,
>
> - we did not want to require a version number of a specific form,
>
> - we did not want to hardcode the URL in the plugin's .jar file itself,
>
> - many advanced plugins require 3rd-party libraries in addition to ImageJ.
>    With the JarUpdater, this need can only be addressed by packing the
>    3rd-party libraries' files into the plugin's .jar file which can lead to
>    http://en.wikipedia.org/wiki/Dependency_hell when multiple plugins
>    require the same 3rd-party library.
>
> - many advanced plugins require a certain minimal version of ImageJ (and
>    sometimes even a certain maximal version of ImageJ; 1.44n broke the
>    Image Expression Parser, for example). If all plugin information is to
>    be stored in the manifest, the manifests can get very bloated (because
>    they have to store redundant information separately) and even worse: the
>    manifests need to be updated _even if the plugin version does not
>    change_! Therefore it is hard to tell which version of a .jar file is
>    current even if the version number is known.
>
> - The load required to inspect whether all local .jar files are up-to-date
>    does not scale well, both locally and especially on the server, when for
>    each and every .jar file, a connection has to be established with the
>    server, quite possibly requiring 3 handshakes: getting the length of the
>    file, seeking to the TOC, then seeking to the actual bytes of the
>    compressed manifest.
>
> - there are serious problems when you overwrite .jar files that are still
>    in use.
>
> Two problems would be fixable in the JarUpdater:
>
> - .jar files can be quite large, if they bundle binary data such as
>    installers or large image collections. Reading in the complete .jar
>    file's contents into memory might fail quite easily.
>
> - when there are errors during writing the file -- e.g. when the disk is
>    full or the quota is reached -- the resulting .jar file is invalid.
>
> The multi-site feature of the Fiji Updater (which is currently in beta
> test phase) might be a preferable mechanism to distribute your plugins.
> You just need a web server which you can access using SSH.
>
> Hth,
> Johannes
>
> P.S.: An updater which addresses at least two previously mentioned
> problems is here:
> http://groups.google.com/group/fiji-devel/browse_thread/thread/422ce3a1206b6090/e3dd7a63c2fe8841#e3dd7a63c2fe8841
> You might be happier with the Plugins_Updater than with your own Fiji
> Updater site, although it is less convenient for users.
>

[dscho]

Hi Jarek,

On Thu, 3 Mar 2011, Jarek Sacha wrote:

> Johannes:

please Cc: me when you address me directly, unless you _want_ me to miss
your mail.

> I am looking forward to a plugins updater that have the features you
> mention and can run with or without Fiji. Updating plugins and their
> dependencies is a pain right now. A flexible mechanism that works with
> "plain" ImageJ and custom distributions is really needed.

The whole point of Fiji is what you claim you want for "plain" ImageJ. I
know what a pain dependencies are. That's why we started Fiji. Please
understand that porting the facilities you asked for to ImageJ would
accomplish nothing else than recreating Fiji.

Hth,
Johannes

[Jarek Sacha-2]

Johannes:


Fiji is much more than the updater; it is also a distribution of large
amount of plugins. What would be nice, is to separate the updater mechanism
from the other Fiji plugins. Something similar to the Update Centers
provided, for instance, by Eclipse and NetBeans. User could select which
update centers to use. Even better which plugin clusters to use. User could
control which types of plugins he/she is interested in. Having a standard
way to provide plugins, other projects that provide plugins (including the
main ImageJ website) could setup their update centers that would be
registered with ImageJ website.


I do appreciate the complexity of resolving dependencies and conflicts
between library versions, and creating update descriptions for the updater.
That is the reason why solution to this so appealing to many users of
ImageJ. Unless you are using Fiji with all of its plugins and only its
plugins, it has to be done manually.


Jarek

On Thu, Mar 3, 2011 at 8:15 PM, Johannes Schindelin <
[hidden email]> wrote:

> Hi Jarek,
>
> On Thu, 3 Mar 2011, Jarek Sacha wrote:
>
> > Johannes:
>
> please Cc: me when you address me directly, unless you _want_ me to miss
> your mail.
>
> > I am looking forward to a plugins updater that have the features you
> > mention and can run with or without Fiji. Updating plugins and their
> > dependencies is a pain right now. A flexible mechanism that works with
> > "plain" ImageJ and custom distributions is really needed.
>
> The whole point of Fiji is what you claim you want for "plain" ImageJ. I
> know what a pain dependencies are. That's why we started Fiji. Please
> understand that porting the facilities you asked for to ImageJ would
> accomplish nothing else than recreating Fiji.
>
> Hth,
> Johannes
>

[dscho]

Hi Jarek,

On Fri, 4 Mar 2011, Jarek Sacha wrote:

> Fiji is much more than the updater; it is also a distribution of large
> amount of plugins. What would be nice, is to separate the updater
> mechanism from the other Fiji plugins. Something similar to the Update
> Centers provided, for instance, by Eclipse and NetBeans. User could
> select which update centers to use. Even better which plugin clusters to
> use. User could control which types of plugins he/she is interested in.
> Having a standard way to provide plugins, other projects that provide
> plugins (including the main ImageJ website) could setup their update
> centers that would be registered with ImageJ website.

This is the multi-site updater feature of the upcoming Fiji release. There
could even be minimal distributions which set up update sites already.

> I do appreciate the complexity of resolving dependencies and conflicts
> between library versions, and creating update descriptions for the
> updater. That is the reason why solution to this so appealing to many
> users of ImageJ. Unless you are using Fiji with all of its plugins and
> only its plugins, it has to be done manually.

Unfortunately, I made myself misunderstood. For the updater to work, it is
absolutely crucial that there is a custom launcher. We put a lot of work
and thought into this (as well as some unfortunate trial and error), but
if you think you can do better without Fiji (but still handling updating
in-use files and dependencies properly), do go ahead.

Hth,
Johannes