Problems uploading jars to update site

[Birgit Möller]

Dear all,
we are trying to set up an ImageJ update site for our plugin collection MiToBo. Since we have a lot of dependencies we would like to test the setup first on our own local server, but in the end we plan to provide Mitobo via an ImageJ Wiki update site. Unfortunately we encountered some problems. Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both jars seem to have been provided by the Fiji Update Site in former days, but are now declared obsolete. Since we need both of them we tried to upload them to our own site, however, this fails. First Fiji claims about changed checksums for both jar files. After recalculating the checksums and trying to upload them again, the updater throws the following exception:

Upload failed: java.lang.NoClassDefFoundError: org/apache/xml/serializer/TreeWalker

Then the updater can only be canceled and trying to re-run it requires recalculation of checksums again. Nevertheless the upload fails again with the above exception. Does anyone have an idea why it is not possible to upload both jars to our own site? For other jars unseen by Fiji before there were no problems.

And finally two general questions:

- What is Fiji's strategy to deal with cases where the same jar in different versions is provided by two update sites?

- How does the updater try to determine depencies of plugins? In our case while checking our local jars it claimed to detect a cyclic dependency which was not there.

Thanks and best regards,

 Birgit


------------------------------------------------------------------------
Dr. rer. nat. Birgit Moeller

Pattern Recognition & Bioinformatics
Institute of Computer Science / Faculty of Natural Sciences III
Martin Luther University Halle-Wittenberg

office:     Room 4.12
phone:      [phone]
fax:        [phone]
snail mail: Von-Seckendorff-Platz 1, 06120 Halle / Saale (Germany)
e-mail:     [hidden email]
www:        http://www.informatik.uni-halle.de/moeller/
------------------------------------------------------------------------

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Rasband, Wayne (NIH/NIMH) [E]]

Dear Birgit,

The Fiji-devel mailing list [1] is the best place to ask questions about update sites and the updater.

[1] https://groups.google.com/forum/#!forum/fiji-devel

-wayne

> On Mar 15, 2015, at 8:34 AM, Birgit Möller <[hidden email]> wrote:
>
> Dear all,
> we are trying to set up an ImageJ update site for our plugin collection MiToBo. Since we have a lot of dependencies we would like to test the setup first on our own local server, but in the end we plan to provide Mitobo via an ImageJ Wiki update site. Unfortunately we encountered some problems. Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both jars seem to have been provided by the Fiji Update Site in former days, but are now declared obsolete. Since we need both of them we tried to upload them to our own site, however, this fails. First Fiji claims about changed checksums for both jar files. After recalculating the checksums and trying to upload them again, the updater throws the following exception:
>
> Upload failed: java.lang.NoClassDefFoundError: org/apache/xml/serializer/TreeWalker
>
> Then the updater can only be canceled and trying to re-run it requires recalculation of checksums again. Nevertheless the upload fails again with the above exception. Does anyone have an idea why it is not possible to upload both jars to our own site? For other jars unseen by Fiji before there were no problems.
>
> And finally two general questions:
>
> - What is Fiji's strategy to deal with cases where the same jar in different versions is provided by two update sites?
>
> - How does the updater try to determine depencies of plugins? In our case while checking our local jars it claimed to detect a cyclic dependency which was not there.
>
> Thanks and best regards,
>
> Birgit

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi Wayne,

I would suggest the imagej-devel list since the updater is not a
Fiji-specific feature.

Regardless: Birgit, I'll reply to your questions tomorrow when I'm back in
the office.

Regards,
Curtis
On Mar 15, 2015 10:26 AM, "Rasband, Wayne (NIH/NIMH) [E]" <
[hidden email]> wrote:

> Dear Birgit,
>
> The Fiji-devel mailing list [1] is the best place to ask questions about
> update sites and the updater.
>
> [1] https://groups.google.com/forum/#!forum/fiji-devel
>
> -wayne
>
> > On Mar 15, 2015, at 8:34 AM, Birgit Möller <
> [hidden email]> wrote:
> >
> > Dear all,
> > we are trying to set up an ImageJ update site for our plugin collection
> MiToBo. Since we have a lot of dependencies we would like to test the setup
> first on our own local server, but in the end we plan to provide Mitobo via
> an ImageJ Wiki update site. Unfortunately we encountered some problems.
> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both jars
> seem to have been provided by the Fiji Update Site in former days, but are
> now declared obsolete. Since we need both of them we tried to upload them
> to our own site, however, this fails. First Fiji claims about changed
> checksums for both jar files. After recalculating the checksums and trying
> to upload them again, the updater throws the following exception:
> >
> > Upload failed: java.lang.NoClassDefFoundError:
> org/apache/xml/serializer/TreeWalker
> >
> > Then the updater can only be canceled and trying to re-run it requires
> recalculation of checksums again. Nevertheless the upload fails again with
> the above exception. Does anyone have an idea why it is not possible to
> upload both jars to our own site? For other jars unseen by Fiji before
> there were no problems.
> >
> > And finally two general questions:
> >
> > - What is Fiji's strategy to deal with cases where the same jar in
> different versions is provided by two update sites?
> >
> > - How does the updater try to determine depencies of plugins? In our
> case while checking our local jars it claimed to detect a cyclic dependency
> which was not there.
> >
> > Thanks and best regards,
> >
> > Birgit
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi Birgit,

> we are trying to set up an ImageJ update site for our plugin
> collection MiToBo.

Sounds great!

> Since we have a lot of dependencies we would like to test the setup
> first on our own local server

Good idea.

> but in the end we plan to provide Mitobo via an ImageJ Wiki update
> site.

Well, if you have a dedicated server, you can just use that. There is no
requirement to use a personal update site. But I guess the URL "
http://sites.imagej.net/MiToBo" would be pretty friendly. :-)

> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
> jars seem to have been provided by the Fiji Update Site in former
> days, but are now declared obsolete.

Indeed.

> Since we need both of them we tried to upload them to our own site,
> however, this fails. First Fiji claims about changed checksums for
> both jar files. After recalculating the checksums and trying to
> upload them again, the updater throws the following exception:
>
> Upload failed: java.lang.NoClassDefFoundError:
> org/apache/xml/serializer/TreeWalker

How bizarre. I wonder if this is something xalan-specific, since xalan
ships classes that are also part of the JRE itself. As a test, you could
temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
the sezpoz JAR to your update site, and see whether you encounter the same
issue.

> What is Fiji's strategy to deal with cases where the same jar in
> different versions is provided by two update sites?

IIRC, the ImageJ Updater (which is part of ImageJ2, and not specific to the
Fiji distribution) favors versions further down in the list of update
sites. That is: there is a linear order to the update sites, such that
files from sites further down the chain are considered to "shadow" the same
file from sites further up the chain. The reason I say "IIRC" is because I
am not 100% certain that chain order is only defined by the ordering listed
on the "List of update sites" wiki page. It may be that the order changes
for a local installation depending on when sites are toggled on and off. If
you are curious to dig in further, there is a unit test that verifies that
various multi-update-site features work properly [1], which you could play
with.

The gist is that there is currently _no_ mechanism for defining
update-site-level dependencies, beyond just building on top of the ImageJ
and Fiji update sites. It is currently the user's responsibility to enable
update sites upon which your update site depends. Some day we may address
that issue as the number of update sites continues to grow, but it is not a
simple feature to add.

> How does the updater try to determine depencies of plugins? In our
> case while checking our local jars it claimed to detect a cyclic
> dependency which was not there.

It uses byte-code analysis to detect the dependencies. However, you can
tweak them before uploading by editing the list of dependencies directly in
the right-hand text box of the graphical updater window. This is sometimes
necessary in cases where there are undesired circular dependencies (e.g.,
slf4j-api and various slf4j bindings always detect as circular, due to the
design of SLF4J).

Regards,
Curtis

[1]
https://github.com/imagej/imagej-updater/blob/imagej-updater-0.7.1/src/test/java/net/imagej/updater/MultipleSitesTest.java

On Sun, Mar 15, 2015 at 7:34 AM, Birgit Möller <
[hidden email]> wrote:

> Dear all,
> we are trying to set up an ImageJ update site for our plugin collection
> MiToBo. Since we have a lot of dependencies we would like to test the setup
> first on our own local server, but in the end we plan to provide Mitobo via
> an ImageJ Wiki update site. Unfortunately we encountered some problems.
> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both jars
> seem to have been provided by the Fiji Update Site in former days, but are
> now declared obsolete. Since we need both of them we tried to upload them
> to our own site, however, this fails. First Fiji claims about changed
> checksums for both jar files. After recalculating the checksums and trying
> to upload them again, the updater throws the following exception:
>
> Upload failed: java.lang.NoClassDefFoundError:
> org/apache/xml/serializer/TreeWalker
>
> Then the updater can only be canceled and trying to re-run it requires
> recalculation of checksums again. Nevertheless the upload fails again with
> the above exception. Does anyone have an idea why it is not possible to
> upload both jars to our own site? For other jars unseen by Fiji before
> there were no problems.
>
> And finally two general questions:
>
> - What is Fiji's strategy to deal with cases where the same jar in
> different versions is provided by two update sites?
>
> - How does the updater try to determine depencies of plugins? In our case
> while checking our local jars it claimed to detect a cyclic dependency
> which was not there.
>
> Thanks and best regards,
>
>  Birgit
>
>
> ------------------------------------------------------------------------
> Dr. rer. nat. Birgit Moeller
>
> Pattern Recognition & Bioinformatics
> Institute of Computer Science / Faculty of Natural Sciences III
> Martin Luther University Halle-Wittenberg
>
> office:     Room 4.12
> phone:      [phone]
> fax:        [phone]
> snail mail: Von-Seckendorff-Platz 1, 06120 Halle / Saale (Germany)
> e-mail:     [hidden email]
> www:        http://www.informatik.uni-halle.de/moeller/
> ------------------------------------------------------------------------
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Birgit Möller]

Hi Curtis!

On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden <[hidden email]> wrote:

>Hi Birgit,
>
>> we are trying to set up an ImageJ update site for our plugin
>> collection MiToBo.
>
>Sounds great!
>
>> Since we have a lot of dependencies we would like to test the setup
>> first on our own local server
>
>Good idea.
>
>> but in the end we plan to provide Mitobo via an ImageJ Wiki update
>> site.
>
>Well, if you have a dedicated server, you can just use that. There is no
>requirement to use a personal update site. But I guess the URL "
>http://sites.imagej.net/MiToBo" would be pretty friendly. :-)

yes, I agree, and we will go for it soon :-)

>> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> jars seem to have been provided by the Fiji Update Site in former
>> days, but are now declared obsolete.
>
>Indeed.
>
>> Since we need both of them we tried to upload them to our own site,
>> however, this fails. First Fiji claims about changed checksums for
>> both jar files. After recalculating the checksums and trying to
>> upload them again, the updater throws the following exception:
>>
>> Upload failed: java.lang.NoClassDefFoundError:
>> org/apache/xml/serializer/TreeWalker
>
>How bizarre. I wonder if this is something xalan-specific, since xalan
>ships classes that are also part of the JRE itself. As a test, you could
>temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
>the sezpoz JAR to your update site, and see whether you encounter the same
>issue.

It looks like it is really a Xalan issue. Skipping the Xalan jar everything works fine. Meanwhile I also figured out that the Xalan jar is included in our dependencies only to due loci_tools declaring that jar as a dependency. As during the update procedure we were also told that loci_tools is now part of the bioformats package, I wonder if we can get rid of that dependency by switching from loci_tools to bioformats_package. Moreover, if the Xalan jar is not present, our operators and plugins seem to work anyway, so maybe we even do not really need any stuff contained in that jar - but we will still need to investigate that further.

>[...]
>
>Regards,
>Curtis
>
>[...]

For now, thanks for your elaborate reply.  If we encounter additional problems, I will get back to you and the list again. But, as suggested, next time I will post questions regarding update sites on the imagej-devel list instead of this one.
Thanks and kind regards,

 Birgit

---

On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden <[hidden email]> wrote:

>Hi Birgit,
>
>> we are trying to set up an ImageJ update site for our plugin
>> collection MiToBo.
>
>Sounds great!
>
>> Since we have a lot of dependencies we would like to test the setup
>> first on our own local server
>
>Good idea.
>
>> but in the end we plan to provide Mitobo via an ImageJ Wiki update
>> site.
>
>Well, if you have a dedicated server, you can just use that. There is no
>requirement to use a personal update site. But I guess the URL "
>http://sites.imagej.net/MiToBo" would be pretty friendly. :-)
>
>> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> jars seem to have been provided by the Fiji Update Site in former
>> days, but are now declared obsolete.
>
>Indeed.
>
>> Since we need both of them we tried to upload them to our own site,
>> however, this fails. First Fiji claims about changed checksums for
>> both jar files. After recalculating the checksums and trying to
>> upload them again, the updater throws the following exception:
>>
>> Upload failed: java.lang.NoClassDefFoundError:
>> org/apache/xml/serializer/TreeWalker
>
>How bizarre. I wonder if this is something xalan-specific, since xalan
>ships classes that are also part of the JRE itself. As a test, you could
>temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
>the sezpoz JAR to your update site, and see whether you encounter the same
>issue.
>
>> What is Fiji's strategy to deal with cases where the same jar in
>> different versions is provided by two update sites?
>
>IIRC, the ImageJ Updater (which is part of ImageJ2, and not specific to the
>Fiji distribution) favors versions further down in the list of update
>sites. That is: there is a linear order to the update sites, such that
>files from sites further down the chain are considered to "shadow" the same
>file from sites further up the chain. The reason I say "IIRC" is because I
>am not 100% certain that chain order is only defined by the ordering listed
>on the "List of update sites" wiki page. It may be that the order changes
>for a local installation depending on when sites are toggled on and off. If
>you are curious to dig in further, there is a unit test that verifies that
>various multi-update-site features work properly [1], which you could play
>with.
>
>The gist is that there is currently _no_ mechanism for defining
>update-site-level dependencies, beyond just building on top of the ImageJ
>and Fiji update sites. It is currently the user's responsibility to enable
>update sites upon which your update site depends. Some day we may address
>that issue as the number of update sites continues to grow, but it is not a
>simple feature to add.
>
>> How does the updater try to determine depencies of plugins? In our
>> case while checking our local jars it claimed to detect a cyclic
>> dependency which was not there.
>
>It uses byte-code analysis to detect the dependencies. However, you can
>tweak them before uploading by editing the list of dependencies directly in
>the right-hand text box of the graphical updater window. This is sometimes
>necessary in cases where there are undesired circular dependencies (e.g.,
>slf4j-api and various slf4j bindings always detect as circular, due to the
>design of SLF4J).
>
>Regards,
>Curtis
>
>[1]
>https://github.com/imagej/imagej-updater/blob/imagej-updater-0.7.1/src/test/java/net/imagej/updater/MultipleSitesTest.java
>
>On Sun, Mar 15, 2015 at 7:34 AM, Birgit Möller <
>[hidden email]> wrote:
>
>> Dear all,
>> we are trying to set up an ImageJ update site for our plugin collection
>> MiToBo. Since we have a lot of dependencies we would like to test the setup
>> first on our own local server, but in the end we plan to provide Mitobo via
>> an ImageJ Wiki update site. Unfortunately we encountered some problems.
>> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both jars
>> seem to have been provided by the Fiji Update Site in former days, but are
>> now declared obsolete. Since we need both of them we tried to upload them
>> to our own site, however, this fails. First Fiji claims about changed
>> checksums for both jar files. After recalculating the checksums and trying
>> to upload them again, the updater throws the following exception:
>>
>> Upload failed: java.lang.NoClassDefFoundError:
>> org/apache/xml/serializer/TreeWalker
>>
>> Then the updater can only be canceled and trying to re-run it requires
>> recalculation of checksums again. Nevertheless the upload fails again with
>> the above exception. Does anyone have an idea why it is not possible to
>> upload both jars to our own site? For other jars unseen by Fiji before
>> there were no problems.
>>
>> And finally two general questions:
>>
>> - What is Fiji's strategy to deal with cases where the same jar in
>> different versions is provided by two update sites?
>>
>> - How does the updater try to determine depencies of plugins? In our case
>> while checking our local jars it claimed to detect a cyclic dependency
>> which was not there.
>>
>> Thanks and best regards,
>>
>>  Birgit
>>
>>
>> ------------------------------------------------------------------------
>> Dr. rer. nat. Birgit Moeller
>>
>> Pattern Recognition & Bioinformatics
>> Institute of Computer Science / Faculty of Natural Sciences III
>> Martin Luther University Halle-Wittenberg
>>
>> office:     Room 4.12
>> phone:      [phone]
>> fax:        [phone]
>> snail mail: Von-Seckendorff-Platz 1, 06120 Halle / Saale (Germany)
>> e-mail:     [hidden email]
>> www:        http://www.informatik.uni-halle.de/moeller/
>> ------------------------------------------------------------------------
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
>--
>ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html