Scripting languages and API

[ctrueden]

Hi everyone,

In recent years a wealth of scripting languages have emerged (Clojure,
Javascript, Groovy, Jython, JRuby, Jaskell, etc.). And starting with Java 6,
Java includes a scripting framework for supporting these various languages (
https://scripting.dev.java.net/; or see
http://groovy.codehaus.org/JSR-223+access+to+other+JVM+languages for a great
example of calling from Groovy to other scripting languages). ImageJ has
traditionally provided scripting through its macro language, but Fiji has
added support for other means as well.

We are interested in improving this scripting support—specifically, coupling
a redesigned Java API more directly with a scripting framework and ImageJ's
macro recorder (as Johannes Schindelin and Adrian Daerr describe below).
This scheme makes it easier to develop scripts by providing more flexibility
in how they can be written, as well as the potential to call into existing
libraries written in those same scripting languages.

However, some people have objected to having more scripting choices (e.g.,
Johan Henriksson below), arguing that these additional languages provide
minimal gains while confusing end users who may need to debug scripts in
unfamiliar languages.

Do others agree that this potential for confusion is a greater concern than
the increased flexibility in scripting?

Lastly, regarding Wilhelm Burger's question below about whether a redesigned
API would be worthwhile, the ImageJDev project strongly asserts that it
would be. Scripting is great for many tasks within the ImageJ application,
but we are looking to expand the project into a general-purpose library
usable from other applications as well, which means a robust Java API. The
very rough software architecture plan I posted to the ImageJDev web site (
http://imagejdev.org/plan) lists several layers and components that we hope
to establish, so that the code can be used to facilitate image processing
and visualization at a variety of levels.

-Curtis

[was: API vs. Scripting?]
On Wed, Oct 7, 2009 at 10:58 AM, Wilhelm Burger <[hidden email]> wrote:

> I only now had the time to look at Curtis' document (http://
> imagejx.googlegroups.com/web/ImageJ+Hardening+aims.rtf) submitted at
> the end of August - I really like it and many of the issues agree
> perfectly with those in one of my earlier postings.
>
> I am still having second thoughts about the possible return of such an
> investement and would like to raise a few fundamental questions,
> hoping not to upset anyone:
>
> 1) From the main IJ newslist my experience is that the vast (and
> growing) majority of postings deals with macros of some sort. Are
> there enough IJ users that still write Java code and would possibly
> benefit from an improved API?
>

[was: Outsider's opinion]
On Fri, Oct 23, 2009 at 4:38 PM, Johan Henriksson <[hidden email]>wrote:

> but the worst problem is that we are building a tower of babel by using
> additional languages for minimal gains, that does not provide any gains
> for larger programs (no surprise, I've veto'ed the addition of
> jython/clojure/whatever-support for my tool Endrov). I consider Fiji the
> prime worst example, with Clojure, JRuby, Javascript and Jython. great,
> so now biologists have to know 5 languages instead of 1 to be able to
> read and modify all the scripts/code out there. hackers who fight over
> syntax might like that but I have trouble enough convincing my coworkers
> to just look at java...
>

[was: Performance, was Re: API vs. Scripting?]
On Wed, Dec 16, 2009 at 11:21 AM, Adrian Daerr <[hidden email]> wrote:

> I am not Johannes, but what I think he is saying is that a good scripting
> interface will essentially mirror the API into the scripting language. That
> is, instead of forcing the developers to manually add a macro command every
> time some API element should be made available for scripting (as Wayne does
> currently, if I understand correctly), a "good" scripting interface can and
> will automatically expose the API calls to the scripting language and thus
> cause no additionnal work when said API is extended. The logic, coherence,
> names etc are chosen only once, scripting follows practically for free.
>

[Gluender-3]

By comment re: IJ-scripting, see below!

>Hi everyone,
>
>In recent years a wealth of scripting languages have emerged (Clojure,
>Javascript, Groovy, Jython, JRuby, Jaskell, etc.). And starting with Java 6,
>Java includes a scripting framework for supporting these various languages (
>https://scripting.dev.java.net/; or see
>http://groovy.codehaus.org/JSR-223+access+to+other+JVM+languages for a great
>example of calling from Groovy to other scripting languages). ImageJ has
>traditionally provided scripting through its macro language, but Fiji has
>added support for other means as well.
>
>We are interested in improving this scripting support-specifically, coupling
>a redesigned Java API more directly with a scripting framework and ImageJ's
>macro recorder (as Johannes Schindelin and Adrian Daerr describe below).
>This scheme makes it easier to develop scripts by providing more flexibility
>in how they can be written, as well as the potential to call into existing
>libraries written in those same scripting languages.
>
>However, some people have objected to having more scripting choices (e.g.,
>Johan Henriksson below), arguing that these additional languages provide
>minimal gains while confusing end users who may need to debug scripts in
>unfamiliar languages.
>
>Do others agree that this potential for confusion is a greater concern than
>the increased flexibility in scripting?
>
>Lastly, regarding Wilhelm Burger's question below about whether a redesigned
>API would be worthwhile, the ImageJDev project strongly asserts that it
>would be. Scripting is great for many tasks within the ImageJ application,
>but we are looking to expand the project into a general-purpose library
>usable from other applications as well, which means a robust Java API. The
>very rough software architecture plan I posted to the ImageJDev web site (
>http://imagejdev.org/plan) lists several layers and components that we hope
>to establish, so that the code can be used to facilitate image processing
>and visualization at a variety of levels.
>
>-Curtis
>
>[was: API vs. Scripting?]
>On Wed, Oct 7, 2009 at 10:58 AM, Wilhelm Burger <[hidden email]> wrote:
>
>>  I only now had the time to look at Curtis' document (http://
>>  imagejx.googlegroups.com/web/ImageJ+Hardening+aims.rtf) submitted at
>>  the end of August - I really like it and many of the issues agree
>>  perfectly with those in one of my earlier postings.
>>
>>  I am still having second thoughts about the possible return of such an
>>  investement and would like to raise a few fundamental questions,
>>  hoping not to upset anyone:
>>
>>  1) From the main IJ newslist my experience is that the vast (and
>>  growing) majority of postings deals with macros of some sort. Are
>>  there enough IJ users that still write Java code and would possibly
>>  benefit from an improved API?
>>
>
>[was: Outsider's opinion]
>On Fri, Oct 23, 2009 at 4:38 PM, Johan Henriksson <[hidden email]>wrote:
>
>>  but the worst problem is that we are building a tower of babel by using
>>  additional languages for minimal gains, that does not provide any gains
>>  for larger programs (no surprise, I've veto'ed the addition of
>>  jython/clojure/whatever-support for my tool Endrov). I consider Fiji the
>>  prime worst example, with Clojure, JRuby, Javascript and Jython. great,
>>  so now biologists have to know 5 languages instead of 1 to be able to
>>  read and modify all the scripts/code out there. hackers who fight over
>>  syntax might like that but I have trouble enough convincing my coworkers
>>  to just look at java...
>>
>
>[was: Performance, was Re: API vs. Scripting?]
>On Wed, Dec 16, 2009 at 11:21 AM, Adrian Daerr <[hidden email]> wrote:
>
>>  I am not Johannes, but what I think he is saying is that a good scripting
>>  interface will essentially mirror the API into the scripting language. That
>>  is, instead of forcing the developers to manually add a macro command every
>>  time some API element should be made available for scripting (as Wayne does
>>  currently, if I understand correctly), a "good" scripting interface can and
>  > will automatically expose the API calls to the scripting language and thus
>>  cause no additionnal work when said API is extended. The logic, coherence,
>>  names etc are chosen only once, scripting follows practically for free.
>>

Please keep in mind that most (and I guess roughly 90%) of the
current IJ-users have little to no programming knowledge, not even
with macro-languages!

Personally, I very much like the classic IJ-macro language. I don't
like Javascript at all and would even prefer to use plain Java (for
plug-ins) instead which I think is no option for most of the IJ-users.

That said, please keep the classic IJ-macro language (that is only in
some minor details OOP).
You may remember that Alan Turing in the 1930 used his famous
approach because he previously thought a lot about human thinking and
reasoning. No doubt OOP has advantages for programming but for
signal/image processing it is less intuitive and may lead to problems
(e.g. threading with Java; experts will understand what I mean).

!!!!!!!!!!!!!!!!!!!!!!
I'm convinced that a procedural looking macro language (such as the
classic IJ-macro language) is of great advantage for most of the
IJ-users.
!!!!!!!!!!!!!!!!!!!!!!

In principle I see no problem with additional (and OOP) macro
languages but my plea is for a choice that is easy to maintain and
that doesn't impose any restrictions on IJ and its classic macro
language!

Happy holidays!

Best
--

                   Herbie

          ------------------------
          <http://www.gluender.de>

[David Webster]

I agree with Herbie. I started to learn Javascript, then decided that it
woud be less confusing to just learn Java. I'm generally not sure why there
is such as proliferation of scripting langauages that seem to do more or
less then same thing. Perhaps rather than expending efort to maintain other
scripting langauges for ImageJ, the effort should go toward upgrading the
use of Java, ImageJ macro langauge, and JavaScript.

David Webster

On Thu, Dec 24, 2009 at 4:04 AM, Gluender <[hidden email]> wrote:

> By comment re: IJ-scripting, see below!
>
>
> Hi everyone,
>>
>> In recent years a wealth of scripting languages have emerged (Clojure,
>> Javascript, Groovy, Jython, JRuby, Jaskell, etc.). And starting with Java
>> 6,
>> Java includes a scripting framework for supporting these various languages
>> (
>> https://scripting.dev.java.net/; or see
>> http://groovy.codehaus.org/JSR-223+access+to+other+JVM+languages for a
>> great
>> example of calling from Groovy to other scripting languages). ImageJ has
>> traditionally provided scripting through its macro language, but Fiji has
>> added support for other means as well.
>>
>> We are interested in improving this scripting support-specifically,
>> coupling
>> a redesigned Java API more directly with a scripting framework and
>> ImageJ's
>> macro recorder (as Johannes Schindelin and Adrian Daerr describe below).
>> This scheme makes it easier to develop scripts by providing more
>> flexibility
>> in how they can be written, as well as the potential to call into existing
>> libraries written in those same scripting languages.
>>
>> However, some people have objected to having more scripting choices (e.g.,
>> Johan Henriksson below), arguing that these additional languages provide
>> minimal gains while confusing end users who may need to debug scripts in
>> unfamiliar languages.
>>
>> Do others agree that this potential for confusion is a greater concern
>> than
>> the increased flexibility in scripting?
>>
>> Lastly, regarding Wilhelm Burger's question below about whether a
>> redesigned
>> API would be worthwhile, the ImageJDev project strongly asserts that it
>> would be. Scripting is great for many tasks within the ImageJ application,
>> but we are looking to expand the project into a general-purpose library
>> usable from other applications as well, which means a robust Java API. The
>> very rough software architecture plan I posted to the ImageJDev web site (
>> http://imagejdev.org/plan) lists several layers and components that we
>> hope
>> to establish, so that the code can be used to facilitate image processing
>> and visualization at a variety of levels.
>>
>> -Curtis
>>
>> [was: API vs. Scripting?]
>> On Wed, Oct 7, 2009 at 10:58 AM, Wilhelm Burger <[hidden email]> wrote:
>>
>>  I only now had the time to look at Curtis' document (http://
>>>  imagejx.googlegroups.com/web/ImageJ+Hardening+aims.rtf) submitted at
>>>  the end of August - I really like it and many of the issues agree
>>>  perfectly with those in one of my earlier postings.
>>>
>>>  I am still having second thoughts about the possible return of such an
>>>  investement and would like to raise a few fundamental questions,
>>>  hoping not to upset anyone:
>>>
>>>  1) From the main IJ newslist my experience is that the vast (and
>>>  growing) majority of postings deals with macros of some sort. Are
>>>  there enough IJ users that still write Java code and would possibly
>>>  benefit from an improved API?
>>>
>>>
>> [was: Outsider's opinion]
>> On Fri, Oct 23, 2009 at 4:38 PM, Johan Henriksson <[hidden email]
>> >wrote:
>>
>>  but the worst problem is that we are building a tower of babel by using
>>>  additional languages for minimal gains, that does not provide any gains
>>>  for larger programs (no surprise, I've veto'ed the addition of
>>>  jython/clojure/whatever-support for my tool Endrov). I consider Fiji the
>>>  prime worst example, with Clojure, JRuby, Javascript and Jython. great,
>>>  so now biologists have to know 5 languages instead of 1 to be able to
>>>  read and modify all the scripts/code out there. hackers who fight over
>>>  syntax might like that but I have trouble enough convincing my coworkers
>>>  to just look at java...
>>>
>>>
>> [was: Performance, was Re: API vs. Scripting?]
>> On Wed, Dec 16, 2009 at 11:21 AM, Adrian Daerr <[hidden email]>
>> wrote:
>>
>>  I am not Johannes, but what I think he is saying is that a good scripting
>>>  interface will essentially mirror the API into the scripting language.
>>> That
>>>  is, instead of forcing the developers to manually add a macro command
>>> every
>>>  time some API element should be made available for scripting (as Wayne
>>> does
>>>  currently, if I understand correctly), a "good" scripting interface can
>>> and
>>>
>>  > will automatically expose the API calls to the scripting language and
>> thus
>>
>>>  cause no additionnal work when said API is extended. The logic,
>>> coherence,
>>>  names etc are chosen only once, scripting follows practically for free.
>>>
>>>
> Please keep in mind that most (and I guess roughly 90%) of the current
> IJ-users have little to no programming knowledge, not even with
> macro-languages!
>
> Personally, I very much like the classic IJ-macro language. I don't like
> Javascript at all and would even prefer to use plain Java (for plug-ins)
> instead which I think is no option for most of the IJ-users.
>
> That said, please keep the classic IJ-macro language (that is only in some
> minor details OOP).
> You may remember that Alan Turing in the 1930 used his famous approach
> because he previously thought a lot about human thinking and reasoning. No
> doubt OOP has advantages for programming but for signal/image processing it
> is less intuitive and may lead to problems (e.g. threading with Java;
> experts will understand what I mean).
>
> !!!!!!!!!!!!!!!!!!!!!!
> I'm convinced that a procedural looking macro language (such as the classic
> IJ-macro language) is of great advantage for most of the IJ-users.
> !!!!!!!!!!!!!!!!!!!!!!
>
> In principle I see no problem with additional (and OOP) macro languages but
> my plea is for a choice that is easy to maintain and that doesn't impose any
> restrictions on IJ and its classic macro language!
>
> Happy holidays!
>
> Best
> --
>
>                  Herbie
>
>         ------------------------
>         <http://www.gluender.de>
>

[Jean-Yves Tinevez-2]

On Dec 24, 2009, at 8:19 PM, David Webster wrote:

> I agree with Herbie. I started to learn Javascript, then decided  
> that it
> woud be less confusing to just learn Java. I'm generally not sure  
> why there
> is such as proliferation of scripting langauages that seem to do  
> more or
> less then same thing. Perhaps rather than expending efort to  
> maintain other
> scripting langauges for ImageJ, the effort should go toward  
> upgrading the
> use of Java, ImageJ macro langauge, and JavaScript.
>

Hi all.
Merry Christmas by the way, hope you are enjoying it as much as we do :)

I would like very much to advocate for scripting in ImageJ, being a  
great fan and user of Jython.

David and Herbie seem to point out that scripting languages  
(Javascript is one of them) are not worth the work, when there is  
Java. Being relatively ok with both, I think yes tough.

* First, they are quick to develop in. Remember that sometime ago,  
Python was described as a language you could teach yourself in one  
day. It became slightly deeper recently, but the ease of learning and  
of use are not compromised.
Learning Java is a different thing. For my experience, it is  
fantastically thrilling, but also much more involving. It is a  
compiled language, and the prototyping and debugging processes take  
much longer and are much subtile than in a scripting language.
There are plenty of differences that make Java and scripting languages  
unique and non-overlapping.

It is then not surprising to meet in biology labs grad-students or  
even undergrads that are pure biologists (and the amount of time it  
takes is enormous and leaves little place to anything else), and  
completely fluent in one of these scripting languages. For instance,  
because of bioinformatics courses, students got to learn Ruby or  
Python or more. Having them in ImageJ is like saying "Hey you are  
familiar with that, just kick in and start developing in ImageJ".

* Second, they do not cost that much to develop or maintain. Ask  
Albert (Cordona) and the Fijiers how they implemented scripting  
languages in Fiji. Once the first was done, the others were easy. You  
can see them as optional plugins for instance.
Macro language does not have to feel threatened by scripting languages.

What is more involving probably is having plugins and classes that  
have public methods well thought for access by these languages or  
other plugins. But if this issue arise also for developing in Java.  
Note that this is a serious issue.

* Third, they look like they are *meant to write macros in*. Just as  
Python is often used as a glue to make C binaries act together, a  
scripting language can be used to build macros that automate tasks.  
Much more like the macro language, but with the extras they offer:
- getting the return values of plugin's methods when they are more  
than one number
- support for arrays
- convoluted loops,
- while, try/catch
- low level APIs to deal with files, searching....
- other external APIs e.g. for graphing, stats....
All of these are freebies as soon as these scripting languages are  
plugged into ImageJ.

Cheers to you all
jyt




--
Jean-Yves Tinevez, PhD
postdoc in PFID - Imagopole
Institut Pasteur
25-28 rue du Docteur Roux
75015 Paris, France
tel: [phone]

[Johan Henriksson-2]

> * First, they are quick to develop in. Remember that sometime ago, Python
> was described as a language you could teach yourself in one day. It became
> slightly deeper recently, but the ease of learning and of use are not
> compromised.
> Learning Java is a different thing. For my experience, it is fantastically
> thrilling, but also much more involving. It is a compiled language, and the
> prototyping and debugging processes take much longer and are much subtile
> than in a scripting language.
> There are plenty of differences that make Java and scripting languages
> unique and non-overlapping.
>
> It is then not surprising to meet in biology labs grad-students or even
> undergrads that are pure biologists (and the amount of time it takes is
> enormous and leaves little place to anything else), and completely fluent in
> one of these scripting languages. For instance, because of bioinformatics
> courses, students got to learn Ruby or Python or more. Having them in ImageJ
> is like saying "Hey you are familiar with that, just kick in and start
> developing in ImageJ".
>

I'm putting a lot of faith in beanshell for scripting. it's easy to convert
to proper java (and the reverse) but has the properties of a scripting
language. it has the disadvantage of being a bit java-ish; java is a
notoriously difficult and messy language. I remember our 7 week university
intro programming courses barely touched making interfaces (implementing,
yes, but only for swing). I have no experience of teaching python but one
guy usually taught all of haskell(98) the first lecture.

it's nice to hear that they don't mainly teach perl for bioinformatics where
you come from :P but that's a point to consider, java has otherwise been The
language taught around here (sweden, gothenburg).

/Johan

--
-----------------------------------------------------------
Johan Henriksson
PhD student, Karolinska Institutet
http://mahogny.areta.org  http://www.endrov.net

[Gluender-3]

Jean-Yves,

as you are referring also to my post by stating

"David and Herbie seem to point out that scripting languages
(Javascript is one of them) are not worth the work, when there is
Java. Being relatively ok with both, I think yes tough."

I got the impression that you didn't read my post properly.

Anyway, have a pleasant holiday!

>On Dec 24, 2009, at 8:19 PM, David Webster wrote:
>
>>I agree with Herbie. I started to learn Javascript, then decided that it
>>woud be less confusing to just learn Java. I'm generally not sure why there
>>is such as proliferation of scripting langauages that seem to do more or
>>less then same thing. Perhaps rather than expending efort to maintain other
>>scripting langauges for ImageJ, the effort should go toward upgrading the
>>use of Java, ImageJ macro langauge, and JavaScript.
>>
>
>Hi all.
>Merry Christmas by the way, hope you are enjoying it as much as we do :)
>
>I would like very much to advocate for scripting in ImageJ, being a
>great fan and user of Jython.
>
>David and Herbie seem to point out that scripting languages
>(Javascript is one of them) are not worth the work, when there is
>Java. Being relatively ok with both, I think yes tough.
>
>* First, they are quick to develop in. Remember that sometime ago,
>Python was described as a language you could teach yourself in one
>day. It became slightly deeper recently, but the ease of learning
>and of use are not compromised.
>Learning Java is a different thing. For my experience, it is
>fantastically thrilling, but also much more involving. It is a
>compiled language, and the prototyping and debugging processes take
>much longer and are much subtile than in a scripting language.
>There are plenty of differences that make Java and scripting
>languages unique and non-overlapping.
>
>It is then not surprising to meet in biology labs grad-students or
>even undergrads that are pure biologists (and the amount of time it
>takes is enormous and leaves little place to anything else), and
>completely fluent in one of these scripting languages. For instance,
>because of bioinformatics courses, students got to learn Ruby or
>Python or more. Having them in ImageJ is like saying "Hey you are
>familiar with that, just kick in and start developing in ImageJ".
>
>* Second, they do not cost that much to develop or maintain. Ask
>Albert (Cordona) and the Fijiers how they implemented scripting
>languages in Fiji. Once the first was done, the others were easy.
>You can see them as optional plugins for instance.
>Macro language does not have to feel threatened by scripting languages.
>
>What is more involving probably is having plugins and classes that
>have public methods well thought for access by these languages or
>other plugins. But if this issue arise also for developing in Java.
>Note that this is a serious issue.
>
>* Third, they look like they are *meant to write macros in*. Just as
>Python is often used as a glue to make C binaries act together, a
>scripting language can be used to build macros that automate tasks.
>Much more like the macro language, but with the extras they offer:
>- getting the return values of plugin's methods when they are more
>than one number
>- support for arrays
>- convoluted loops,
>- while, try/catch
>- low level APIs to deal with files, searching....
>- other external APIs e.g. for graphing, stats....
>All of these are freebies as soon as these scripting languages are
>plugged into ImageJ.
>
>Cheers to you all
>jyt

Best
--

                   Herbie

          ------------------------
          <http://www.gluender.de>

[dscho]

Hi,

On Wed, 23 Dec 2009, Curtis Rueden wrote:

> However, some people have objected to having more scripting choices
> (e.g., Johan Henriksson below), arguing that these additional languages
> provide minimal gains while confusing end users who may need to debug
> scripts in unfamiliar languages.

Just for the record: IMHO it is not wise to disallow scripting languages
(be that scripting in general, or specific language choices, such as
Javascript).  Those who do not want to use them, fine, but let's not
restrict others' choices (especially as the Fiji project has established
that a variety of scripting languages actually attracts developers).

IOW I fully agree with Jean-Yves.

Ciao,
Dscho