bug report: -batch command adds txt extension and macros directory automatically

[Carnë Draug]

Hi

The title says it all. It would appear that ImageJ automatically adds
a txt extension when there is none, even if the file without file
extension exists. How to reproduce

1) create a simple macro

$ echo 'print ("Hello world!");' > ./macro_test

2) get an error from ImageJ about file not existing

$ ImageJ-linux32 -batch macro_test

Macro or script not found:

$path for imagej$macros/macro_test.txt

Note that this works fine if the file exists in the current directory
with a txt file extension. A temporary workaround for this is to use
"-batch ./macro_test" but I don't think that ImageJ should be trying
to make any guess at all. Anyone calling through the command line
should know how to specify the path for the macro file. And even if it
should be guessing, then it should only do it after checking if the
file does exist in the current directory.

I am using ImageJ via FIJI in Debian Squeeze (current stable version)
  IJ.getVersion: 1.48s
  IJ.getFullVersion: 1.48s99

Carnë

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[dscho]

Hi Carnë,

On Fri, 14 Mar 2014, Carnë Draug wrote:

> The title says it all.

Call me dumb, but I still required the explanation below to understand the
issue.

> 1) create a simple macro
>
> $ echo 'print ("Hello world!");' > ./macro_test
>
> 2) get an error from ImageJ about file not existing
>
> $ ImageJ-linux32 -batch macro_test
>
> Macro or script not found:
>
> $path for imagej$macros/macro_test.txt

While you might argue that this is a bug, or that it is a bug that ImageJ
looks into the current directory first before looking into the macros/
directory when the file name contains no slash, by now it is an
established API (application programmers' interface).

Therefore I doubt that this bug -- if Wayne agrees that it is a bug -- can
be fixed without seriously breaking existing workflows. And backwards
compatibility is what Wayne states as one of his primary goals with ImageJ
when you get the chance to listen to one of his presentations.

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Carnë Draug]

On 14 March 2014 15:59, Johannes Schindelin <[hidden email]> wrote:

> Hi Carnë,
>
> On Fri, 14 Mar 2014, Carnë Draug wrote:
>
>> 1) create a simple macro
>>
>> $ echo 'print ("Hello world!");' > ./macro_test
>>
>> 2) get an error from ImageJ about file not existing
>>
>> $ ImageJ-linux32 -batch macro_test
>>
>> Macro or script not found:
>>
>> $path for imagej$macros/macro_test.txt
>
> While you might argue that this is a bug, or that it is a bug that ImageJ
> looks into the current directory first before looking into the macros/
> directory when the file name contains no slash, by now it is an
> established API (application programmers' interface).
>
> Therefore I doubt that this bug -- if Wayne agrees that it is a bug -- can
> be fixed without seriously breaking existing workflows. And backwards
> compatibility is what Wayne states as one of his primary goals with ImageJ
> when you get the chance to listen to one of his presentations.

The bug is not limited to search into the macros folder first although
it would be nice if that was documented. The documentation states that
the "-batch" option takes path as argument [1] and a 20 years old bug
should not be made into a feature. But, I am sure everyone has seen
that happening for much worse cases, all in the name of backwards
compatibility, so I'm not going to argue with that (but please update
the documentation).

The bigger problem, which if fixed would not break backwards
compatibility, is that after looking into the macros directory with a
file extension that was never defined, it should at least check if the
given argument was the filepath all along. You will notice it still
looks into the current directory but with a file extension:

## this fails
$ echo 'print ("Hello world!");' > ./macro_test
$ ImageJ-linux32 -batch macro_test

## this works
$ echo 'print ("Hello world!");' > ./macro_test.txt
$ ImageJ-linux32 -batch macro_test

Experimenting with file extensions, this seems to be the behaviour:
1. looks into the macros directory with txt extension
2. looks into the macros directory with ijm extension
3. looks into the current directory with txt extension
4. fails because it doesn't find a file in current directory with txt

So there is a consistency problem as well because it does not try the
ijm file extension in the current directory. In addition, it doesn't
look for a file without file extension in the macros directory which
would be nice since after all, I specifically didn't specify a file
extension.

If before throwing an error, there is a check for a file named exactly
as it was meant in the input arguments and in the current working
directory, it would make it work as expected without breaking anyone's
code. This may still catch some people by surprise if a macro with the
same name happens to exist in the macros directory.

Carnë

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[dscho]

Hi Carnë,

On Fri, 14 Mar 2014, Carnë Draug wrote:

> [...] a 20 years old bug should not be made into a feature. But, I am
> sure everyone has seen that happening for much worse cases, all in the
> name of backwards compatibility, so I'm not going to argue with that
> [...]

That opinion is a comfortable one to have, unless, of course, when you are
relying on scripts and macros that have been developed over the course of
twenty years.

However, that scenario is quite real, and therefore it is the duty of a
good maintainer to pay attention to backwards compatibility, even if it
offends our sense of elegance from time to time. After all, ImageJ is a
tool to be used, heavily, not an art project.

Having said that, major new releases are exactly what maintainers do to
allow a partial redesign.

> The bigger problem, which if fixed would not break backwards
> compatibility, is that after looking into the macros directory with a
> file extension that was never defined,

Please specify the exact file name, then, and do not let ImageJ guess what
you want.

For that matter, I also would like to recommend prefixing the path
argument with a "./" to be explicit in your wish to load this file from
the current directory.

Further, I would *strongly* recommend using the file extension ".ijm"
instead of ".txt". The latter is a well-established file extension
denoting human-readable text, while ".ijm" is the recommended file
extension for ImageJ macros for some years already. ImageJ 1.x' support to
interpret ".txt" as macros' file extension is a historical wart of the
same sort as criticised above; there is no need to make use of it.

Ciao,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Carnë Draug]

On 14 March 2014 17:59, Johannes Schindelin <[hidden email]> wrote:

> Hi Carnë,
>
> On Fri, 14 Mar 2014, Carnë Draug wrote:
>
>> [...]
>>
>> The bigger problem, which if fixed would not break backwards
>> compatibility, is that after looking into the macros directory with a
>> file extension that was never defined,
>
> Please specify the exact file name, then, and do not let ImageJ guess what
> you want.

I am sorry if I am not being clear but that is exactly what I am
doing. I am specifying the exact file name so that ImageJ does not
have to make any guess. The macro file does *not* have a file
extension hence why I am passing ImageJ a filepath with *no* file
extension. But ImageJ seems to assume that if there is no file
extension it must be because I forgot it. So it looks for a file with
a file extension and then complains that it does not exist. The file
does exist, it simply has no file extension. I specifically said that
the file has no file extension by omitting it from the command. Is
there a syntax to specify no file extension?

I do not want ImageJ to guess, I want ImageJ to not guess. I specified
the exact and correct path. Why is it assuming that the path is
incorrect even before checking for it? I understand that if I'm making
things ambiguous by not specifying extension and leaving that up to
ImageJ, things can go wrong. But the argument I'm passing is correct
and unambiguous.

Why my macro files don't have a file extension? Because my macro files
are temporary and are being generated automatically on the run. The
creation and removal of the temporary file is being handled by other
libraries. What I am doing now, as a workaround this bug, is using "
'.' + filesep() + tempname" as filename to the system call.

Anyway, a solution to this while maintaining backwards compatibility:
after looking for files with the extension txt and ijm, look for the
actual file as defined in the input arguments.

> Further, I would *strongly* recommend using the file extension ".ijm"
> instead of ".txt". The latter is a well-established file extension
> denoting human-readable text, while ".ijm" is the recommended file
> extension for ImageJ macros for some years already. ImageJ 1.x' support to
> interpret ".txt" as macros' file extension is a historical wart of the
> same sort as criticised above; there is no need to make use of it.

Again, I apologise if I am being unclear. When I do write macros
myself, I use the ijm extension. What I pointed out was that the
current behaviour goes against what you just strongly recommended. The
current behaviour looks in the macro folder for files with txt and ijm
extension. If that fails, it looks into the current directory for a
file with the txt extension *only*. A file in the current directory
with the ijm extension (the one you strongly recommend) will be
ignored by ImageJ.

Carnë

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[dscho]

Hi Carnë,

On Fri, 14 Mar 2014, Carnë Draug wrote:

> On 14 March 2014 17:59, Johannes Schindelin <[hidden email]> wrote:
> > On Fri, 14 Mar 2014, Carnë Draug wrote:
> >
> >> The bigger problem, which if fixed would not break backwards
> >> compatibility, is that after looking into the macros directory with a
> >> file extension that was never defined,
> >
> > Please specify the exact file name, then, and do not let ImageJ guess
> > what you want.
>
> I am sorry if I am not being clear but that is exactly what I am doing.

Sorry, I did not mean "exact", I meant "correct". If it is a macro, use
the ".ijm" extension.

I apologize for my mistake,
Johannes

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Kenneth Sloan-2]

I’v got to agree with the OP here - if you have a file name, and a file with that EXACT filename exists, then that’s what you should get.

Changing the file lookup to do this should not break old stuff.  

If you want to be stiff-necked and declare that only “.ijm” is correct, then also ban “.txt”.

If would be (marginally) acceptable (to me) if the EXACT filename were delivered ONLY if the same filename with either “.ijm” or “.txt” did NOT exist.  I think this is sub-optimal, but OK.

But, complaining that the file does not exist is clearly (to me) broken.

--
Kenneth Sloan
[hidden email]


On Mar 14, 2014, at 14:44 , Johannes Schindelin <[hidden email]> wrote:

> Hi Carnë,
>
> On Fri, 14 Mar 2014, Carnë Draug wrote:
>
>> On 14 March 2014 17:59, Johannes Schindelin <[hidden email]> wrote:
>>> On Fri, 14 Mar 2014, Carnë Draug wrote:
>>>
>>>> The bigger problem, which if fixed would not break backwards
>>>> compatibility, is that after looking into the macros directory with a
>>>> file extension that was never defined,
>>>
>>> Please specify the exact file name, then, and do not let ImageJ guess
>>> what you want.
>>
>> I am sorry if I am not being clear but that is exactly what I am doing.
>
> Sorry, I did not mean "exact", I meant "correct". If it is a macro, use
> the ".ijm" extension.
>
> I apologize for my mistake,
> Johannes
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Carnë Draug]

On 15 March 2014 18:56, Rasband, Wayne (NIH/NIMH) [E]
<[hidden email]> wrote:

> Dear Carnë,
>
> On Mar 14, 2014, at 9:28 AM, Carnë Draug wrote:
>
>> Hi
>>
>> The title says it all. It would appear that ImageJ automatically adds
>> a txt extension when there is none, even if the file without file
>> extension exists. How to reproduce
>>
>> 1) create a simple macro
>>
>> $ echo 'print ("Hello world!");' > ./macro_test
>>
>> 2) get an error from ImageJ about file not existing
>>
>> $ ImageJ-linux32 -batch macro_test
>>
>> Macro or script not found:
>>
>> $path for imagej$macros/macro_test.txt
>>
>> Note that this works fine if the file exists in the current directory
>> with a txt file extension. A temporary workaround for this is to use
>> "-batch ./macro_test" but I don't think that ImageJ should be trying
>> to make any guess at all. Anyone calling through the command line
>> should know how to specify the path for the macro file. And even if it
>> should be guessing, then it should only do it after checking if the
>> file does exist in the current directory.
>
> This bug should be fixed in the latest daily build (1.48t14).
>
> Best regards,
>
> -wayne

Hi

I confirm this is also fixed using 1.48t14 via FIJI.

For future reference, the new behaviour when the file has no file extension is:

1) macros directory with the txt extension
2) macros directory with ijm extension
3) macros directory without extension
4) current directory without file extension
5) current directory with txt extension
6) current directory with ijm extension

Thank you,
Carnë

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html