exec() bug in handling embedded blanks

[Neil Fazel]

I'm using exec() to run a native command from the ImageJ macro code. This code (note the single quotes inside the double quotes)

   cmd="ls '"+directory+"/'*avg.txt";
   print(cmd);
   templist = exec(cmd);

doesn't seem to work (templist is blank) whereas if I copy the value of cmd from the log window to the Unix prompt, it works. I think it's because the variable 'directory' has embedded blanks in it and exec() can't handle blanks in path names.

I'm working on Mac and it is not uncommon for directory paths to have blanks in them. Is there a way around this?

BTW, the reason I'm using exec() is because getFileList() doesn't work with wildcards, so exec() seems to be the only way I can get a list of the files matching a certain pattern.

Thanks,
Neil

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Neil,

you can use quotes in the unix (linux, etc.) command line because the unix shell handles them: An item in quotes is taken as a single argument for the program's arg list (in the current case, the program is "ls").

ImageJ does not remove the quotes, so the 'ls' command will look for something like
  '/home/user1/'*avg.txt
including the quotes, thus directory name ' with respect to the current directory, then subdirectory home, subdirectory user, filename '*avg.txt
Obviously, that's not what you want.

You can pass the call to the shell:
  cmd="sh -c ls '"+directory+"/'*avg.txt";

Then the shell will care about removing the quotes and creating the proper list of arguments for the ls program.

---

The nicer way of doing it would be using getFileList() and filtering the result (this is independent of the operating system):

   list = getFileList(dir);
   for (i=0; i<list.length; i++) {
      if (endsWith(list[i], "avg.txt")) {
          //handle the file
      }
   }


Michael
________________________________________________________________
On Jul 22, 2014, at 04:59, Neil Fazel wrote:

> I'm using exec() to run a native command from the ImageJ macro code. This code (note the single quotes inside the double quotes)
>
>   cmd="ls '"+directory+"/'*avg.txt";
>   print(cmd);
>   templist = exec(cmd);
>
> doesn't seem to work (templist is blank) whereas if I copy the value of cmd from the log window to the Unix prompt, it works. I think it's because the variable 'directory' has embedded blanks in it and exec() can't handle blanks in path names.
>
> I'm working on Mac and it is not uncommon for directory paths to have blanks in them. Is there a way around this?
>
> BTW, the reason I'm using exec() is because getFileList() doesn't work with wildcards, so exec() seems to be the only way I can get a list of the files matching a certain pattern.
>
> Thanks,
> Neil

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Neil Fazel]

Hi Michael,

    Thanks for the useful tip; I switched to using "sh -c" and was able to get it to work. A couple of remarks:

1) "sh -c" seems to require that the argument passed to it be itself a string, i.e. quoted. This would require 3 levels of quoting. I tried this (note the escaping of the innermost double quotes):

  cmd="sh -c \"ls '" + directory + "/'*avg.txt\"";
  exec(cmd);

and it didn't work (even though copying & pasting the value of "cmd" to the Unix prompt did work). What finally worked inside the macro was this syntax:

exec("sh", "-c", "ls '" + directory + "/'*.txt"),

2) If the "sh -c" syntax causes the Unix shell to take control, what happens without it? In order to run a native command, doesn't ImageJ have to pass on exec()'s argument to the Unix shell, even in the absence of "sh -c"?


Thanks for the code snippet.

Best regards,
Neil

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Michael Schmid]

Hi Neil,

oops, I forgot two points:

(1) The wildcards ("*") are expanded by the shell, not by 'ls'.
So, with 'ls *avg.txt' the 'ls' command will actually receive a list of filepaths (directory/filename) from the shell. The 'ls' program would otherwise take the asterisk literal, reporting only files with an asterisk exactly as you write it in the filename.

(2) Yes, sorry, I agree, I should have used the multi-argument version of 'exec'.  With this version of exec, the called program (in this case, the shell) gets its arguments as they should be. The argument following the "-c" should be exactly what you would type into the unix shell.

When not using the multi-argument version of 'exec' but simple string concatenation into one exec argument, Java will split the string at all blanks:
  exec('sh -c ls "/home/user/any file"');
will split the arguments of the shell as
  -c               <this is fine, it tells to use the next argument as command
  ls               <this will be taken as argument of '-c', so this will be executed
  "/home/user/any  <the shell won't understand this and ignore it
  file"            <the shell won't understand this and ignore it

The quotes won't help, because Java does not interpret them when splitting the string using a StringTokenizer
  See
http://docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html#exec%28java.lang.String,%20java.lang.String[],%20java.io.File%29

Maybe this helps a bit...

Michael
________________________________________________________________
On Jul 22, 2014, at 17:53, Neil Fazel wrote:

> Hi Michael,
>
>    Thanks for the useful tip; I switched to using "sh -c" and was able to get it to work. A couple of remarks:
>
> 1) "sh -c" seems to require that the argument passed to it be itself a string, i.e. quoted. This would require 3 levels of quoting. I tried this (note the escaping of the innermost double quotes):
>
>  cmd="sh -c \"ls '" + directory + "/'*avg.txt\"";
>  exec(cmd);
>
> and it didn't work (even though copying & pasting the value of "cmd" to the Unix prompt did work). What finally worked inside the macro was this syntax:
>
> exec("sh", "-c", "ls '" + directory + "/'*.txt"),
>
> 2) If the "sh -c" syntax causes the Unix shell to take control, what happens without it? In order to run a native command, doesn't ImageJ have to pass on exec()'s argument to the Unix shell, even in the absence of "sh -c"?
>
>
> Thanks for the code snippet.
>
> Best regards,
> Neil

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html