Hello!
I am trying to use FIJI/ImageJ on a cluster and I need to run headless. I have to process a large number of images. I have a macro that I made for my local computer and I was wondering if anyone could help me adapt it to jython which seems to not need a GUI? The Macro I have is this (fijimacro1.ijm): input = "/test/fiji1in/"; output = "/test/fiji1out/"; setBatchMode(true); list = getFileList(input); for (i = 0; i < list.length; i++) action(input, output, list[i]); setBatchMode(false); function action(input, output, filename) { open(input + filename); run("16-bit"); run("TransformJ Scale", "x-factor=4.0 y-factor=4.0 z-factor=1.0 interpolation=[quintic B-spline]"); saveAs("TIFF", output + filename); close(); } setBatchMode(false); //setTool("angle"); I am able to get this to run from the command line with ‘fiji-linux64 –Dplugins.dir=. –macro fijimacro1.ijm –batch &’. But I am connected through xwindows and the display is being exported. I have been reading forum posts and the main suggestion is to write a jython script that doesn’t interact with a GUI at all. I have very little experience in programming, and from what I found at Albert Cardona’s python and ImageJ tutorial this was what I came up with. My first attempt is this (fiji1_.py): from ij import IJ from ij.io import FileSaver import os # folder to read all images from: folderin = "/test/fiji1in" folderout = "/test/fiji1out" for filename in os.listdir(folderin): if filename.endswith(".tif"): print "Processing", filename imp =IJ.openImage(folderin + "/" + filename) IJ.run(imp, "16-bit") IJ.run(imp, "TransformJ Scale", "x-factor=2.0 y-factor=2.0 z-factor=1.0 interpolation=[quintic B-spline]") filepath = folderout + "/" +filename else: print "Ignoring", filename Run from the command line with: ‘fiji-linux64 --headless --jython ./scripts/fiji1_.py -batch &’ The error message that I get is: “Processing 6_FA3-120217_26_2_6_DAPI.tif Traceback (most recent call last): File "./scripts/fiji1_.py", line 13, in <module> IJ.run(imp, "16-bit") TypeError: run(): 1st arg can't be coerced to String” Any advice or assistance is greatly appreciated!! Matt -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Matt,
Your error message has the key, but it's not obvious since you're a novice programmer. It reads: “Processing 6_FA3-120217_26_2_6_DAPI.tif > Traceback (most recent call last): > File "./scripts/fiji1_.py", line 13, in <module> > IJ.run(imp, "16-bit") > TypeError: run(): 1st arg can't be coerced to String” > It's telling you that the first argument you give the IJ.run method needs to be a string--that is, text, either in the form of a text variable, or text between quotes. However, the first argument you gave it was an image (more strictly, an ImagePlus, which is ImageJ's container for holding images). The IJ.run method can be called in 3 ways (see note below**), but the way you want is described about halfway down Albert's very nice tutorial that you're using, under "Running a command on an image''. He says The IJ namespace has a function, *run*, that accepts an ImagePlus as first > argument, then the name of the command to run, and then the macro-ready > list of arguments that the command requires. > When you recorded the macro, it only recorded "run("16-bit");" This is because that command doesn't take any parameters. However, you need to tell your jython script there aren't any. So when you convert it to a python command, it is IJ.run(imp, "16-bit", "") > Note here I just added an empty set of quotes, so the IJ.run command has the 3 arguments it needs. I think this should avoid your error. **Now for the promised note explaining the ways to call IJ.run and your error message. This is a bit complicated if you're just starting to program, but stick with me. If you look up the documentation (or "API") at http://imagej.nih.gov/ij/developer/api/index.html and find the IJ.run method (in lower-left-hand box, click on "IJ", then scroll down the large frame on the right to find "run" in the "Method Summary" section) you will see the method can be called in 3 different ways. run(ImagePlus imp, java.lang.String command, java.lang.String options) > Runs an ImageJ command using the specified image and options. > run(java.lang.String command) > Runs an ImageJ command. > run(java.lang.String command, java.lang.String options) > Runs an ImageJ command, with options that are passed to the > GenericDialog and OpenDialog classes. > You can give it 1 argument (a string command), 2 arguments (a string command, then a string of that command's options), or 3 arguments (an ImagePlus, a string command, then a string of that command's options). Since you want to specify the image, you'll have to use the 3-argument version, which requires the options string. Keep at it. You're on the right track! *Jim Passmore* Research Associate Sealed Air Corporation -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
In reply to this post by Matt Thornton
With some help from Albert (thank you) and your advice Jim(!) I was able to write a proper script for processing my images (fiji1_.py):
from ij import IJ from ij.io import FileSaver import os # folder to read all images from: folderin = "/test/fiji1in" folderout = "/test/fiji1out" for filename in os.listdir(folderin): if filename.endswith(".tif"): print "Processing", filename imp =IJ.openImage(os.path.join(folderin, filename)) IJ.run(imp, "16-bit", "") imp.setProcessor(imp.title, imp.getProcessor() .convertToShort (True)) IJ.run(imp, "TransformJ Scale", "x-factor=2.0 y-factor=2.0 z-factor=1.0 interpolation=[quintic B-spline]") IJ.save(imp, os.path.join(folderout, filename)) else: print "Ignoring", filename ### Now the problem seems to be a failure of TransformJ Scale with --Headless. So, if I run the program form command line with 'fiji-linux64 -macro fijimacro1.ijm -batch, using xwindows and exporting the display, the FIJI symbol is sent to the display and the image is processed. If I run the script 'fiji-linux64 --headless --jython fiji1_.py -batch' an image is saved in folderout, but the scaling is not applied and the following error is given: Processing 6_FA3-120217_26_2_6_DAPI.tif java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at ij.Command.runPlugIn(Command.java:146) at ij.Command.runCommand(Command.java:95) at ij.Executer.run(Executer.java:64) at ij.IJ.run(IJ.java:251) at ij.IJ.run(IJ.java:300) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:175) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190) at org.python.core.PyObject.__call__(PyObject.java:432) at org.python.core.PyObject.__call__(PyObject.java:436) at org.python.pycode._pyx0.f$0(../Fiji.app/scripts/fiji1_.py:9) at org.python.pycode._pyx0.call_function(../Fiji.app/scripts/fiji1_.py) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyCode.call(PyCode.java:18) at org.python.core.Py.runCode(Py.java:1204) at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:200) at org.python.util.jython.run(jython.java:246) at org.python.util.jython.main(jython.java:129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at imagej.ClassLauncher.launch(ClassLauncher.java:220) at imagej.ClassLauncher.run(ClassLauncher.java:158) at imagej.ClassLauncher.main(ClassLauncher.java:71) Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: showHelp signature: ()V) Incompatible argument to function at TJ_Scale.run(TJ_Scale.java:39) at ij.IJ.runUserPlugIn(IJ.java:185) at ij.IJ.runPlugIn(IJ.java:152) ... 32 more Does anyone have any suggestions? -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Hi Matt,
> Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: > showHelp signature: ()V) Incompatible argument to function You may have run up against a limitation and/or bug in Fiji's headless support. When running in headless mode, Fiji tries to hack the GenericDialog class so that it does not extend an AWT dialog. However, in some cases these VerifyErrors occur as a consequence. I am guessing your script works if you comment out the "TransformJ Scale" command? The easiest workaround is to install a virtual framebuffer such as xvfb on your server. This works well; the only thing to watch out for is to avoid macros/scripts/plugins that wait for user input. For details, see this page: http://fiji.sc/wiki/index.php/Headless Unfortunately, there are certain operations in ImageJ 1.x that simply cannot run headless. One of the design goals of ImageJ2 is to reduce the dependency on AWT such that all processing can be run headless. Regards, Curtis On Tue, Jun 5, 2012 at 1:30 PM, Matt Thornton <[hidden email]> wrote: > With some help from Albert (thank you) and your advice Jim(!) I was able > to write a proper script for processing my images (fiji1_.py): > > from ij import IJ > from ij.io import FileSaver > import os > > # folder to read all images from: > folderin = "/test/fiji1in" > folderout = "/test/fiji1out" > > for filename in os.listdir(folderin): > if filename.endswith(".tif"): > print "Processing", filename > imp =IJ.openImage(os.path.join(folderin, filename)) > IJ.run(imp, "16-bit", "") > imp.setProcessor(imp.title, imp.getProcessor() > .convertToShort (True)) > IJ.run(imp, "TransformJ Scale", "x-factor=2.0 y-factor=2.0 > z-factor=1.0 interpolation=[quintic B-spline]") > IJ.save(imp, os.path.join(folderout, filename)) > else: > print "Ignoring", filename > > ### > > Now the problem seems to be a failure of TransformJ Scale with --Headless. > So, if I run the program form command line with 'fiji-linux64 -macro > fijimacro1.ijm -batch, using xwindows and exporting the display, the FIJI > symbol is sent to the display and the image is processed. If I run the > script 'fiji-linux64 --headless --jython fiji1_.py -batch' an image is > saved in folderout, but the scaling is not applied and the following error > is given: > > Processing 6_FA3-120217_26_2_6_DAPI.tif > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at ij.Command.runPlugIn(Command.java:146) > at ij.Command.runCommand(Command.java:95) > at ij.Executer.run(Executer.java:64) > at ij.IJ.run(IJ.java:251) > at ij.IJ.run(IJ.java:300) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:175) > at > org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190) > at org.python.core.PyObject.__call__(PyObject.java:432) > at org.python.core.PyObject.__call__(PyObject.java:436) > at org.python.pycode._pyx0.f$0(../Fiji.app/scripts/fiji1_.py:9) > at > org.python.pycode._pyx0.call_function(../Fiji.app/scripts/fiji1_.py) > at org.python.core.PyTableCode.call(PyTableCode.java:165) > at org.python.core.PyCode.call(PyCode.java:18) > at org.python.core.Py.runCode(Py.java:1204) > at > org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:200) > at org.python.util.jython.run(jython.java:246) > at org.python.util.jython.main(jython.java:129) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at imagej.ClassLauncher.launch(ClassLauncher.java:220) > at imagej.ClassLauncher.run(ClassLauncher.java:158) > at imagej.ClassLauncher.main(ClassLauncher.java:71) > Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: > showHelp signature: ()V) Incompatible argument to function > at TJ_Scale.run(TJ_Scale.java:39) > at ij.IJ.runUserPlugIn(IJ.java:185) > at ij.IJ.runPlugIn(IJ.java:152) > ... 32 more > > Does anyone have any suggestions? > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
In reply to this post by Matt Thornton
Hi Matt,
On Tue, 5 Jun 2012, Matt Thornton wrote: > Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: showHelp signature: ()V) Incompatible argument to function > at TJ_Scale.run(TJ_Scale.java:39) > at ij.IJ.runUserPlugIn(IJ.java:185) > at ij.IJ.runPlugIn(IJ.java:152) > ... 32 more I think I managed to fix this: https://github.com/fiji/fiji/commit/93f65e5ca76c2462dbbb1a43b7a93679ed582a47 Please update your Fiji (in particular, jars/fiji-compat.jar) and try again. Thanks, Johannes -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
In reply to this post by ctrueden
Hi again Matt,
> Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: > showHelp signature: ()V) Incompatible argument to function Johannes just fixed a suspiciously similar problem to yours: https://groups.google.com/d/msg/fiji-devel/wEPgHOroopk/fQgvSFMnPHsJ So it may be that after updating Fiji, your particular script will work headless after all! Regards, Curtis On Tue, Jun 5, 2012 at 2:42 PM, Curtis Rueden <[hidden email]> wrote: > Hi Matt, > > >> Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: >> showHelp signature: ()V) Incompatible argument to function > > > You may have run up against a limitation and/or bug in Fiji's headless > support. > > When running in headless mode, Fiji tries to hack the GenericDialog class > so that it does not extend an AWT dialog. However, in some cases these > VerifyErrors occur as a consequence. I am guessing your script works if you > comment out the "TransformJ Scale" command? > > The easiest workaround is to install a virtual framebuffer such as xvfb > on your server. This works well; the only thing to watch out for is to > avoid macros/scripts/plugins that wait for user input. > > For details, see this page: > http://fiji.sc/wiki/index.php/Headless > > Unfortunately, there are certain operations in ImageJ 1.x that simply > cannot run headless. One of the design goals of ImageJ2 is to reduce the > dependency on AWT such that all processing can be run headless. > > Regards, > Curtis > > > On Tue, Jun 5, 2012 at 1:30 PM, Matt Thornton <[hidden email]>wrote: > >> With some help from Albert (thank you) and your advice Jim(!) I was able >> to write a proper script for processing my images (fiji1_.py): >> >> from ij import IJ >> from ij.io import FileSaver >> import os >> >> # folder to read all images from: >> folderin = "/test/fiji1in" >> folderout = "/test/fiji1out" >> >> for filename in os.listdir(folderin): >> if filename.endswith(".tif"): >> print "Processing", filename >> imp =IJ.openImage(os.path.join(folderin, filename)) >> IJ.run(imp, "16-bit", "") >> imp.setProcessor(imp.title, imp.getProcessor() >> .convertToShort (True)) >> IJ.run(imp, "TransformJ Scale", "x-factor=2.0 >> y-factor=2.0 z-factor=1.0 interpolation=[quintic B-spline]") >> IJ.save(imp, os.path.join(folderout, filename)) >> else: >> print "Ignoring", filename >> >> ### >> >> Now the problem seems to be a failure of TransformJ Scale with >> --Headless. So, if I run the program form command line with 'fiji-linux64 >> -macro fijimacro1.ijm -batch, using xwindows and exporting the display, the >> FIJI symbol is sent to the display and the image is processed. If I run >> the script 'fiji-linux64 --headless --jython fiji1_.py -batch' an image is >> saved in folderout, but the scaling is not applied and the following error >> is given: >> >> Processing 6_FA3-120217_26_2_6_DAPI.tif >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at ij.Command.runPlugIn(Command.java:146) >> at ij.Command.runCommand(Command.java:95) >> at ij.Executer.run(Executer.java:64) >> at ij.IJ.run(IJ.java:251) >> at ij.IJ.run(IJ.java:300) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:175) >> at >> org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190) >> at org.python.core.PyObject.__call__(PyObject.java:432) >> at org.python.core.PyObject.__call__(PyObject.java:436) >> at org.python.pycode._pyx0.f$0(../Fiji.app/scripts/fiji1_.py:9) >> at >> org.python.pycode._pyx0.call_function(../Fiji.app/scripts/fiji1_.py) >> at org.python.core.PyTableCode.call(PyTableCode.java:165) >> at org.python.core.PyCode.call(PyCode.java:18) >> at org.python.core.Py.runCode(Py.java:1204) >> at >> org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:200) >> at org.python.util.jython.run(jython.java:246) >> at org.python.util.jython.main(jython.java:129) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at imagej.ClassLauncher.launch(ClassLauncher.java:220) >> at imagej.ClassLauncher.run(ClassLauncher.java:158) >> at imagej.ClassLauncher.main(ClassLauncher.java:71) >> Caused by: java.lang.VerifyError: (class: ij/gui/GenericDialog, method: >> showHelp signature: ()V) Incompatible argument to function >> at TJ_Scale.run(TJ_Scale.java:39) >> at ij.IJ.runUserPlugIn(IJ.java:185) >> at ij.IJ.runPlugIn(IJ.java:152) >> ... 32 more >> >> Does anyone have any suggestions? >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
In reply to this post by Matt Thornton
Hi Johannes!
I updated Headless.java, but when I try to run the script I get this error. [methornt@hpc0970 ~]$ $FIJI -Dplugins.dir=${FDIR} --headless --jython ${FDIR}/scripts/fiji1_.py Segmentation fault [methornt@hpc0970 ~]$ $FIJI -Dplugins.dir=${FDIR} -macro ${FDIR}/macros/fijimacro1.ijm -batch ...runs and processes the data. We just downloaded and installed FIJI on 06/04/2012 ImageJ 1.46J using Java 1.6.0_24 Here is the script again fiji1_.py from ij import IJ from ij.io import FileSaver import os # folder to read all images from: folderin = "/home/rcf-proj3/met1/test/fiji1in" folderout = "/home/rcf-proj3/met1/test/fiji1out" for filename in os.listdir(folderin): if filename.endswith(".tif"): print "Processing", filename imp =IJ.openImage(os.path.join(folderin, filename)) IJ.run(imp, "16-bit", "") imp.setProcessor(imp.title, imp.getProcessor() .convertToShort (True)) IJ.run(imp, "TransformJ Scale", "x-factor=4.0 y-factor=4.0 z-factor=1.0 interpolation=[quintic B-spline]") IJ.save(imp, os.path.join(folderout, filename)) else: print "Ignoring", filename ### Thank you for all of your help so far! Matt -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Hi Matt,
On Wed, 6 Jun 2012, Matt Thornton wrote: > I updated Headless.java, but when I try to run the script I get this error. > > [methornt@hpc0970 ~]$ $FIJI -Dplugins.dir=${FDIR} --headless --jython ${FDIR}/scripts/fiji1_.py > Segmentation fault This is an unfortunate by-product of a recent change in Jython (to be precise, version 2.5.2). Our work-around in the ImageJ launcher made it into the source code repository only recently: http://fiji.sc/cgi-bin/gitweb.cgi?p=imagej2/.git;a=commitdiff;h=a24da9e77483487665ed6dcf94ab894e6a02b1bb;hb=refs/heads/master And of course, I forgot to upload the current ImageJ launcher until I read your mail. So please update, that should fix the problem. Ciao, Johannes -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
In reply to this post by Matt Thornton
Hi Johannes,
I am now using FIJI/ImageJ 1.46p. I updated FIJI but when I restarted the program, I am getting the error: /lib64/libc.so.6: version `GLIBC_2.14` not found (required by /software/FIJI/ImageJ-linux64). Is there an easy patch or upgrade to fix this error? Thank you very much for your help! -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Hi Matt,
On Thu, 7 Jun 2012, Matt Thornton wrote: > I am now using FIJI/ImageJ 1.46p. I updated FIJI but when I restarted > the program, I am getting the error: > > /lib64/libc.so.6: version `GLIBC_2.14` not found (required by /software/FIJI/ImageJ-linux64). Very sorry. This happened because we upgraded the server performing the automated builds for the ImageJ launcher. And that upgrade included an upgrade to the GNU libc which now contains an incompatibly-versioned memcpy function. Now, we have a mechanism in place with which we can force backwards-compatibility despite GNU libc versioning rather vital functions (and without tricks, the resulting executables will show exactly the error you experienced when running with an older GNU libc). However, so far this process is purely manual: I have to compile the launcher, run the script to detect incompatibly-versioned functions and re-compile. And since I did not have to do that in a long time, I simply forgot. In the near future, I will automate this process so that the server building the launcher for the platforms we support can run that script and fix things before the launcher gets deployed. > Is there an easy patch or upgrade to fix this error? Well, since you cannot start Fiji, you cannot run the updater ;-) Alas, salvation is nigh! The same server which is building the launcher also keeps a Fiji directory up-to-date. You can download the launcher here: http://jenkins.imagej.net/job/Stable-Fiji/ws/Fiji.app/ImageJ-linux64 Please move this into the correct place (and fix the permissions so it is executable; on the command-line it is the call "chmod a+x /path/to/ImageJ-linux64"). Sorry for the breakage, Johannes -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Free forum by Nabble | Edit this page |