Adjust image filters with TraKEM2

[L'assegnista]

Dear List,
I'm using TrakEM2 to reconstruct bundles of nerve fibers from four channels
confocal images ( a huge serial section reconstruction). As suggested by
Stephan Saalfeld (some yeras ago!) I imported the channels as separate
8-bit patches, applyed a LUT with "adjust image filters" and set the
composite mode to Add. However if I close and re-open the project the color
of the LUTs turns to gray and I have to set everithing again and wait the
(long) regeneration of the minimaps. Moreover, since the RAM is apparently
not released after the process (running the garbage collector is
ineffective) scrolling through the sections for annotation become very
slow. Is this a bug? Are there alterantive ways to change the LUT of the
channels?

Thank you

Federico

--
Federico Luzzati, PhD
Assistant Professor
University of Turin,
Dept. LIfe Sciences and Systems Biology (DBIOS)
Via Accademia Albertina, 13 - 10123 Torino - IT

Researcher at
Neuroscience Institute Cavalieri Ottolenghi (NICO)
Regione Gonzole, 10 - 10043 Orbassano (To) - IT

tel. [phone]/ -6631
http://www.nico.ottolenghi.unito.it/index.php/it/adult-neurogenesis

Sostieni con il tuo 5xmille il NICO e la ricerca sulle malattie
neurodegenerative
Firma nel riquadro "Finanziamento della ricerca scientifica e
dell'Università"
CF 97564560015

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Saalfeld, Stephan]

I confirm it's a bug.  I will see what I can do over the next days but
may not have enough time to get into this.  If you love the risky life
(you use Fiji and TrakEM2, so you do!), you can get it to work right
now by editing the project XML file.

In the XML, you will find a property

type="0"

for each 8-bit

<t2_patch ...>

entry (the tiles).  If you replace this by

type="3"

TrakEM2 seems to be sufficiently convinced that these are color images
to render them with the LUT.  In vim, the correct global replace
command is

:%s/type="0"/type="3"/g

I have tested this only on a small set of images and it may have
unintended side effects, so you should probably do this in a copy of
your XML file.

Let me know.

Best,
Stephan
 

On Mon, 2016-02-15 at 15:29 +0100, Federico Luzzati wrote:

> Dear List,
> I'm using TrakEM2 to reconstruct bundles of nerve fibers from four
> channels
> confocal images ( a huge serial section reconstruction). As suggested
> by
> Stephan Saalfeld (some yeras ago!) I imported the channels as
> separate
> 8-bit patches, applyed a LUT with "adjust image filters" and set the
> composite mode to Add. However if I close and re-open the project the
> color
> of the LUTs turns to gray and I have to set everithing again and wait
> the
> (long) regeneration of the minimaps. Moreover, since the RAM is
> apparently
> not released after the process (running the garbage collector is
> ineffective) scrolling through the sections for annotation become
> very
> slow. Is this a bug? Are there alterantive ways to change the LUT of
> the
> channels?
>
> Thank you
>
> Federico
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Albert Cardona-2]

Dear Federico,

About the memory build up: in some systems, TrakEM2 mipmaps accumulate in
the internal cache beyond what would be optimal. To release them, run the
following from e.g. a jython interpreter:

from ini.trakem2 import Project
Project.getProjects()[0].getLoader().releaseAll()


The above can run as well from a ScheduledService every minute if
necessary. For example, from the interpreter run this:

###
from java.util.concurrent import Executors, TimeUnit

def releaseAll():
  Project.getProjects()[0].getLoader().releaseAll()

exe = Executors.newSingleThreadScheduledExecutor()
exe.scheduleAtFixedRate(releaseAll, 0, 60, TimeUnit.SECONDS)
###

To stop it, type:

exe.shutdown()


Hope this helps.

Albert Cardona



2016-02-15 9:29 GMT-05:00 Federico Luzzati <[hidden email]>:

> Dear List,
> I'm using TrakEM2 to reconstruct bundles of nerve fibers from four channels
> confocal images ( a huge serial section reconstruction). As suggested by
> Stephan Saalfeld (some yeras ago!) I imported the channels as separate
> 8-bit patches, applyed a LUT with "adjust image filters" and set the
> composite mode to Add. However if I close and re-open the project the color
> of the LUTs turns to gray and I have to set everithing again and wait the
> (long) regeneration of the minimaps. Moreover, since the RAM is apparently
> not released after the process (running the garbage collector is
> ineffective) scrolling through the sections for annotation become very
> slow. Is this a bug? Are there alterantive ways to change the LUT of the
> channels?
>
> Thank you
>
> Federico
>
> --
> Federico Luzzati, PhD
> Assistant Professor
> University of Turin,
> Dept. LIfe Sciences and Systems Biology (DBIOS)
> Via Accademia Albertina, 13 - 10123 Torino - IT
>
> Researcher at
> Neuroscience Institute Cavalieri Ottolenghi (NICO)
> Regione Gonzole, 10 - 10043 Orbassano (To) - IT
>
> tel. [phone]/ -6631
> http://www.nico.ottolenghi.unito.it/index.php/it/adult-neurogenesis
>
> Sostieni con il tuo 5xmille il NICO e la ricerca sulle malattie
> neurodegenerative
> Firma nel riquadro "Finanziamento della ricerca scientifica e
> dell'Università"
> CF 97564560015
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
http://albert.rierol.net
http://www.janelia.org/lab/cardona-lab
http://www.ini.uzh.ch/~acardona/

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[L'assegnista]

Thank you Stephan and Albert,
the type="3" trick magically solved the problem! The RAM still tend to
accumulate in a strange way (altough I never run out of memory), but this
seem rather a general problem of fiji, not specifically of TraKEM2, indeed
the script suggested by Albert have no effect in my system.
Thank you very much!

Federico

2016-02-15 18:23 GMT+01:00 Saalfeld, Stephan <[hidden email]>:

> I confirm it's a bug.  I will see what I can do over the next days but
> may not have enough time to get into this.  If you love the risky life
> (you use Fiji and TrakEM2, so you do!), you can get it to work right
> now by editing the project XML file.
>
> In the XML, you will find a property
>
> type="0"
>
> for each 8-bit
>
> <t2_patch ...>
>
> entry (the tiles).  If you replace this by
>
> type="3"
>
> TrakEM2 seems to be sufficiently convinced that these are color images
> to render them with the LUT.  In vim, the correct global replace
> command is
>
> :%s/type="0"/type="3"/g
>
> I have tested this only on a small set of images and it may have
> unintended side effects, so you should probably do this in a copy of
> your XML file.
>
> Let me know.
>
> Best,
> Stephan
>
>
> On Mon, 2016-02-15 at 15:29 +0100, Federico Luzzati wrote:
> > Dear List,
> > I'm using TrakEM2 to reconstruct bundles of nerve fibers from four
> > channels
> > confocal images ( a huge serial section reconstruction). As suggested
> > by
> > Stephan Saalfeld (some yeras ago!) I imported the channels as
> > separate
> > 8-bit patches, applyed a LUT with "adjust image filters" and set the
> > composite mode to Add. However if I close and re-open the project the
> > color
> > of the LUTs turns to gray and I have to set everithing again and wait
> > the
> > (long) regeneration of the minimaps. Moreover, since the RAM is
> > apparently
> > not released after the process (running the garbage collector is
> > ineffective) scrolling through the sections for annotation become
> > very
> > slow. Is this a bug? Are there alterantive ways to change the LUT of
> > the
> > channels?
> >
> > Thank you
> >
> > Federico
> >
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
Federico Luzzati, PhD
Assistant Professor
University of Turin,
Dept. LIfe Sciences and Systems Biology (DBIOS)
Via Accademia Albertina, 13 - 10123 Torino - IT

Researcher at
Neuroscience Institute Cavalieri Ottolenghi (NICO)
Regione Gonzole, 10 - 10043 Orbassano (To) - IT

tel. [phone]/ -6631
http://www.nico.ottolenghi.unito.it/index.php/it/adult-neurogenesis

Sostieni con il tuo 5xmille il NICO e la ricerca sulle malattie
neurodegenerative
Firma nel riquadro "Finanziamento della ricerca scientifica e
dell'Università"
CF 97564560015

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Saalfeld, Stephan]

Glad to hear.  I assume that you mean RAM usage as reported by some
system administration tool (top, task manager?)?  Seeing this is the
typical behavior of the JVM, i.e. all Java/Scala/Groovy/...
applications.  You can limit the amount of memory used by the JVM using
the -Xmx parameter.  But you will keep seeing JVM memory usage to
approach this limit and then stay there for its lifetime.  If you want
to observe the 'real' memory usage of Fiji/ TrakEM2, use a JVM monitor
such as jvisualvm.  There are anumber of threads covering this in the
mailing list and forum.

Cheers,
Stephan


On Mon, 2016-02-15 at 23:37 +0100, Federico Luzzati wrote:

> Thank you Stephan and Albert,
> the type="3" trick magically solved the problem! The RAM still tend
> to
> accumulate in a strange way (altough I never run out of memory), but
> this
> seem rather a general problem of fiji, not specifically of TraKEM2,
> indeed
> the script suggested by Albert have no effect in my system.
> Thank you very much!
>
> Federico
>
> 2016-02-15 18:23 GMT+01:00 Saalfeld, Stephan <
> [hidden email]>:
>
> > I confirm it's a bug.  I will see what I can do over the next days
> > but
> > may not have enough time to get into this.  If you love the risky
> > life
> > (you use Fiji and TrakEM2, so you do!), you can get it to work
> > right
> > now by editing the project XML file.
> >
> > In the XML, you will find a property
> >
> > type="0"
> >
> > for each 8-bit
> >
> > <t2_patch ...>
> >
> > entry (the tiles).  If you replace this by
> >
> > type="3"
> >
> > TrakEM2 seems to be sufficiently convinced that these are color
> > images
> > to render them with the LUT.  In vim, the correct global replace
> > command is
> >
> > :%s/type="0"/type="3"/g
> >
> > I have tested this only on a small set of images and it may have
> > unintended side effects, so you should probably do this in a copy
> > of
> > your XML file.
> >
> > Let me know.
> >
> > Best,
> > Stephan
> >
> >
> > On Mon, 2016-02-15 at 15:29 +0100, Federico Luzzati wrote:
> > > Dear List,
> > > I'm using TrakEM2 to reconstruct bundles of nerve fibers from
> > > four
> > > channels
> > > confocal images ( a huge serial section reconstruction). As
> > > suggested
> > > by
> > > Stephan Saalfeld (some yeras ago!) I imported the channels as
> > > separate
> > > 8-bit patches, applyed a LUT with "adjust image filters" and set
> > > the
> > > composite mode to Add. However if I close and re-open the project
> > > the
> > > color
> > > of the LUTs turns to gray and I have to set everithing again and
> > > wait
> > > the
> > > (long) regeneration of the minimaps. Moreover, since the RAM is
> > > apparently
> > > not released after the process (running the garbage collector is
> > > ineffective) scrolling through the sections for annotation become
> > > very
> > > slow. Is this a bug? Are there alterantive ways to change the LUT
> > > of
> > > the
> > > channels?
> > >
> > > Thank you
> > >
> > > Federico
> > >
> >
> > --
> > ImageJ mailing list: http://imagej.nih.gov/ij/list.html
> >
>
>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[ctrueden]

Hi,

> Seeing this is the typical behavior of the JVM, i.e. all
> Java/Scala/Groovy/... applications.  You can limit the amount of
> memory used by the JVM using the -Xmx parameter.  But you will keep
> seeing JVM memory usage to approach this limit and then stay there for
> its lifetime.

See also this question on StackOverflow:
* http://stackoverflow.com/a/22937867/1207769

And this section of the ImageJ troubleshooting guide:
* http://imagej.net/Troubleshooting#OutOfMemoryError (right-hand sidebar)

Regards,
Curtis

--
Curtis Rueden
LOCI software architect - http://loci.wisc.edu/software
ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden
Did you know ImageJ has a forum? http://forum.imagej.net/


On Mon, Feb 15, 2016 at 5:08 PM, Saalfeld, Stephan <
[hidden email]> wrote:

> Glad to hear.  I assume that you mean RAM usage as reported by some
> system administration tool (top, task manager?)?  Seeing this is the
> typical behavior of the JVM, i.e. all Java/Scala/Groovy/...
> applications.  You can limit the amount of memory used by the JVM using
> the -Xmx parameter.  But you will keep seeing JVM memory usage to
> approach this limit and then stay there for its lifetime.  If you want
> to observe the 'real' memory usage of Fiji/ TrakEM2, use a JVM monitor
> such as jvisualvm.  There are anumber of threads covering this in the
> mailing list and forum.
>
> Cheers,
> Stephan
>
>
> On Mon, 2016-02-15 at 23:37 +0100, Federico Luzzati wrote:
> > Thank you Stephan and Albert,
> > the type="3" trick magically solved the problem! The RAM still tend
> > to
> > accumulate in a strange way (altough I never run out of memory), but
> > this
> > seem rather a general problem of fiji, not specifically of TraKEM2,
> > indeed
> > the script suggested by Albert have no effect in my system.
> > Thank you very much!
> >
> > Federico
> >
> > 2016-02-15 18:23 GMT+01:00 Saalfeld, Stephan <
> > [hidden email]>:
> >
> > > I confirm it's a bug.  I will see what I can do over the next days
> > > but
> > > may not have enough time to get into this.  If you love the risky
> > > life
> > > (you use Fiji and TrakEM2, so you do!), you can get it to work
> > > right
> > > now by editing the project XML file.
> > >
> > > In the XML, you will find a property
> > >
> > > type="0"
> > >
> > > for each 8-bit
> > >
> > > <t2_patch ...>
> > >
> > > entry (the tiles).  If you replace this by
> > >
> > > type="3"
> > >
> > > TrakEM2 seems to be sufficiently convinced that these are color
> > > images
> > > to render them with the LUT.  In vim, the correct global replace
> > > command is
> > >
> > > :%s/type="0"/type="3"/g
> > >
> > > I have tested this only on a small set of images and it may have
> > > unintended side effects, so you should probably do this in a copy
> > > of
> > > your XML file.
> > >
> > > Let me know.
> > >
> > > Best,
> > > Stephan
> > >
> > >
> > > On Mon, 2016-02-15 at 15:29 +0100, Federico Luzzati wrote:
> > > > Dear List,
> > > > I'm using TrakEM2 to reconstruct bundles of nerve fibers from
> > > > four
> > > > channels
> > > > confocal images ( a huge serial section reconstruction). As
> > > > suggested
> > > > by
> > > > Stephan Saalfeld (some yeras ago!) I imported the channels as
> > > > separate
> > > > 8-bit patches, applyed a LUT with "adjust image filters" and set
> > > > the
> > > > composite mode to Add. However if I close and re-open the project
> > > > the
> > > > color
> > > > of the LUTs turns to gray and I have to set everithing again and
> > > > wait
> > > > the
> > > > (long) regeneration of the minimaps. Moreover, since the RAM is
> > > > apparently
> > > > not released after the process (running the garbage collector is
> > > > ineffective) scrolling through the sections for annotation become
> > > > very
> > > > slow. Is this a bug? Are there alterantive ways to change the LUT
> > > > of
> > > > the
> > > > channels?
> > > >
> > > > Thank you
> > > >
> > > > Federico
> > > >
> > >
> > > --
> > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html
> > >
> >
> >
> >
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html