Open a folder of Stacks as a big Virtual Stack ?

[lechristophe]

HI,

I'm trying to use the FolderOpener class in a javascript to make a big
virtual stack form several stacks in a folder.
The FolderOpener (and the File > Import > Image Sequence command)
allows to open a sequence of stacks in a folder as a non virtual
stack. If the folders contains three 100-slices stacks, the result is
a 300-slice non virtual stack.

However, if I set the virtual stack option to true (in the Import
Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
object), what I get is only a 3-slice virtual stack containing the
first image of each 100-slice stacks.

Is that intended? If so, how can I get a virtual stack concatenating
all stacks in a folder in javascript?

Thanks for your help,

Christophe

[Albert Cardona-2]

El 16 de març de 2012 7:15, Christophe Leterrier
<[hidden email]> ha escrit:

> HI,
>
> I'm trying to use the FolderOpener class in a javascript to make a big
> virtual stack form several stacks in a folder.
> The FolderOpener (and the File > Import > Image Sequence command)
> allows to open a sequence of stacks in a folder as a non virtual
> stack. If the folders contains three 100-slices stacks, the result is
> a 300-slice non virtual stack.
>
> However, if I set the virtual stack option to true (in the Import
> Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
> object), what I get is only a 3-slice virtual stack containing the
> first image of each 100-slice stacks.
>
> Is that intended? If so, how can I get a virtual stack concatenating
> all stacks in a folder in javascript?

There is related code in python here:

http://fiji.sc/wiki/index.php/Jython_Scripting#Visualize_any_number_of_TIFF_stacks_in_a_single_composite_multi-color_image_stack

In the above case, the stacks are used as independent channels in a
multicolor composite.

Albert

--
http://albert.rierol.net
http://www.ini.uzh.ch/~acardona/

[Michael Schmid]

Hi Christophe,

(after some off-list conversation because I did not understand the problem):

It won't work if your stacks are not made from single files, but you have just 3 tiffs (or zipped tiffs), each containing a stack.
The Import>Image Sequence command expects one image per file, not one stack per file.

It would work if you split your stacks into separate files, say A000.tif-A099.tif for stack A, B000.tif-B099.tif for stack B, etc., all in the same directory.  File>Save As>Image Sequence.

The reason why it won't work:
When opening a virtual stack, the individual files are not really opened unless the respective stack slice is accessed.
It must be known in advance how many images one has, but for some files being stacks this could be determined only by reading all the files, which would take quite long for a directory with many images.
Without knowing how many slices there are in each file, hone could not go to, say, slice 10 of a virtual stack.  That would be file #10 if all are single files, but file #3 if file #1 has a single image and file 2# happens to be an 8-image stack.

Michael
________________________________________________________________
On Mar 16, 2012, at 12:15, Christophe Leterrier wrote:

> HI,
>
> I'm trying to use the FolderOpener class in a javascript to make a big
> virtual stack form several stacks in a folder.
> The FolderOpener (and the File > Import > Image Sequence command)
> allows to open a sequence of stacks in a folder as a non virtual
> stack. If the folders contains three 100-slices stacks, the result is
> a 300-slice non virtual stack.
>
> However, if I set the virtual stack option to true (in the Import
> Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
> object), what I get is only a 3-slice virtual stack containing the
> first image of each 100-slice stacks.
>
> Is that intended? If so, how can I get a virtual stack concatenating
> all stacks in a folder in javascript?
>
> Thanks for your help,
>
> Christophe

[lechristophe]

Hi Michael,

I understand the problem, but the result is that the "Import Image
Sequence" does not behave consistently when the "use virtual stack"
option is checked or not. If not checked, it effectlively concatenate
the stacks with all their slices, but if checked, it will just
concatenate the first slice of each stack.


I don't know if it should be made consistent. If so, I can see three
possibilities:
- The non-virtual stack case could also take only the first slice of
each stack (although it does not make much sense)
- The Importer could ignore multi-slice Tiffs, or throw an error
- The virtual stack case could handle all multi-slice TIFFs like in
the non-virtual case, i.e. retrieve slice number via TiffInfo (not
sure about the feasability?) and generate a virtual stack that
concatenates all slices from the stacks.

Do you think one of these solutions would be preferable to the current
situation?

On Fri, Mar 16, 2012 at 17:12, Michael Schmid <[hidden email]> wrote:

> Hi Christophe,
>
> (after some off-list conversation because I did not understand the problem):
>
> It won't work if your stacks are not made from single files, but you have just 3 tiffs (or zipped tiffs), each containing a stack.
> The Import>Image Sequence command expects one image per file, not one stack per file.
>
> It would work if you split your stacks into separate files, say A000.tif-A099.tif for stack A, B000.tif-B099.tif for stack B, etc., all in the same directory.  File>Save As>Image Sequence.
>
> The reason why it won't work:
> When opening a virtual stack, the individual files are not really opened unless the respective stack slice is accessed.
> It must be known in advance how many images one has, but for some files being stacks this could be determined only by reading all the files, which would take quite long for a directory with many images.
> Without knowing how many slices there are in each file, hone could not go to, say, slice 10 of a virtual stack.  That would be file #10 if all are single files, but file #3 if file #1 has a single image and file 2# happens to be an 8-image stack.
>
> Michael
> ________________________________________________________________
> On Mar 16, 2012, at 12:15, Christophe Leterrier wrote:
>
>> HI,
>>
>> I'm trying to use the FolderOpener class in a javascript to make a big
>> virtual stack form several stacks in a folder.
>> The FolderOpener (and the File > Import > Image Sequence command)
>> allows to open a sequence of stacks in a folder as a non virtual
>> stack. If the folders contains three 100-slices stacks, the result is
>> a 300-slice non virtual stack.
>>
>> However, if I set the virtual stack option to true (in the Import
>> Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
>> object), what I get is only a 3-slice virtual stack containing the
>> first image of each 100-slice stacks.
>>
>> Is that intended? If so, how can I get a virtual stack concatenating
>> all stacks in a folder in javascript?
>>
>> Thanks for your help,
>>
>> Christophe

[ctrueden]

Hi Christophe,

Did you try the Bio-Formats importer with "Group files with similar names"
and "Use virtual stacks" checked? Or are you files not numbered properly
for that? The BF file pattern logic is quite sophisticated; perhaps a
manually defined pattern could work for you in the latter case.

-Curtis
 On Mar 16, 2012 6:26 AM, "Christophe Leterrier" <
[hidden email]> wrote:

> HI,
>
> I'm trying to use the FolderOpener class in a javascript to make a big
> virtual stack form several stacks in a folder.
> The FolderOpener (and the File > Import > Image Sequence command)
> allows to open a sequence of stacks in a folder as a non virtual
> stack. If the folders contains three 100-slices stacks, the result is
> a 300-slice non virtual stack.
>
> However, if I set the virtual stack option to true (in the Import
> Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
> object), what I get is only a 3-slice virtual stack containing the
> first image of each 100-slice stacks.
>
> Is that intended? If so, how can I get a virtual stack concatenating
> all stacks in a folder in javascript?
>
> Thanks for your help,
>
> Christophe
>

[lechristophe]

Hi Curtis,

Indeed, after spending a couple of hours struggling with the virtual
stacks class, it is now clear that they can only refer to individual
image files on disk, not to slices into a stack file. By the way it
took me a long time to get that the addSlice() parameter (the only way
to add content to a Virtual Stack ?) has to do with the path to the
image file (is there a good documentation / programming exemples /
tips on working with virtual stacks beyond the IJ Javadoc?).

BioFormats seems to be able to override this limit and make a virtual
stack that refers to images that are slices inside stack files. How is
that done? Is it the use of the BFVirtualStacks class that allows to
do it? Is the resulting virtual stack a real one, i.e. can I just use
it as any Virtual Stack for further processing?

In the end, working with BioFormat seems the way to go. Where can I
find BioFormat's API / Javadoc? Can I simply record a call to the BF
Importer with "groupe similar names", "use virtual stack" checked and
use it in a scrip to get a silent opening? How can I specify name
patterns in a script?

Thanks for your help,

Christophe



On Sat, Mar 17, 2012 at 21:12, Curtis Rueden <[hidden email]> wrote:

> Hi Christophe,
>
> Did you try the Bio-Formats importer with "Group files with similar names"
> and "Use virtual stacks" checked? Or are you files not numbered properly
> for that? The BF file pattern logic is quite sophisticated; perhaps a
> manually defined pattern could work for you in the latter case.
>
> -Curtis
>  On Mar 16, 2012 6:26 AM, "Christophe Leterrier" <
> [hidden email]> wrote:
>
>> HI,
>>
>> I'm trying to use the FolderOpener class in a javascript to make a big
>> virtual stack form several stacks in a folder.
>> The FolderOpener (and the File > Import > Image Sequence command)
>> allows to open a sequence of stacks in a folder as a non virtual
>> stack. If the folders contains three 100-slices stacks, the result is
>> a 300-slice non virtual stack.
>>
>> However, if I set the virtual stack option to true (in the Import
>> Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
>> object), what I get is only a 3-slice virtual stack containing the
>> first image of each 100-slice stacks.
>>
>> Is that intended? If so, how can I get a virtual stack concatenating
>> all stacks in a folder in javascript?
>>
>> Thanks for your help,
>>
>> Christophe
>>

[ctrueden]

Hi Christophe,

Indeed, after spending a couple of hours struggling with the virtual
> stacks class, it is now clear that they can only refer to individual
> image files on disk, not to slices into a stack file. By the way it
> took me a long time to get that the addSlice() parameter (the only way
> to add content to a Virtual Stack ?) has to do with the path to the
> image file (is there a good documentation / programming exemples /
> tips on working with virtual stacks beyond the IJ Javadoc?).
>

Not that I know of. Looking at the Javadoc and source code is a reasonably
reliable, if slow, means of understanding how things work.

BioFormats seems to be able to override this limit and make a virtual
> stack that refers to images that are slices inside stack files. How is
> that done? Is it the use of the BFVirtualStacks class that allows to
> do it? Is the resulting virtual stack a real one, i.e. can I just use
> it as any Virtual Stack for further processing?
>

Yes, this is possible thanks to the BFVirtualStack class in
loci.plugins.util, which extends ImageJ's VirtualStack class so that the
data is backed by a Bio-Formats reader. It would be similarly possible to
create another extension of VirtualStack that provides the data by any
mechanism you need (e.g., from a database online). These are indeed "real"
virtual stacks, in that they are ij.VirtualStack objects, and will work in
ImageJ as any other virtual stack would.

In the end, working with BioFormat seems the way to go. Where can I
> find BioFormat's API / Javadoc? Can I simply record a call to the BF
> Importer with "groupe similar names", "use virtual stack" checked and
> use it in a scrip to get a silent opening? How can I specify name
> patterns in a script?
>

Online Javadoc is here:
   http://hudson.openmicroscopy.org.uk/job/BIOFORMATS-trunk/javadoc

You can certainly record a call to the Bio-Formats Importer. If you want it
to run silently (i.e., no dialog popups), use the "Bio-Formats Windowless
Importer" instead.

Alternately, since you are working from Javascript, you can call the
loci.plugins.BF utility class directly to do what you want:

   importClass(Packages.loci.plugins.BF);
   importClass(Packages['loci.plugins.in.ImporterOptions']);
   options = new ImporterOptions();
   options.setId("/data/toucan.png");
   options.setGroupFiles(true);
   options.setVirtual(true);
   imps = BF.openImagePlus(options);
   imps[0].show();

However, if you want to manually specify a pattern as an ID string, it
becomes more complicated. Easiest is if your file numbering can be detected
properly as-is.

HTH,
Curtis





On Sun, Mar 18, 2012 at 1:04 PM, Christophe Leterrier <
[hidden email]> wrote:

> Hi Curtis,
>
> Indeed, after spending a couple of hours struggling with the virtual
> stacks class, it is now clear that they can only refer to individual
> image files on disk, not to slices into a stack file. By the way it
> took me a long time to get that the addSlice() parameter (the only way
> to add content to a Virtual Stack ?) has to do with the path to the
> image file (is there a good documentation / programming exemples /
> tips on working with virtual stacks beyond the IJ Javadoc?).
>
> BioFormats seems to be able to override this limit and make a virtual
> stack that refers to images that are slices inside stack files. How is
> that done? Is it the use of the BFVirtualStacks class that allows to
> do it? Is the resulting virtual stack a real one, i.e. can I just use
> it as any Virtual Stack for further processing?
>
> In the end, working with BioFormat seems the way to go. Where can I
> find BioFormat's API / Javadoc? Can I simply record a call to the BF
> Importer with "groupe similar names", "use virtual stack" checked and
> use it in a scrip to get a silent opening? How can I specify name
> patterns in a script?
>
> Thanks for your help,
>
> Christophe
>
>
>
> On Sat, Mar 17, 2012 at 21:12, Curtis Rueden <[hidden email]> wrote:
> > Hi Christophe,
> >
> > Did you try the Bio-Formats importer with "Group files with similar
> names"
> > and "Use virtual stacks" checked? Or are you files not numbered properly
> > for that? The BF file pattern logic is quite sophisticated; perhaps a
> > manually defined pattern could work for you in the latter case.
> >
> > -Curtis
> >  On Mar 16, 2012 6:26 AM, "Christophe Leterrier" <
> > [hidden email]> wrote:
> >
> >> HI,
> >>
> >> I'm trying to use the FolderOpener class in a javascript to make a big
> >> virtual stack form several stacks in a folder.
> >> The FolderOpener (and the File > Import > Image Sequence command)
> >> allows to open a sequence of stacks in a folder as a non virtual
> >> stack. If the folders contains three 100-slices stacks, the result is
> >> a 300-slice non virtual stack.
> >>
> >> However, if I set the virtual stack option to true (in the Import
> >> Image Sequence GUI or using openAsVirtualStack() on the FolderOpener
> >> object), what I get is only a 3-slice virtual stack containing the
> >> first image of each 100-slice stacks.
> >>
> >> Is that intended? If so, how can I get a virtual stack concatenating
> >> all stacks in a folder in javascript?
> >>
> >> Thanks for your help,
> >>
> >> Christophe
> >>
>