HDF5 support
Locked
 
|
Hi,
I was wondering if there is a way to read in an hdf5 (.h5) dataset into ImageJ. I already tried using the plugin in the page below but it seems not to work for me for files above 4GB in size. I am using win7 64-bit along with 64-bit imagej&java so there should not be a 32-bit limitation in that respect. http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plugins/hdf5.html I had a quick look in the bioformats plugin but could not find anything compatible to plain hdf5 format. Kind regards, Lucas -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi,
I think the question is how your HDF5 file is internally formatted. HDF5 has not many constraints an internal structure, so it leaves full freedom to you, to store your datasets in different internal paths or names or ... It can not know, for example, how multiple channels would be stored, if you have that, or how your x/y/z dimensions are stored, if you have that. All this is free to decide when writing the file, and no reader can know it in advance. So there is not really a "plain" HDF5 reader because it would need to know at least something about your internal structure of the file. If you have that, it should be trivial to adjust one of the existing readers to read your files. All the best, Mario On 26.06.2014 13:15, Hadjilucas, Lucas wrote: > Hi, > > I was wondering if there is a way to read in an hdf5 (.h5) dataset into ImageJ. > > I already tried using the plugin in the page below but it seems not to work for me for files above 4GB in size. I am using win7 64-bit along with 64-bit imagej&java so there should not be a 32-bit limitation in that respect. > > http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plugins/hdf5.html > > I had a quick look in the bioformats plugin but could not find anything compatible to plain hdf5 format. > > Kind regards, > Lucas > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting annoying in email? Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch D-81669 München http://www.marioemmenlauer.de/ -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi,
The HDF group has developed a software named Hdf5 view <http://www.hdfgroup.org/products/java/index.html>. You can use this soft to read and extract your dataset and then use imageJ to analyse the data. ++ Antoine Bergamaschi 2014-06-26 15:37 GMT+02:00 Mario Emmenlauer <[hidden email]>: > Hi, > > I think the question is how your HDF5 file is internally formatted. HDF5 > has not many constraints an internal structure, so it leaves full freedom > to you, to store your datasets in different internal paths or names or ... > It can not know, for example, how multiple channels would be stored, if > you have that, or how your x/y/z dimensions are stored, if you have that. > All this is free to decide when writing the file, and no reader can know > it in advance. > > So there is not really a "plain" HDF5 reader because it would need to > know at least something about your internal structure of the file. > If you have that, it should be trivial to adjust one of the existing > readers to read your files. > > All the best, > > Mario > > > > On 26.06.2014 13:15, Hadjilucas, Lucas wrote: > > Hi, > > > > I was wondering if there is a way to read in an hdf5 (.h5) dataset into > ImageJ. > > > > I already tried using the plugin in the page below but it seems not to > work for me for files above 4GB in size. I am using win7 64-bit along with > 64-bit imagej&java so there should not be a 32-bit limitation in that > respect. > > > > > http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plugins/hdf5.html > > > > I had a quick look in the bioformats plugin but could not find anything > compatible to plain hdf5 format. > > > > Kind regards, > > Lucas > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > A: Yes. > > Q: Are you sure? > >> A: Because it reverses the logical flow of conversation. > >>> Q: Why is top posting annoying in email? > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.marioemmenlauer.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Antoine,
The actual reason for using hdf5 is because I have some very large arrays I need to pass from Matlab -> Fiji. I tried using Miji with the command CreateImage to instantiate a new image from within Matlab but because the array is so big I get Java Heap Out of Memory errors. This is despite having adjusted the Matlab Java heap space and having enough memory on my machine. It is also painfully slow to instantiate a large array with MIJ.createImage. For some reason it appears to be faster to just write the data on disk and load it with MIJ.run('Open...', 'path=[filepath]') I used to use FITS for writing the large volume as a single file but the FITS libraries on windows Matlab have an issue reading files over 4GB (despite being 64-bit). Therefore I worked around that by replacing FITS with the HDF5 format which Matlab appears to be more comfortable with reading/writing when it comes to file size of 4GB+. The problem now is that ImageJ cannot read in that HDF5 data... Kind regards, Lucas -----Original Message----- From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of Antoine Bergamaschi Sent: 26 June 2014 15:03 To: [hidden email] Subject: Re: HDF5 support Hi, The HDF group has developed a software named Hdf5 view <http://www.hdfgroup.org/products/java/index.html>. You can use this soft to read and extract your dataset and then use imageJ to analyse the data. ++ Antoine Bergamaschi 2014-06-26 15:37 GMT+02:00 Mario Emmenlauer <[hidden email]>: > Hi, > > I think the question is how your HDF5 file is internally formatted. > HDF5 has not many constraints an internal structure, so it leaves full > freedom to you, to store your datasets in different internal paths or names or ... > It can not know, for example, how multiple channels would be stored, > if you have that, or how your x/y/z dimensions are stored, if you have that. > All this is free to decide when writing the file, and no reader can > know it in advance. > > So there is not really a "plain" HDF5 reader because it would need to > know at least something about your internal structure of the file. > If you have that, it should be trivial to adjust one of the existing > readers to read your files. > > All the best, > > Mario > > > > On 26.06.2014 13:15, Hadjilucas, Lucas wrote: > > Hi, > > > > I was wondering if there is a way to read in an hdf5 (.h5) dataset > > into > ImageJ. > > > > I already tried using the plugin in the page below but it seems not > > to > work for me for files above 4GB in size. I am using win7 64-bit along > with 64-bit imagej&java so there should not be a 32-bit limitation in > that respect. > > > > > http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plug > ins/hdf5.html > > > > I had a quick look in the bioformats plugin but could not find > > anything > compatible to plain hdf5 format. > > > > Kind regards, > > Lucas > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > A: Yes. > > Q: Are you sure? > >> A: Because it reverses the logical flow of conversation. > >>> Q: Why is top posting annoying in email? > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > D-81669 München http://www.marioemmenlauer.de/ > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
Hi Lucas,
> The problem now is that ImageJ cannot read in that HDF5 data... So, MATLAB is the tool responsible for writing out this HDF5 data? If you really want it supported in ImageJ, someone would need to create a SCIFIO file format plugin. If you file an issue on the SCIFIO bug tracker, along with instructions on how to generate some sample HDF5 data using MATLAB, the core SCIFIO developers can at least keep it on the radar -- though no promises on timeline. Or if you (or a colleague) want to take a crack at implementing such a file format yourself, we would be delighted to help with any technical hurdles. Regards, Curtis On Thu, Jun 26, 2014 at 2:30 PM, Hadjilucas, Lucas < [hidden email]> wrote: > Hi Antoine, > > The actual reason for using hdf5 is because I have some very large arrays > I need to pass from Matlab -> Fiji. > I tried using Miji with the command CreateImage to instantiate a new image > from within Matlab but because the array is so big I get Java Heap Out of > Memory errors. This is despite having adjusted the Matlab Java heap space > and having enough memory on my machine. It is also painfully slow to > instantiate a large array with MIJ.createImage. For some reason it appears > to be faster to just write the data on disk and load it with > MIJ.run('Open...', 'path=[filepath]') > > I used to use FITS for writing the large volume as a single file but the > FITS libraries on windows Matlab have an issue reading files over 4GB > (despite being 64-bit). Therefore I worked around that by replacing FITS > with the HDF5 format which Matlab appears to be more comfortable with > reading/writing when it comes to file size of 4GB+. > > The problem now is that ImageJ cannot read in that HDF5 data... > > Kind regards, > Lucas > > -----Original Message----- > From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of > Antoine Bergamaschi > Sent: 26 June 2014 15:03 > To: [hidden email] > Subject: Re: HDF5 support > > Hi, > > The HDF group has developed a software named Hdf5 view < > http://www.hdfgroup.org/products/java/index.html>. You can use this soft > to read and extract your dataset and then use imageJ to analyse the data. > > ++ > > Antoine Bergamaschi > > > 2014-06-26 15:37 GMT+02:00 Mario Emmenlauer <[hidden email]>: > > > Hi, > > > > I think the question is how your HDF5 file is internally formatted. > > HDF5 has not many constraints an internal structure, so it leaves full > > freedom to you, to store your datasets in different internal paths or > names or ... > > It can not know, for example, how multiple channels would be stored, > > if you have that, or how your x/y/z dimensions are stored, if you have > that. > > All this is free to decide when writing the file, and no reader can > > know it in advance. > > > > So there is not really a "plain" HDF5 reader because it would need to > > know at least something about your internal structure of the file. > > If you have that, it should be trivial to adjust one of the existing > > readers to read your files. > > > > All the best, > > > > Mario > > > > > > > > On 26.06.2014 13:15, Hadjilucas, Lucas wrote: > > > Hi, > > > > > > I was wondering if there is a way to read in an hdf5 (.h5) dataset > > > into > > ImageJ. > > > > > > I already tried using the plugin in the page below but it seems not > > > to > > work for me for files above 4GB in size. I am using win7 64-bit along > > with 64-bit imagej&java so there should not be a 32-bit limitation in > > that respect. > > > > > > > > http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plug > > ins/hdf5.html > > > > > > I had a quick look in the bioformats plugin but could not find > > > anything > > compatible to plain hdf5 format. > > > > > > Kind regards, > > > Lucas > > > > > > -- > > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > > > > -- > > A: Yes. > > > Q: Are you sure? > > >> A: Because it reverses the logical flow of conversation. > > >>> Q: Why is top posting annoying in email? > > > > Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 > > Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch > > D-81669 München http://www.marioemmenlauer.de/ > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Curtis and others (Mark?),
if there is going to be any development in the HDF5 direction, we'd absolutely love to see the Image Standard [1] of HDF5 being supported! This would actually solve quite some of our time-consuming format-juggling exercises... I'm pretty sure we're not the only ones out there that would really benefit from an HDF5 reader in SCIFIO. :-) Regards, Niko [1] http://www.hdfgroup.org/HDF5/doc/ADGuide/ImageSpec.html On 16.07.2014 21:45, Curtis Rueden wrote: > Hi Lucas, > >> The problem now is that ImageJ cannot read in that HDF5 data... > > So, MATLAB is the tool responsible for writing out this HDF5 data? If you > really want it supported in ImageJ, someone would need to create a SCIFIO > file format plugin. If you file an issue on the SCIFIO bug tracker, along > with instructions on how to generate some sample HDF5 data using MATLAB, > the core SCIFIO developers can at least keep it on the radar -- though no > promises on timeline. Or if you (or a colleague) want to take a crack at > implementing such a file format yourself, we would be delighted to help > with any technical hurdles. > > Regards, > Curtis > > > On Thu, Jun 26, 2014 at 2:30 PM, Hadjilucas, Lucas < > [hidden email]> wrote: > >> Hi Antoine, >> >> The actual reason for using hdf5 is because I have some very large arrays >> I need to pass from Matlab -> Fiji. >> I tried using Miji with the command CreateImage to instantiate a new image >> from within Matlab but because the array is so big I get Java Heap Out of >> Memory errors. This is despite having adjusted the Matlab Java heap space >> and having enough memory on my machine. It is also painfully slow to >> instantiate a large array with MIJ.createImage. For some reason it appears >> to be faster to just write the data on disk and load it with >> MIJ.run('Open...', 'path=[filepath]') >> >> I used to use FITS for writing the large volume as a single file but the >> FITS libraries on windows Matlab have an issue reading files over 4GB >> (despite being 64-bit). Therefore I worked around that by replacing FITS >> with the HDF5 format which Matlab appears to be more comfortable with >> reading/writing when it comes to file size of 4GB+. >> >> The problem now is that ImageJ cannot read in that HDF5 data... >> >> Kind regards, >> Lucas >> >> -----Original Message----- >> From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of >> Antoine Bergamaschi >> Sent: 26 June 2014 15:03 >> To: [hidden email] >> Subject: Re: HDF5 support >> >> Hi, >> >> The HDF group has developed a software named Hdf5 view < >> http://www.hdfgroup.org/products/java/index.html>. You can use this soft >> to read and extract your dataset and then use imageJ to analyse the data. >> >> ++ >> >> Antoine Bergamaschi >> >> >> 2014-06-26 15:37 GMT+02:00 Mario Emmenlauer <[hidden email]>: >> >>> Hi, >>> >>> I think the question is how your HDF5 file is internally formatted. >>> HDF5 has not many constraints an internal structure, so it leaves full >>> freedom to you, to store your datasets in different internal paths or >> names or ... >>> It can not know, for example, how multiple channels would be stored, >>> if you have that, or how your x/y/z dimensions are stored, if you have >> that. >>> All this is free to decide when writing the file, and no reader can >>> know it in advance. >>> >>> So there is not really a "plain" HDF5 reader because it would need to >>> know at least something about your internal structure of the file. >>> If you have that, it should be trivial to adjust one of the existing >>> readers to read your files. >>> >>> All the best, >>> >>> Mario >>> >>> >>> >>> On 26.06.2014 13:15, Hadjilucas, Lucas wrote: >>>> Hi, >>>> >>>> I was wondering if there is a way to read in an hdf5 (.h5) dataset >>>> into >>> ImageJ. >>>> >>>> I already tried using the plugin in the page below but it seems not >>>> to >>> work for me for files above 4GB in size. I am using win7 64-bit along >>> with 64-bit imagej&java so there should not be a 32-bit limitation in >>> that respect. >>>> >>>> >>> http://lmb.informatik.uni-freiburg.de/resources/opensource/imagej_plug >>> ins/hdf5.html >>>> >>>> I had a quick look in the bioformats plugin but could not find >>>> anything >>> compatible to plain hdf5 format. >>>> >>>> Kind regards, >>>> Lucas >>>> >>>> -- >>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>>> >>> >>> -- >>> A: Yes. >>>> Q: Are you sure? >>>>> A: Because it reverses the logical flow of conversation. >>>>>> Q: Why is top posting annoying in email? >>> >>> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >>> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >>> D-81669 München http://www.marioemmenlauer.de/ >>> >>> -- >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>> >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- Niko Ehrenfeuchter Imaging Core Facility Kragenbau, Room G1055 Biozentrum, University of Basel Klingelbergstrasse 50/70 CH-4056 Basel (Switzerland) Office: +41 (61) 26 72673 Email: [hidden email] -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
|
|
Hi Niko,
> if there is going to be any development in the HDF5 direction, we'd > absolutely love to see the Image Standard [1] of HDF5 being supported! > This would actually solve quite some of our time-consuming > format-juggling exercises... > > I'm pretty sure we're not the only ones out there that would really > benefit from an HDF5 reader in SCIFIO. :-) We would love that too, but unfortunately we are not funded to do it right now. Care to take a crack at it? We would be delighted to help overcome any technical obstacles, offer advice and assistance, etc. https://github.com/scifio/scifio-tutorials You might also reach out to the OME developers since it is something that has been on the OME team's radar since at least 2008. Regards, Curtis On Thu, Jul 17, 2014 at 2:38 AM, Niko Ehrenfeuchter < [hidden email]> wrote: > Hi Curtis and others (Mark?), > > if there is going to be any development in the HDF5 direction, we'd > absolutely love to see the Image Standard [1] of HDF5 being supported! This > would actually solve quite some of our time-consuming format-juggling > exercises... > > I'm pretty sure we're not the only ones out there that would really > benefit from an HDF5 reader in SCIFIO. :-) > > Regards, > Niko > > [1] http://www.hdfgroup.org/HDF5/doc/ADGuide/ImageSpec.html > > > On 16.07.2014 21:45, Curtis Rueden wrote: > >> Hi Lucas, >> >> The problem now is that ImageJ cannot read in that HDF5 data... >>> >> >> So, MATLAB is the tool responsible for writing out this HDF5 data? If you >> really want it supported in ImageJ, someone would need to create a SCIFIO >> file format plugin. If you file an issue on the SCIFIO bug tracker, along >> with instructions on how to generate some sample HDF5 data using MATLAB, >> the core SCIFIO developers can at least keep it on the radar -- though no >> promises on timeline. Or if you (or a colleague) want to take a crack at >> implementing such a file format yourself, we would be delighted to help >> with any technical hurdles. >> >> Regards, >> Curtis >> >> >> On Thu, Jun 26, 2014 at 2:30 PM, Hadjilucas, Lucas < >> [hidden email]> wrote: >> >> Hi Antoine, >>> >>> The actual reason for using hdf5 is because I have some very large arrays >>> I need to pass from Matlab -> Fiji. >>> I tried using Miji with the command CreateImage to instantiate a new >>> image >>> from within Matlab but because the array is so big I get Java Heap Out of >>> Memory errors. This is despite having adjusted the Matlab Java heap space >>> and having enough memory on my machine. It is also painfully slow to >>> instantiate a large array with MIJ.createImage. For some reason it >>> appears >>> to be faster to just write the data on disk and load it with >>> MIJ.run('Open...', 'path=[filepath]') >>> >>> I used to use FITS for writing the large volume as a single file but the >>> FITS libraries on windows Matlab have an issue reading files over 4GB >>> (despite being 64-bit). Therefore I worked around that by replacing FITS >>> with the HDF5 format which Matlab appears to be more comfortable with >>> reading/writing when it comes to file size of 4GB+. >>> >>> The problem now is that ImageJ cannot read in that HDF5 data... >>> >>> Kind regards, >>> Lucas >>> >>> -----Original Message----- >>> From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of >>> Antoine Bergamaschi >>> Sent: 26 June 2014 15:03 >>> To: [hidden email] >>> Subject: Re: HDF5 support >>> >>> Hi, >>> >>> The HDF group has developed a software named Hdf5 view < >>> http://www.hdfgroup.org/products/java/index.html>. You can use this soft >>> to read and extract your dataset and then use imageJ to analyse the data. >>> >>> ++ >>> >>> Antoine Bergamaschi >>> >>> >>> 2014-06-26 15:37 GMT+02:00 Mario Emmenlauer <[hidden email]>: >>> >>> Hi, >>>> >>>> I think the question is how your HDF5 file is internally formatted. >>>> HDF5 has not many constraints an internal structure, so it leaves full >>>> freedom to you, to store your datasets in different internal paths or >>>> >>> names or ... >>> >>>> It can not know, for example, how multiple channels would be stored, >>>> if you have that, or how your x/y/z dimensions are stored, if you have >>>> >>> that. >>> >>>> All this is free to decide when writing the file, and no reader can >>>> know it in advance. >>>> >>>> So there is not really a "plain" HDF5 reader because it would need to >>>> know at least something about your internal structure of the file. >>>> If you have that, it should be trivial to adjust one of the existing >>>> readers to read your files. >>>> >>>> All the best, >>>> >>>> Mario >>>> >>>> >>>> >>>> On 26.06.2014 13:15, Hadjilucas, Lucas wrote: >>>> >>>>> Hi, >>>>> >>>>> I was wondering if there is a way to read in an hdf5 (.h5) dataset >>>>> into >>>>> >>>> ImageJ. >>>> >>>>> >>>>> I already tried using the plugin in the page below but it seems not >>>>> to >>>>> >>>> work for me for files above 4GB in size. I am using win7 64-bit along >>>> with 64-bit imagej&java so there should not be a 32-bit limitation in >>>> that respect. >>>> >>>>> >>>>> >>>>> http://lmb.informatik.uni-freiburg.de/resources/ >>>> opensource/imagej_plug >>>> ins/hdf5.html >>>> >>>>> >>>>> I had a quick look in the bioformats plugin but could not find >>>>> anything >>>>> >>>> compatible to plain hdf5 format. >>>> >>>>> >>>>> Kind regards, >>>>> Lucas >>>>> >>>>> -- >>>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>>>> >>>>> >>>> -- >>>> A: Yes. >>>> >>>>> Q: Are you sure? >>>>> >>>>>> A: Because it reverses the logical flow of conversation. >>>>>> >>>>>>> Q: Why is top posting annoying in email? >>>>>>> >>>>>> >>>> Mario Emmenlauer BioDataAnalysis Mobil: +49-(0)151-68108489 >>>> Balanstrasse 43 mailto: mario.emmenlauer * unibas.ch >>>> D-81669 München http://www.marioemmenlauer.de/ >>>> >>>> -- >>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>>> >>>> >>> -- >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>> >>> -- >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >>> >>> >> -- >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html >> >> > > -- > Niko Ehrenfeuchter > Imaging Core Facility > Kragenbau, Room G1055 > Biozentrum, University of Basel > Klingelbergstrasse 50/70 > CH-4056 Basel (Switzerland) > > Office: +41 (61) 26 72673 > Email: [hidden email] > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |