Bug with Batch Covert scale factor?
Locked
Bug with Batch Covert scale factor?
 
|
Hi,
I tried to use ImageJ and Fiji Batch Convert to downsize some large 2D images (close to but smaller than 2 GB) using the scale factor, I ran into an error: >>>>>>>>>>>>>>>>>>>>>>>>>>> <Out of memory> <All available memory (12271MB) has been> <used. To make more available, use the> <Edit>Options>Memory & Threads command.> >>>>>>>>>>>>>>>>>>>>>>>>>>> The error occurs when I choose to downsize the images with scale factor < 1.0; it does not happen with scale factor 1.0 (e.g. without downsizing/resizing). The error could show up before any image is downsized, or after 2-3 images are downsized. Thanks Yuekan Jiao -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Bug with Batch Covert scale factor?
|
|
Hi Yuekan,
(or Jiao? sorry, I don't know which is your first name) If you choose "Average when downsizing", Image>Adjust>Size converts the image to 32 bits (floating point), then resizes it and converts it back. 2 GPixels floating-point data need already 8 GB, in addition you need memory for the original image and the downsized floating-point image. Thus it is easily possible that 12 GB is not enough. Suggestions: (1) increase the memory in Edit>Options (if your machine has enough RAM) (2) deselect "Average when downsizing": Then it just takes interpolated pixel positions; input pixels that are not at the position of an output pixels will be discarded. Convolution with a suitable kernel before averaging would help. (3) if you downsize by an integer factor, you can use the Averaging Reducer plugin. http://rsb.info.nih.gov/ij/plugins/reducer.html Michael ________________________________________________________________ On Apr 14, 2015, at 00:50, Yuekan Jiao wrote: > Hi, > > I tried to use ImageJ and Fiji Batch Convert to downsize some large 2D > images (close to but smaller than 2 GB) using the scale factor, I ran > into an error: > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > <Out of memory> > <All available memory (12271MB) has been> > <used. To make more available, use the> > <Edit>Options>Memory & Threads command.> >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > The error occurs when I choose to downsize the images with scale factor < > 1.0; it does not happen with scale factor 1.0 (e.g. without > downsizing/resizing). The error could show up before any image is > downsized, or after 2-3 images are downsized. > > Thanks > > Yuekan Jiao > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Bug with Batch Covert scale factor?
|
|
Hi
The error is with Batch Convert, in more details, if I choose: Output format: TIFF Interpolation: None Scale factor: 1.0 Read images using Bio-Formats: unchecked Batch Convert works fine. But if I do downsizing, for example: Output format: TIFF Interpolation: None Scale factor: 0.5 Read images using Bio-Formats: unchecked The error shows up. The (batch) downsizing I am trying to do, is to reduce the width and height of some large images, it does not involve image type conversion, Thanks Michael Yuekan On Tue, Apr 14, 2015 at 12:50 AM, Michael Schmid <[hidden email]> wrote: > Hi Yuekan, > > (or Jiao? sorry, I don't know which is your first name) > > If you choose "Average when downsizing", Image>Adjust>Size converts the > image to 32 bits (floating point), then resizes it and converts it back. 2 > GPixels floating-point data need already 8 GB, in addition you need memory > for the original image and the downsized floating-point image. Thus it is > easily possible that 12 GB is not enough. > > Suggestions: > (1) increase the memory in Edit>Options (if your machine has enough RAM) > (2) deselect "Average when downsizing": Then it just takes interpolated > pixel positions; input pixels that are not at the position of an output > pixels will be discarded. Convolution with a suitable kernel before > averaging would help. > (3) if you downsize by an integer factor, you can use the Averaging > Reducer plugin. > http://rsb.info.nih.gov/ij/plugins/reducer.html > > Michael > ________________________________________________________________ > On Apr 14, 2015, at 00:50, Yuekan Jiao wrote: > > > Hi, > > > > I tried to use ImageJ and Fiji Batch Convert to downsize some large 2D > > images (close to but smaller than 2 GB) using the scale factor, I ran > > into an error: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > <Out of memory> > > <All available memory (12271MB) has been> > > <used. To make more available, use the> > > <Edit>Options>Memory & Threads command.> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > The error occurs when I choose to downsize the images with scale factor < > > 1.0; it does not happen with scale factor 1.0 (e.g. without > > downsizing/resizing). The error could show up before any image is > > downsized, or after 2-3 images are downsized. > > > > Thanks > > > > Yuekan Jiao > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Bug with Batch Covert scale factor?
|
|
Hi Yuekan,
ok, sorry, I missed the title saying that it's Batch Convert. Anyhow, Process>Batch>Convert always uses "Average when downsizing"=true (it does not allow you to deselect that option), so the problem is the same: Internally the images are converted to 32-bit (float images) to do the downsizing, then they are converted back to their original data type, 8 bits, 16 bits or RGB. It is not easy to change this. I have quickly created an external plugin (I'll attach it to a private mail; the mailing list does not allow it as attachment). It is identical to the internal Process>Batch>Convert except for a minor change that slightly reduces the memory requirements in some cases. Could you please try it (put it into your Plugins Folder and use Plugins>Compile&Run). If it works with your images, please let me know; then one could incorporate this modification into ImageJ. Otherwise, if you are satisfied with reducing the size by an integer factor, e.g. a factor of 2 (half size), I would propose that you use the Averaging Reducer plugin and call it in Process>Batch>Macro. It needs much less memory. With reduction by a factor of 2, its result should be the same as what you get with bilinear interpolation. Or, if you have enough RAM, use Edit>Options>Memory&Threads to increase the memory for ImageJ (you have to restart ImageJ thereafter). Michael ________________________________________________________________ On Apr 14, 2015, at 18:30, Yuekan Jiao wrote: > Hi > > The error is with Batch Convert, in more details, if I choose: > > Output format: TIFF > Interpolation: None > Scale factor: 1.0 > Read images using Bio-Formats: unchecked > > Batch Convert works fine. But if I do downsizing, for example: > > Output format: TIFF > Interpolation: None > Scale factor: 0.5 > Read images using Bio-Formats: unchecked > > The error shows up. > > The (batch) downsizing I am trying to do, is to reduce the width and height > of some large images, it does not involve image type conversion, > > Thanks Michael > > Yuekan > > > > > > > > > On Tue, Apr 14, 2015 at 12:50 AM, Michael Schmid <[hidden email]> > wrote: > >> Hi Yuekan, >> >> (or Jiao? sorry, I don't know which is your first name) >> >> If you choose "Average when downsizing", Image>Adjust>Size converts the >> image to 32 bits (floating point), then resizes it and converts it back. 2 >> GPixels floating-point data need already 8 GB, in addition you need memory >> for the original image and the downsized floating-point image. Thus it is >> easily possible that 12 GB is not enough. >> >> Suggestions: >> (1) increase the memory in Edit>Options (if your machine has enough RAM) >> (2) deselect "Average when downsizing": Then it just takes interpolated >> pixel positions; input pixels that are not at the position of an output >> pixels will be discarded. Convolution with a suitable kernel before >> averaging would help. >> (3) if you downsize by an integer factor, you can use the Averaging >> Reducer plugin. >> http://rsb.info.nih.gov/ij/plugins/reducer.html >> >> Michael >> ________________________________________________________________ >> On Apr 14, 2015, at 00:50, Yuekan Jiao wrote: >> >>> Hi, >>> >>> I tried to use ImageJ and Fiji Batch Convert to downsize some large 2D >>> images (close to but smaller than 2 GB) using the scale factor, I ran >>> into an error: >>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>> <Out of memory> >>> <All available memory (12271MB) has been> >>> <used. To make more available, use the> >>> <Edit>Options>Memory & Threads command.> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>> >>> The error occurs when I choose to downsize the images with scale factor < >>> 1.0; it does not happen with scale factor 1.0 (e.g. without >>> downsizing/resizing). The error could show up before any image is >>> downsized, or after 2-3 images are downsized. >>> >>> Thanks >>> >>> Yuekan Jiao >>> -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
Re: Bug with Batch Covert scale factor?
|
|
On Apr 14, 2015, at 2:49 PM, Michael Schmid <[hidden email]> wrote:
> > Hi Yuekan, > > ok, sorry, I missed the title saying that it's Batch Convert. > > Anyhow, Process>Batch>Convert always uses "Average when downsizing"=true (it does not allow you to deselect that option), so the problem is the same: > Internally the images are converted to 32-bit (float images) to do the downsizing, then they are converted back to their original data type, 8 bits, 16 bits or RGB. It is not easy to change this. In the latest ImageJ daily build (1.49r19), the Process>Batch>Convert dialog has an "Average when downsizing” checkbox. It now also has a working progress bar. -wayne > I have quickly created an external plugin (I'll attach it to a private mail; the mailing list does not allow it as attachment). It is identical to the internal Process>Batch>Convert except for a minor change that slightly reduces the memory requirements in some cases. Could you please try it (put it into your Plugins Folder and use Plugins>Compile&Run). If it works with your images, please let me know; then one could incorporate this modification into ImageJ. > > Otherwise, if you are satisfied with reducing the size by an integer factor, e.g. a factor of 2 (half size), I would propose that you use the Averaging Reducer plugin and call it in Process>Batch>Macro. It needs much less memory. With reduction by a factor of 2, its result should be the same as what you get with bilinear interpolation. > > Or, if you have enough RAM, use Edit>Options>Memory&Threads to increase the memory for ImageJ (you have to restart ImageJ thereafter). > > Michael > ________________________________________________________________ > On Apr 14, 2015, at 18:30, Yuekan Jiao wrote: > >> Hi >> >> The error is with Batch Convert, in more details, if I choose: >> >> Output format: TIFF >> Interpolation: None >> Scale factor: 1.0 >> Read images using Bio-Formats: unchecked >> >> Batch Convert works fine. But if I do downsizing, for example: >> >> Output format: TIFF >> Interpolation: None >> Scale factor: 0.5 >> Read images using Bio-Formats: unchecked >> >> The error shows up. >> >> The (batch) downsizing I am trying to do, is to reduce the width and height >> of some large images, it does not involve image type conversion, >> >> Thanks Michael >> >> Yuekan >> >> >> >> >> >> >> >> >> On Tue, Apr 14, 2015 at 12:50 AM, Michael Schmid <[hidden email]> >> wrote: >> >>> Hi Yuekan, >>> >>> (or Jiao? sorry, I don't know which is your first name) >>> >>> If you choose "Average when downsizing", Image>Adjust>Size converts the >>> image to 32 bits (floating point), then resizes it and converts it back. 2 >>> GPixels floating-point data need already 8 GB, in addition you need memory >>> for the original image and the downsized floating-point image. Thus it is >>> easily possible that 12 GB is not enough. >>> >>> Suggestions: >>> (1) increase the memory in Edit>Options (if your machine has enough RAM) >>> (2) deselect "Average when downsizing": Then it just takes interpolated >>> pixel positions; input pixels that are not at the position of an output >>> pixels will be discarded. Convolution with a suitable kernel before >>> averaging would help. >>> (3) if you downsize by an integer factor, you can use the Averaging >>> Reducer plugin. >>> http://rsb.info.nih.gov/ij/plugins/reducer.html >>> >>> Michael >>> ________________________________________________________________ >>> On Apr 14, 2015, at 00:50, Yuekan Jiao wrote: >>> >>>> Hi, >>>> >>>> I tried to use ImageJ and Fiji Batch Convert to downsize some large 2D >>>> images (close to but smaller than 2 GB) using the scale factor, I ran >>>> into an error: >>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> <Out of memory> >>>> <All available memory (12271MB) has been> >>>> <used. To make more available, use the> >>>> <Edit>Options>Memory & Threads command.> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> >>>> The error occurs when I choose to downsize the images with scale factor < >>>> 1.0; it does not happen with scale factor 1.0 (e.g. without >>>> downsizing/resizing). The error could show up before any image is >>>> downsized, or after 2-3 images are downsized. >>>> >>>> Thanks >>>> >>>> Yuekan Jiao >>>> > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html |
«
Return to ImageJ
|
1 view|%1 views
| Free forum by Nabble | Edit this page |