Re: Incorrect density values when quantifying PNG files exported from GIMP
Posted by Garvalov, Boyan on Nov 20, 2012; 8:38pm
URL: http://imagej.273.s1.nabble.com/Incorrect-density-values-when-quantifying-PNG-files-exported-from-GIMP-tp5000862p5000905.html
Dear Rob,
Just as I had finished putting together some detailed feedback to your troubleshooting suggestions, the big man himself, who had asked me for sample images, provided a working solution, which I paste below.
From: Rasband, Wayne (NIH/NIMH) [E] [mailto:[hidden email]]
Sent : 20 November, 2012 17:38
To: Garvalov, Boyan
Subject: Re: Incorrect density values when quantifying PNG filesexportedfrom GIMP
Dear Boyan,
The PNG and TIFF images you sent have two channels with the second channel blank. The built in ImageJ Tiff reader does not know how to read two channel images but you can work around the problem by opening the images using the Bio-Formats plugin. In the Bio-Formats importer dialog set "View stack as" to "Standard ImageJ". You can probably also work around the problem by "flattening" the Images in Gimp before saving. I suspect this is a bug in GIMP since it does not make any sense to save a grayscale image in this format.
Best regards,
-wayne
I tried both of these approaches, they both work, but I thought some more specific feedback might be helpful to others:
1) Flattening both the png and tif images in GIMP does the trick - when they are opened in ImageJ they are recognized as 8-bit and produce correct density values.
2) Importing the images with the LOCI Bio-Formats Importer displays the normal and the blank channel as a stack. When quantifying the slice with the normal channel I get correct density values under RawIntDen, but the area and IntDen values were several orders of magnitude higher than the sum of all pixels. This appears to have been caused by a setting under Analyze --> Set Scale where a scale of 0.0118 pixels per micron was set. After entering 0 under Distance in pixels the measured area corresponded to the number of pixels and IntDen was equal to RawIntDen.
I think it is clear now that GIMP 2.8 is doing something strange when exporting png and tif files, and I will post this to the GIMP developers as a bug report. Thanks to all and particularly to Wayne for helping to get to the bottom of this. I hope the GIMP people react appropriately and that the next version wont have these issues (or at least that there will be an option not to include a blank channel, which I do not find at the moment).
Below I copy my initial response, which is now mostly of historical interest, but may help others with similar issues find this thread through search engines:
Thanks for the reply! I totally share your opinion - GIMP and ImageJ are, I believe, the two most important open source programs for the imaging community and making them work more smoothly with each other will be of great benefit.
I already started researching a bit in the directions in which you are asking and here is what I can tell you (I also attach 3 versions of the same image in jpg, png and tif formats exported from GIMP 2.8, which illustrate the issues):
>As I understand it, png supports bit depths of 2, 4, 8 or 16 bits. Do you know which bit depth your png files had when you saved them in GIMP? When you open your .jpg and .png images in ImageJ and look at *Image < Type* how is ImageJ interpreting the .png and .jpg images? Is it the same in both cases? If not, is the difference happening at import or export time. This is where I would go first with trouble shooting.
- This issue indeed appears to be at the heart of the problem, but I actually cannot give you a straight answer right now. I can only tell you the following:
1) Both GIMP and Photoshop see the GIMP-generated png file as grayscale (Photoshop - 8-bits/channel; GIMP does not explicitly mention the bits, or maybe I dont know where to look for that, but as I understand GIMP 2.8 does not support 16-bit grayscale images).
IrfanView says for the png file Original colors: 65536 (16 BitsPerPixel), Current colors: 16.7 Millions (24 BitsPerPixel), for the jpg file: Original colors: 256 (8 BitsPerPixel), Current colors: 256 (8 BitsPerPixel). For the tif file it says Original colors: 65536 (16 BitsPerPixel), Current colors: 256 (8 BitsPerPixel).
2) When I open the png file exported from GIMP with ImageJ it is recognized as an RGB file, whereas the jpg (or Photoshop-generated png) is 8-bit.
3) The problem only occurs with grayscale images, color images produce practically identical values when exported from GIMP in either jpg, png or tif format and analyzed in ImageJ.
4) The problem only occurs when I copy (a part of) one image and paste it into a new file in GIMP which I then export as png from GIMP. In that case it doesnt help if I tell GIMP to open the new file as grayscale and it makes no difference if I define the fill color as white, foreground (black) or transparent.
5) If I first export the file as jpg in GIMP and then re-export it as png in GIMP, I get correct density values in ImageJ. However, if I first export the file as tif and then re-export it as png I again get a file that is seen as RGB in ImageJ and gives me wrong values.
6) One other important point that I discovered in the process: grayscale tif files exported from GIMP cannot be opened in ImageJ at all. The following error message appears: TiffDecoder Unsupported SamplesPerPixel: 2. Thus I cannot really work with lossless files exported from GIMP (at least not with the two most popular formats). The problem is again restricted to grayscale images, color images open fine and give correct density values.
> The other minor thing that bothers me is that you talk about rotating the images. This could be just "a part of your process" or it could be critical to the difference in the .png and .jpg export. Is rotation step critical to the difference in .jpg and .png behavior?
- The rotation is not relevant for this problem, it also occurs if I dont rotate, what is required for the problem to occur is that I copy image information and paste it in a new file (see point 4 above).
>Suppose you bring your original .jpg image into ImageJ and save it as a .png BEFORE your processing. Then you open the ImageJ .png in GIMP, do your thing, and resave as a .png. Does this produce the same difference if you compare to .jpg, that is, does it matter where the initial conversion to .png takes place?
- The answer is essentially analogous to point 4 above. If I take the original jpg image and save it as png in ImageJ then both images give me identical density values and are 8-bit. If I then open this ImageJ-generated png image in GIMP and directly re-export it as png, then the density values are correct and the image is recognized as 8-bit back in ImageJ. But if I copy the whole image or a part of it and paste it in a new file, then export this file as png from GIMP, ImageJ sees the new png file as RGB and gives me wrong density values.
Best wishes
Boyan
-----Original Message-----
From: ImageJ Interest Group on behalf of Robert Baer
Sent: Tue 20/11/2012 16:25
To: [hidden email]
Subject: Re: Incorrect density values when quantifying PNG files exported from GIMP
On 11/19/2012 8:17 AM, Garvalov, Boyan wrote:
> Dear Gabriel,
> Thanks a lot for the reply! It is nice to get one from you, I have already read many helpful pieces of advice on ImageJ from you and have used a number of your plugins for years! Thank you for all that!
> I am aware of the limitations of lossy formats like jpeg and the pitfalls of film scanning and quantification, and I included a couple of comments on that below. But I just wanted to note that the question still remains why in contrast to other programs ImageJ doesn't show and quantify correctly PNG files exported from GIMP. This may be an important question, as millions of people use GIMP and PNG is the default export format in that program. So it may be that there are many people who are quantifying GIMP-generated PNG files with ImageJ and getting substantially incorrect results without realizing it. I myself realized it by accident, after I had already prepared a quantification graph for publication. And the errors, as I mentioned, can be quite substantial, orders of magnitude higher than the differences between a tif scan and a jpg scan.
> Regarding scanning, we do pay careful attention to getting the scanner settings right (which is not the default situation), but the scanner itself is not calibrated. Following your tip I found some instructions on how to do that at http://imagej.nih.gov/ij/docs/examples/calibration/ and will look into the possibility of doing this for our lab scanner.
> Regarding the information loss when saving to jpeg, I have found that it can definitely be an issue e.g. when thresholding and analysing thresholded particles, so there using tif there is better. On the other hand, tests I did with films scanned using our scanner indicate that the differences in density between a tiff scan and a jpeg scan are in the order of 0.01-0.12%. We so far found this to be an acceptable compromise between information preservation and file size.
> Best wishes
> Boyan
Understanding how programs like GIMP interact with ImageJ IS of great
importance so thanks for sharing. However, I can't quite tell which of
a number of things might be going on.
As I understand it, png supports bit depths of 2, 4, 8 or 16 bits. Do
you know which bit depth your png files had when you saved them in
GIMP? When you open your .jpg and .png images in ImageJ and look at
*Image < Type* how is ImageJ interpreting the .png and .jpg images? Is
it the same in both cases? If not, is the difference happening at
import or export time. This is where I would go first with trouble
shooting.
The other minor thing that bothers me is that you talk about rotating
the images. This could be just "a part of your process" or it could be
critical to the difference in the .png and .jpg export. Is rotation step
critical to the difference in .jpg and .png behavior?
Suppose you bring your original .jpg image into ImageJ and save it as a
.png BEFORE your processing. Then you open the ImageJ .png in GIMP, do
your thing, and resave as a .png. Does this produce the same difference
if you compare to .jpg, that is, does it matter where the initial
conversion to .png takes place?
Anyway, just some troubleshooting ideas!
Rob
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dr. Boyan K. Garvalov
> Institut für Neuropathologie
> Justus-Liebig-Universität
> Aulweg 123, 3. OG
> 35392 Giessen
>
> Tel: (+49-641) 99-41401
> Fax: (+49-641) 99-41689
> E-mail: [hidden email]
> Website: http://www.ukgm.de/ugm_2/deu/ugi_npa/5424.html
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> -----Ursprüngliche Nachricht-----
> Von: ImageJ Interest Group [mailto:[hidden email]] Im Auftrag von Gabriel Landini
> Gesendet: 19 November, 2012 13:40
> An: [hidden email]
> Betreff: Re: Incorrect density values when quantifying PNG files exported from GIMP
>
> On Monday 19 Nov 2012 11:24:04 you wrote:
>> I have been using ImageJ to quantify some images (western blot scans,
>> grayscale). The blots were first scanned in JPG format, then were
>> rotated and cropped using GIMP and finally exported from GIMP either
>> in JPG or PNG format.
> Jpegs (unless the non-lossy version) for imaging are a very bad idea. There are non-lossy formats (like TIFF or PNG) that retain all the scanned information.
> If you load the jpeg to rotate it and save it again in jpeg, you get more lost information. Once saved in jpeg, there is no benefit in converting to PNG, as the loss of information already took place.
>
> It would be better to scan directly to TIFF, then load in IJ and use the
> Image>Transform menu entry to rotate it (no need for the GIMP step).
> It is also necessary to calibrate the scanner too, otherwise the meaning of the darkness in terms of optical density units is unknown.
>
> Regards
>
> Gabriel
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
--
__________________
Robert W. Baer, Ph.D.
Professor of Physiology
Kirksille College of Osteopathic Medicine
A. T. Still University of Health Sciences
Kirksville, MO 63501 USA
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
URL: http://imagej.273.s1.nabble.com/Incorrect-density-values-when-quantifying-PNG-files-exported-from-GIMP-tp5000862p5000905.html
Dear Rob,
Just as I had finished putting together some detailed feedback to your troubleshooting suggestions, the big man himself, who had asked me for sample images, provided a working solution, which I paste below.
From: Rasband, Wayne (NIH/NIMH) [E] [mailto:[hidden email]]
Sent : 20 November, 2012 17:38
To: Garvalov, Boyan
Subject: Re: Incorrect density values when quantifying PNG filesexportedfrom GIMP
Dear Boyan,
The PNG and TIFF images you sent have two channels with the second channel blank. The built in ImageJ Tiff reader does not know how to read two channel images but you can work around the problem by opening the images using the Bio-Formats plugin. In the Bio-Formats importer dialog set "View stack as" to "Standard ImageJ". You can probably also work around the problem by "flattening" the Images in Gimp before saving. I suspect this is a bug in GIMP since it does not make any sense to save a grayscale image in this format.
Best regards,
-wayne
I tried both of these approaches, they both work, but I thought some more specific feedback might be helpful to others:
1) Flattening both the png and tif images in GIMP does the trick - when they are opened in ImageJ they are recognized as 8-bit and produce correct density values.
2) Importing the images with the LOCI Bio-Formats Importer displays the normal and the blank channel as a stack. When quantifying the slice with the normal channel I get correct density values under RawIntDen, but the area and IntDen values were several orders of magnitude higher than the sum of all pixels. This appears to have been caused by a setting under Analyze --> Set Scale where a scale of 0.0118 pixels per micron was set. After entering 0 under Distance in pixels the measured area corresponded to the number of pixels and IntDen was equal to RawIntDen.
I think it is clear now that GIMP 2.8 is doing something strange when exporting png and tif files, and I will post this to the GIMP developers as a bug report. Thanks to all and particularly to Wayne for helping to get to the bottom of this. I hope the GIMP people react appropriately and that the next version wont have these issues (or at least that there will be an option not to include a blank channel, which I do not find at the moment).
Below I copy my initial response, which is now mostly of historical interest, but may help others with similar issues find this thread through search engines:
Thanks for the reply! I totally share your opinion - GIMP and ImageJ are, I believe, the two most important open source programs for the imaging community and making them work more smoothly with each other will be of great benefit.
I already started researching a bit in the directions in which you are asking and here is what I can tell you (I also attach 3 versions of the same image in jpg, png and tif formats exported from GIMP 2.8, which illustrate the issues):
>As I understand it, png supports bit depths of 2, 4, 8 or 16 bits. Do you know which bit depth your png files had when you saved them in GIMP? When you open your .jpg and .png images in ImageJ and look at *Image < Type* how is ImageJ interpreting the .png and .jpg images? Is it the same in both cases? If not, is the difference happening at import or export time. This is where I would go first with trouble shooting.
- This issue indeed appears to be at the heart of the problem, but I actually cannot give you a straight answer right now. I can only tell you the following:
1) Both GIMP and Photoshop see the GIMP-generated png file as grayscale (Photoshop - 8-bits/channel; GIMP does not explicitly mention the bits, or maybe I dont know where to look for that, but as I understand GIMP 2.8 does not support 16-bit grayscale images).
IrfanView says for the png file Original colors: 65536 (16 BitsPerPixel), Current colors: 16.7 Millions (24 BitsPerPixel), for the jpg file: Original colors: 256 (8 BitsPerPixel), Current colors: 256 (8 BitsPerPixel). For the tif file it says Original colors: 65536 (16 BitsPerPixel), Current colors: 256 (8 BitsPerPixel).
2) When I open the png file exported from GIMP with ImageJ it is recognized as an RGB file, whereas the jpg (or Photoshop-generated png) is 8-bit.
3) The problem only occurs with grayscale images, color images produce practically identical values when exported from GIMP in either jpg, png or tif format and analyzed in ImageJ.
4) The problem only occurs when I copy (a part of) one image and paste it into a new file in GIMP which I then export as png from GIMP. In that case it doesnt help if I tell GIMP to open the new file as grayscale and it makes no difference if I define the fill color as white, foreground (black) or transparent.
5) If I first export the file as jpg in GIMP and then re-export it as png in GIMP, I get correct density values in ImageJ. However, if I first export the file as tif and then re-export it as png I again get a file that is seen as RGB in ImageJ and gives me wrong values.
6) One other important point that I discovered in the process: grayscale tif files exported from GIMP cannot be opened in ImageJ at all. The following error message appears: TiffDecoder Unsupported SamplesPerPixel: 2. Thus I cannot really work with lossless files exported from GIMP (at least not with the two most popular formats). The problem is again restricted to grayscale images, color images open fine and give correct density values.
> The other minor thing that bothers me is that you talk about rotating the images. This could be just "a part of your process" or it could be critical to the difference in the .png and .jpg export. Is rotation step critical to the difference in .jpg and .png behavior?
- The rotation is not relevant for this problem, it also occurs if I dont rotate, what is required for the problem to occur is that I copy image information and paste it in a new file (see point 4 above).
>Suppose you bring your original .jpg image into ImageJ and save it as a .png BEFORE your processing. Then you open the ImageJ .png in GIMP, do your thing, and resave as a .png. Does this produce the same difference if you compare to .jpg, that is, does it matter where the initial conversion to .png takes place?
- The answer is essentially analogous to point 4 above. If I take the original jpg image and save it as png in ImageJ then both images give me identical density values and are 8-bit. If I then open this ImageJ-generated png image in GIMP and directly re-export it as png, then the density values are correct and the image is recognized as 8-bit back in ImageJ. But if I copy the whole image or a part of it and paste it in a new file, then export this file as png from GIMP, ImageJ sees the new png file as RGB and gives me wrong density values.
Best wishes
Boyan
-----Original Message-----
From: ImageJ Interest Group on behalf of Robert Baer
Sent: Tue 20/11/2012 16:25
To: [hidden email]
Subject: Re: Incorrect density values when quantifying PNG files exported from GIMP
On 11/19/2012 8:17 AM, Garvalov, Boyan wrote:
> Dear Gabriel,
> Thanks a lot for the reply! It is nice to get one from you, I have already read many helpful pieces of advice on ImageJ from you and have used a number of your plugins for years! Thank you for all that!
> I am aware of the limitations of lossy formats like jpeg and the pitfalls of film scanning and quantification, and I included a couple of comments on that below. But I just wanted to note that the question still remains why in contrast to other programs ImageJ doesn't show and quantify correctly PNG files exported from GIMP. This may be an important question, as millions of people use GIMP and PNG is the default export format in that program. So it may be that there are many people who are quantifying GIMP-generated PNG files with ImageJ and getting substantially incorrect results without realizing it. I myself realized it by accident, after I had already prepared a quantification graph for publication. And the errors, as I mentioned, can be quite substantial, orders of magnitude higher than the differences between a tif scan and a jpg scan.
> Regarding scanning, we do pay careful attention to getting the scanner settings right (which is not the default situation), but the scanner itself is not calibrated. Following your tip I found some instructions on how to do that at http://imagej.nih.gov/ij/docs/examples/calibration/ and will look into the possibility of doing this for our lab scanner.
> Regarding the information loss when saving to jpeg, I have found that it can definitely be an issue e.g. when thresholding and analysing thresholded particles, so there using tif there is better. On the other hand, tests I did with films scanned using our scanner indicate that the differences in density between a tiff scan and a jpeg scan are in the order of 0.01-0.12%. We so far found this to be an acceptable compromise between information preservation and file size.
> Best wishes
> Boyan
Understanding how programs like GIMP interact with ImageJ IS of great
importance so thanks for sharing. However, I can't quite tell which of
a number of things might be going on.
As I understand it, png supports bit depths of 2, 4, 8 or 16 bits. Do
you know which bit depth your png files had when you saved them in
GIMP? When you open your .jpg and .png images in ImageJ and look at
*Image < Type* how is ImageJ interpreting the .png and .jpg images? Is
it the same in both cases? If not, is the difference happening at
import or export time. This is where I would go first with trouble
shooting.
The other minor thing that bothers me is that you talk about rotating
the images. This could be just "a part of your process" or it could be
critical to the difference in the .png and .jpg export. Is rotation step
critical to the difference in .jpg and .png behavior?
Suppose you bring your original .jpg image into ImageJ and save it as a
.png BEFORE your processing. Then you open the ImageJ .png in GIMP, do
your thing, and resave as a .png. Does this produce the same difference
if you compare to .jpg, that is, does it matter where the initial
conversion to .png takes place?
Anyway, just some troubleshooting ideas!
Rob
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dr. Boyan K. Garvalov
> Institut für Neuropathologie
> Justus-Liebig-Universität
> Aulweg 123, 3. OG
> 35392 Giessen
>
> Tel: (+49-641) 99-41401
> Fax: (+49-641) 99-41689
> E-mail: [hidden email]
> Website: http://www.ukgm.de/ugm_2/deu/ugi_npa/5424.html
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> -----Ursprüngliche Nachricht-----
> Von: ImageJ Interest Group [mailto:[hidden email]] Im Auftrag von Gabriel Landini
> Gesendet: 19 November, 2012 13:40
> An: [hidden email]
> Betreff: Re: Incorrect density values when quantifying PNG files exported from GIMP
>
> On Monday 19 Nov 2012 11:24:04 you wrote:
>> I have been using ImageJ to quantify some images (western blot scans,
>> grayscale). The blots were first scanned in JPG format, then were
>> rotated and cropped using GIMP and finally exported from GIMP either
>> in JPG or PNG format.
> Jpegs (unless the non-lossy version) for imaging are a very bad idea. There are non-lossy formats (like TIFF or PNG) that retain all the scanned information.
> If you load the jpeg to rotate it and save it again in jpeg, you get more lost information. Once saved in jpeg, there is no benefit in converting to PNG, as the loss of information already took place.
>
> It would be better to scan directly to TIFF, then load in IJ and use the
> Image>Transform menu entry to rotate it (no need for the GIMP step).
> It is also necessary to calibrate the scanner too, otherwise the meaning of the darkness in terms of optical density units is unknown.
>
> Regards
>
> Gabriel
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
--
__________________
Robert W. Baer, Ph.D.
Professor of Physiology
Kirksille College of Osteopathic Medicine
A. T. Still University of Health Sciences
Kirksville, MO 63501 USA
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
winmail.dat (632K) | Free forum by Nabble | Disable Popup Ads | Edit this page |