Fwd: possible bug in the ImageJ Coloc 2

[Daniel White]

Hi Andrii

This is interesting. I remember working hard on this part of the maths
trying to make sure it was right.

Our implementation seems verbose compared to what you suggest from
Wikipedia. I don't know the reason for that. Perhaps Tom K remembers?

I'm not able to spend time on this sadly
At least I can't promise.

But perhaps you can investigate where the difference is between the two
methods. The scatter plot visually suggests that a gradient of close to 3
is not right. But that's no proof.

That the implementations in scipy and numpy give same answer as you have is
encouraging. But perhaps we miss something I forgot over time.

The sample image data pixel values are very far from normally distributed.
So perhaps some assumptions fail.....????

Also I added a large positive offset to both images to simulate data from a
camera that adds a positive offset. Just to make the threshold algorithm
struggle harder.... And in fact fail. As a demonstration of why the offsets
need to be subtracted first. Bit this should not affect a regression or
correlation calculation???????

cheers, Dan

http://chalkie.org.uk
---------- Forwarded message ----------
From: "Andrii Rogov" <[hidden email]>
Date: 5 Dec 2014 17:21
Subject: possible bug in the ImageJ Coloc 2
To: <[hidden email]>
Cc:

Dear Mr White,

>
> I am working on the colocalisation analysis of images and used Costes
> algorithm implemented in ImageJ Coloc 2 plugin.
> I've tried to reproduce the results in my python program, but did not
> succeed.
>
>  The problem is in calculating linear regression - the first step of
> Costes algorithm.
>
> I found your code on github and after translating java code into python
> code was able to obtain the same m and b values as calculated in ImageJ.
> But they are different from values calculated by linear regression
> expressions proposed by Wikipedia. ( http://en.wikipedia.org/wiki/
> Simple_linear_regression )
>
> Maybe you don't calculate the linear regression, but use some other
> value... - what is the reason for this?
>
> Please find attached  2 tif files (the examples from the Coloc2 webpage),
> python program, output of the program and a screenshot of ImageJ Coloc
> output.
> Software used - ImageJ 1.49g, Python 3.4
>
> Channel 1 - red.tif
> Channel 2 - green.tif
>
> Thank you in advance,
>
> Best regards,
> Andrii
>
> --
> ------------------------------------
> Andrii Rogov
> University of Geneva
> GAP - Biophotonics
>
> 22, chemin de Pinchat
> CH-1211 Geneva 4
>
> Phone: [phone]
> Fax:   [phone]
>
> http://www.gap.unige.ch/biophotonics/
>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Daniel White]

Hi

I'm not sure if it's bug or not. It's certainly possible. But I haven't got
time to look into it. Perhaps Tom has. Perhaps not.

If you can locate the where a bug might be exactly in our long verbose way
of calculating the regression vs yours,  then it would be much easier to
get any existing bug fixed.

I do remember that we hit a numerical problem. That caused a divide by zero
or something equally bad.... In the case where the mean value of the images
was identical. Unlike in real images but not impossible. The verbose
implementation.might be to do with that. Or that might have been some other
bit if maths in coloc2

Best is for you to look through the commit.messages in the git history and
look for the explanations of why that code was done that way.

This will allow you to read what we were thinking at the time. Maybe that
will help.
I will also have a look to see what we wrote at the time.  But do have a
look yourself. It's a good was to see why git is awesome

Best

D

cheers, Dan

http://chalkie.org.uk
On 8 Dec 2014 12:02, "Andrii Rogov" <[hidden email]> wrote:

>  Dear Dan,
>
> Thank you for your response! It is the first time that I analyzed the code
> of an open source project, and it is great to get the response from the
> authors of the code.
>
> In the notation that Wikipedia proposed the slope is beta : 0.99. So the
> difference in the slope is 0.99 instead of 1.26, proposed by Coloc2.
> Attached is a plot, showing the difference between two lines -
> comparison.png .
>
> Also I've checked  the square error to be sure, and it shows that
> coefficients m and b, given by Coloc 2 plugin are not coefficients of the
> linear regression.
> So there might be a bug in the  calculation of the linear regression, or
> you've used the different approach, from the one proposed by Costes in his
> paper.
>
> What to you think?
> I appreciate any possible input from Tom.
>
> Thanks again for your feedback.
>
> Best regards,
> Andrii
>
> ------------------------------------
> Andrii Rogov
> University of Geneva
> GAP - Biophotonics
>
> 22, chemin de Pinchat
> CH-1211 Geneva 4
>
> Phone: [phone]
> Fax:   [phone]
> http://www.gap.unige.ch/biophotonics/
>
> On 06/12/2014 15:16, Daniel White wrote:
>
> Hi Andrii
>
> This is interesting. I remember working hard on this part of the maths
> trying to make sure it was right.
>
> Our implementation seems verbose compared to what you suggest from
> Wikipedia. I don't know the reason for that. Perhaps Tom K remembers?
>
> I'm not able to spend time on this sadly
> At least I can't promise.
>
> But perhaps you can investigate where the difference is between the two
> methods. The scatter plot visually suggests that a gradient of close to 3
> is not right. But that's no proof.
>
> That the implementations in scipy and numpy give same answer as you have
> is encouraging. But perhaps we miss something I forgot over time.
>
> The sample image data pixel values are very far from normally distributed.
> So perhaps some assumptions fail.....????
>
> Also I added a large positive offset to both images to simulate data from
> a camera that adds a positive offset. Just to make the threshold algorithm
> struggle harder.... And in fact fail. As a demonstration of why the offsets
> need to be subtracted first. Bit this should not affect a regression or
> correlation calculation???????
>
> cheers, Dan
>
> http://chalkie.org.uk
> ---------- Forwarded message ----------
> From: "Andrii Rogov" <[hidden email]>
> Date: 5 Dec 2014 17:21
> Subject: possible bug in the ImageJ Coloc 2
> To: <[hidden email]>
> Cc:
>
> Dear Mr White,
>>
>> I am working on the colocalisation analysis of images and used Costes
>> algorithm implemented in ImageJ Coloc 2 plugin.
>> I've tried to reproduce the results in my python program, but did not
>> succeed.
>>
>>  The problem is in calculating linear regression - the first step of
>> Costes algorithm.
>>
>> I found your code on github and after translating java code into python
>> code was able to obtain the same m and b values as calculated in ImageJ.
>> But they are different from values calculated by linear regression
>> expressions proposed by Wikipedia. (
>> http://en.wikipedia.org/wiki/Simple_linear_regression )
>>
>> Maybe you don't calculate the linear regression, but use some other
>> value... - what is the reason for this?
>>
>> Please find attached  2 tif files (the examples from the Coloc2 webpage),
>> python program, output of the program and a screenshot of ImageJ Coloc
>> output.
>> Software used - ImageJ 1.49g, Python 3.4
>>
>> Channel 1 - red.tif
>> Channel 2 - green.tif
>>
>> Thank you in advance,
>>
>> Best regards,
>> Andrii
>>
>> --
>> ------------------------------------
>> Andrii Rogov
>> University of Geneva
>> GAP - Biophotonics
>>
>> 22, chemin de Pinchat
>> CH-1211 Geneva 4
>>
>> Phone: [phone]
>> Fax:   [phone]
>>
>> http://www.gap.unige.ch/biophotonics/
>>
>>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Tom Kazimiers]

Hi,

@Dan: Thanks for putting me on CC!

On Mon, Dec 08, 2014 at 03:01:08PM +0100, Daniel White wrote:
> I'm not sure if it's bug or not. It's certainly possible. But I haven't got
> time to look into it. Perhaps Tom has. Perhaps not.

Sorry, I probably won't have time to look into this until the end of
this week, likely not before Friday.

I also remember vaguely that we had a discussion around August last
year, where we spotted a problem with the linear regression as well. The
problem there was that different thresholds are found if the order of
input images is reversed. Now that this comes back to my mind, I don't
think we actually fixed this. I'll have a look at both problems, maybe
they are related.

Best,
Tom

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Daniel White]

Hi guys,

Perhaps we should be using a simple linear regression (if that's what
Costes says to use)  from a stats package instead of rolling our own?
Apache commons math? Or does that just add overhead....

Honestly the maths is beyond me here. I get what a least squares method
should do. But the calculations doing it all in one go stretch my maths
past its limit. So I will leave it to those who have better grasp of it.

As usual, watch out for numerical weirdness eg if image means are the same.
Write a unit test if you can and should, which takes generated data as
input with a well known regression result or set of results. Eg we can
generate pairs if values for a known linear function. Add a but of Gaussian
white noise. And we should get a good fit and the same linear equations
back out.

Hope that helps.

cheers, Dan

http://chalkie.org.uk
On 8 Dec 2014 17:57, "Tom Kazimiers" <[hidden email]> wrote:

> Hi,
>
> @Dan: Thanks for putting me on CC!
>
> On Mon, Dec 08, 2014 at 03:01:08PM +0100, Daniel White wrote:
> > I'm not sure if it's bug or not. It's certainly possible. But I haven't
> got
> > time to look into it. Perhaps Tom has. Perhaps not.
>
> Sorry, I probably won't have time to look into this until the end of
> this week, likely not before Friday.
>
> I also remember vaguely that we had a discussion around August last
> year, where we spotted a problem with the linear regression as well. The
> problem there was that different thresholds are found if the order of
> input images is reversed. Now that this comes back to my mind, I don't
> think we actually fixed this. I'll have a look at both problems, maybe
> they are related.
>
> Best,
> Tom
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html