Java SIFT plugin for ImageJ : Question on parametage

[Xin ZHOU]

Hello, everybody,

Is there anybody who knows a lot about the Java SIFT plugin provided by
Stephan Saalfeld <mailto:[hidden email]>?
I also send this message to Stephan Saalfeld
<mailto:[hidden email]> but I need some quick answers to make my
project move.

I'm using the SIFT plugin to extract the salient points and I played
with the parameters.

In the "Java SIFT parameters explanation.doc" I find some inconsistency.
*        Steps per scale octave*: Keypoint candidates are extracted at
all scales between /maximum image size/ and /minimum image size/.

    * *Minimum image size*: The Scale Space stops if the size of the
      octave would be smaller than /minimum image size/. *Tip*: Increase
      the minimum size to discard large features (i.e. those extracted
      from looking at an image from far, such as the overall shape).
    * *Maximum image size*: The Scale Space starts with the first octave
      equal or smaller than the /maximum image size/. *Tip*: By reducing
      the size, fine scaled features will be discarded, favoring larger
      features. Larger features are more likely to remain constant
      across serial sections. Increasing the size beyond that of the
      actual images has no effect.

I don't understand why if I enlarger the maximum image size, I'll
discard the larger features ....
I presume that the larger features are for the larger scale, right?
Then larger scale can only be detected in larger window.

I think I just badly understand these two parameters.

Can anyone explain a little bit?

cheers, Xin

[Xin ZHOU]

Hello,

Maybe I just make it to be precise in discussion.
For example, I want larger features, which means those show the salient
point in a "low resolution".
How should I set the max and min image size value?

cheers, Xin

Xin ZHOU a écrit :

> Hello, everybody,
>
> Is there anybody who knows a lot about the Java SIFT plugin provided
> by Stephan Saalfeld <mailto:[hidden email]>?
> I also send this message to Stephan Saalfeld
> <mailto:[hidden email]> but I need some quick answers to make my
> project move.
>
> I'm using the SIFT plugin to extract the salient points and I played
> with the parameters.
>
> In the "Java SIFT parameters explanation.doc" I find some inconsistency.
> *        Steps per scale octave*: Keypoint candidates are extracted at
> all scales between /maximum image size/ and /minimum image size/.
>
>    * *Minimum image size*: The Scale Space stops if the size of the
>      octave would be smaller than /minimum image size/. *Tip*: Increase
>      the minimum size to discard large features (i.e. those extracted
>      from looking at an image from far, such as the overall shape).
>    * *Maximum image size*: The Scale Space starts with the first octave
>      equal or smaller than the /maximum image size/. *Tip*: By reducing
>      the size, fine scaled features will be discarded, favoring larger
>      features. Larger features are more likely to remain constant
>      across serial sections. Increasing the size beyond that of the
>      actual images has no effect.
>
> I don't understand why if I enlarger the maximum image size, I'll
> discard the larger features ....
> I presume that the larger features are for the larger scale, right?
> Then larger scale can only be detected in larger window.
>
> I think I just badly understand these two parameters.
>
> Can anyone explain a little bit?
>
> cheers, Xin
>
>
>

[Albert Cardona]

Hi Xin,


> I don't understand why if I enlarger the maximum image size, I'll
> discard the larger features ....
> I presume that the larger features are for the larger scale, right?
> Then larger scale can only be detected in larger window.


Exactly the opposite. Keep in mind that features are found by moving
around a fixed size NxN pixel window. If the image is larger, the window
is looking at comparatively a smaller region of the overall image, and
thus it's extracting small features.

Hope that helps,

Albert

--
Albert Cardona
http://www.mcdb.ucla.edu/Research/Hartenstein/acardona

[Xin ZHOU]

Hi, Albert,

What is the exact definition of this "image size"?
I means, for me it is the size of the part that we want to study. Which
is, the study window size.
Because basically the image size depends on image itself, we cannot set
a value for that.
If I'm wrong please tell me.

In this case, I will be confused by "If the image is larger, the window
is looking at comparatively a smaller region"
I'm enlarging the window, isn't it?

"If the image is larger" means somehow we are able to enlarge the
original image.
If this is true, than I just want the size to be the original size of
image, what should I do?

Albert Cardona a écrit :

> Hi Xin,
>> I don't understand why if I enlarger the maximum image size, I'll
>> discard the larger features ....
>> I presume that the larger features are for the larger scale, right?
>> Then larger scale can only be detected in larger window.
> Exactly the opposite. Keep in mind that features are found by moving
> around a fixed size NxN pixel window. If the image is larger, the
> window is looking at comparatively a smaller region of the overall
> image, and thus it's extracting small features.
>
> Hope that helps,
>
> Albert
>

[Stephan Saalfeld]

Hi Xin,

SIFT interest points are blobs of arbitrary size.  They are detected by
searching the whole scale space of the image for such points.  You can
think about the scale space as shrinking the image continuously.  All
blob like structures have at some point the size of exactly one pixel
and this is where they are detected.  Obviously it doesn't make sense to
shrink the image smaller than 1 pixel, so the natural limits of the
scale space of an image are the original pixel resolution as the
"smallest scale" and the whole image in 1 pixel as the "largest scale".

That is, allowing some minimal image size larger than 1 pixel would not
detect blobs that are still larger than 1 pixel at this resolution.
Limiting the maximal image size will not detect blobs that are already
smaller than one pixel at this resolution.  The explanation at Albert's
page is correct, but here we have a different formulation :)

In detail---the "shrinkage" is "simulated" by Gaussian smoothing and
actual resizing is only performed at doubling of the Gaussian kernel
size.  This means that only image sizes of the original image size
divided by a power of two will occur.  The "shrinking" between these
steps are in the scale octaves whereas each octave has the mentioned
number of steps.  If you are interested in more detail, you can have a
look into the original paper of the inventor of SIFT, David Lowe, which
is very nice to read:

http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf

or have a look at this figure where two exemplary scale octaves are
shown:

http://fly.mpi-cbg.de/~saalfeld/scale_octaves.png

I beg your pardon for not having a comprehensive documentation written
yet.  This will happen at some point---I promise!

If you are interested, you could have a look at fiji:

http://pacific.mpi-cbg.de/wiki/index.php/Main_Page

It contains a slightly more mature implementation and some plugins for
further use of the detection results in ImageJ.

Best regards,
Stephan


On Tue, 2008-09-30 at 16:57 +0200, Xin ZHOU wrote:

> Hello, everybody,
>
> Is there anybody who knows a lot about the Java SIFT plugin provided by
> Stephan Saalfeld <mailto:[hidden email]>?
> I also send this message to Stephan Saalfeld
> <mailto:[hidden email]> but I need some quick answers to make my
> project move.
>
> I'm using the SIFT plugin to extract the salient points and I played
> with the parameters.
>
> In the "Java SIFT parameters explanation.doc" I find some inconsistency.
> *        Steps per scale octave*: Keypoint candidates are extracted at
> all scales between /maximum image size/ and /minimum image size/.
>
>     * *Minimum image size*: The Scale Space stops if the size of the
>       octave would be smaller than /minimum image size/. *Tip*: Increase
>       the minimum size to discard large features (i.e. those extracted
>       from looking at an image from far, such as the overall shape).
>     * *Maximum image size*: The Scale Space starts with the first octave
>       equal or smaller than the /maximum image size/. *Tip*: By reducing
>       the size, fine scaled features will be discarded, favoring larger
>       features. Larger features are more likely to remain constant
>       across serial sections. Increasing the size beyond that of the
>       actual images has no effect.
>
> I don't understand why if I enlarger the maximum image size, I'll
> discard the larger features ....
> I presume that the larger features are for the larger scale, right?
> Then larger scale can only be detected in larger window.
>
> I think I just badly understand these two parameters.
>
> Can anyone explain a little bit?
>
> cheers, Xin
>
>

[Stephan Saalfeld]

Hi again,

short answer: for getting only larger features reduce the maximum image
size, for getting only small features increase the minimum image size.

Best regards,
Stephan


On Tue, 2008-09-30 at 16:59 +0200, Xin ZHOU wrote:

> Hello,
>
> Maybe I just make it to be precise in discussion.
> For example, I want larger features, which means those show the
> salient
> point in a "low resolution".
> How should I set the max and min image size value?
>
> cheers, Xin
>
> Xin ZHOU a écrit :
> > Hello, everybody,
> >
> > Is there anybody who knows a lot about the Java SIFT plugin
> provided
> > by Stephan Saalfeld <mailto:[hidden email]>?
> > I also send this message to Stephan Saalfeld
> > <mailto:[hidden email]> but I need some quick answers to make
> my
> > project move.
> >
> > I'm using the SIFT plugin to extract the salient points and I
> played
> > with the parameters.
> >
> > In the "Java SIFT parameters explanation.doc" I find some
> inconsistency.
> > *        Steps per scale octave*: Keypoint candidates are extracted
> at
> > all scales between /maximum image size/ and /minimum image size/.
> >
> >    * *Minimum image size*: The Scale Space stops if the size of the
> >      octave would be smaller than /minimum image size/. *Tip*:
> Increase
> >      the minimum size to discard large features (i.e. those
> extracted
> >      from looking at an image from far, such as the overall shape).
> >    * *Maximum image size*: The Scale Space starts with the first
> octave
> >      equal or smaller than the /maximum image size/. *Tip*: By
> reducing
> >      the size, fine scaled features will be discarded, favoring
> larger
> >      features. Larger features are more likely to remain constant
> >      across serial sections. Increasing the size beyond that of the
> >      actual images has no effect.
> >
> > I don't understand why if I enlarger the maximum image size, I'll
> > discard the larger features ....
> > I presume that the larger features are for the larger scale, right?
> > Then larger scale can only be detected in larger window.
> >
> > I think I just badly understand these two parameters.
> >
> > Can anyone explain a little bit?
> >
> > cheers, Xin
> >
> >
> >
>

[Xin ZHOU]

Hello, Stephan,

Thanks for all your answers.
The article I've already red but I myself feel that I didn't understand
it totally.
This is the reason why I have still following questions.

Q1:
I'm actually working on a project which use the salient points detector
based on wavelet transformation.

My question comes from the fact that now I want to use SIFT detector and
I just wonder what will be the difference between the these two.
I'm only talking about detection, not the alignment, so no RASNAN
transformation to reduce the points.

I explain :
Salient points detector based on wavelet transformation is doing the
wavelet transformation with multi-resolution.
SIFT for me is something that using the DoG for different resolution and
detect salient points.
And DoG is very similar to the Mexical hat wavelet,
*so can we say that SIFT is just a approximatively the same thing that
if I use a Mexical hat wavelet to detect the salient points? *

I mean, *only for the detection*, *is there any difference?* or big
difference? (For example, the 500 first salient points, will they be in
the same position?)

That's why I tried your SIFT detector. And in reality it seems to be a
difference. I think it is a parameter setting problem, because for
wavelet, it performs on scale T, and the DoG, in fact it performs on a
resolution which is the difference of two octaves (say T0-T1).

I try to make T0-T1=T, and then the result should be similar, no?

Q2 :
How did SIFT sort the features? In the result, it seems that it sorts
the features by scale.
Does this mean that larger features first and then small features?
(means salient points in low frequency domain and then high frequency
domain)

Any help will be appreciated. Especially for the difference between
these two detectors.

cheers, Xin


Stephan Saalfeld a écrit :

> Hi Xin,
>
> SIFT interest points are blobs of arbitrary size.  They are detected by
> searching the whole scale space of the image for such points.  You can
> think about the scale space as shrinking the image continuously.  All
> blob like structures have at some point the size of exactly one pixel
> and this is where they are detected.  Obviously it doesn't make sense to
> shrink the image smaller than 1 pixel, so the natural limits of the
> scale space of an image are the original pixel resolution as the
> "smallest scale" and the whole image in 1 pixel as the "largest scale".
>
> That is, allowing some minimal image size larger than 1 pixel would not
> detect blobs that are still larger than 1 pixel at this resolution.
> Limiting the maximal image size will not detect blobs that are already
> smaller than one pixel at this resolution.  The explanation at Albert's
> page is correct, but here we have a different formulation :)
>
> In detail---the "shrinkage" is "simulated" by Gaussian smoothing and
> actual resizing is only performed at doubling of the Gaussian kernel
> size.  This means that only image sizes of the original image size
> divided by a power of two will occur.  The "shrinking" between these
> steps are in the scale octaves whereas each octave has the mentioned
> number of steps.  If you are interested in more detail, you can have a
> look into the original paper of the inventor of SIFT, David Lowe, which
> is very nice to read:
>
> http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf
>
> or have a look at this figure where two exemplary scale octaves are
> shown:
>
> http://fly.mpi-cbg.de/~saalfeld/scale_octaves.png
>
> I beg your pardon for not having a comprehensive documentation written
> yet.  This will happen at some point---I promise!
>
> If you are interested, you could have a look at fiji:
>
> http://pacific.mpi-cbg.de/wiki/index.php/Main_Page
>
> It contains a slightly more mature implementation and some plugins for
> further use of the detection results in ImageJ.
>
> Best regards,
> Stephan
>
>  

[Stephan Saalfeld]

Hi Xin,

> And DoG is very similar to the Mexical hat wavelet,

Yes---both approximate a Laplacian of Gaussian.

> *so can we say that SIFT is just a approximatively the same thing that
> if I use a Mexical hat wavelet to detect the salient points? *
>

Depends on if you compare across scale too.  The DoG detector gives you
points that are salient in x,y, scale, which means that they have a
specific location and size.  Furthermore, in the DoG detector proposed
by Lowe, the detections are localized with subpixel accuracy in x,y and
scale.

> I mean, *only for the detection*, *is there any difference?* or big
> difference? (For example, the 500 first salient points, will they be in
> the same position?)
>

That depends very much on implementation details of the detector.  Do
you compare across scales or not?  Do you filter detections by some
criteria?  How do you handle the border region of the image?  How
accurate is your kernel function?  How was your image resized?  In which
value range do you work? ...

> That's why I tried your SIFT detector. And in reality it seems to be a
> difference. I think it is a parameter setting problem, because for
> wavelet, it performs on scale T, and the DoG, in fact it performs on a
> resolution which is the difference of two octaves (say T0-T1).
>
> I try to make T0-T1=T, and then the result should be similar, no?
>

If you adapt all the remaining free parameters of the implementation
too---could be so :)

Actually---for comparison of SIFT against other techniques I suggest to
use the original implementation of Lowe, this is the reference.
Alternatively, Andrea Vedaldi has free implementations of SIFT in Matlab
and C++, that were designed to be exactly compatible to this reference:

http://vision.ucla.edu/~vedaldi/

In my implementation, I did not care about this level of compatibility.
I was interested in having it running in principle for use in our
registration attempts.

> Q2 :
> How did SIFT sort the features? In the result, it seems that it sorts
> the features by scale.
> Does this mean that larger features first and then small features?
> (means salient points in low frequency domain and then high frequency
> domain)
>

I sort them by size, because this helped matching the features in lack
of scale difference.  But for your comparison as I understand it, you
need the results of the DoG detector only which is totally independent
from the SIFT descriptor extraction.

Best regards,
Stephan