Could you please consider helping us with this project....

[EProce]

Dear Sir and Madam,
 
We are in need of software which technical
requirements are outlined below. We also hoped that
most of the software's functionality we need might
have possibly been implemented in ImageJ.
 
I have recently contacted few people on this mail
list. Dr. Dimeter Prodanov has kindly reviewed our
technical requirements and advised that, in his
opinion, the only potential problem lies in
establishing correlation between the same objects
captured by three cameras (please see below). Please
also see my comments on this matter. Dr. Michael
Schmid kindly advised to post this message. Dr. Walter
Odell kindly offered his comments. However, nobody of
those few people I have contacted has agreed to help
us with this development. In order to prevent
duplication of efforts I will let you know if somebody
will agree to help us. Moreover, it would be great if
this software cold become of interest of more than one
person.
 
This project is a continuation of research conducted
by Ottawa U group within a project: "Lake Sediment
Structure and Evolution"
http://www.science.uottawa.ca/LSSE/summary.html
This project has not been funded as of more than one
year ago. However, some scientists believe that it
makes sense to continue this research. If successful,
the research being conducted could possibly be
commercialized. Therefore, currently the research is
not commercially paid for. Some participants of the
research are personally contributing funds towards
continuation of this development. If required, I will
connect you to the professor leading this development.
 
Talking about the application environment, the final
application will focus on studying growth of fish in a
semi-isolated area of a lake. As a preliminary step,
at some point we will make images of fish in an
aquarium. If you believe that you strongly need those
images for your considerations please let us know
(please note that we will have to remove fish from the
aquarium to image the background and then place it
back in order to make those images). I also would like
to mention that this software could be very useful in
many other applications.
 
Should you or one of your colleagues are interested in
helping us to develop this software on a
freelance/contract basis please kindly provide me with
the preliminary quotation (unless you are willing to
help us on a voluntarily basis).
 
 
I look forward to hearing from you.
 
Sincerely,
Ema Proce.
              Software Technical Requirements:
 
Introduction: You have three 2-d black&white images of
a group of objects. The objects are of random
dimensions and are randomly oriented in 3-d. The
images have been taken simultaneously with three
cameras producing the front, the side and the top
views of this group of the objects.
 
Software to be developed must do the following:
 
a1. Take three base-line images (with three cameras)
before objects are located in the field of view of the
cameras (UI – a pushbutton, also, alternatively, a
DOS-based command – example “XXX.exe /abc”). This
requirement is optional (please see below)
 
a2. Automatically store the mentioned above three
base-line images on a computer’s hard drive. This
requirement is optional (please see below)
 
a3. Retrieve stored images and Apply image masking
technique pixel-by-pixel to remove artifacts from the
basic images and memorize the gray-level of every
pixel for these three basic images by creating three
mask-images
 
a4. Store these three mask-images
 
a5. Then, after user places a group of objects such
that those object appear in the field of view of the
mentioned above three cameras, the user clicks another
button (also, alternatively, after user enter a
DOS-based command) such that:
           a5.1 Software takes simultaneously three
images of these objects (optionally, software will
retrieved these images from a known directory if the
images have been made and stored by another software,
for example, a standard software which comes with
cameras), then applies mask images respectively to
subtract artifacts, and then applies threshold such
that there is a certain amount of noise left. Noise is
calculated as the number of single-pixel events left
on the image (Through UI User defines that after the
mask images are applied and the threshold is applied
to all three derived images, the counted number of
single-pixels “objects” on each of three resulting
images image must be less than, let say, 1000 per
frame)
           a5.2 Software automatically establishes
relationships between those three images such as the
object number 1 on the front-view image would be
correlated to the same object on the side view and the
top view images respectively. Then software calculates
width, height, and depth of the object, and then
multiplying those pixels counts to produce volumes of
every of these object. Note that we need three cameras
only because we might have a situation when one object
is located, let say, besides another object and the
objects are of different sizes, and yet a smaller
object could be in a shadow of a larger object. If we
had only two cameras (although every camera produces a
2–d image), we could miscalculate the objects’
volumes. Note that software must properly handle
different situations which might occur with objects’
orientation and possible overlapping.
 
a6. Measured volumes of objects have to be presented
in the UI as a table such that user can define the
ranges of object volumes (let say, 0 to 5 cm^3, 5.1 to
7 cm^3, etc), and the number of objects counted within
the certain range should fell into the table. In order
to do that the UI must provide user a text box to
enter the pixel size. For example, a user might need
to define that one pixel corresponds 2.5 mm width and
3 mm height of an actual object, based on the optics
attached to digital cameras.
 
a.7 Also, the software must create an excel file with
the record of width, height, depth, and the volumes of
each objects. Software must also generate a text file
with indication of date and time of all measurements
and the related files locations on the PC.
 
a8. Finally, UI must allow user to repeat the entire
process of making pictures of the objects using the
same basic images and mask-images, and then conducting
the mentioned above calculations, as well as an option
of resetting the software such that basic images are
re-taken and the new mask images are created. User
must also have an option of deleting all old files.
 
Talking about the application environment, the final
application will focus on studying growth of fish in a
semi-isolated area of a lake. As a preliminary step,
at some point we will make images of fish in an
aquarium. If you believe that you strongly need those
images for your considerations please let us know
(please note that we will have to remove fish from the
aquarium to image the background and then place it
back in order to make those images).
 


Discussion on establishing correlation between the
same objects captured by three cameras:
 
 An example of the challenges related to this
development. It's just an example but a good one:
Let's assume that you have several objects which are
swimming your aquarium: let say, one small fish, 3
medium-size fishes, and 4 large fishes.
 
You have three cameras to capture the front view, the
side view, and the top view of the aquarium.
 
Our goal is to calculate volumes of all these objects.
 
Below I would like to share with you ideas of our
group on this matter:
1. To start with, we can use three cameras and their
software and hardware to capture B&W and (optionally)
RGB images simultaneously taken by these three
cameras. The images will be stored in the specific
folder. The image format will be 8 (to start with),
and also 12 and 16 bit TIFF. The software to be
developed will be running on WinXP platform (+Java,
when required). Therefore we will not need to analyze
images in real time while objects move.
2. On establishing the correlation between the same
objects captured by three cameras, we have been
thinking about possible complications. We think it
will not be a problem for the following reasons:
a. All three cameras are permanently mounted at 90
degree angle (front view camera F, side view camera S
and top view camera T).
b. We can have many objects captured at the same time.
In order to clarify three cameras matter let's assume,
to begin with, that we have just one object captured.
Let’s also assume a simple scenario where the object
is just a rectangular parallelepiped. The captured
object has width X, height Y, and depth Z.
c. Let’s assume that the front surface of this object
is located parallel to the front view camera.
d. Front view camera F captures the object such that
horizontal x and vertical y coordinates of the image
taken by camera F allows to calculate X and Y of the
object: F(x2 - x1) = X; F(y2 -y1) = Y
e. The image of the same object taken by the camera S
will produce the following equations: S(x2 - x1) = Z;
S(y2 - y1) = Y.
f. The image of the same object taken by the camera T
will produce the following equations: T(x2 - x1) = X;
T(y2 - y1) = Z.
 
g. Also, our object may, for example, appear in the
middle of the camera F, but (let say) near the right
side of the image taken by camera S. Therefore,
generically saying:              Tx1 is not equal to
Fx1; Ty1 is not equal to Sx1; Fy1 is not equal to Sy1;
Tx2 is not equal to Fx2; Ty2 is not equal to Sx2; Fy2
is not equal to Sy2;  This problem can be easily
solved if you place one time a phantom with known
dimensions, make three images: F, S, and T, and then
calculate permanent shift of horizontal and vertical
coordinates: a, b, and c between three cameras:
a = Tx1 - Fx1; b = Ty1 - Sx1; c = Fy1 - Sy1;
 
h. After you place the phantom you can one time
manually identify pixels numbers for S, T, and F
images for the same object and via UI in the software
we are trying to develop use can enter constants a, b,
and c.
 
i. Also, because three cameras do not have identical
magnification, before you proceed as below, after
images with phantom are taken, you can manually (just
one time) calculate magnification/demagnification
constants for two of three images. Let say, you
started form camera F. You made an image F and then
measured width of the phantom in pixels on the image
of the object and you also measured the object itself
with a caliber. Then you take images S and T and
manually define magnification/demagnification
coefficients Sk and Tk. Then you manually enter Sk and
Tk into the software. Then before proceeding as below,
you automatically magnify (demagnify) images S and T.
 
j. Now the process of correlating the same object
between three images becomes easy:
 
J1. You make three images: F, S, and T
J2. You subtract background images Fb, Tb, and Sb
taking ahead of time with no object inserted. Then you
apply threshold in order to being able to identify our
object on all three images. Note: in order to
normalize threshold for all three images you can, for
example, first adjust for different illumination by
comparing pixels grey-level of all three images, and
then calculate noises by counting number of single
pixel on all three images such after you apply
threshold the noises on all three images will become
the same.
J3. Now you scan all three images pixel-by-pixel and
line-by line in order to identify the object, such
that as soon as you find, let say, Tx1 you can
immediately identify Fx1; as soon as you identify Ty1
you can identify Sx1, and then you scan, let say side
view image S and identify Sy1 (because you already
know Sx1) and then you can easily identify Fy1. This
way you can correlate the same object on all three
images.
 
Finally, optimization of volume V of the object
becomes also easy. Ideally, V = F(x2 - x1) * F(y2 -
y1)*S(x2-x1) = F(x2 - x1) * S(y2 - y1)*S(x2-x1) = T(x2
- x1) * F(y2 - y1)*S(x2-x1), etc. You have 8 possible
combinations of calculating volume of the same object
obtained from three cameras. In real environment,
because t object may be tilted towards all three
cameras, the estimated volume could be chosen, for
example, as the smallest result obtained within those
8 possible combinations. Better result of calculation
the volume gives minimum of three combinations of
calculating the number of pixels corresponding the
same object taking by one camera and then multiplying
by the max (or average) of the third coordinate. For
example, camera F captured the object. Number of
pixels corresponding the object captured by F
represents the area of the object in the field (Fx *
Fy). Then if you multiply this number of pixel to
MAX((S(x2 - x1)) you will calculate V of the object
under the assumption that the object has the same
depth Sx in all different locations of the object.
 
Ideally, we also have to handle situations when
because you have several objects captured at the same
time (in real environment), some of them may be
partially or completely overlapped.
 
We are also very limited in the size, shape and
configuration of the 3 objects for which a solution
exists.  For example, we could never deduce the object
as being a cup from an object that is a solid cylinder
-- we could never detect a concavity.   However, in
our application the above limitations are acceptable.
It is sufficient to measure volume of objects just
based on their projections to three cameras.

***
I would also like to address financial matters.
Notwithstanding financial situation of our group, we
understand that your help and/or help of your
colleagues needs to be financially compensated, unless
you find it possible to help us on a voluntarily
basis. At the same time, we have to be sure that we
can afford paying for your work. Therefore, before you
spend your time on helping us with software
development, could you please let me know your
financial requirements, if any?
 
In the end of this message I would like to thank you
all for your time and attention. I look forward to
hearing from you.
 
Sincerely,
Ema Proce.


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

[Mikhail Umorin]

Ema --

Identifying fish and estimating their volumes seems to be more a AI
(Artificial Intelligence) problem then IA (Image Analysis) problem.

I think, the problem can be formalized as follows:

you have a 3-D density distribution and you have three 2-D projections of this
3-D volume distribution. You want to recreate the 3-D density from the three
2-D projections.

The problem is not solvable in such general case. The problem is not solvable
even if you had marginal 2-D distributions which carry more information than
projections (I define here that marginal distribution gives you a
cumulative/integrated density along a projection path while projection simply
gives yes/no if there is a non-zero density along a projection path).

The example you give uses very strong assumptions:
1) the shape of objects (rectangular parallelepiped);
2) the number of objects (1);

Firstly, even if the object looks square on projections that does not mean
it's a cube. The three projection images you have simply cannot tell you the
shape. In fact, you know that your fish are not rectangular parallelepipeds
which are defined by 3 numbers. You will have to assume the shape of the
objects and then calculate their volumes based on 1, 2, or 3 measurements for
each object (say max length in each dimension). You may be able to assume the
shapes based, say, on the species of fish which will have to be determined
from the images. This may be a very complicated problem in itself.

Secondly, the number of objects cannot be generally determined from three
projections you have. Consider 8 objects (same shape) placed at the vertices
of a cube whose sides a parallel to the projection axis. You can take out any
one or any two diagonally opposite objects and your projections will not show
the difference. You will have to estimate the number of objects on the images
from some other information or assume that there is no triple-overlap, i.e.
that an object does not overlap with others on all three projections. (I am
not sure about the double-overlap -- it may be solvable but may be very
tough). Assumption of small number of objects (<?) may also help.

So, the bottom line is: you can reconstruct a 3-D picture only if you have an
MRI of a volume, projections won't do. To estimate the number of objects you
will have to make assumptions that may or may not hold true. To estimate the
volume of each object you will have to assume its shape and you may not have
sufficient information to do that. So, the problem may be solvable in
specific instances (which, fortunately, you may just pick from the total set
you have), but not in general.  

Good luck!

Mikhail.

[Mikhail Umorin]

To clarify --
> So, the bottom line is: you can reconstruct a 3-D picture only if you have
> an MRI of a volume, projections won't do.
 I meant the just the three projections you have

But, thinking further, may be you can do something like a tomography imaging
by using multiple cameras arranged in a circle looking inwards. Essentially
taking pictures of a cylindrical piece of volume from all around the curved
side. Taking multiple  (> 3) projections will give you better chances at
resolving the number of objects and their shapes. Tomographic methods are
well developed. You have a much better chances here.

Mikhail.