Re: Parallel image processing using ImageJ in the Cloud

Posted by Jimmy Su on Apr 07, 2011; 3:20pm
URL: http://imagej.273.s1.nabble.com/Parallel-image-processing-using-ImageJ-in-the-Cloud-tp3685113p3685121.html

On Wed, Apr 6, 2011 at 1:18 PM, Dean Kossives
<[hidden email]> wrote:
> Every now and then something comes along that catches my eye. A
> reduction in processing time from 5 hours to 15 min is great.
>
> 1) can you get the processing time down to under 1 minute?
>

The short answer is yes.  It would require throwing more processors at
the problem.  This would do the trick if applying this set of
operations to a single image would take under one minute.  Otherwise,
we would need to further parallelize the problem by using multiple
processors/cores on each image.

Jimmy

> Dean P. Kossives
> Principal Engineer
> Clear Align
> 2550 Boulevard of the Generals, Suite 280
> Eagleville, PA 19403
> phone [phone] X185
> fax  [phone]
> mailto:[hidden email]
> www.clearalign.com
>
> Clear Align is certified as a SB, 8(a) SDB & WOSB.
> See us at SPIE Defense, Security, and Sensing (Booth 1017), April 25-29,
> 2011
>
> This message and any attachments are solely for the use of the
> individual or entity to which it is addressed and may contain
> information that is privileged or confidential.  If you are not the
> intended recipient, any disclosure, use or distribution of the
> information contained herein is prohibited. If you have received this
> communication in error, please notify the sender by reply e-mail and
> immediately delete this message and any attachments.  In the event this
> document(s) contains technical data within the definition of the
> International Traffic in Arms Regulations, it is subject to the export
> control laws of the U.S. Government.  Transfer of this data by any means
> to a foreign person, whether in the United States or abroad, without an
> export license or other approval from the U.S. Department of State, is
> prohibited.
>
>  P Please consider your environmental responsibilities before printing
> this e-mail
>
>
> -----Original Message-----
> From: ImageJ Interest Group [mailto:[hidden email]] On Behalf Of
> Johannes Schindelin
> Sent: Wednesday, April 06, 2011 6:25 AM
> To: [hidden email]
> Subject: Re: Parallel image processing using ImageJ in the Cloud
>
> Hi Jimmy,
>
> On Mon, 4 Apr 2011, Jimmy Su wrote:
>
>> We recently completed a Phase 1 SBIR project with OSD on the topic of
>> analytic tools in the Cloud.
>
> I only understood half the words in that sentence, but I'm quite used to
>
> that :-)
>
>> To demonstrate our tool's ability to construct image processing
> workflow
>> and deploy the generated code to the Cloud, we took ImageJ and added
>> some Cloud processing capabilities by using the MapReduce framework.
>
> What exactly did you do in terms of image processing? Some Gaussian
> Blur,
> or Find Edges, or some advanced plug-in? From a technical point of view,
>
> there are huge differences there.
>
>> We added Cloud processing capability to ImageJ by adding a Hadoop
>> InputFormat to handle image types in HDFS (Hadoop Distributed File
>> System) and encapsulating ImageJ operations in map and reduce methods.
>
> That is _very_ interesting. For a long time I have been wanting to play
> with Hadoop now.
>
>> This significantly increases ImageJ's throughput in processing
>> images.  Attached is the running time chart showing processing time
>> decreases from over 5 hours on two nodes to 15 minutes on 64 nodes on
>> Amazon EC2.  Are there any interests in the ImageJ community for
>> parallel processing in the Cloud?  What kind of applications are you
>> developing that needs ImageJ processing in the Cloud?  We would love
>> to hear your feedback.
>
> Two feedbacks from my side:
>
> 1) fantastic!
> 2) where can I get it?
>
> Ciao,
> Johannes
>