Avisynth and ImageJ

[Holger Goerlitz]

Hi all,

I am establishing a 3D animal tracking project using video recordings in
mpg2 formate. mpg2 streams seem to be tricky, but I have to use them...
For this project, the position of the animals should be tracked by a
user from frame to frame. Several tracking plugins exist for ImageJ; I
was using MTrackJ for images so far. However, now I would like to be
able to track the position not on subsequent images, but I would like to
use directly the video-files.

ImageJ allows to open avi-files via AVI Reader plugin, but not
mpg2-files. However, there exists a very powerful video-editing software
called Avisynth which can read a variety of video-files, process and
edit them, and serve them to any other program, e.g. your favourite
media player. Effectively, what you do is to open an Avisynth file
(*.avs) in your media player. This *.avs-file only contains a few very
simple lines of code containing the video-file name and all relevant
video editing steps (e.g. deinterlacing, bobbing, cropping,
contrast-adjustment,...). When opening this *.avs-file, Avisynth will in
real-time process the video-stream and serve the video frame by frame to
your media player, displaying you the processed video. The big advantage
is that you can write one *.avs-file and easily edit each video, while
watching or analysing them, without the need to do this manually for
every single video.

I was wondering whether anybody knows of plugins for ImageJ that allow
to open such *.avs-files of Avisynth. Effectively, this would give
ImageJ the capability to process all types of video-files that can be
read by Avisynth, including all editing capability of Avisynth
(difference frames, colour -, contrast -, luminance adjustment,... -
simply everything Avisynth can do). I think that this would be an
incredibly powerful tool for ImageJ, but I haven't found anything about
it anywhere.

I would be most grateful for any ideas, suggestions and help.

I have never ever written any plugins for ImageJ (and don't yet know how
to do it), but was wondering it it would be worth doing that?

Best,
Holger



--
Dr. Holger R. Goerlitz

School of Biological Sciences
University of Bristol
Woodland Road
Bristol BS8 1UG
UK

phone: [phone]
fax: [phone]
[hidden email]

[Michael Schmid]

Hi Holger,

ImageJ can easily do many of the processing steps that you can  
specify in an AVS file. So the question is how to get your files into  
ImageJ.

As I understand it, the output of Avisynth is Video for Windows, so  
you need some software that can read Video for Windows. I am not  
aware of any ImageJ plugin that can do this (ImageJ reads AVI files  
without using any Windows-specific routines - ImageJ  works on many  
operating systems that don't have these routines).

ImageJ does not interface to Windows codecs, so also the option of a  
"fake AVI file" mentioned in the Avisynth FAQ won't work. Again, I am  
not aware of any ImageJ plugin that could do this.

As far as I could find out, Avisynth can output to VirtualDub; so you  
could have VirtualDub write an uncompressed avi format that ImageJ  
can read. This will create very large files, however.

Currently, ImageJ cannot read YV12 video, which seems to be the  
output of Avisynth without applying any format conversion. If  
VirtualDub cannot convert to a raw format different from YV12, adding  
YV12 support would not be too difficult.

Michael
________________________________________________________________

On 11 Dec 2008, at 16:05, Holger Goerlitz wrote:

> Hi all,
>
> I am establishing a 3D animal tracking project using video  
> recordings in mpg2 formate. mpg2 streams seem to be tricky, but I  
> have to use them... For this project, the position of the animals  
> should be tracked by a user from frame to frame. Several tracking  
> plugins exist for ImageJ; I was using MTrackJ for images so far.  
> However, now I would like to be able to track the position not on  
> subsequent images, but I would like to use directly the video-files.
>
> ImageJ allows to open avi-files via AVI Reader plugin, but not mpg2-
> files. However, there exists a very powerful video-editing software  
> called Avisynth which can read a variety of video-files, process  
> and edit them, and serve them to any other program, e.g. your  
> favourite media player. Effectively, what you do is to open an  
> Avisynth file (*.avs) in your media player. This *.avs-file only  
> contains a few very simple lines of code containing the video-file  
> name and all relevant video editing steps (e.g. deinterlacing,  
> bobbing, cropping, contrast-adjustment,...). When opening this  
> *.avs-file, Avisynth will in real-time process the video-stream and  
> serve the video frame by frame to your media player, displaying you  
> the processed video. The big advantage is that you can write one  
> *.avs-file and easily edit each video, while watching or analysing  
> them, without the need to do this manually for every single video.
>
> I was wondering whether anybody knows of plugins for ImageJ that  
> allow to open such *.avs-files of Avisynth. Effectively, this would  
> give ImageJ the capability to process all types of video-files that  
> can be read by Avisynth, including all editing capability of  
> Avisynth (difference frames, colour -, contrast -, luminance  
> adjustment,... - simply everything Avisynth can do). I think that  
> this would be an incredibly powerful tool for ImageJ, but I haven't  
> found anything about it anywhere.
>
> I would be most grateful for any ideas, suggestions and help.
>
> I have never ever written any plugins for ImageJ (and don't yet  
> know how to do it), but was wondering it it would be worth doing that?
>
> Best,
> Holger
>
>
>
> --
> Dr. Holger R. Goerlitz
>
> School of Biological Sciences
> University of Bristol
> Woodland Road
> Bristol BS8 1UG
> UK
>
> phone: [phone]
> fax: [phone]
> [hidden email]

[Holger Goerlitz]

Hi Michael,

thanks for your help. You're right, many of the processing steps of
Avisynth can also be done in ImageJ; the biggest advantage of connecting
Avisynth to ImageJ would probably be the possibility to load different
video formates into ImageJ. However, as I understand you, using Avisynth
seems to be problematic as ImageJ is not using the Windows-specific
routines (which is actually a good thing...).

Saving *.avi-files via VirtualDub would easily work, but as you say,
creates very large files, which would be one of the problems I would
encounter with that approach.

I just found a program which can mount Avisynth *.avs-files into
*.avi-files, called AVFS (http://www.turtlewar.org/avfs/). I am still
about to test this.

Thanks for the hint with YV12 - I have to look into that. Probably
format conversion is possible in Avisynth (I'm not yet really experienced).

Holger



--
Dr. Holger R. Goerlitz

School of Biological Sciences
University of Bristol
Woodland Road
Bristol BS8 1UG
UK

phone: [phone]
fax: [phone]
[hidden email]

[Michael Schmid]

Hi Holger,

AVFS looks good, it might work.

Note, however, that ImageJ is restricted to AVI 1 (maximum size 2  
GB); for AVI 2 (which supports larger files) it will probably read a  
bit less than the first 1 GB. There was a recent thread in this  
mailinglist about extending ImageJ to read full AVI 2.

Michael
________________________________________________________________

On 11 Dec 2008, at 17:35, Holger Goerlitz wrote:

> Hi Michael,
>
> thanks for your help. You're right, many of the processing steps of  
> Avisynth can also be done in ImageJ; the biggest advantage of  
> connecting Avisynth to ImageJ would probably be the possibility to  
> load different video formates into ImageJ. However, as I understand  
> you, using Avisynth seems to be problematic as ImageJ is not using  
> the Windows-specific routines (which is actually a good thing...).
>
> Saving *.avi-files via VirtualDub would easily work, but as you  
> say, creates very large files, which would be one of the problems I  
> would encounter with that approach.
>
> I just found a program which can mount Avisynth *.avs-files into  
> *.avi-files, called AVFS (http://www.turtlewar.org/avfs/). I am  
> still about to test this.
>
> Thanks for the hint with YV12 - I have to look into that. Probably  
> format conversion is possible in Avisynth (I'm not yet really  
> experienced).
>
> Holger
>
>
>
> --
> Dr. Holger R. Goerlitz
>
> School of Biological Sciences
> University of Bristol
> Woodland Road
> Bristol BS8 1UG
> UK
>
> phone: [phone]
> fax: [phone]
> [hidden email]