Creating DataSet after building classification with Trainable Weka Segmentation

[marcelo_chong]

Subject: Creating DataSet after building classification with Trainable Weka
Segmentation
Hi,

I am new to imageJ, and I have 2 questions would like to ask

1. When i use imageJ Fiji with plugin in Trainable Weka Segmentation, it
only use one picture to define different class and build up a
classification. Can we use more pictures combine together to build up a
classification?

2. After I build up the classification, or I output as the data.arff. I took
a look in the arff and it has extract alot of features. Now, I would like to
take some new images, and test with the classification, but my images is in
jpg or other image format, with still havent extract the features yet. How
can i extract the same features as the previous "data.arff" and use it as my
Test Dataset ?

Please help, thanks a lot.

Regards,
Marcelo

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Ignacio Arganda-Carreras]

Hello Marcelo,

1. When i use imageJ Fiji with plugin in Trainable Weka Segmentation, it
> only use one picture to define different class and build up a
> classification. Can we use more pictures combine together to build up a
> classification?
>

Yes, you need to open them as a stack (File > Import > Image Sequence...)

>
> 2. After I build up the classification, or I output as the data.arff. I
> took
> a look in the arff and it has extract alot of features. Now, I would like
> to
> take some new images, and test with the classification, but my images is in
> jpg or other image format, with still havent extract the features yet. How
> can i extract the same features as the previous "data.arff" and use it as
> my
> Test Dataset ?
>

The ARFF file is meant to be used on either the main Weka GUI or in the
Trainable Weka Segmentation plugin to help in the segmentation of other
images.

If you don't change any of the settings, the same features are used every
time you call the plugin.

If what you want is to have an ARFF file for training and an ARFF for
testing, just call the plugin twice with two different images, trace some
samples of each class, save the feature data as ARFF with different names
and open them later in the Weka GUI.

Let me know if you need more help!

ignacio


>
> Please help, thanks a lot.
>
> Regards,
> Marcelo
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>



--
Ignacio Arganda-Carreras, Ph.D.
Seung's lab, 46-5065
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
43 Vassar St.
Cambridge, MA 02139
USA

Phone: [phone]
Website: http://bioweb.cnb.csic.es/~iarganda/index_EN.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[marcelo_chong]

Hi Ignacio,

Thanks for your reply, I tried immediately after reading your reply

1. For the stack image sequence, it works. Just one question abt it, do i have

to load in all the same dimension images? Coz some images are not in the same size.



2. This one I don't readily get it. First I train 15 images and it save to data-train.arff,. I check the arff file and it got 119 attributes with 4867 instances, which the last attribute is the classification. Then now, I got another 10 images, and I want to know if i use the previous training data to build up a Tree or whatever, will it correctly classification this 10 images. So first, I want to extract the 118 attributes(-1 for the classification attribute) from the 10 images. I open the 10 images in Fiji, and open the TWS, in the TWS, I "Load Data" and open the data-train.arff.  Then, what shall I do to turn the 10 images into 10 instances with 118 attribute ARFF file ?


thanks.

Regards,
Marcelo

[Ignacio Arganda-Carreras]

Hello again, Marcelo!


> 1. For the stack image sequence, it works. Just one question abt it, do i
> have to load in all the same dimension images? Coz some images are not in
> the
> same size.
>
>
Unfortunately the plugin only takes stacks, so all the images should be of
the same size.

A solution would be to

1. load one image and call the plugin
2. do the training and save the traces (as ARFF)
3. exit the plugin
4. load the second image and call the plugin
5. load the ARFF file you just save
6. train with the new traces

The old traces will be taken into account as soon as you load the ARFF file.

>
> 2. This one I don't readily get it. First I train 15 images and it save to
> data-train.arff,. I check the arff file and it got 119 attributes with 4867
> instances, which the last attribute is the classification. Then now, I got
> another 10 images, and I want to know if i use the previous training data
> to
> build up a Tree or whatever, will it correctly classification this 10
> images. So first, I want to extract the 118 attributes(-1 for the
> classification attribute) from the 10 images. I open the 10 images in Fiji,
> and open the TWS, in the TWS, I "Load Data" and open the data-train.arff.
> Then, what shall I do to turn the 10 images into 10 instances with 118
> attribute ARFF file ?
>

You almost got it! This is what you have to do:

1. save the information from the 15 images you loaded in the plugin as
data-train.arff
2. exit the plugin
3. open the test images as a stack and call the plugin
4. trace some pixels for all classes you have (you must have exactly the
same classes as before)
5. "save data" as "data-test.arff"
6. click on the Weka button
7. launch the Weka explorer
8. in the explorer, open the training arff file
9. click on classify and choose the classifier you want (the equivalent to
the plugin is a hr.irb.fastRandomForest.FastRandomForest with numTrees =
200 trees and numFeatures = 2).
10. select as test the data-test.arff you saved and run the classification

ignacio

>
>
> thanks.
>
> Regards,
> Marcelo
>
>
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Creating-DataSet-after-building-classification-with-Trainable-Weka-Segmentation-tp5009283p5009313.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
Ignacio Arganda-Carreras, Ph.D.
Seung's lab, 46-5065
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
43 Vassar St.
Cambridge, MA 02139
USA

Phone: [phone]
Website: http://bioweb.cnb.csic.es/~iarganda/index_EN.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[marcelo_chong]

Hello Ignacio,

Thanks for the reply again !

1. Solved, thanks a lot.



2. Sorry for my poor understanding. I still cannot success.

As I follow your steps:
------------------------------------------------------------------------

1. save the information from the 15 images you loaded in the plugin as
data-train.arff

DONE !
I got 4867 instances with 119 attributes  in the arff file.

2. exit the plugin

DONE !


3. open the test images as a stack and call the plugin

DONE !

4. trace some pixels for all classes you have (you must have exactly the
same classes as before)

As I have to extract the same features. So i need to LOAD the data-train.arff again, right?
I load the data-train.arff,  then I start to trace some pixels and in each images and classed it as A or B after the trace.


5. "save data" as "data-test.arff"

After trace all 10 images and make classification with the trace, I press save data and save as data-test.arff

Then the log is :
Creating feature stack...
Updating features of slice number 1...
Updating features of slice number 2...
Updating features of slice number 3...
Updating features of slice number 4...
Updating features of slice number 5...
Updating features of slice number 6...
Updating features of slice number 7...
Updating features of slice number 8...
Updating features of slice number 9...
Updating features of slice number 10...
Feature stack is now updated.
Training input:
# of pixels selected as class 1: 1772
# of pixels selected as class 2: 593
Merging data...
Finished: total number of instances = 7232
Writing training data: 7232 instances...

It created 7232 instance .....what i expect is 10 instance with 119 attributes.


6. click on the Weka button

Anyway, I still continue and try. I click on the weka button to load weka.

7. launch the Weka explorer

launch the explorer.


8. in the explorer, open the training arff file

open my data-trian.arff

9. click on classify and choose the classifier you want (the equivalent to
the plugin is a hr.irb.fastRandomForest.FastRandomForest with numTrees =
200 trees and numFeatures = 2).

done.

10. select as test the data-test.arff you saved and run the classification

the "Supplied test set", i choose my data-test.arff, and the outcome is :

=== Run information ===

Scheme:       weka.classifiers.trees.RandomForest -I 200 -K 2 -S 1 -num-slots 1
Relation:     segment
Instances:    4867
Attributes:   119
              [list of attributes omitted]
Test mode:    user supplied test set:  size unknown (reading incrementally)

=== Classifier model (full training set) ===

Random forest of 200 trees, each constructed while considering 2 random features.
Out of bag error: 0.0643



Time taken to build model: 4.98 seconds

=== Evaluation on test set ===
=== Summary ===

Correctly Classified Instances        6707               92.7406 %
Incorrectly Classified Instances       525                7.2594 %
Kappa statistic                          0.8363
Mean absolute error                      0.1242
Root mean squared error                  0.2294
Relative absolute error                 26.4056 %
Root relative squared error             47.8019 %
Coverage of cases (0.95 level)          99.8202 %
Mean rel. region size (0.95 level)      73.4098 %
Total Number of Instances             7232    

=== Detailed Accuracy By Class ===

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area   PRC Area  Class
                 0.976     0.162      0.918     0.976     0.946      0.978      0.986    class 1
                 0.838     0.024      0.949     0.838     0.89       0.978      0.967    class 2
Weighted Avg.    0.927     0.114      0.929     0.927     0.926      0.978      0.98

=== Confusion Matrix ===

    a    b   <-- classified as
 4579  114 |    a = class 1
  411 2128 |    b = class 2



The question is, is the above step correct ? Coz i want to have 10 instances to be test to see rather it is class 1 or class 2, but now I don't know how to interpret the result.

Or I did wrong with some steps?

Thanks.


Regards,
Marcelo

Ignacio Arganda-Carreras wrote

Hello again, Marcelo!


> 1. For the stack image sequence, it works. Just one question abt it, do i
> have to load in all the same dimension images? Coz some images are not in
> the
> same size.
>
>
Unfortunately the plugin only takes stacks, so all the images should be of
the same size.

A solution would be to

1. load one image and call the plugin
2. do the training and save the traces (as ARFF)
3. exit the plugin
4. load the second image and call the plugin
5. load the ARFF file you just save
6. train with the new traces

The old traces will be taken into account as soon as you load the ARFF file.


>
> 2. This one I don't readily get it. First I train 15 images and it save to
> data-train.arff,. I check the arff file and it got 119 attributes with 4867
> instances, which the last attribute is the classification. Then now, I got
> another 10 images, and I want to know if i use the previous training data
> to
> build up a Tree or whatever, will it correctly classification this 10
> images. So first, I want to extract the 118 attributes(-1 for the
> classification attribute) from the 10 images. I open the 10 images in Fiji,
> and open the TWS, in the TWS, I "Load Data" and open the data-train.arff.
> Then, what shall I do to turn the 10 images into 10 instances with 118
> attribute ARFF file ?
>

You almost got it! This is what you have to do:

1. save the information from the 15 images you loaded in the plugin as
data-train.arff
2. exit the plugin
3. open the test images as a stack and call the plugin
4. trace some pixels for all classes you have (you must have exactly the
same classes as before)
5. "save data" as "data-test.arff"
6. click on the Weka button
7. launch the Weka explorer
8. in the explorer, open the training arff file
9. click on classify and choose the classifier you want (the equivalent to
the plugin is a hr.irb.fastRandomForest.FastRandomForest with numTrees =
200 trees and numFeatures = 2).
10. select as test the data-test.arff you saved and run the classification

ignacio


>
>
> thanks.
>
> Regards,
> Marcelo
>
>
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Creating-DataSet-after-building-classification-with-Trainable-Weka-Segmentation-tp5009283p5009313.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>



--
Ignacio Arganda-Carreras, Ph.D.
Seung's lab, 46-5065
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
43 Vassar St.
Cambridge, MA 02139
USA

Phone: [phone]
Website: http://bioweb.cnb.csic.es/~iarganda/index_EN.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Ignacio Arganda-Carreras]

Hello again, Marcelo,


> 4. trace some pixels for all classes you have (you must have exactly the
> same classes as before)
>
> As I have to extract the same features. So i need to LOAD the
> data-train.arff again, right?
> I load the data-train.arff,  then I start to trace some pixels and in each
> images and classed it as A or B after the trace.
>

That's not correct. The ARFF file contains the feature values only for the
training traces. The features you use each time you call the plugin are
selected in the Settings menu. If you don't change them at all, the default
values are used (I think you are in this situation).

>
>
> 5. "save data" as "data-test.arff"
>
> After trace all 10 images and make classification with the trace, I press
> save data and save as data-test.arff
>
> Then the log is :
> Creating feature stack...
> Updating features of slice number 1...
> Updating features of slice number 2...
> Updating features of slice number 3...
> Updating features of slice number 4...
> Updating features of slice number 5...
> Updating features of slice number 6...
> Updating features of slice number 7...
> Updating features of slice number 8...
> Updating features of slice number 9...
> Updating features of slice number 10...
> Feature stack is now updated.
> Training input:
> # of pixels selected as class 1: 1772
> # of pixels selected as class 2: 593
> Merging data...
> Finished: total number of instances = 7232
> Writing training data: 7232 instances...
>
> It created 7232 instance .....what i expect is 10 instance with 119
> attributes.
>
>

OK, I see why you're confused. The number of instances is the *number of
pixels you traced not the number of input images*. In this case, the number
of training pixels (4867) plus the new test pixels you traced on the 10
test images.

>
> 6. click on the Weka button
>
> Anyway, I still continue and try. I click on the weka button to load weka.
>
> 7. launch the Weka explorer
>
> launch the explorer.
>
>
> 8. in the explorer, open the training arff file
>
> open my data-trian.arff
>
> 9. click on classify and choose the classifier you want (the equivalent to
> the plugin is a hr.irb.fastRandomForest.FastRandomForest with numTrees =
> 200 trees and numFeatures = 2).
>
> done.
>
> 10. select as test the data-test.arff you saved and run the classification
>
> the "Supplied test set", i choose my data-test.arff, and the outcome is :
>
> === Run information ===
>
> Scheme:       weka.classifiers.trees.RandomForest -I 200 -K 2 -S 1
> -num-slots 1
> Relation:     segment
> Instances:    4867
> Attributes:   119
>               [list of attributes omitted]
> Test mode:    user supplied test set:  size unknown (reading incrementally)
>
> === Classifier model (full training set) ===
>
> Random forest of 200 trees, each constructed while considering 2 random
> features.
> Out of bag error: 0.0643
>
>
>
> Time taken to build model: 4.98 seconds
>
> === Evaluation on test set ===
> === Summary ===
>
> Correctly Classified Instances        6707               92.7406 %
> Incorrectly Classified Instances       525                7.2594 %
> Kappa statistic                          0.8363
> Mean absolute error                      0.1242
> Root mean squared error                  0.2294
> Relative absolute error                 26.4056 %
> Root relative squared error             47.8019 %
> Coverage of cases (0.95 level)          99.8202 %
> Mean rel. region size (0.95 level)      73.4098 %
> Total Number of Instances             7232
>
> === Detailed Accuracy By Class ===
>
>                TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area
> PRC Area  Class
>                  0.976     0.162      0.918     0.976     0.946      0.978
> 0.986    class 1
>                  0.838     0.024      0.949     0.838     0.89       0.978
> 0.967    class 2
> Weighted Avg.    0.927     0.114      0.929     0.927     0.926      0.978
> 0.98
>
> === Confusion Matrix ===
>
>     a    b   <-- classified as
>  4579  114 |    a = class 1
>   411 2128 |    b = class 2
>
>

By doing this, you're using 4867 samples for training and 7232 for testing,
but those 7232 include the 4867, which is not correct.


>
>
> The question is, is the above step correct ? Coz i want to have 10
> instances
> to be test to see rather it is class 1 or class 2, but now I don't know how
> to interpret the result.
>

You need to understand that you're classifying pixels not whole images. So
the number of test instances is not 10, but the number of pixels you traced
on them. Does it make sense, now?

ignacio


--
Ignacio Arganda-Carreras, Ph.D.
Seung's lab, 46-5065
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
43 Vassar St.
Cambridge, MA 02139
USA

Phone: [phone]
Website: http://bioweb.cnb.csic.es/~iarganda/index_EN.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[marcelo_chong]

Hello Ignacio,

Thanks for the reply again.

I understand now about the instance it create after i load in the 10 images.

thanks for the explanation.

So i would like to ask, hope you won't mind :P

If I want to use TWS with Fiji to build a training arff dataset with a set of images

so i can build up a classification model in weka

Then i want to use it to classify some images, let say Lion and other animals.

So am i work on the correct way with TWS now, or i go the wrong direction?

coz now even I load in 10 images and create about 3000 instances for test. But I don't know

which image is classify as LION or not LION, coz it is in pixel level.

Sorry a little bit out of the scope.

Thx.

[Ignacio Arganda-Carreras]

OK, I see what you're trying to do now.

TWS is designed for pixel classification, not image classification. For the
latest, you will need to create your own plugin and features. The idea
would be very similar to TWS but you'll need a single final class per image
and features that are image-based.

You can also use existing software such as PiXiT:
http://www.montefiore.ulg.ac.be/~maree/pixit.html

Best,

ignacio


On Thu, Aug 28, 2014 at 11:34 AM, marcelo_chong <[hidden email]> wrote:

> Hello Ignacio,
>
> Thanks for the reply again.
>
> I understand now about the instance it create after i load in the 10
> images.
>
> thanks for the explanation.
>
> So i would like to ask, hope you won't mind :P
>
> If I want to use TWS with Fiji to build a training arff dataset with a set
> of images
>
> so i can build up a classification model in weka
>
> Then i want to use it to classify some images, let say Lion and other
> animals.
>
> So am i work on the correct way with TWS now, or i go the wrong direction?
>
> coz now even I load in 10 images and create about 3000 instances for test.
> But I don't know
>
> which image is classify as LION or not LION, coz it is in pixel level.
>
> Sorry a little bit out of the scope.
>
> Thx.
>
>
>
> --
> View this message in context:
> http://imagej.1557.x6.nabble.com/Creating-DataSet-after-building-classification-with-Trainable-Weka-Segmentation-tp5009283p5009394.html
> Sent from the ImageJ mailing list archive at Nabble.com.
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
Ignacio Arganda-Carreras, Ph.D.
Seung's lab, 46-5065
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
43 Vassar St.
Cambridge, MA 02139
USA

Phone: [phone]
Website: http://bioweb.cnb.csic.es/~iarganda/index_EN.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[Ignacio Arganda-Carreras]

Here you are another approach:

http://sujitpal.blogspot.fr/2012/04/image-classification-photo-or-drawing.html


On Thu, Aug 28, 2014 at 12:00 PM, Ignacio Arganda-Carreras <
[hidden email]> wrote:

> OK, I see what you're trying to do now.
>
> TWS is designed for pixel classification, not image classification. For
> the latest, you will need to create your own plugin and features. The idea
> would be very similar to TWS but you'll need a single final class per image
> and features that are image-based.
>
> You can also use existing software such as PiXiT:
> http://www.montefiore.ulg.ac.be/~maree/pixit.html
>
> Best,
>
> ignacio
>
>
> On Thu, Aug 28, 2014 at 11:34 AM, marcelo_chong <[hidden email]>
> wrote:
>
>> Hello Ignacio,
>>
>> Thanks for the reply again.
>>
>> I understand now about the instance it create after i load in the 10
>> images.
>>
>> thanks for the explanation.
>>
>> So i would like to ask, hope you won't mind :P
>>
>> If I want to use TWS with Fiji to build a training arff dataset with a set
>> of images
>>
>> so i can build up a classification model in weka
>>
>> Then i want to use it to classify some images, let say Lion and other
>> animals.
>>
>> So am i work on the correct way with TWS now, or i go the wrong direction?
>>
>> coz now even I load in 10 images and create about 3000 instances for test.
>> But I don't know
>>
>> which image is classify as LION or not LION, coz it is in pixel level.
>>
>> Sorry a little bit out of the scope.
>>
>> Thx.
>>
>>
>>
>> --
>> View this message in context:
>> http://imagej.1557.x6.nabble.com/Creating-DataSet-after-building-classification-with-Trainable-Weka-Segmentation-tp5009283p5009394.html
>> Sent from the ImageJ mailing list archive at Nabble.com.
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
>
>
> --
> Ignacio Arganda-Carreras, Ph.D.
> Seung's lab, 46-5065
> Department of Brain and Cognitive Sciences
> Massachusetts Institute of Technology
> 43 Vassar St.
> Cambridge, MA 02139
> USA
>
> Phone: [phone]
> Website: http://bioweb.cnb.csic.es/~iarganda/index_EN.html
>
> --
> Ignacio Arganda-Carreras, Ph.D.
> Seung's lab, 46-5065
> Department of Brain and Cognitive Sciences
> Massachusetts Institute of Technology
> 43 Vassar St.
> Cambridge, MA 02139
> USA
>
> Phone: [phone]
> Website: <http://bioweb.cnb.csic.es/~iarganda/index_EN.html>
> http://bioweb.cnb.csic.es/~iarganda/index_EN.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html

[marcelo_chong]

Hello Ignacio,

you are very kind ! Thanks for your info.

Best Wishes,
Marcelo