Benchmark Macro

[Alan Brooks]

Hi Wayne & Gabriel,

I was curious about benchmarking with various JDKs on my 2015 Intel MacBook versus my 2020 M1 ARM MacBook. To that end, I updated Gabriel’s macro to run both the old 512x512 and new 5000x5000 benchmark a few times and print a bit more about the architecture/JRE.

On the Java 11 Azul Zulu JDK, I found the following performance for the best case run:
* 2015 MacBook on x86_64: 0.52 sec for 512x512, 39 sec for 5000x5000
* 2020 MacBook on x86_64 (Rosetta2): 0.23 sec for 512x512, 18 sec for 5000x5000
* 2020 MacBook on aarch64: 0.11 sec for 512x512, 11 sec for 5000x5000

It appears that using a native JDK/JRE is almost a 2x speedup for these benchmarks on the new M1 Macs.


Example updated macro output looks like this:
- - -
ImageJ: 1.53i
OS : Mac OS X 11.2.3 on aarch64
Java: 11.0.10, vm: 11.0.10+9-LTS Azul Systems, Inc.
JRE: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib
Benchmark result 1: 0.114 sec
Benchmark result 2: 0.108 sec
Benchmark result 3: 0.109 sec
Benchmark result 4: 0.115 sec
Benchmark result 5: 0.110 sec
Benchmark result 6: 0.110 sec
Benchmark result 7: 0.111 sec
Benchmark best : 0.108 seconds
Benchmark mean : 0.111 seconds
Benchmark worst: 0.115 seconds
- - -
ImageJ: 1.53i
OS : Mac OS X 11.2.3 on aarch64
Java: 11.0.10, vm: 11.0.10+9-LTS Azul Systems, Inc.
JRE: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib
Benchmark result 1: 10.530 sec
Benchmark result 2: 10.515 sec
Benchmark result 3: 10.531 sec
Benchmark result 4: 10.579 sec
Benchmark best : 10.515 seconds
Benchmark mean : 10.539 seconds
Benchmark worst: 10.579 seconds


And the updated macro Bench.ijm is as follows:
--------------------------------------------------------------
// Benchmark test macro
// based on https://imagejdocu.list.lu/faq/technical/are_there_performance_statistics_to_compare_against_my_system

runBenchmark(7, true); // Legacy fast benchmark on 512x512 RGB image
runBenchmark(4, false); // New 1.53j 5000x5000 image benchmark

function runBenchmark(trials, doLegacy) {
  if (doLegacy) {
    run("Lena (68K)");
    id = getImageID;
  }
  print("- - - ");
  print("ImageJ: " + getVersion());
  print("OS : " + getInfo("os.name") + " " + getInfo("os.version") + " on " + getInfo("os.arch"));
  print("Java: "+ getInfo("java.version") + ", vm: " + getInfo("java.vm.version") + " " + getInfo("java.vm.vendor"));
  print("JRE: "+ getInfo("sun.boot.library.path"));

  best = 100000;
  worst = -1;
  mean = 0;

  for(i=0; i<trials; i++) {
    t = getTime();
    run("Benchmark");
    thisrun = getTime() - t;
    mean += thisrun/trials;
    if (thisrun<best) best = thisrun;
    if (thisrun>worst) worst = thisrun;
    print("Benchmark result "+i+1+": " + d2s(thisrun/1000, 3) + " sec");
  }
  print("Benchmark best : " + d2s(best/1000, 3) + " seconds");
  print("Benchmark mean : " + d2s(mean/1000, 3) + " seconds");
  print("Benchmark worst: " + d2s(worst/1000, 3) + " seconds");

  if (doLegacy) {
    selectImage(id);
    close();
  }
}
--------------------------------------------------------------



Best,
Alan

> On Mar 23, 2021, at 2:40 PM, Wayne Rasband <[hidden email]> wrote:
>
> Hi Gabrial,
>
> Your benchmark macro at
>
>    https://imagejdocu.list.lu/faq/technical/are_there_performance_statistics_to_compare_against_my_system
>
> is working again with the 1.53i43 daily build. The Plugins>Utilities>Benchmark command runs the previous version when the current image is 512x512 and RGB. Here is the output from running the macro on an M1 (ARM64) MacBook Air with 8GB of RAM:
>
>  ImageJ: 1.53i
>  OS : Mac OS X 11.2
>  Java: 13.0.6, vm: 13.0.6+5-MTS Azul Systems, Inc.
>  Benchmark best: 0.119
>  Benchmark worst: 0.129
>
> -wayne
>
>
>> On Mar 23, 2021, at 7:26 AM, Gabriel Landini <[hidden email]> wrote:
>>
>>> The daily build also has an improved version of the
>>> Plugins>Utilities>Benchmark command.  It uses two little known, but very
>>> useful, features contributed by Norbert Vischer that allow subordinate
>>> status bar messages and progress bars to be suppressed.
>>
>> Hi, I was a bit confused  by this. It seems that the Benchmark command now  
>> use a 5kx5k image instead of the image one has open?
>>
>> Perhaps this old benchmark macro should be removed as it does not do what it
>> did before:
>> https://imagejdocu.list.lu/faq/technical/
>> are_there_performance_statistics_to_compare_against_my_system
>>
>> Cheers
>>
>> Gabriel
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html