Running OpenMPI Ops though HPC Workflow Manager

HPC Workflow Manager can also be used with OpenMPI Ops.

This tutorial aims to show you how to run a Jython script that contains call to parallel image operations of OpenMPI Ops and not how to write one.

We are going to use the famous peppers photo which is considered a standard test image for image processing and use a Canny edge detector to produce a new output image.

• Start fiji.
• Click on Plugins > HPC-ParallelTools > HPC Workflow Manager.
• right click and click on the "Create a new job" menu item.
• At this stage select "Script" instead of "Macro" in the job type section.
• Create an empty directory anywhere.
• Download peppers.tiff image and place it in the directory you created.
• Create an empty Jython script that contains the OpenMPI Ops. For example name it "peppers.py". (Note the .py extension instead of .ijm)
• Copy this script in the directory you created were you placed "peppers.tiff".
• Browse to the Jython script.
• Press the create button.
• Right click on the job's row and click upload.
• Once the upload is done, right click on the job's row and click the "Start job" menu item.

#@ OpService ops
#@ SCIFIO scifio
#@ UIService ui
#@ DatasetService datasets

from net.imglib2.type.numeric.real import DoubleType
from net.imglib2.view import Views

# Input and output paths:
input_path = "./peppers.tiff"
output_path = "./peppers_edges.tiff"

rgb = scifio.datasetIO().open(input_path);
print("Loaded image");

grayscale = ops.run("convert.rgb2grayscale", rgb);
grayscale = ops.convert().float32(grayscale);
rgb = None;

gauss_kernel = ops.create().kernel([
    [1 / 256.0, 4 / 256.0, 6 / 256.0, 4 / 256.0, 1 / 256.0],
    [4 / 256.0, 14 / 256.0, 24 / 256.0, 14 / 256.0, 4 / 256.0],
    [6 / 256.0, 24 / 256.0, 36 / 256.0, 24 / 256.0, 6 / 256.0],
    [4 / 256.0, 14 / 256.0, 24 / 256.0, 14 / 256.0, 4 / 256.0],
    [1 / 256.0, 4 / 256.0, 6 / 256.0, 4 / 256.0, 1 / 256.0]
], DoubleType());

print("Created kernel");

without_noise = ops.create().img(grayscale);
ops.filter().convolve(without_noise, Views.extendMirrorSingle(grayscale), gauss_kernel);

edges = ops.run("edgeDetector", without_noise);
scifio.datasetIO().save(datasets.create(ops.convert().uint8(ops.math().multiply(edges, 255))), output_path);
print("Done");

• Download the results by right-clicking the job and clicking the "Download results" menu item.
• Right-click the job tow and click the "Open job sub directory" menu item.

In the folder you should now see the edge detected image of peppers. Now you are ready to run Jython jobs with parallel ops in HPC Workflow Manager.

Advanced feature: Parallel Macro and OpenMPI Ops can also be used together either for different parts of the code or the same code using MPIUtils.split(color, myRank).

video version 📺

main hub:house: previous 👈