The Nimrod Toolkit

At the University of Queensland’s Research Computing Centre, I worked on The Nimrod Toolkit, a suite of utilities for distributed computing. Originally developed at Monash University by David Abramson, Nimrod enables parameter sweep management, optimisation, and workflow integration for high-performance computing environments.

Main Sites

• UQ Research Computing Center
• Monash eScience and Grid Engineering Laboratory (MeSsAGE Lab)
• ARDC Nectar Research Cloud

Downloads

ℹ️ Note

Source code for the pre-rewrite version of Nimrod/G, as well as Nimrod/O, is not publicly available.

Components & Features

Nimrod/G

Core grid scheduler responsible for distributing work across multiple resources and aggregating results.

• Supported parameter sweeps with the ability to execute millions of jobs.
• Jobs defined via planfiles specifying parameters, tasks, and data management.
• Originally implemented in early Python, later rewritten in C.

Sample planfile

parameter aircraft_model files select anyof "A3??.dat" "737-*.dat"
parameter AoA label "Angle of attack" float range from -45 to 45 step 2.5
parameter winglets text select anyof "none" "fence" "blended" "raked"
parameter airspeed integer range from 50 to 600 step 50
parameter turbulence label "Normalized Reynolds" float random from 1 to 2

task main
	copy root:${aircraft_model} node:.
	copy root:wing_test.zip node:.
	exec unzip wing_test.zip
	shexec "./run_wing_test.sh ${aircraft_model} ${winglets} ${AoA} ${airspeed} ${turbulence} >> output.${jobindex}"
	shexec "zip results.${jobindex} *"
	copy node:results.${jobindex}.zip root:.
endtask

Nimrod/O

Optimisation engine built on top of Nimrod/G.

• Supported single- and multi-objective optimisation.
• Extended planfile syntax with optimisation directives (schedule files).
• Written in C.

Sample Schedule File

parameter p integer range from 35 to 60
parameter q integer range from 2 to 17
parameter r integer range from 15 to 45

task main
   copy  bead.data node:bead.data
   copy  beader node:beader
   node:execute  ./beader $p $q $r > final
   copy  node:final output.$jobname
endtask

method simplex
  starts 2 named "Simplex"
    starting points random
    tolerance 0.000
  endstarts
endmethod

Nimrod/K

Reimplementation of Nimrod/G for the Kepler Project.

• Enabled visual workflow construction with Kepler’s actor–director model.
• Allowed parameter sweeps and distributed execution via a GUI.
• Written in Java.

Nimrod/OK

Optimisation integration for Kepler and Nimrod/K.

• Partial reimplementation of Nimrod/O.
• Introduced an optimisation actor to support feedback loops and optimisation pipelines.
• Written in Java.

Embedded Nimrod

Lightweight variant of Nimrod/G designed for use within a single HPC cluster.

• Configured via inline comments in job scripts, similar to PBSPro and SLURM.
• Provided a simpler on-ramp for parameter sweeps without complex configuration.
• Implemented in C++, coordinating a local Nimrod/G instance.
• Originally developed as a shell script to bootstrap Nimrod/G instances for testing.
• See also: Embedded Nimrod: Straightforward HTC in HPC environments.

Sample Job Script

1
2
3
4
5
6
7
8
9

#!/usr/bin/env nimexec
#PBS -lselect=4:ncpus=4:ompthreads=2:mem=16gb
#PBS -lwalltime=10:00:00

#NIM shebang /bin/bash
#NIM parameter x integer range from 1 to 100 step 1
#NIM parameter y integer range from 1 to 100 step 1

expr ${NIMROD_VAR_x} \* ${NIMROD_VAR_y}

Development History

Assessment and Modernisation

• Inherited a legacy codebase that had been largely unmaintained for nearly 20 years.

• Identified major issues including:

  • Unsafe database queries using string concatenation.
  • Poor error handling that left orphaned jobs and resources.
  • Rudimentary, regex-based parsing of planfiles.

• Determined the codebase was unsalvageable and initiated a complete reimplementation.

Reimplementation of Nimrod/G

• Reimplemented Nimrod/G as a modern Java-based scheduler with improved safety, maintainability, and portability.
• Introduced structured error handling, resource cleanup, and consistent parsing.
• Ensured cross-platform compatibility for deployment across diverse HPC environments.

Outcome

• Delivered an updated and maintainable foundation for distributed computing at RCC.
• Simplified user adoption through Embedded Nimrod, reducing reliance on staff intervention.