3

10 Preface Introduction to Distributed Processing
Batches can be distributed to a cluster by the cluster controller in one or both of the
following ways. (Apple Qmaster determines which way is the most efficient for specific
batches, depending on the circumstances.)
 The batch is subdivided into data segments: For example, for a render batch, the
cluster controller could divide the frames into groups ( segments ). Each segment
would be processed in parallel on the service nodes in the cluster.
 The batch is subdivided into tasks: For example, for a render batch, the cluster
controller could subdivide the rendering work into different processing tasks.
Different tasks would be run on different service nodes.
Rather than actually moving segments, Apple Qmaster tells the service nodes which
segments to read via the network, where to find them, and what to do with them. Below
is an example of how one batch could be processed in an Apple Qmaster system.
In distributing batches, Apple Qmaster uses the technology built in to Mac OS X to
locate services in a cluster on the same IP subnet and to dynamically share and receive
information. Because the computers can continually transmit their current processing
availability status, Apple Qmaster can distribute (load-balance) the workload evenly
across the cluster.
Batch submitted by
Compressor or
Apple Qmaster (job request
for frames 1–30)
Cluster controller
divides and distributes
job to available
service nodes
Service node 1
Service node 2
Service node 3
Instructions specifying
locations of source files
and frames 11–20
Instructions specifying
locations of source files
and frames 21–30
Instructions specifying
locations of source files
and frames 01–10
Processed file (frames 1-30)
placed in specified
destination