YADE

YADE was developed to distribute calculations across multiple devices using PYTHON. It was developed for Prof. Oswin Aichholzer at the Institute for Software Technology at the Technical University Graz. During his research activities thousands of calculations, each taking up to several months to complete, have to be executed. YADEwas developed to support this requirement. Simple lists of execution commands can be added to the YADE jobs list, which are then sequentially executed by multiple computers.  These lists of execution commands can be any type of shell command, such as an application call with parameters. Each list is a simple text document with one command per line. One text document is here after referred to as job and each line in this job file is referred to as command. Each job can be given a priority. Yade will execute jobs with higher priority first.

Yade is a combination of two pieces of software.

  1. Managing Application
  2. Processing Application

Managing Application

The command application is installed on every computer which is supposed to be able to add new jobs to the YADE system. This part removes the necessity of a single root server for managing the jobs to be executed on YADE. The managing application provides following functionality:

    Adding/Removing Jobs

  • Adding/Removing Devices
  • Get information about current state of one or all devices.
    • List of currently running commands
  • Starting/Stopping devices
    • Stop and wait for current command finished
    • Stop and terminate current command

Processing Application

This part is responsible for the actual execution of the commands from the job files. Additionally to the execution of commands, the processing application provides simple ways of communication using sockets. This provides a way of controlling the processor using the managing application.

The process of executing a command is very simple yet flexible due to the usage of priorities. The basic process consists of 6 steps:

  1. Get job file with highest priority:
  2. Select command which is not marked as finished or computing
  3. Mark selected command as computing
  4. Execute command
  5. Mark command as finished
  6. Return to 1