Aynchronous Communications

AC has many possible implementations and the main danger is to overcomplicate a solution (trying to facilitate all wishes and real user requirements).

Typical use case

Typically, now, a user creates a webservice (HTTP) and needs a way of executing a job that may take longer than the time a browser times out (think of BLAST). Potentially the job can take a month to run and both the client, server and network may be occasionaly down. Finally the user gets the output from the job - with output on success, error on failure.

The webservice author will want a clean API which allows job submission and passing the feedback mechanism.

Minimal Specification

A only job of the AC is to provide the messaging service between client and job.

What it should do:

* Receive info from web-service (jobreceiver) * Start remote job on cluster (jobmonitor) * Pipe input data to job (jobmonitor) * Monitor job status (on failure perhaps restart) * Get output (jobmonitor) * Push output to client (jobsender)

What it should *not* do:

* Authentication/authorisation (is responsibility of the webservice) * Filtering of data (modifying the stream) is responsibility of webservice) * Job control (handled by cluster tools)

In its minimalistic fashio it does not even handle status reports (i.e. percentage of job executed).

API

* jobreceiver may be able to receive jobs through SOAP, a pipe, or mail * jobmonitor may be able to give updates on percentage executed (SOAP, RPC) * jobsender may be able to return results through SOAP, pipe or mail

Notes

* The jobmonitor has to be stateful as the server or node running the job may go down * Piping is an incredible useful mechanism. The complication with webservices is the asynchronous nature (while the web protocol tends to by synchronous) - inspiration could be modglue (a Plan9-inspired extension of the Unix pipe concept).