Changes between Version 1 and Version 2 of PjotrAsync

Show
Ignore:
Timestamp:
2008/02/12 15:06:46 (11 years ago)
Author:
pjotr2007
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PjotrAsync

    v1 v2  
    1 == Aynchronous Communications == 
     1== Aynchronous Communications (Pjotr) == 
    22 
    33AC has many possible implementations and the main danger is to overcomplicate a solution (trying to facilitate all wishes and real user requirements). 
     
    1515What it should do: 
    1616 
    17 * Receive info from web-service (jobreceiver) 
    18 * Start remote job on cluster (jobmonitor) 
    19 * Pipe input data to job (jobmonitor) 
    20 * Monitor job status (on failure perhaps restart)  
    21 * Get output (jobmonitor) 
    22 * Push output to client (jobsender) 
     17  * Receive info from web-service (jobreceiver) 
     18  * Start remote job on cluster (jobmonitor) 
     19  * Pipe input data to job (jobmonitor) 
     20  * Monitor job status (on failure perhaps restart)  
     21  * Get output (jobmonitor) 
     22  * Push output to client (jobsender) 
    2323 
    2424What it should *not* do: 
    2525 
    26 * Authentication/authorisation (is responsibility of the webservice) 
    27 * Filtering of data (modifying the stream) is responsibility of webservice) 
    28 * Job control (handled by cluster tools) 
     26  * Authentication/authorisation (is responsibility of the webservice) 
     27  * Filtering of data (modifying the stream) is responsibility of webservice) 
     28  * Job control (handled by cluster tools) 
    2929 
    30 In its minimalistic fashio it does not even handle status reports (i.e. percentage of job executed). 
     30In its minimalistic fashion it does not even handle status reports (i.e. percentage of job executed). 
    3131 
    3232=== API === 
    3333 
    34 * jobreceiver may be able to receive jobs through SOAP, a pipe, or mail 
    35 * jobmonitor may be able to give updates on percentage executed (SOAP, RPC) 
    36 * jobsender may be able to return results through SOAP, pipe or mail 
     34  * jobreceiver may be able to receive jobs through SOAP, a pipe, or mail 
     35  * jobmonitor may be able to give updates on percentage executed (SOAP, RPC) 
     36  * jobsender may be able to return results through SOAP, pipe or mail 
    3737 
    3838=== Notes === 
    3939 
    40 * The jobmonitor has to be stateful as the server or node running the job may go down 
    41 * Piping is an incredible useful mechanism. The complication with webservices is the asynchronous nature (while the web protocol tends to by synchronous) - inspiration could be modglue (a Plan9-inspired extension of the Unix pipe concept). 
     40  * The jobmonitor has to be stateful as the server or node running the job may go down 
     41  * Piping is an incredibly useful mechanism. The complication with webservices is the asynchronous nature (while the web protocol tends to by synchronous) - inspiration could be [http://www.aei.mpg.de/~peekas/modglue/ modglue] (a Plan9-inspired extension of the Unix pipe concept). What I would like to do is: 
     42 
     43  acjob -t pipe "dbfetch URI:gb -t fasta|clustalw"|phyloanalyse > tree.ph 
     44 
     45where dbfetch and clustalw run on the cluster and phyloanalyse locally - just as an example. Obviously with pipes you expect the client to keep running uninterrupted. The AC version could be (simplistically): 
     46 
     47  acjob -t email me@waste.com "dbfetch URI:gb -t fasta|clustalw" 
     48  cat email|phyloanalyse > tree.ph 
     49 
     50or: 
     51 
     52  acjob -t poll "dbfetch URI:gb -t fasta|clustalw"|acpoll -i 180|phyloanalyse > tree.ph 
     53 
     54where acpoll polls the jobmonitor every 3 minutes for results. 
    4255 
    4356 
    4457 
    4558 
     59 
     60 
     61 
     62 
     63