IF-5 WorkFlow Plan

Who

• MarkGreenwood
• JustinFerris
• DarrenMarvin
• TomOinn

Flipchart Summary

1. Investigating Workflow "patterns of use"
   • examplar patterns for new features PAPER!
   • test harness
2. Conditionals and loops in workflow scripts
3. Long-running workflows
   • asynchronous interaction
   • resource(?illegible) correlation
   • iteration
4. Large datasets - relevant to Grid infrastructure PAPER!
   • pipelining
   • streaming
   • staging
5. RPC vs. Document style - see IfVArchitecturePlan
   • ability to call bioinformatics services which are provided with a document rather than an RPC style interface (e.g. services developed using .Net use document style)
     • achieved in Taverna beta6 release October 2003
6. Dynamic binding of service endpoints during enactment
   • Scufl enhancements
   • service discovery
7. Representing data collections in Freefluo and Taverna
   • implicit mapping of services/workflows over collections
8. Using the FindService^? during workflow construction
   • integration with Taverna
9. Interface for biologists?

Planning towards IF-6 (jan 2004)

Initial thoughts MarkGreenwood - 28 Sep 2003 revised 10 Oct 2003

• NeSC workflow workshop, priority high, 3-5 Dec 2003, effort 1 week
• workflow papers, priority very high, requires draft to promote comments, effort 5 weeks ( Peter Li to lead bioinformatics angle paper. MarkGreenwood to lead eScience angle paper )
  • Peter Li has circulated outline, getting feedback from co-authors 10 Oct 2003
  • MarkGreenwood eScience extended abstract draft for Carole while at GGF 6-10 Oct 2003
• workflow information on web sites and TWiki - priority high, ongoing, effort 1 week
• patterns of use (exploitation examples - why features are needed and how they can be used) - priority medium, (requires further thought and planning)
  • examples can be used to test provenance - automatic running of examples would be useful - will need new initial examples using conditionals and iteration
  • knowledge in workflows - explicit links of input and outputs, workflow producing not just data but also annotations relating data, use of knowledge in capturing the intention of a workflow (e.g. this workflow produces annotations about its input gene) becoming a major issue in the InformationModel and therefore probably needs separate recognition and priority.

Initial thoughts TomOinn - 29 Sep 2003

• resolve language issues for conditionals to allow use of the facilities in the next version of the enactor, IF6 release should support looping. BIG issues with provenance for that one...
• make better use of the ontologies, we started this already and need to investigate further. I'd want to have Taverna using some kind of ontology based find service by IF6, ideally some time before.
• workflow portal for biologists who don't actually want to edit the things, just use them. Relatively simple and we're pursuing this at the HGMP, I'd expect it to be there by IF6 although of course it isn't technically a myGrid component as such.
• I still haven't seen a convincing case for runtime service binding in our problem domain, therefore no plans to include this facility.
• 'baclava' object model and language spec for collections should be finalised by IF6, with documentation and an efficient implementation.
• collaboration portal server, speculative at the moment but more detail will appear at some point, maybe for IF6 maybe not.

Initial Thoughts JustinFerris - 29th September 2003

In order of priority with rough estimates:

1. refactoring and adding explicit iteration and conditional branching to Taverna and Freefluo (2 weeks)
2. large data sets and service pipelining (8-10 weeks effort)
   • analysis (2 weeks)
   • changes to workflow core (2 weeks)
   • adding asynchronous interactions and message correlation
   • data staging services (2 weeks)
   • testing and fixes (2 weeks)
3. architecture for supporting long running workflows (10-12 weeks)
   • analysis (2 weeks)
   • domain layer design and implementation (2 weeks)
   • service design and implementation (2 weeks)
   • front end modifications (2 weeks)
   • testing and fixes (2 weeks)

-- NickSharman - 23 Sep 2003