Attending: Shoaib, David Norman
feedback from usecases workshop and some discussions:
Finn:
Got a good overview of the various parts of the SCAPE project. More complex/heterogeneous than I first imagined, lots of people doing lots of different things, with lots of overlap in some areas and none in others. Talked to several people from KEEPS (Portuguese SCAPE partners) who were interested in registering their services on the SCAPE catalogue. They wanted to register a large number of services at once via the API, but POST support to the Biocat API is undocumented and only loosely supported. I made a simple workflow to register a set of services with the SCAPE Biocatalogue instance, but need to spend more time testing whether or not writing to the API presents any issues before I send it to them.
Those requirements, along with others have been collected on this wiki page:
http://wiki.opf-labs.org/display/SP/PT.WP4.Task+3+Requirements+for+the+Preservation+Components+Catalogue
Also had some feedback on myExperiment. Andrew wanted to be able to see whether a workflow on myExperiment could be run as-is or whether it had dependencies that needed to be installed. He also would have liked a system for "tagging" your profile with interests, with the ability to find people with similar interests. Such a system already exists on SysMO, so could perhaps be adopted for myExp.
David:
Scenarios, datasets and issues -
There is a site with the list of scenarios.
Lots of discussions of the Demo for the review
Various tools they wanted to show of
Detecting and fixing broken images
Debian packaging for easier install
Feedback for the demos - asked question about packaging - only support debian (yes) - need a good answer - lots of the work focused on improving existing tools - improve packaging - easier for community
Demo was themed being around the tool - not a good idea - might give impression only one tool
Norman - short term focus (Scenarios for the review)
Map reduce - concerns processing text files and medium size image files - and not designed for large files and issues with HDFS - blocksize for collections of items that already exist - still in discussions whether map reduce is the way to go - lots of partners questioned the suitability.
Rainer holds ticket for scape platform and scalability
Lots of issues installing tool service on Taverna - - Java 7 issue - fixed and ready to ship in next release.
Andrew Jackson - scape group on myExperiment - keep it closed - need a public group - could call it Digital Preservation and put things out there - skinning the group - decide whether we brand public group (Norman - might be a good outreach idea)
Norman - issue with putting workflows can't run (Digital Preservation.com) - releases workflows - testing work and tools - quality bar - scape techy group -
Lots of tools that David not heard of - Norman said would expect that
Microsoft are implementing map reduce on Azure - sharepoints have tools to convert to text document - no one was sure how useful this was.
Risk - Andrew Jackson - moving onto a different role - Technical coordinator - not known generally - BL - a couple of people - might take over the role - Senior and less senior. ( Andrew not at review in beginning of March) - not sure who will lead - maybe Karl Wilson as senior - Paul Wheatly (junior) - All staff meeting (review focused only a part)
Norman - like to have more clarity at All staff meeting on provenance - within work package slot - map reduce - Berlin - offer our stuff and hear their stuff - (covered some planning for the all staff meeting) - provenance: present where we are at and standards - work out who to work with to get provenance requirements. Goal - want a requirements doc like wf4ever or a designated person to lead on putting together this requirements document.
David said one the scenarios was What level of compression and still have sensible image to analyse the text on.
Actions:
David - Location of wikis and infrastructure add to Alex's page - what things are where and where (location wise) infrastructure is moving to in the future.
Shoaib - send Norman instructions on getting onto the wiki
David - add to UI returning an error from the tool service - make it easier for users (add to Jira)
David - send around scenrios workshop writeup url
David - send a reminder to scape local list when next platform call is.
David - send e-mail about SCAPE new public myExperiment group - and details on publishing (e.g. workflow must run or have instructions for how to install pre-requisites and config) and example data if applicable - none scape members can register for the data or the data is public
David - Investigate a SCAPE Jira and link to issues in BioCatalogue, myExperiment and Taverna - or higher level linked to scape
Norman - generate a proposal for the scape list for the session - at the all staff meeting (will send around a draft to local Manchester list ) - Mcr will present provenance and map reduce - David to present provenance - and Norman - to present map reduce (David to source provenance material on standards from Sean ?)
All - stock take next week on where we are upto wrt to prep for All staff meeting