myGrid

More like a bug, suggest demand factor 5 or 6.

Stian - can you chase down the origin of this from the archive to see if it was a nice-to-have or a genuine request (which would alter the demand factor). Thanks. June.

Sent an email to Edward Kawas who originally reported this.

From: "Edward Kawas" <edward.kawas@gmail.com>
To: "'Stian Soiland'" <ssoiland@cs.man.ac.uk>
Subject: RE: Shim for flattening lists?
Date: Thu, 4 Jan 2007 07:09:19 -0800

> Do you remember this thread (then called "Save to excel")
> from August 2006? Check out
> http://www.mygrid.org.uk/jira/browse/TAV-118 - note that I
> think I solved most of the save-excel-issues from back then -
> but do you still think we need such a local processor to
> bundle as a shim?
>
I do! That would be fantastic.

> In most cases it should not be needed, as Taverna handles
> iterations pretty well, but of course it's a mess when the
> end result is a list of a list of a list of a list of a list
> that contains some lists that actually contains the values.
> But I think I remembered you had a use- case where Biomoby
> produced these super-deep lists.. Do you know why this
> happens, and should we try to fix that or just put in this
> simple shim?
>
Moby services have output ports that produce a single item and/or a list of
items. When you make workflows that mix these 2 together, you get the list of a
list ... At least, that is what I think. A while back, I tried making our
processors produce just a list of items, but I ran into problems that made the
plugin unusable. I am not sure how you could fix it (I am still trying to get my
head back into things after a long break, so I may seem even more out to lunch
than usual .

A single flattener that flattens everything would be great!

Eddie

Another email to taverna-users from Jeffrey Tilson at 2007-02-28:

Given: A simple beanshell processor.

I have as input ports to this processor 12 input ports. Each port is a
string type. The real use-case, however, is for each of these 12 input
ports to pass a list of data. No port explicitly conforms to any other
port. So if the ports are labeled A,B,C,etc then the data entry may look
like

A=(1,2,3), B=(x,y), C=(a,b,c,d,e),etc. Thus, each are 1D lists.

Implicit iteration is used to cross ( and in some cases dot ) as
necessary.

The output consists of a list of entries constructed from a particular
combination as defined by Implicit iteration.

Now here is the issue. It seems the "depth" of the Java list for this
output corresponds to the number of entries. What I want is to flatten
this list to a one-dimensional list. However, it seems I need to use a
"flatten" processor 12 times for a 12 deep list.

I hope this sufficiently explained the issue.

How can I "easily" get this data to a 1D list without lots of
flattening.

Modified FlattenList so it support arbitrary levels of flattening. The level can be adjusted through Right click on the processor. The level is stored in the XSCUFL through the <extensions> element.

The tests have been extended to test for deeper lists, and also tests that it does not flatten too much. (Was not tested before, although the previous behaviour didn't flatten too much)

Changed name to "Flatten list" as it is not just l(l( to l(.

Small bug left: The mime type is showed as l(l( (or whatever was the current depth on XScufl Load) in the Advanced Model Explorer, however running the workflow is OK because of the implisit serialization of the XSCufl.. (Not sure how clever it is to rely on that, though)

Screenshot of "Set list flattener depth" dialogue attached.

Fixed for the 1.5.1.5 release