myGrid

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4, 1.5, 1.5.1, 1.5.1.1, 1.5.1.2, 1.5.1.3, 1.5.1.4
• Fix Version/s: 1.5.2
• Component/s: Taverna Core
• Labels:

  None

When launching folders through the API using WorkflowLauncher, the thread leak identified in TAV-127 is still alive, because the WorkflowLauncher does not do workflowInstance.destroy();

In addition for every workflow run, a listener is added (which I guess keeps a reference to the instance as well), but never removed.

This causes problems for people trying to run several (say more than hundred) workflows through the API, as in en email from James Sinnot to taverna-hackers on 2007-04-04:

I'm using the Taverna API to execute workflows outside of the GUI and
I've noticed that every time I run a workflow the number of threads
used by Taverna progressively increases, to the point where I
eventually run out of memory (my app might run several 100 workflows
a day!).

I've since investigated this in Taverna proper (1.5.1.6), on Windows,
Linux & OS X and found that it also happens.

Browsing the hackers list, I came across this post:

http://sourceforge.net/mailarchive/message.php?msg_id=44EDCEF8.7030504%40cs.man.ac.uk

from August of last year, which seems to concern the same issue.

Is a fix for this problem likely to appear in the near future?

Stian replied:

We believe we have solved the thread leak problem from within the GUI. What happens is that for each workflow, in addition to threads involved in showing the results (usually die immediately after clicking "Close" on the result), there's two NotifyThreads hanging around.

When the workflow is destroyed, which happens when the [X Close] button is pressed, we run the following code:

public void onDispose() {
try

Unknown macro: { workflowEditor.detachFromModel(); // In case it is still running, we'll stop it workflowInstance.cancelExecution(); // FIXME}

catch (Exception e)

Unknown macro: { logger.error("Could not detach EnactorInvocation", e); }

}

The main points here is workflowInstance.destroy(); - which will put the workflow instance in a "To be destroyed" state (the workflowToBeDestroyed is sent), which when captured by a temporary listener will do the actual destruction with "engine.destroy(engineId);" and freeing resources referenced from the workflow instance.

This should break the instance --> NotifyThread --> instance loop and allow everything to be garbage collected (when that time comes, should be before you run out of memory!), and thereby stopping the two threads.

Now, as you are invoking the workflows through the API, it might be that this destruction is not called. WorkflowLauncher don't do the destroy() call, but it probably should, and this is something we should trivially be able to fix.

In addition I see the code does things like for every execute:
WorkflowEventDispatcher.DISPATCHER.addListener(completionListener);

Clearly this does not scale too well, and WorkflowLauncher needs to be shaken up a bit.

Are you using WorkflowLauncher? I imagine that adding a try..finally and do workflowInstance.destroy(); would fix it, as well as removing that listener (as it would keep a reference to the final workflowInstance).

• Assignee:

  Stuart Owen

  Reporter:

  Stian Soiland-Reyes

• Created:

  2007-04-05 11:05

  Updated:

  2007-05-21 10:52

  Resolved:

  2007-05-21 10:52