Re: Assigning multiple pools to a task
Yes, after looking at the code, it looks like implementing a many to many
between pools and tasks requires significant redesign.
My workaround is to implement it externally by borrowing required resources
from an external pool using a sensor, and returning them back once the task
is done using another sensor. They are basically entry/exit sensors for a
On Sat, May 12, 2018, 2:28 PM Dennis O'Brien <dennis@xxxxxxxxxxxxxxxx>
> A task is assigned to a pool by the task specifying the name of the pool.
> The docs suggest that the pool argument is a string, not a list of strings.
> And looking at the code it does seem like this relationship of one task
> assigned to zero or one pools is strict.
> It's an interesting idea of trying to associated a task with multiple
> pools. I think things would get complicated fast to avoid deadlock
> situations. Do you know of any other work flow systems that support
> one-to-many relationship?
> Is it possible to bundle the resources into a single resource? I'm
> guessing this is not an option, and that tasks use different combinations
> of these resources.
> On Fri, May 11, 2018 at 1:04 PM Neelesh <neeleshs@xxxxxxxxx> wrote:
> > Hello,
> > Is it possible to assign multiple pools to a task?
> > I'm wrapping a bash operator around some heavy-weight custom data
> > processing script that cannot be modified. This script cannot be
> > and it uses multiple resources. Im trying to use pools as proxies for
> > resources.
> > Thanks