Re: jobs with nodes=x x>1 are packing on single node

On Thu, Apr 12, 2007 at 04:57:16PM -0400, P Spencer Davis wrote:
> Hi,
>    In a nutshell, I'm trying to configure Torque(2.1.6)/Maui(3.2.6p19) 
> so that jobs will be spread over multiple nodes instead of bunched up on 
> a single node.
> 
> Currently when I submit
> 
> echo "cat $PBS_NODESFILE" | qsub -l nodes=4
> 
> I get back something like this
> 
> n31
> n31
> n31
> n31
> 
> I've tried a variety of options in my maui.cfg file, such as setting the 
> BACKFILLPOLICY to NONE (no effect) and NODEALLOCATIONPOLICY to 
> MAXBALANCE (which caused jobs requesting more than one node/cpu to 
> hang), but honestly I'm not even sure that was the right place to start 
> after reading some of the previous posts around this same topic.
> 
> Any ideas on what needs to be tweaked?

I believe you want to request nodes=4:ppn=1 and set JOBNODEMATCHPOLICY to
EXACTNODE in your maui.cfg.

HTH,

Gabe
-- 
Gabe Turner                                             gabe@xxxxxxxxxxx
UNIX System Administrator,
University of Minnesota
Supercomputing Institute                          http://www.msi.umn.edu