Re: [gt-user] Run a wrapped command with MPI

Hello:

Thank you all for your help. The problem was indeed the missing
command line arguments.

I patched mi pbs.pm script from the
$GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager directory in the way
I describe below. It is a bit untidy, but I got it to work.

Replaced:

            print CMD "#!/bin/sh\n";
            print CMD 'cd ', $description->directory(), "\n";
            print CMD "$rsh_env\n";
            print CMD $description->executable(), " $args\n";
            close(CMD);
           chmod 0700, $cmd_script_name;

With:

            print CMD "#!/bin/sh\n";
            print CMD 'cd ', $description->directory(), "\n";
            print CMD "$rsh_env\n";

       if ($description->jobtype() eq "mpi")
       {
           if ($mpiexec ne 'no')
           {
               # below is the line as it originally was
               print CMD $description->executable(), " $args\n";
           }
           else
           {
               # mpirun case
               print CMD $description->executable(), " \"\$@\"\n";
           }
       }
       else
       {
           # below is the line as it originally was
           print CMD $description->executable(), " $args\n";
       }

            close(CMD);
           chmod 0700, $cmd_script_name;


And replaced:

      if ($description->jobtype() eq "mpi")
       {
           if ($mpiexec ne 'no')
           {
               my $machinefilearg = "";
               if ($cluster)
               {
                   $machinefilearg = ' -machinefile $PBS_NODEFILE';
               }
               print JOB "$mpiexec $machinefilearg -n " .
$description->count();
           }
           else
           {
               print JOB "$mpirun -np " . $description->count();
               if ($cluster)
               {
                   print JOB ' -machinefile $PBS_NODEFILE';
               }
           }

           print JOB " $cmd_script_name < " .  $description->stdin() . "\n";
       }

With:

       if ($description->jobtype() eq "mpi")
       {
           if ($mpiexec ne 'no')
           {
               my $machinefilearg = "";
               if ($cluster)
               {
                   $machinefilearg = ' -machinefile $PBS_NODEFILE';
               }
               print JOB "$mpiexec $machinefilearg -n " .
$description->count();
               # this line is copied from below, I separated the two
cases and only modified the mpirun case
               print JOB " $cmd_script_name < " .
$description->stdin() . "\n";
           }
           else
           {
               print JOB "$mpirun -np " . $description->count();
               if ($cluster)
               {
                   print JOB ' -machinefile $PBS_NODEFILE';
               }
               print JOB " $cmd_script_name $args < " .
$description->stdin() . "\n";
           }

           # I broke the following line into two cases above
           # print JOB " $cmd_script_name < " .  $description->stdin() . "\n";
       }

Thanks again,

Matt


On 5/15/07, Brian R. Toonen <toonen@xxxxxxxxxxx> wrote:
> Some implementations of mpirun pass command line arguments to the executable
> which are then used during MPI_Init() to determine how many processes to
> start, etc.  Your script isn't passing those command line arguments to the
> cpi executable.
>
> Try the following script.  It's a bit ugly, but it should work.
>
> #!/bin/sh
>
> IFS=
> args=""
> while test $# -gt 0 ; do
>     args="$args '"`echo "$1" | sed -e "s/'/'"'"'"'"'"'"'/g"`"'"
>     shift
> done
> args=`echo "$args" | sed 's/^ *//'`
>
> eval ./cpi "$args"
>
> --brian
>
> |-----Original Message-----
> |From: owner-gt-user@xxxxxxxxxx [mailto:owner-gt-user@xxxxxxxxxx] On Behalf
> |Of Matias Alberto Gavinowich
> |Sent: Monday, May 14, 2007 14:43
> |To: mpich-discuss@xxxxxxxxxxx; torqueusers@xxxxxxxxxxxxxxxx; gt-
> |user@xxxxxxxxxx
> |Subject: [gt-user] Run a wrapped command with MPI
> |
> |Hello:
> |
> |I am having the following problem.
> |
> |I can run a comand through mpi invoking:
> |
> |mpirun -np 2 ./cpi   (I am using my machines.LINUX default file).
> |
> |Two processes are started, as expected.
> |
> |Then, I write a wrapper script that looks like:
> |
> |#!/bin/sh
> |./cpi             (I also tried with a full path)
> |
> |and I run:
> |
> |mpirun -np 2 ./cpiwrapper
> |
> |Only one process is started in this case.
> |
> |The trick is I need it to work with a wrapper, because jobmanager-pbs
> |from globus invokes it this way (with a wrapper script).
> |
> |AFAIK, I am running MPICH1.
> |
> |Has anyone come accross this?
> |
> |Thank you,
> |
> |Matt