osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Multiprocessing on a remote host


On Tue, Mar 20, 2018 at 11:15 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 21 Mar 2018 02:20:16 +0000, Larry Martell wrote:
>
>> Is there a way to use the multiprocessing lib to run a job on a remote
>> host?
>
> Don't try to re-invent the wheel. This is a solved problem.
>
> https://stackoverflow.com/questions/1879971/what-is-the-current-choice-for-doing-rpc-in-python
>
> I've used both rpyc and pyro, they work fine and are easy to learn.

Yeah, I saw that and I wasn't trying to reinvent the wheel. On this
page https://docs.python.org/2/library/multiprocessing.html it says
this:

The multiprocessing package offers both local and remote concurrency,
effectively side-stepping the Global Interpreter Lock by using
subprocesses instead of threads. Due to this, the multiprocessing
module allows the programmer to fully leverage multiple processors on
a given machine.

I took 'remote concurrency' to mean I could use it run a process on
another host. But I don't see how to do that, and I was asking if it
was possible or am I misinterpreting the docs.