[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

George On 21/02/19 1:15 PM, george trojan wrote: > def create_box(x_y): > return geometry.box(x_y[0] - 1, x_y[1], x_y[0], x_y[1] - 1) > > x_range = range(1, 1001) > y_range = range(1, 801) > x_y_range = list(itertools.product(x_range, y_range)) > > grid = list(map(create_box, x_y_range)) > > Which creates and populates an 800x1000 ?grid? (represented as a flat list > at this point) of ?boxes?, where a box is a shapely.geometry.box(). This > takes about 10 seconds to run. > > Looking at this, I am thinking it would lend itself well to > parallelization. Since the box at each ?coordinate" is independent of all > others, it seems I should be able to simply split the list up into chunks > and process each chunk in parallel on a separate core. To that end, I > created a multiprocessing pool: I recall a similar discussion when folk were being encouraged to move away from monolithic and straight-line processing to modular functions - it is more (CPU-time) efficient to run in a straight line; than it is to repeatedly call, set-up, execute, and return-from a function or sub-routine! ie there is an over-head to many/all constructs! Isn't the 'problem' that it is a 'toy example'? That the amount of computing within each parallel process is small in relation to the inherent 'overhead'. Thus, if the code performed a reasonable analytical task within each box after it had been defined (increased CPU load), would you then notice the expected difference between the single- and multi-process implementations? From AKL to AK -- Regards =dn

- Prev by Date:
**Multiprocessing performance question** - Next by Date:
**Multiprocessing performance question** - Previous by thread:
**Multiprocessing performance question** - Next by thread:
**Multiprocessing performance question** - Index(es):