Home Original page

submit jobs on multi-core

Ian Kelly ian.g.kelly at gmail.com
Tue Sep 11 14:41:34 EDT 2012 
On Mon, Sep 10, 2012 at 11:53 PM, Laszlo Nagy <gandalf at shopzeus.com> wrote:
> On 2012-09-11 06:16, Dhananjay wrote:
>>
>> Dear all,
>>
>> I have a python script in which I have a list of files to input one by one
>> and for each file I get a number as an output.
>> I used for loop to submit the file to script.
>> My script uses one file at a time and returns the output.
>>
>> My computers has 8 cores.
>> Is there any way that I could submit 8 jobs at a time and get all the
>> output faster ?
>> In other words, how can I modify my script so that I could submit 8 jobs
>> together on 8 different processors ?
>>
>> I am bit new to this stuff, please suggest me some directions.
>
> You should first look at the multiprocessing module. It is part of the
> standard library.
>
> http://docs.python.org/library/multiprocessing.html

Or if Python 3.2 is an option, the concurrent.futures module would be
very well suited for this task.
More information about the Python-list mailing list