Back

Explore Courses Blog Tutorials Interview Questions
0 votes
2 views
in Python by (47.6k points)

Is there a Pool class for worker threads, similar to the multiprocessing module's Pool class?

I like for example the easy way to parallelize a map function

def long_running_func(p):

c_func_no_gil(p)

p = multiprocessing.Pool(4)

xs = p.map(long_running_func, range(100))

however, I would like to do it without the overhead of creating new processes.

I know about the GIL. However, in my use case, the function will be an IO-bound C function for which the python wrapper will release the GIL before the actual function call.

Do I have to write my own threading pool?

1 Answer

0 votes
by (106k points)

Yes, there is a threading pool similar to the multiprocessing Pool, however, it is hidden somewhat and not properly documented. You can import it by following way:-

from multiprocessing.pool import ThreadPool

Related questions

0 votes
1 answer
asked Jun 27, 2019 in Python by Sammy (47.6k points)
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
asked Jun 27, 2019 in Python by Sammy (47.6k points)
Welcome to Intellipaat Community. Get your technical queries answered by top developers!

30.5k questions

32.5k answers

500 comments

108k users

Browse Categories

...