모든 스레드가 끝날 때까지 파이썬 다중 스레딩 대기

모든 스레드가 끝날 때까지 파이썬 다중 스레딩 대기

---

비슷한 맥락에서 질문을 받았을 지 모르지만 20 분 정도 검색해도 답을 찾을 수 없어서 물어 보겠습니다.

Python 스크립트 (scriptA.py)와 스크립트 (scriptB.py)를 작성했습니다.

scriptB에서 저는 다른 인수로 scriptA를 여러 번 호출하고 싶습니다. 매번 실행하는 데 약 한 시간이 걸립니다 (대용량 스크립트이며 많은 작업을 수행합니다. 걱정하지 마십시오). 모든 다른 인수를 동시에 사용하는 scriptA이지만 계속하기 전에 모든 인수가 완료 될 때까지 기다려야합니다. 내 코드 :

import subprocess

#setup
do_setup()

#run scriptA
subprocess.call(scriptA + argumentsA)
subprocess.call(scriptA + argumentsB)
subprocess.call(scriptA + argumentsC)

#finish
do_finish()

한꺼번에 모두 실행하고 싶고 subprocess.call(), 모두 끝날 때까지 기다립니다. 어떻게해야합니까?

여기 예제와 같은 스레딩을 사용하려고했습니다 .

from threading import Thread
import subprocess

def call_script(args)
    subprocess.call(args)

#run scriptA   
t1 = Thread(target=call_script, args=(scriptA + argumentsA))
t2 = Thread(target=call_script, args=(scriptA + argumentsB))
t3 = Thread(target=call_script, args=(scriptA + argumentsC))
t1.start()
t2.start()
t3.start()

그러나 나는 이것이 옳다고 생각하지 않습니다.

내로 가기 전에 모두 실행을 마쳤는지 어떻게 알 수 do_finish()있습니까?

---

스크립트의 끝에서 객체의 결합 방법 을 사용해야 Thread합니다.

t1 = Thread(target=call_script, args=(scriptA + argumentsA))
t2 = Thread(target=call_script, args=(scriptA + argumentsB))
t3 = Thread(target=call_script, args=(scriptA + argumentsC))

t1.start()
t2.start()
t3.start()

t1.join()
t2.join()
t3.join()

따라서 메인 스레드가 때까지 기다리는 t1, t2그리고 t3실행을 완료합니다.

---

스레드를 목록에 넣은 다음 Join 메서드 를 사용합니다.

 threads = []

 t = Thread(...)
 threads.append(t)

 ...repeat as often as necessary...

 # Start all threads
 for x in threads:
     x.start()

 # Wait for all of them to finish
 for x in threads:
     x.join()

---

입력 목록을 기반으로 목록 이해를 사용하는 것을 선호합니다.

inputs = [scriptA + argumentsA, scriptA + argumentsB, ...]
threads = [Thread(target=call_script, args=(i)) for i in inputs]
[t.start() for t in threads]
[t.join() for t in threads]

---

Python3에서는 Python 3.2부터 동일한 결과에 도달하는 새로운 접근 방식이 있습니다. 저는 개인적으로 전통적인 스레드 생성 / 시작 / 결합, 패키지를 선호합니다 concurrent.futures. https://docs.python.org/3/library/concurrent.futures .html

ThreadPoolExecutor코드를 사용하면 다음 과 같습니다.

from concurrent.futures.thread import ThreadPoolExecutor
import time

def call_script(ordinal, arg):
    print('Thread', ordinal, 'argument:', arg)
    time.sleep(2)
    print('Thread', ordinal, 'Finished')

args = ['argumentsA', 'argumentsB', 'argumentsC']

with ThreadPoolExecutor(max_workers=2) as executor:
    ordinal = 1
    for arg in args:
        executor.submit(call_script, ordinal, arg)
        ordinal += 1
print('All tasks has been finished')

이전 코드의 출력은 다음과 같습니다.

Thread 1 argument: argumentsA
Thread 2 argument: argumentsB
Thread 1 Finished
Thread 2 Finished
Thread 3 argument: argumentsC
Thread 3 Finished
All tasks has been finished

One of the advantages is that you can control the throughput setting the max concurrent workers.

---

You can have class something like below from which you can add 'n' number of functions or console_scripts you want to execute in parallel passion and start the execution and wait for all jobs to complete..

from multiprocessing import Process

class ProcessParallel(object):
    """
    To Process the  functions parallely

    """    
    def __init__(self, *jobs):
        """
        """
        self.jobs = jobs
        self.processes = []

    def fork_processes(self):
        """
        Creates the process objects for given function deligates
        """
        for job in self.jobs:
            proc  = Process(target=job)
            self.processes.append(proc)

    def start_all(self):
        """
        Starts the functions process all together.
        """
        for proc in self.processes:
            proc.start()

    def join_all(self):
        """
        Waits untill all the functions executed.
        """
        for proc in self.processes:
            proc.join()


def two_sum(a=2, b=2):
    return a + b

def multiply(a=2, b=2):
    return a * b


#How to run:
if __name__ == '__main__':
    #note: two_sum, multiply can be replace with any python console scripts which
    #you wanted to run parallel..
    procs =  ProcessParallel(two_sum, multiply)
    #Add all the process in list
    procs.fork_processes()
    #starts  process execution 
    procs.start_all()
    #wait until all the process got executed
    procs.join_all()

---

Maybe, something like

for t in threading.enumerate():
    if t.daemon:
        t.join()

---

From the threading module documentation

There is a “main thread” object; this corresponds to the initial thread of control in the Python program. It is not a daemon thread.

There is the possibility that “dummy thread objects” are created. These are thread objects corresponding to “alien threads”, which are threads of control started outside the threading module, such as directly from C code. Dummy thread objects have limited functionality; they are always considered alive and daemonic, and cannot be join()ed. They are never deleted, since it is impossible to detect the termination of alien threads.

So, to catch those two cases when you are not interested in keeping a list of the threads you create:

import threading as thrd


def alter_data(data, index):
    data[index] *= 2


data = [0, 2, 6, 20]

for i, value in enumerate(data):
    thrd.Thread(target=alter_data, args=[data, i]).start()

for thread in thrd.enumerate():
    if thread.daemon:
        continue
    try:
        thread.join()
    except RuntimeError as err:
        if 'cannot join current thread' in err.args[0]:
            # catchs main thread
            continue
        else:
            raise

Whereupon:

>>> print(data)
[0, 4, 12, 40]

---

I just came across the same problem where I needed to wait for all the threads which were created using the for loop.I just tried out the following piece of code.It may not be the perfect solution but I thought it would be a simple solution to test:

for t in threading.enumerate():
    try:
        t.join()
    except RuntimeError as err:
        if 'cannot join current thread' in err:
            continue
        else:
            raise

참고URL : https://stackoverflow.com/questions/11968689/python-multithreading-wait-till-all-threads-finished