TERM을 트래핑하고 QUIT를 보낸 후 Heroku에서 Unicorn 종료 시간 초과

TERM을 트래핑하고 QUIT를 보낸 후 Heroku에서 Unicorn 종료 시간 초과

---

unicorn 및 sidekiq을 실행하는 Heroku 앱에 대해 R12 종료 시간 초과 오류가 발생합니다. 이러한 오류는 배포 할 때마다 하루에 1-2 번 발생합니다. 유니콘이 올바르게 응답하려면 Heroku의 종료 신호를 변환해야한다는 것을 이해하지만 아래 유니콘 구성에서 그렇게했다고 생각했습니다.

worker_processes 3
timeout 30
preload_app true

before_fork do |server, worker|
  Signal.trap 'TERM' do
    puts "Unicorn master intercepting TERM and sending myself QUIT instead. My PID is #{Process.pid}"
    Process.kill 'QUIT', Process.pid
  end

  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.connection.disconnect!
    Rails.logger.info('Disconnected from ActiveRecord')
  end
end

after_fork do |server, worker|
  Signal.trap 'TERM' do
    puts "Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is #{Process.pid}"
  end

  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    Rails.logger.info('Connected to ActiveRecord')
  end

  Sidekiq.configure_client do |config|
    config.redis = { :size => 1 }
  end
end

오류를 둘러싼 내 로그는 다음과 같습니다.

Stopping all processes with SIGTERM
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 7
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 11
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 15
Unicorn master intercepting TERM and sending myself QUIT instead. My PID is 2
Started GET "/manage"
reaped #<Process::Status: pid 11 exit 0> worker=1
reaped #<Process::Status: pid 7 exit 0> worker=0
reaped #<Process::Status: pid 15 exit 0> worker=2
master complete
Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
Stopping remaining processes with SIGKILL
Process exited with status 137

시간 초과 전에 모든 하위 프로세스가 성공적으로 회수 된 것으로 보입니다. 마스터가 아직 살아있을 수 있습니까? 또한 로그에 표시된 것처럼 라우터가 종료 중에 dyno에 웹 요청을 계속 보내야합니까?

FWIW, 저는 Heroku의 제로 다운 타임 배포 플러그인 ( https://devcenter.heroku.com/articles/labs-preboot/ )을 사용하고 있습니다.

---

I think your custom signal handling is what's causing the timeouts here.

EDIT: I'm getting downvoted for disagreeing with Heroku's documentation and I'd like to address this.

Configuring your Unicorn application to catch and swallow the TERM signal is the most likely cause of your application hanging and not shutting down correctly.

Heroku seems to argue that catching and transforming a TERM signal into a QUIT signal is the right behavior to turn a hard shutdown into a graceful shutdown.

However, doing this seems to introduce the risk of no shutdown at all in some cases - the root of this bug. Users experiencing hanging dynos running Unicorn should consider the evidence and make their own decision based on first principles, not just documentation.

참고URL : https://stackoverflow.com/questions/17450732/unicorn-exit-timeout-on-heroku-after-trapping-term-and-sending-quit