Use God with multiple applications and start them automatically after a reboot

I'm currently trying to monitor various processes/daemons of in total three Rails/Rack Applications using god. Monitoring works great, the problem is that i'm not able to configure god to autostart all processes after a reboot.

My Setup: I'm running a Linux VPS with Centos & Plesk. I have a non-root linux user "deployer" which is used to deploy & run the three Rails/Rack Applications. Two applications are running with the passenger apache module, the third application uses a thin Server (that's necessary because the application doesen't work with apache). The two Rails applications, that are using passenger have additional rake tasks that run in the background - these and the thin Server are monitored by god.

The god gem is specified in the Gem File of all three Applications.

In every deploy.rb file i have a method that looks like

namespace :misc do
    desc "restart woekers using gog; restart webserver"
    task :restart, roles: [:web, :resque] do
        run "touch #{current_path}/tmp/restart.txt"
        god.all.start
        god.all.reload
        god.all.terminate
        god.all.start
    end
end

After a reboot of the server, if i run the cap misc:restart for all three applications manually, all processes are booted up and monitored correctly.

Every try to start god automatically on boot and start all necessary processes failed so far. I tried many different things, but nothing worked. My approach so far was to create a cron task with @reboot that runs three of the following script:

#!/bin/bash -l

cd /path/to/app/ && bundle exec god -c /path/to/app/config/god/resque.god && bundle exec god load /path/to/app/config/god/resque.god && bundle exec god start resque

This works great for the first application: god and all processes are started. When the script is executed for the second application (of course with the with the correct paths), god is not able to start the tasks. I enabled logging in god and the error message (in case of the Rack Application) was "thin: command not found". When I'm starting the Rack Application first, thin is started correctly and the commands of the other task are not found.

I don't get whats wrong with my configuration. I added the bundle exec command in front of the god calls as you can see above (so the commands should be executed in the environment of their respective application) - nevertheless, it just doesen't work.

I would really appreciate if anyone could help me getting god to start automatically.

If you need further information please don't hesitate to ask!

Thanks in Advance!

Answers


Am working on something similar and took this approach:

Use upstart or something similar to launch the god daemon on system boot, for me this is done like so:

/etc/init/god.conf

description "god"

start on runlevel [2]
stop on runlevel [016]

console owner

exec /usr/local/rvm/bin/rvm_god -c /etc/god

respawn

That guy runs god specifying one ruby god configuration file with the -c option:

/etc/god

# Load the configs

God.load "/home/dangerousbeans/kitten_smusher/config/config.god"
God.load "/home/dangerousbeans/irc_nommer/config/config.god"

This ruby dude loads in the individual application god configs and running God.load causes them to boot up.

The individual files look like this I guess as I'm using RVM: /home/dangerousbeans/irc_nommer/config/config.god

God.watch do |w|
  w.dir = "/home/dangerousbeans/irc_nommer"
  w.name = "IRCnommer"

  # scary rvm magic begins
  gemsets_path = [
        "/home/dangerousbeans/.rvm/gems/ruby-1.9.3-p125@irc_nommer/bin",
        "/home/dangerousbeans/.rvm/rubies/ruby-1.9.3-p125/bin",
    "/home/dangerousbeans/.rvm/bin",
      ENV['PATH'] # inherit this
    ].join(':')

   w.env      = {
    "PATH"        => gemsets_path,
        "GEM_PATH"    => "/home/dangerousbeans/.rvm/gems/ruby-1.9.3-p125@irc_nommer"
    }
  # scary rvm magic ends

  w.log = "/tmp/ircnommer.log"

  w.start = "ruby /home/dangerousbeans/irc_nommer/irc_nommer.rb"
  w.keepalive
end

The key point is the environments is different between manual and automatic while god execute the [start] command. So you can add command env to the command. like:

God.watch do |w|
  w.start    = "cd #{your_app_directory}; env >> log/god.log; your-real-command >> log/god.log 2>&1"
end

There'll be some differences as you type env in the same directory. Check the difference and add required/correct paragraph to god's env.

Today I encounter an issue, I deployed 2 rails apps in 1 server, both uses god. The App#2 can't startup the command correctly. After do above test I found the cause: God hold an environment variable [BUNDLE_GEMFILE] that points to App#1. So I add a simple line then error gone away:

God.watch do |w|
  w.env = {
    "BUNDLE_GEMFILE" => "#{$rails_root}/Gemfile"
  }
end

Need Your Help

How to use COUNT() with MYSQLi and Prepared Statements?

php mysql count mysqli

I need to count the number of persons with the same status from a table for a specific user.. I want to use prepared statements so I did something like this (not working):

Going from Abstract Generic to Non-Abstract Non-Generic

java algorithm generics abstract-class bounded-wildcard

I've been working on a project in Java lately that uses a Generic-Best Search Algorithm. To make the Algorithm itself Generic, I used a bunch of Generics on all of the Classes used in the Algorithm.