I will be adding to this doc as time goes on, now that AWS, my home laptops, DNS, MySQL (rep and cluster), and mongo are setup. I have to redo some of this in SALT however.
WARNING: you are opening up several ports for salt on servers. Make sure they are firewalled and those ports are not open to the outside. Make sure in AWS you don't let anybody use those ports and block off access to those ports for people not in your local are network.
Links:
- http://docs.saltstack.com/en/latest/topics/installation/ubuntu.html
- http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html
- https://www.youtube.com/user/SaltStack/videos
- TODO: run as non-root user: http://docs.saltstack.com/en/latest/ref/configuration/nonroot.html
- chown -R user /etc/salt /var/cache/salt /var/log/salt /var/run/salt
- But will this affect the restart scripts?
- http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html
- http://docs.saltstack.com/en/latest/ref/modules/all/
- Yes, the mysql module allows connection parameters to be overridden. Basically, I want to be able to connect with read-only connection, and maybe at some point make a writable connection.
- http://bencane.com/2013/09/03/getting-started-with-saltstack-by-example-automatically-installing-nginx/
- Wish I would have found this first.
- https://salt.readthedocs.org/en/v2014.1.0rc1/ref/clients/index.html
Steps
- Add salt.mylocaldomain to the DNS
- Add "salt IN CNAME ns2" to /etc/bind/db.mylocaldomain
- Restart DNS: service bind9 restart
- The Master (which will also be a minion). We do everything to the master, and then one section for each other service.
-
sudo add-apt-repository ppa:saltstack/salt sudo apt-get update sudo apt-get install salt-master salt-minion salt-syndic
-
sed -i 's/\#master: salt/master: salt.mylocaldomain/' /etc/salt/minion # It asks a question I can't seem to get rid of here. Good in a way. service salt-master restart service salt-minion restart service salt-syndic restart
Now setup the minion part of this server -- use the master as its own minion.
- When minion is restarted above it will attempt to make its own key and connect to the master.
- salt-call key.finger --local @ Should look like
- local:
- 28:01:06:52:9e:e6:97:7d:cc:00:2d:7f:fa:16:93:a3
- Now accept the key on the master:
- List the key status: salt-key -L
- Accept the key either as
- salt-key -A
- or, salt-key -a NAME
- Install it:
-
sudo add-apt-repository ppa:saltstack/salt # It asks a question I can't seem to get rid of here. Good in a way. sudo apt-get -y update sudo apt-get -y install salt-minion sed -i 's/\#master: salt/master: salt.mylocaldomain/' /etc/salt/minion service salt-minion restart
- Do an "nslookup salt.mylocaldomain" and if it comes up correct.
- If so, when you restarted salt minion it should have submitted a key to the salt master.
- On the master: salt-key -A.
The first commands (some of them are in the docs). Do this on the master.
- salt '*' test.ping
- On the hosts should return "True". If not, the DNS might be wrong, you may not have installed sal-minion on the target, or its config file doesn't point to the master, or the master had not accepted its key
- salt '*' disk.usage
- salt '*' disk.usage --out raw
- salt '*' disk.usage --out yaml
Now let's put the command in a script. Parsing console output is stupid when its already parsed in the language.
Make this script and save it as "salt.py" :
import salt.client
local = salt.client.LocalClient()
result = local.cmd('*', 'test.ping', timeout=2)
print result
result = local.cmd('*', 'disk.usage', timeout=2)
print result
And then execute the script as "python salt1.py". It shows the output (I think in json format) and you can modify the script to deal with the parsed data instead of reparsing it from a cli. I will have examples in scripts for my DAD.
Now let's parse the output of the "result" output. For this module, it seems be a multi hash. Let's just get the capacity of each partitions.
import salt.client
local = salt.client.LocalClient()
# I chose this one since I think it might be multi process, not sure.
# I need to actually test it out.
inter1 = local.cmd_iter('*', 'disk.usage', timeout=2)
for i in inter1:
for host in i:
# print i[host]
retcode = i[host]['retcode']
if retcode != 0:
print "ERROR:", host, retcode
else:
ret = i[host]['ret']
for partition in ret:
capacity = ret[partition]['capacity']
print host, partition, capacity
# which hosts failed. There msust be a better way where above it will return the ones failed.
result = local.cmd('*', 'test.ping', timeout=2)
print result
Now I want to see if this is multi-process (or threaded or whatever)
import salt.client
local = salt.client.LocalClient()
inter1 = local.cmd_iter('*', 'cmd.run', ['echo "test" > /tmp/START; sleep 120; echo "test" > /tmp/STOP'], timeout=2)
for i in inter1:
for host in i:
# print i[host]
retcode = i[host]['retcode']
if retcode != 0:
print "ERROR:", host, retcode
else:
ret = i[host]['ret']
for partition in ret:
capacity = ret[partition]['capacity']
print host, partition, capacity
And looking at the output of the processes on the master server.....
mark2 ~ # ps auwx | grep minion
root 4544 0.0 0.5 522460 20036 ? S 01:14 0:00 /usr/bin/python /usr/bin/salt-minion
root 4725 0.0 0.0 11744 912 pts/2 S+ 01:15 0:00 grep --colour=auto minion
root 7815 0.0 0.2 73712 10196 ? Ss Jun17 0:00 /usr/bin/python /usr/bin/salt-minion
root 7819 0.0 0.5 522460 20296 ? Sl Jun17 7:24 /usr/bin/python /usr/bin/salt-minion
Now looking at the timestamps of the 3 servers.....
mark ~ # ls -al /tmp/S* -rw-r--r-- 1 root root 5 Jun 25 01:14 /tmp/START -rw-r--r-- 1 root root 5 Jun 25 01:16 /tmp/STOP mark2 ~ # ls -al /tmp/S* -rw-r--r-- 1 root root 5 Jun 25 01:14 /tmp/START -rw-r--r-- 1 root root 5 Jun 25 01:16 /tmp/STOP mark3 ~ # ls -al /tmp/S* -rw-r--r-- 1 root root 5 Jun 25 01:14 /tmp/START -rw-r--r-- 1 root root 5 Jun 25 01:16 /tmp/STOP
We see that they all have the same timestamps, and there were 3 processes running, so that command is verified to be what we want. Imagine if we have 1000 computers. Doing this serially would suck.
Now let's organize stuff ---- doing.
