This is going to focus on GIT.
- Develop a home git environment.
- initialize project
- Clone the project locally
- Make available for remote acess
- Do a test run remotely.
- Replicate it to AWS via SSH,
- Integrate with SALT somehow.
This is going to be an ongoing doc. Just started it.
Links:
- https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
- https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository
- https://www.kernel.org/pub/software/scm/git/docs/git-config.html
- http://www.opentaps.org/docs/index.php/How_to_Use_Git:_a_Tutorial
- http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/
- http://www.cyberciti.biz/tips/linux-unix-get-yesterdays-tomorrows-date.html
sudo apt-get install git
addgroup --gid 10000 gitadmin
# We need to use a specific user and group because we are going to ssh
# over everything to another server.
adduser --disabled-password --disabled-login --uid 10000 --gid 10000 gitadmin --gecos GECOS
mkdir -p /opt/git/menprojects
chown -R gitadmin.gitadmin /opt/git/menprojects
su -l gitadmin
git config --global user.name "Git Admin"
git config --global user.email gitadmin@localhost
git config --global core.editor emacs
# Don't ask me why, but I kept screwing this up.
cd /opt/git/menprojects
git init --bare --shared test_project
cd
# Now setup a version this account can make changes to but won't affect
# anything until it is committed. If you mess up this copy, you can delete it
# and who cares.
git clone file:///opt/git/menprojects/test_project
cd test_project
echo date > test_file
# add it local clone
git add test_file
# commit to local clone -- not pushed to main repository yet
git commit -m "test commit"
# push it to the main repository
git push origin master
exit
Test anther account with local copy.
adduser --disabled-password --disabled-login git_test1 --gecos GECOS --gid 10000
su -l git_test1
ssh-keygen -t dsa -N "" -f ~/.ssh/id_dsa
cp .ssh/id_dsa.pub .ssh/authorized_keys
ssh localhost -o StrictHostKeyChecking=no "echo 'ssh worked'"
git config --global user.email "git_test1@localhost"
git config --global user.name "Your Name"
git clone file:///opt/git/menprojects/test_project
cd test_project
echo date > test_file2
# add it local clone
git add test_file2
# commit to local clone -- not pushed to main repository yet
git commit -m "test commit"
# push it to the main repository
git push origin master
# now let's test this over ssh.
cd
git clone git_test1@localhost:/opt/git/menprojects/test_project test_project_ssh
cd test_project_ssh
echo date > test_file3
# add it local clone
git add test_file3
# commit to local clone -- not pushed to main repository yet
git commit -m "test commit"
# push it to the main repository
git push origin master
Now Let's do a backup once a day on another server. This is local, so mark3 is another server. In this step, root already has access to the other servers as root. This is not wise in production in my opinion, things can be done with sudo, but I am making it easy for me.
su -l root
exit
# if you haven't su-l root already, do it. Make sure root has keys setup on the other servers,
ssh mark
# copy the rest AFTER you login in to mark
# This could be be automated better by scp over a file and then executing it
# but I am being lazy.
adduser --disabled-password --disabled-login git_backup --gecos GECOS
su -l git_backup
ssh-keygen -t dsa -N "" -f ~/.ssh/id_dsa
exit
exit
# git_admin needs ssh setup. Let the program do it.
sudo -u gitadmin ssh-keygen -t dsa -N "" -f /home/gitadmin/.ssh/id_dsa
# exchange keys
scp mark:/home/git_backup/.ssh/id_dsa.pub /home/gitadmin/.ssh/authorized_keys
scp /home/gitadmin/.ssh/id_dsa.pub mark:/home/git_backup/.ssh/authorized_keys
# Set permissions.
chown gitadmin /home/gitadmin/.ssh/authorized_keys
ssh mark "chown git_backup /home/git_backup/.ssh/authorized_keys"
# test logins
ssh mark "sudo -u git_backup ssh -o StrictHostKeyChecking=no gitadmin@mark2 'echo it worked'"
# Make backup script and cron it. Use no extra diskspace.
ssh mark
mkdir -p /backups/git/menprojects
chown git_backup /backups/git/menprojects
su -l git_admin
echo "D=\`date +%Y%M%d_%H%m%S\`" > daily_backup.sh
echo "ssh gitadmin@mark2 \"tar -cvzf - -C /opt/git menprojects\" > /backups/git/menprojects/\$D.tgz 2>/backups/git/menprojects/$D.log " >> daily_backup.sh
echo "D2=\`date +%Y%M%d_%H%m%S --date='60 days ago'\`" >> daily_backup.sh
echo " rm -vf /backups/git/menprojects/\$D2.tgz " >> daily_backup.sh
# test it
chmod 755 daily_backup.sh
./daily_backup.sh
# add the crontob, technically it should go under /etc/cron/cron.daily
echo " 1 1 * * * /home/git_backup/daily_backup.sh > /var/tmp/backup.log 2>/var/tmp/backup.err" > menprojects.cron
crontab menprojects.cron
exit
exit
# Should be back at main server.
# Open up another terminal for me because my AWS keys are not on root.
Let's setup the AWServer. That's our production side. It can't affect code change, just read. This is technically a lie, you could write to it. Making it read only is left up to the user.
# On the server, make same account and ssh key. In the future we will only
# be installing stuff with package management, but this is a good practice.
# Copy over local ssh key to AWS server.
# On local server, as root, copy the key to tmp.
su -l root
scp /home/gitadmin/.ssh/id_dsa.pub /tmp/
# Copy over ssh key as other user which has keys to AWS server.
su -l USER_which_has_access_to_AWS
scp -i /PEM_FILE /tmp/id_dsa.pub USER@SERVER.amazonaws.com:/tmp/
# As root on AWS server.
sudo bash
sudo apt-get install git
addgroup --gid 10000 gitadmin
adduser --disabled-password --disabled-login --uid 10000 --gid 10000 gitadmin --gecos GECOS
mkdir -p /opt/git/menprojects
chown -R gitadmin.gitadmin /opt/git/menprojects
su -l gitadmin
git config --global user.name "Git Admin"
git config --global user.email gitadmin@localhost
git config --global core.editor emacs
ssh-keygen -t dsa -N "" -f ~/.ssh/id_dsa
cp /tmp/id_dsa.pub .ssh/authorized_keys
# As gitadmin on local server
su -l gitadmin
rsync -av --delete-after /opt/git/menprojects/* gitadmin@SERVER.amazonaws.com:/opt/git/menprojects/
# This point, it should be useable.
# We will test it only to make sure. We DON'T want to write to it normally.
# on the AWS server
su -l gitadmin
git clone file:///opt/git/menprojects/test_project
cd test_project
echo date > test_file4
# add it local clone
git add test_file4
# commit to local clone -- not pushed to main repository yet
git commit -m "test commit"
# push it to the main repository
git push origin master
# now we have a bogus file on the prod side, remove it.
# On the local server.
su -l gitadmin
rsync -av --delete-after /opt/git/menprojects/* gitadmin@SERVER.amazonaws.com:/opt/git/menprojects/
# on the AWS server
# there is now a file in the clone, not in the master.
# git status ---- DOES NOT WORK.
su -l gitadmin
cd test_project
# Does not work.
git status
# Fetch it, sync it
git fetch
# Now status will report it is one ahead
git status
# This does a reset, but doesnt' eliminate file.
# I need something to check ALL files.
git reset HEAD^ --hard
As a final test, put all automated steps in git under sample code. Everything in here should be automated.
# on local server
cd /opt/git/menprojects
git init --bare --shared samplecode
cd
git clone file:///opt/git/menprojects/samplecode
cd samplecode
#Add files, commit them, then push.
# Now sync to main server
rsync -av --delete-after /opt/git/menprojects/* gitadmin@SERVER.amazonaws.com:/opt/git/menprojects/
# On AWS server under gitadmin, update copy to make sure it is in there.
su -l gitadmin
cd menprojects
git pull
Other stuff:
- I think SALT should be git and git not dependent on SALT. Why? you just need to set git up once. If you want to put SALT configs for git in git, that's fine.
- Figure out a better way to display all differences between clone and master.
- Generation of packages instead of relying on source code for deployment.
- Make git server available through download. Only allow limited number of connections --- maybe run in a separate instance so the main website is secure?
No comments:
Post a Comment