For years, I used shared hosting systems exclusively for all my websites. They were affordable, easy to manage, and required zero effort on my part to set up and maintain. Unfortunately, I have a habit of rapidly outgrowing hosts and usually have to migrate to bigger and better servers about once a year. I finally decided to bite the bullet and purchased a VPS hosting system with a great company called Atum last month.
The advantage of this is that I have complete control over the VPS. Whereas before I could only host websites, I can now host my own development repositories (currently rocking Subersion, Mercurial, and Git all on my server). It protects me from the inevitable crashes of my aging computer systems, and makes life a lot easier for me.
While reveling in my newfound power, though, I ran into a serious issue: I had no idea how to set things up!
I’d read this or that tutorial and get things off the ground. Setting up Apache took a few tries. I’m still working on configuring email. But the biggest challenge I’ve faced thus far is setting up an automated Subversion-to-Git mirror that would run on a cron job.
Recently, I started working on a project that needed to maintain an up-to-date mirror of WordPress’ Subversion repository … in a Git repository. It’s easy enough to use git-svn to synchronize the repositories. That I figured out pretty quick. The challenge lies in automating the process. It took me a month to figure out the right way to set up a cron job. So, to save you the trouble, here’s the process I took, heavily derived from steps outlined by Bob McWhirter.
Set up the hosted repository
My team elected to work with Gitorious. It’s a site that hosts public Git repositories for free, and it’s not as crowded as GitHub. So the first step is to set up an empty repository there. You’ll be pushing all of your Subversion stuff into it. Mine is called “Daybreak:”
Set up a Git repository on your server
Remember, the point here is to automate everything. So we need to set up a Git repository on our server so the server can grab from Subversion and push it into Gitorious. Log in to your server via SSH and make an empty repository somewhere:
[cc lang=”bash”]
cd /var/www/git/
mkdir daybreak
cd daybreak
git init
[/cc]
Next we need to tell the repository that it’s cloning from a Subversion system and which branch to take up.
[cc lang=”bash”]
git svn init -T http://core.svn.wordpress.org/trunk/
[/cc]
The -T reminds Git to grab trunk, only trunk, and nothing but trunk. Yes, you could clone the branches as well, but I only want the latest development version in my repository. That’s what I’m working with and everything else is noise.
At this point, we have a Git repository with nothing in it. We need to tell it to fetch the Subversion revision history, but here is where you need to be careful! WordPress has over 17,000 commits in core. If you try to fetch everything, the task might never complete. I say “never” because I tried … walked away from the computer for several hours … and came back to see I was still on revision 2,000. Trust me, you won’t ever need to revert that far, so grab a recent revision and fetch from there. I picked RC3 (revision 17355).
[cc lang=”bash”]
git svn fetch -r 17355
[/cc]
This command might run for a while, depending on how long it’s been between the revision you’re fetching and trunk’s current head. After it’s done, make sure you run the garbage collector to clean things up a bit.
[cc lang=”bash”]
git gc
[/cc]
Set up your SSH credentials
Once again, we want the server to be the one pushing these commits into the repository, so we have to do a few things to set up permission. The first is set up a public/private key pair for use with Gitorious.
[cc lang=”bash”]
cd ~
ssh-keygen -t dsa -f .ssh/id_dsa_gitorious_mirroring
[/cc]
I usually go back to the root directory before running commands like this. I don’t know if there are any negative consequences of running the command elsewhere, this is just what I did.
Next we want to edit our SSH configuration to use a “fake” host so any connection to Gitorious uses the key we just made.
[cc lang=”bash”]
cd /.ssh
vim config
[/cc]
This opens the config file in vim. You need to add the following:
[cc lang=”vim”]
Host gitoriousmirror
User git
Hostname gitorious.org
IdentityFile /home/eric/.ssh/id_dsa_gitorious_mirroring
[/cc]
Save the config file and open your public key file so you can copy the SSH key to your Gitorious account.
[cc lang=”bash”]
vim id_dsa_gitorious_mirroring.pub
[/cc]
On Gitorious, go to your Dashboard and click “Manage SSH Keys.” Then click “Add SSH Key” and copy the contents of your public key into the textarea provided. This makes it so Gitorious will accept pushes from your server.
Set up Gitorious as a remote
Your server’s repository is now set up to clone from WordPress and it knows what SSH key to send to Gitorious. Now we tell it how to talk to Gitorious.
[cc lang=”bash”]
cd ~
cd /var/www/git/daybreak/
git remote add origin git@gitoriousmirror:daybreak/core.git
[/cc]
Now we push to Gitorious.
[cc lang=”bash”]
git push origin master
[/cc]
Voila! You’ve just pushed from Subversion to Git. Doing so again is as simple as rebasing the repository and pushing back:
[cc lang=”bash”]
git svn rebase
git gc
git push origin master
[/cc]
Configure your cron job
This is the vital step that was missing from all the other tutorials I read. I’m not a Linux guy, so “just add this script” doesn’t tell me where or how to do it. I’d never configured a cron job before, so I was in way over my head. But I did eventually figure it out … so learn from my mistakes.
Lesson 1: Don’t do anything as root
I was running everything as the root user. Coming from Windows, I’m used to always doing things as the system administrator. But everyone I asked for help immediately said, “don’t run as root, that’s a big no-no.” I’m not sure if it is, but I created a new user anyway and did everything from that account instead.
Lesson 2: Remember where you installed git
My first attempt at writing a script was just using the same commands as above and invoking Git directly. But Git wasn’t installed that way and cron was throwing errors because the system didn’t know what I was talking about. So I eventually created a file named `wordpress.sh` in my `/etc` folder. Just use vim to do the same.
[cc lang=”vim”]
#!/bin/bash
cd /var/www/git/daybreak
/usr/bin/git/git checkout wordpress
/usr/bin/git/git svn rebase
/usr/bin/git/git gc
/usr/bin/git/git push origin wordpress
[/cc]
This script jumps to my repository’s folder, makes sure I’m on the WordPress branch, rebases from the remote Subversion repository, cleans up some garbage, and pushes the changes to Gitorious. Quick, simple, easy to understand.
Lesson 3: CentOS Webadmin is your friend
I spent many fruitless hours trying to add my cron job through the command line. Then I remembered CentOS allows cron management from the webadmin control panel. So I logged in and told it to execute the following command every 15 minutes:
[cc lang=”bash”]
sh /etc/wordpress.sh
[/cc]
And just like that, everything works!
You can now follow WordPress core development live (at least within 15 minutes) through the WordPress branch of my Daybreak project on Gitorious!
