mcp - a script for mirroring files on remote servers

If you’re a command-line-happy nerd who likes to use scp (secure copy) to copy files to one or more remote web-servers, you may come to love this script, which I wrote a few years ago, and which I use numerous times every day. It saves a lot of typing when I’m updating files on my web-servers via the command-line (which is how I usually do it).

Here's my (perl) script, mcp, which stands for "mirror copy". You may need to modify the first line to indicate the location of the perl executable on your system.

To date, I've attempted to share this little gem with very few people, because describing the problem it solves is a little complicated, and not that many people actually encounter the problem.

If you use scp as much as I do, you will almost definitely find it useful. If you use ftp a lot to copy files, you may like it even more - because ftp is a comparative hassle, and not particularly secure. But first you have to get in the habit of using scp instead of ftp. Come back after you've accomplished that feat. You'll also need to learn how to use ssh/scp without having to constantly enter passwords. More info here.

If you don't fit the above criteria, stop reading now. I'm about to get nerdy.

Okay. Let's imagine I have a few different web servers that I maintain. On my laptop, the files in the directory

/Users/jbum/development/websites/happyclam/

Are mirrored on a remote server, happyclam.com, on the following path:

/home/happyclam/public_html/

If I want to copy a local file to that remote server, I would issue an scp (secure copy) command like this:

scp index.html happyclam@happyclam.com:/home/happyclam/public_html/

By using my script, which is called mcp or "mirror copy", I can simply type

mcp index.html

And the script "knows" where the file needs to go. It issues the full scp command for me:

$ mcp index.html
scp index.html happyclam@happyclam.com:/home/happyclam/public_html/
index.html                                       100% 1744     1.7KB/s   00:00    
$ _

Moreover, if I need to copy a file from the remote machine to here on the local machine, I can do that with the -r option, and it will reverse the sense of the transfer.

$ mcp -r index.html
scp happyclam@happyclam.com:/home/happyclam/public_html/index.html .
index.html                                       100% 1744     1.7KB/s   00:00    
$ _

Now, there are other sub-trees on my local filesystem, that are associated with other, completely different web servers, in some cases on other computers that have different usernames and passwords. So for example:

/Users/jbum/development/websites/krazyeddy/

goes to

eddy@krazyeddy.com:/home/dad/public_html/

When I use mcp over in this directory, it "knows" that the files need to go to this different location.

mcp myfile.html

results in

scp myfile.html  eddy@krazyeddy.com:/home/dad/public_html/myfile.html

and I have another directory that needs to be mirrored on two different machines.

/Users/jbum/development/websites/testdomain/test

goes to the following two locations:

happyclam@happyclam.com:/home/happyclam/public_html/production_test/
eddy@krazyeddy.com:/home/krazyeddy/public_html/development_test/

When I go to that local directory and use mcp, the file gets copied to both places.

The way mcp knows where to copy files is that it looks for a hidden file with the name ".mcp" in the current directory. This file contains the destination paths that the files need to be copied to, for example, it might contain the following two lines:

happyclam@happyclam.com:/home/happyclam/public_html/production_test/
eddy@krazyeddy.com:/home/krazyeddy/public_html/development_test/

If mcp can't find an .mcp file in the current directory, it goes one level up, to the parent directory, and looks there, and keeps repeating this, until it finds an .mcp file. If you are a few directories down from the .mcp file, it will automatically add the path names necessary to insure the file goes to the right place. So for any sub-tree that is mirrored on some other machine, you only need a single .mcp file in the root-most directory of the tree on your local machine. All the files beneath that directory will go to the correct place on the remote server automatically, when you use mcp in those directories.

Because I use mcp a lot, I have ten or twelve of these .mcp files strewn about my filesystem, which correspond to different projects I'm working on that are also mirrored on a remote server somewhere. In order for you to use mcp, you will need at least one of these .mcp files, typically in a local directory that corresponds to where you put .html files on your remote server.

You can create an .mcp file with the echo command, like so:

echo "happyclam@happyclam.com:/home/happyclam/public_html/" >.mcp

If you're not absolutely sure what mcp is going to do, you can test it with the "-t" option, which causes it to show you what it would do, without acting on it.

mcp -t remotefile.html

Finally, if your .mcp file is set to go to a few different destination servers, but you only want a file go to one of them, you can do that with the -n option, which allows you to specify a particular one, by line number. The following command would only send the file to the first server in the list.

mcp -n=1 remotefile.html

So that's what my mcp script does. I hope one or two of you get some enjoyment out of it.

Finally, I should mention that in order for mcp to be useful, you need to set up your ssh/scp access so you don't need to enter passwords every time you use scp. I do this on all the machines I work with a lot. This webpage explains how to do that.