Copying directories recursively using shell script

Question

Should be an easy question for the gurus here, though it's hard to explain it in text so hopefully this is clear. I've got two directories on a box with some flavor of unix on it. I've got a script that I want to use to move all the files and directories from one location to another.

First, an example of how the directories look:

Directory A: final/results/2012/2012-02/2012-02-25/name/files

Directory B: test/results/2012/2012-02/2012-02-24/name/files

So you see they're very similar. What I want to do is move everything from the Directory B 2012 directory, recursively, to the same level of Directory A. So you'd end up with:

someproject/results/2012/2012-02/2012-02-25/name/files

someproject/results/2012/2012-02/2012-02-24/name/files

etc.

I want this script to be future proof though, meaning I don't want the 2012 hardcoded. Also, towards the end of a month you will potentially have data from two different months and both need to be copied into the 2012 directory. So here is the command I used in the shell script file:

CONS="/someproject";
ROOT="/test";

/bin/cp -r ${ROOT}/results/* ${CONS}/results/*

but this resulted in:

/final/results/2012/2012-02/2012-02-25/name/files

and

/final/results/2012/2012/2012-02/2012-02-24/name/files

So as I hope is clear, it started a level below where I wanted it too. Can anyone fill me in on what I'm doing wrong, if they can understand what I'm even trying to explain. My apologies if it's not clear. I'm sure this is a fairly simple fix but I'm not sure what to do. Shell scripting is not a strong point of mine.

The second * on your cp command is not a good plan. It's expanding to a list of the directories in the destination, and then copying everything into the last one listed (since that's what cp does). Strike the second * and you'll be good to go. — zmccord
– zmccord, Commented Feb 27, 2012 at 16:23
@zmccord Thanks! I knew it was something small and stupid, but as I said, scripting is not my strong point. That worked perfectly. If you add it as an answer I'll accept it. — cardician
– cardician, Commented Feb 27, 2012 at 16:43
@sgibb I tried it this way, without either * and it didn't provide the desire result. Instead it added another result directory (and everything below) to Director A's result directory. But trying it without the second * worked perfectly. Thanks though. — cardician
– cardician, Commented Feb 27, 2012 at 18:13

Perry · Accepted Answer · 2012-02-27 17:15:07Z

5

One poster suggests rsync, which is overkill.

cp -rp will work fine. if you want to move the files, just mv the directory -- it and everything under it will move too.

The only real problem here is the use of terminating *'s in the command line in the original script. You don't need the *, you're just trying to pass directories to the cp command, you aren't trying to pass it the names of all the files already in the source (and more importantly, the destination).

answered Feb 27, 2012 at 17:15

Perry

4,4951 gold badge19 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

cardician Over a year ago

Yep, the last * in the cp command was the problem as another poster pointed out in a comment to my original thread. I'll select your answer as the answer if he doesn't post one. However, I tried it both ways, without either * or with just the first one and I only get the desired result when I leave the first * in and remove the second one.

Perry Over a year ago

That is because * means "all files not starting in .", which will in most cases mean the same thing -- unless, however, you have a file name staring in . -- so if I were you, I would drop the * unless you know why you are using it.

ghoti · Accepted Answer · 2012-02-27 16:34:48Z

4

You could also use a tool like rsync to make sure your source and target are synchronized.

rsync -av ${ROOT}/results/ ${CONS}/results/

You specified that you want to "move" the files, though. Which means deleting the originals after they're copied:

rsync -av --remove-source-files ${ROOT}/results/ ${CONS}/results/

If you start playing around with rsync, be sure to read the man page about how it treats trailing slashes.

answered Feb 27, 2012 at 16:34

ghoti

47.2k8 gold badges70 silver badges108 bronze badges

2 Comments

cardician Over a year ago

Not an available application on whatever platform this script is running on. Solaris perhaps? Dunno. Thanks though.

Perry Over a year ago

Rsync is overkill for this sort of job. Normal cp or mv runs fine. rsync is for synchronizing files over a network -- it is rarely needed for a job like this.

Collectives™ on Stack Overflow

Copying directories recursively using shell script

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related