Bash script to copy a directory from multiple subdirectories with directory structure

Question

New to shell script, have to copy the class files from different subdirectories to another destination directory My directory structure is

Module
    |-- submodule
    |         |--main
    |         |--test
    |         |--target
    |               |__classes
    |               |__src  
    |
    |
    |-- subdirectory
              |--maintest
              |--target
                    |__ classes
                    |__ src

I have to copy all the files from target/classes directory to destination directory "POM" and expecting the directory structure as below

 POM
    |-- submodule
    |         |__ target
    |               |__classes
    |
    |-- subdirectory
              |_target
                    |__ classes

tried using the below command

for dest in POM/; do cp -r modules/*/target/classes $dest; done

But it copied only the submodule2 directories.

Note: we are restricted to use the rsync command.

You can rsync as well.

Riz
– Riz

2022-01-12 14:32:21 +00:00
Commented Jan 12, 2022 at 14:32 — Riz
– Riz, Commented Jan 12, 2022 at 14:32

dawg · Accepted Answer · 2022-01-12 16:22:10Z

Given this file tree:

 tree /tmp/Module
/tmp/Module
├── submodule1
│   └── target
│       └── classes
│           ├── class_1
│           ├── class_2
│           ├── class_3
│           ├── class_4
│           └── class_5
└── submodule2
    └── target
        └── classes
            ├── class_1
            ├── class_2
            ├── class_3
            ├── class_4
            └── class_5

6 directories, 10 files

Just use cp -R [source path] [target path] (ie, no shell loop needed and no glob):

% cp -vR /tmp/Module /tmp/POM
/tmp/Module -> /tmp/POM
/tmp/Module/submodule2 -> /tmp/POM/submodule2
/tmp/Module/submodule2/target -> /tmp/POM/submodule2/target
/tmp/Module/submodule2/target/classes -> /tmp/POM/submodule2/target/classes
/tmp/Module/submodule2/target/classes/class_4 -> /tmp/POM/submodule2/target/classes/class_4
/tmp/Module/submodule2/target/classes/class_3 -> /tmp/POM/submodule2/target/classes/class_3
/tmp/Module/submodule2/target/classes/class_2 -> /tmp/POM/submodule2/target/classes/class_2
/tmp/Module/submodule2/target/classes/class_5 -> /tmp/POM/submodule2/target/classes/class_5
/tmp/Module/submodule2/target/classes/class_1 -> /tmp/POM/submodule2/target/classes/class_1
/tmp/Module/submodule1 -> /tmp/POM/submodule1
/tmp/Module/submodule1/target -> /tmp/POM/submodule1/target
/tmp/Module/submodule1/target/classes -> /tmp/POM/submodule1/target/classes
/tmp/Module/submodule1/target/classes/class_4 -> /tmp/POM/submodule1/target/classes/class_4
/tmp/Module/submodule1/target/classes/class_3 -> /tmp/POM/submodule1/target/classes/class_3
/tmp/Module/submodule1/target/classes/class_2 -> /tmp/POM/submodule1/target/classes/class_2
/tmp/Module/submodule1/target/classes/class_5 -> /tmp/POM/submodule1/target/classes/class_5
/tmp/Module/submodule1/target/classes/class_1 -> /tmp/POM/submodule1/target/classes/class_1

Result:

% tree /tmp/POM
/tmp/POM
├── submodule1
│   └── target
│       └── classes
│           ├── class_1
│           ├── class_2
│           ├── class_3
│           ├── class_4
│           └── class_5
└── submodule2
    └── target
        └── classes
            ├── class_1
            ├── class_2
            ├── class_3
            ├── class_4
            └── class_5

6 directories, 10 files

If you actually want a Bash script to do this (for instance if you want to process some of these files), you would do something along these lines:

#!/bin/bash

cd /tmp || exit
fc=0; dc=0
for fn in /tmp/Module/**/*; do
    if [ -d "$fn" ]; then
        # react to directories here
        printf "directory %s ->t %s\n" "$fn" "${fn/Module/POM}"
        mkdir -p "${fn/Module/POM}"
        (( dc++ ))
    else
        # everything else returned by **/* here
        printf "file %s -> %s\n" "$fn" "${fn/Module/POM}"
        cp "$fn" "${fn/Module/POM}"
        (( fc++ ))
    fi  
done    
printf "\n%'d directories, %'d files" "$dc" "$fc"

Prints:

directory /tmp/Module/submodule1 ->t /tmp/POM/submodule1
directory /tmp/Module/submodule1/target ->t /tmp/POM/submodule1/target
directory /tmp/Module/submodule1/target/classes ->t /tmp/POM/submodule1/target/classes
file /tmp/Module/submodule1/target/classes/class_1 -> /tmp/POM/submodule1/target/classes/class_1
file /tmp/Module/submodule1/target/classes/class_2 -> /tmp/POM/submodule1/target/classes/class_2
file /tmp/Module/submodule1/target/classes/class_3 -> /tmp/POM/submodule1/target/classes/class_3
file /tmp/Module/submodule1/target/classes/class_4 -> /tmp/POM/submodule1/target/classes/class_4
file /tmp/Module/submodule1/target/classes/class_5 -> /tmp/POM/submodule1/target/classes/class_5
directory /tmp/Module/submodule2 ->t /tmp/POM/submodule2
directory /tmp/Module/submodule2/target ->t /tmp/POM/submodule2/target
directory /tmp/Module/submodule2/target/classes ->t /tmp/POM/submodule2/target/classes
file /tmp/Module/submodule2/target/classes/class_1 -> /tmp/POM/submodule2/target/classes/class_1
file /tmp/Module/submodule2/target/classes/class_2 -> /tmp/POM/submodule2/target/classes/class_2
file /tmp/Module/submodule2/target/classes/class_3 -> /tmp/POM/submodule2/target/classes/class_3
file /tmp/Module/submodule2/target/classes/class_4 -> /tmp/POM/submodule2/target/classes/class_4
file /tmp/Module/submodule2/target/classes/class_5 -> /tmp/POM/submodule2/target/classes/class_5

6 directories, 10 files

You can also just use rsync. A basic example would be:

$ rsync -vr /tmp/Module /tmp/POM
# -r for recursive; all targets created and all files copied.

cp: /tmp/Module/**/*: No such file or directory getting this error
Don't do cp /tmp/Module/**/*. Do do cp -R /tmp/Module [tgt _path] with no glob. There are two modes to cp and you are using the file by file mode with a glob. Read the man page.

ceving · Accepted Answer · 2022-01-12 15:26:19Z

0

You can use "install -D" to copy a file and create all leading target directories.

Example:

#! /bin/bash

mockup ()
{
  rm -rf Module
  for s in {1,2}; do
    for d in main test target/classes target/src; do
      mkdir -p Module/submodule$s/$d
      touch Module/submodule$s/$d/f{1..3}
    done
  done
}

mockup

find Module \
     -wholename '*/classes/*' \
     -exec bash -c 'install -D "$1" "${1/#Module/POM}"' _ {} \;

The option -wholename of find is used to find all files in any directory called classes.

Bash's parameter expansion is used to replace "Module" with "POM".

But the parameter expansion requires a variable. The pattern bash -c '...' _ {} is used to assign {} to $1.

answered Jan 12, 2022 at 15:26

ceving

24.1k15 gold badges120 silver badges201 bronze badges

Collectives™ on Stack Overflow

Bash script to copy a directory from multiple subdirectories with directory structure

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related