Skip to main content
Tweeted twitter.com/StackUnix/status/1034908861092831233
deleted 38 characters in body
Source Link
Rui F Ribeiro
  • 58k
  • 28
  • 156
  • 239

Any help is appreciated. Thank you

Any help is appreciated. Thank you

Source Link
beginner
  • 277
  • 2
  • 10

How to create a new file with required columns from different multiple files in linux?

I have a directory ballgown, in which there are around 1000 subdirectories as sample names. Each subdirectory has a file t_data.ctab. The filename is the same in all subdirectories.

ballgown
      |_______TCGA-A2-A0T3-01A
                   |___________ t_data.ctab
      |_______TCGA-A7-A4SA-01A
                   |___________ t_data.ctab
      |_______TCGA-A7-A6VW-01A
                   |___________ t_data.ctab

Like above ballgown has 1000 subdirectories. The t_data.ctab file in all those 1000 subdirectories looks like below with columns:

t_id    chr     strand  start   end     t_name  num_exons       length  gene_id gene_name       cov     FPKM
1       1       -       10060   10614   MSTRG.1.1       1       555     MSTRG.1 .       0.000000        0.000000
2       1       +       11140   30023   MSTRG.10.1      12      3981    MSTRG.10        .       2.052715        0.284182
3       1       -       11694   29342   MSTRG.11.1      8       6356    MSTRG.11        .       0.557588        0.077194
4       1       +       11869   14409   ENST00000456328.2       3       1657    MSTRG.10        DDX11L1 0.000000        0.000000
5       1       +       11937   29347   MSTRG.10.3      12      3544    MSTRG.10        .       0.000000        0.000000
6       1       -       11959   30203   MSTRG.11.2      11      4547    MSTRG.11        .       0.369929        0.051214
7       1       +       12010   13670   ENST00000450305.2       6       632     MSTRG.10        DDX11L1 0.000000        0.000000
8       1       +       12108   26994   MSTRG.10.5      10      5569    MSTRG.10        .       0.057091        0.007904
9       1       +       12804   199997  MSTRG.10.6      12      3567    MSTRG.10        .       0.000000        0.000000
10      1       +       13010   31097   MSTRG.10.7      12      4375    MSTRG.10        .       0.000000        0.000000
11      1       -       13068   26832   MSTRG.11.3      9       5457    MSTRG.11        .       0.995280        0.137788

From all the t_data.ctab files I want to extract only t_name and FPKM column and create a new file. In the new file the FPKM column should be the sample name. It should look like below:

t_name         TCGA-A2-A0T3-01A TCGA-A7-A4SA-01A    TCGA-A7-A6VW-01A
MSTRG.1.1              0            0.028181                 0
MSTRG.10.1         0.284182         0.002072             0.046302
MSTRG.11.1         0.077194         0.685535             0.105849
ENST00000456328.2      0            0.307315             0.038961
MSTRG.10.3             0            0.446015             0.009946
MSTRG.11.2         0.051214         0.053577             0.036081
ENST00000450305.2      0            0.110438             0.040319
MSTRG.10.5         0.007904             0                1.430825
MSTRG.10.6             0                0                0.221105
MSTRG.10.7             0            0.199354                 0
MSTRG.11.3         0.137788         0.004792                 0

If it is two or three files I can use cut -f6,12 on each file and then join them. But I have around 1000 files now.

Any help is appreciated. Thank you