0

Is there a Linux command that can remove the first character on the first line for every file in a folder, and then save it?

Many online resources cover doing this for every line instead of just the first one, for filenames, or they don't save the files. I tried modifying one of these commands to do what I wanted, but it actually made the files blank instead.

4
  • 5
    @zevzek+ technically tail -c does bytes which are not always the same as characters, since countries other than the US came into existence. Linux usually provides GNU sed which has -i inplace, so as long as the first character isn't a newline, sed -i 1s/^.// file... will work on one or more files, including multibyte chars if the locale is suitable, although as you note it does need to copy (the rest of) each file. (At least some BSD sed's also have -i, but POSIX doesn't.) Commented Nov 15, 2021 at 0:36
  • Besides non-us countries, the UTF-8 BOM is 3 bytes. Removing that is a really common reason to want to remove the first character of a file. Commented Nov 16, 2021 at 10:08
  • What are the commands you've tried? Show us both the orginal and modified versions so we perhaps can see your level and where you went wrong. Commented Nov 19, 2021 at 20:49
  • Original: rev input | cut -c2- | rev | cut -c2- Modified: rev * | cut -c2- | rev | cut -c2- rev * | cut -c2- | rev | cut -c2- | tee * I can't find the original versions for these: echo * | sed -r 's/^.{1}//' tail -c +2 'Text File.txt' > 'Text File.txt.new' && mv 'Text File.txt.new' 'Text File.txt' awk '{print substr($1,2) > "*" }' * Commented Nov 19, 2021 at 23:28

3 Answers 3

1

To remove the first character¹ of every non-empty regular file in or below the current working directory, in the zsh shell, you could do:

zmodload zsh/mapfile
for file (**/*(ND.L+0)) mapfile[$file]=${mapfile[$file]#?}

Note that it's not exactly the same as removing the first character of the first line in the special case where the first line is empty (the file starts with a newline character). In that case, the above removes that newline character which means the first (empty) line is removed. If you don't want that, you can replace ? (which matches any character) with [^$'\n'] which matches any character other than newline.

Note that as that ends up loading the whole file in memory, you wouldn't want to do that on very large files.

A more memory friendly approach but that still rewrites the original file in place and preserved all its metadata (as opposed to sed -i which replaces the file with a modified copy) could be:

find . -name '*.jpg' -size +0 -exec ksh93 -c '
  builtin cat
  for file do
    { IFS= read -rN1 && cat; } < "$file" 1<>; "$file"
  done' ksh93 {} +

Where this time we use ksh93's <>; redirection operator that opens a file in read+write mode (initially without truncation) and truncates it only upon exit of the redirected command if that command was successful. Here, the command is a command group that reads one character from stdin and discards it, and then writes the rest of its stdin to stdout.

Beware that GNU find (contrary to zsh) won't find files with names ending in .jpg if the rest of the file name can be decoded as characters in the current locale.


¹ or first byte for those files that start with something that can't be decoded into characters in the locale's character encoding. Compare with sed '1s/.//' which would remove the first character that can be decoded as such, so not necessarily at the start of the file or sed '1s/^.//' which would remove the first character at the start of the first line if there is one there, but do nothing otherwise

0

If you have GNU sed :

find PATH -type f -exec sed -s -i '1s/.//' \{} +
4
  • The first one is wrong for a number of reason: -printf %P (GNU-specific) prints the path of the file relative to PATH. For PATH/some/file, it prints some/file. So sed will not find it unless its current working directory is also PATH. Even then, you'd have sed's output go to _some/file. Also, use find -exec sh -c 'for f do... ; done' sh {} +. find can't be piped to xargs unless you use -print0 and -r0 GNU extensions. Commented Nov 20, 2021 at 7:12
  • @StéphaneChazelas thanks. I've moved "%P" to "%p". Since the question explicitly states coreutils and linux i can't see whats wrong when using GNU extensions. Note that the sed output is redirect to a file and the file is then renamed as the original, what's wrong whith that? Commented Nov 20, 2021 at 9:58
  • Note that find comes from GNU findutils, not GNU coreutils. -printf '%p\n' is the same as the standard -print and -print is even the default action if you don't give an action predicate. But again the output of find -print is not post-processable, and you don't need to post-process it with xargs to execute commands as find can execute commands by itself as you already show in your second (correct) example. Commented Nov 20, 2021 at 10:24
  • $1 contains the full path (PATH/to/some/file). With > "_$1", you're redirecting to _PATH/to/some/file. You probably intended to redirect to PATH/to/some/_file instead. Commented Nov 20, 2021 at 10:25
0

You could start with an awk loop in the folder with your files, then in a secondo moment redirecting the output string to the original file. The awk command you can use for removing the first n characters looks something like this :

awk '{print substr($1,2) > "/output/file" }' /path/to/file

Keep in mind that writing on the same file you read is not the best practice, but it works.

4
  • What would the loop look like, and what file paths would I use to remove the first character from every file? Commented Nov 17, 2021 at 1:46
  • awk automatically loops every line of the file you input, you need to loop all the files in your folder/folders with a script or from command line. The file path is the path all the files you need to edit, I can't know it myself. Commented Nov 17, 2021 at 8:27
  • I don't know much about scripting, but I'm imagining a script that just removes the first character from every file in the current folder. Maybe it would be easier to use a script instead of a command? Commented Nov 19, 2021 at 4:13
  • The command in my initial answer already does what you are asking, try it for a single file. What you'll need is a second command or script to loop effectively into all your needed folders and files. Commented Nov 19, 2021 at 15:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.