Issues redirecting standard output to a file in bash

Question

I've written a short bash script to update some twitter_ebooks bots. The commands involved generate a lot of output and I want to redirect it to a file with the day's date. I tried using the top aswer to this question on ask ubuntu however the vast majority of the output still goes to the terminal.

My bash script:

#!/bin/bash

arc() { ## $1 is account name $2 is log file
    ebooks archive $1 corpus/$1.json >> "$2"
    ebooks consume corpus/$1.json >> "$2"
}

accounts=(some twitter names)

echo "moving to ~/twitter/snare_ebooks"
cd ~/twitter/snare_ebooks

date_var=$(date -d @$(date +%s) +"%m_%d_%Y")
logfile=~/ebooks_archive_logs/$date_var.log
touch "${logfile}"
echo "model update log for $date_var" >> "${logfile}"

echo "archiving and consuming the corpus"

for accountName in ${accounts[*]}
do
    arc $accountName "${logfile}"
done

echo "git adding"
git add --all . >> "${logfile}"

echo "git committing"
git commit -m "model update" >> "${logfile}"

echo "pushing to heroku"
git push heroku master >> "${logfile}"

The log file that is created only contains:

model update log for 07_28_2015
[master 2eb0961] model update
 4 files changed, 134 insertions(+), 134 deletions(-)

Can anyone explain why this isn't working as expected? Also what needs to be done in order for it to work as expected?

The -d @$(date +%s) is unnecessary; you're just telling date to use the current time (which is the default). — chepner
– chepner, Commented Jul 28, 2015 at 19:25
Also, you may need to redirect standard error as well as standard output, depending on what the various programs use and what, exactly, you want in the log. — chepner
– chepner, Commented Jul 28, 2015 at 19:29
arc "$accountname" >> "$logfile" is better than doing the redirection in the function twice. Similarly doing the redirection once on the entire contents of the script would be even better. — Etan Reisner
– Etan Reisner, Commented Jul 28, 2015 at 19:35
What output is showing up on the terminal instead of going to the logfile? — Barmar
– Barmar, Commented Jul 28, 2015 at 19:47
@chepner I redirected standard error with &>> and it worked like a charm! If you write an answer I'd love to mark it as the correct answer — Will Norvelle
– Will Norvelle, Commented Jul 28, 2015 at 19:48

Barmar · Accepted Answer · 2015-07-28 19:56:42Z

3

Instead of redirecting every command, you can do this at the beginning of the script:

date_var=$(date -d @$(date +%s) +"%m_%d_%Y")
logfile=~/ebooks_archive_logs/$date_var.log
exec >"$logfile" 2>&1

exec with no command and just redirection operators simply changes the I/O connections for the remainder of the script. 2>&1 additionally redirects standard error. If you were seeing output, those commands were probably writing it to stderr instead of stdout.

If you want to be able to occasionally write to the original sdtdout, like your echo "git adding" lines, you could do:

exec 3>&1 >"$logfile" 2>&1

This moves the original stdout to FD 3, and then you can do:

echo "git adding" >&3

It's possible that some of the commands were writing to /dev/tty instead of stdout, but this is unlikely (this is generally only done for things like prompts for input).

edited Jul 28, 2015 at 19:56

answered Jul 28, 2015 at 19:51

Barmar

789k57 gold badges554 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

chepner Over a year ago

One tiny issue with the global redirect: there is one echo statement in the script that wasn't originally redirected (but that might have been an oversight).

Barmar Over a year ago

Actually, I now see that there are a bunch of echo statements that are used for showing the step of the script. It's not clear whether he wants those in the log or not, and maybe they were just temporary.

Will Norvelle Over a year ago

The undirected echo statements are intentional, I want to have a general idea of where the script is at in its execution

Barmar Over a year ago

@WillNorvelle I updated my answer to show how to deal with exceptions.

Will Norvelle Over a year ago

@Barmar this works and makes my script much cleaner, thanks

Collectives™ on Stack Overflow

Issues redirecting standard output to a file in bash

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related