Python Script Not Executing When Run By a Bash Script Called Periodically by Cron

Question

My Problem is Solved: I provided execute permission to the python scripts, added cron.allow file in the /etc/ directory (with the file containing the only entry "pi", without quotes, which is my username) and provided a full path to the python script. I am unable to determine which solution made it work, so thanks to glenn jackman and Abhijit Pritam.

I have two python scripts which are executed by a bash script. Since I want this script to be run every few hours, I have set up a Cron Job. The Bash script's (run_awsscrubber.sh) job is as follows:

Set up the environment.
Write the Bash script run time to two log files (one for all the previous runs and one exclusive to this run).
Check using wget whether an IP address is up or not.
If its up:
Run script imd_aws_parse_rc4.py and capture its output in a variable and write it to the log files.
Run script gmail_run_alert.py and capture its output in a variable and write it to the log files.
Clean up some generated files
If it's not up:
Run script gmail_run_alert.py and capture its output in a variable and write it to the log files.
Clean up some generated files

imd_aws_parse_rc4.py: The code in this script scrubs a website for information using Selenium WebDriver, Firefox, Gecko driver and Beautiful Soup and stores the parsed data as CSV in a file.

gmail_run_alert.py: The code in this script emails the CSV file (if it exists) and current generated error log to an ID.

I am using a Raspberry Pi 3 running Raspbian Stretch.

The bash script when run manually is able to run both the python scripts but when it is run by Cron the python scripts are not executed. The Bash script and python scripts are located in the directory ~/CronJobs and their output is stored in the directory ~/CronOutput

Following is some code/output which might be useful for finding the solution:

Contents of the Cron Job in /etc/cron.d Directory and The File Permissions

pi@RPI_IronHide:/etc/cron.d $ ls -al AWSScriptCJ
-rw-r--r-- 1 root root 53 May  1 16:56 AWSScriptCJ
pi@RPI_IronHide:/etc/cron.d $ cat AWSScriptCJ
*/2 * * * * pi /home/pi/CronJobs/run_awsscrubber.sh

File Permissions of Bash Script and the Python Scripts

pi@RPI_IronHide:~/CronJobs $ ls -al
total 28
-rw-rw-r--  1 pi pi 2264 May  1 10:41 gmail_run_alert.py
-rw-rw-r--  1 pi pi 7166 May  1 10:50 imd_aws_parse_rc4.py
-rwxr-xr-x  1 pi pi 1812 May  1 17:26 run_awsscrubber.sh

Permissions of CronJobs and CronOutput Directories

pi@RPI_IronHide:~ $ ls -al
total 148
drwxr-xr-x  2 pi   pi    4096 May  1 17:27 CronJobs
drwxr-xr-x  2 pi   pi    4096 May  1 16:55 CronOutput

Single Run Log File Contents

pi@RPI_IronHide:~/CronOutput $ cat AWS_Data_Scrub_Latest.log
Tue 1 May 17:56:01 IST 2018
LANG=en_GB.UTF-8
PWD=/home/pi
HOME=/home/pi
SHELL=/bin/bash
SHLVL=1
LOGNAME=pi
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/printenv
Script Begin!
Online

Removing geckodriver.log File
Removing aws_html.html File

Removing Single Run CSV File
Script End Success!

Cron Logs from /var/log/syslog

pi@RPI_IronHide:~ $ grep CRON /var/log/syslog | tail -10
May  1 17:48:01 RPI_IronHide CRON[2555]: (pi) CMD (/home/pi/CronJobs/RunACronTask.sh)
May  1 17:48:01 RPI_IronHide CRON[2556]: (pi) CMD (/home/pi/CronJobs/run_awsscrubber.sh)
May  1 17:50:01 RPI_IronHide CRON[2590]: (pi) CMD (/home/pi/CronJobs/run_awsscrubber.sh)
May  1 17:50:01 RPI_IronHide CRON[2591]: (pi) CMD (/home/pi/CronJobs/RunACronTask.sh)
May  1 17:52:01 RPI_IronHide CRON[2625]: (pi) CMD (/home/pi/CronJobs/RunACronTask.sh)
May  1 17:52:01 RPI_IronHide CRON[2626]: (pi) CMD (/home/pi/CronJobs/run_awsscrubber.sh)
May  1 17:54:01 RPI_IronHide CRON[2675]: (pi) CMD (/home/pi/CronJobs/run_awsscrubber.sh)
May  1 17:54:01 RPI_IronHide CRON[2674]: (pi) CMD (/home/pi/CronJobs/RunACronTask.sh)
May  1 17:56:01 RPI_IronHide CRON[2725]: (pi) CMD (/home/pi/CronJobs/run_awsscrubber.sh)
May  1 17:56:01 RPI_IronHide CRON[2726]: (pi) CMD (/home/pi/CronJobs/RunACronTask.sh)

Bash Script Contents

pi@RPI_IronHide:~/CronJobs $ cat run_awsscrubber.sh
#!/bin/bash

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SHELL=/bin/bash
MAILTO=""

LOG_FILE_N="/home/pi/CronOutput/AWS_Data_Scrub.log"
CURR_LOG_FILE="/home/pi/CronOutput/AWS_Data_Scrub_Latest.log"

echo $(/bin/date) >> $LOG_FILE_N
printenv >> $LOG_FILE_N
echo $(/bin/date) > $CURR_LOG_FILE
printenv >>$CURR_LOG_FILE

echo "Script Begin!" >> $LOG_FILE_N
echo "Script Begin!" >> $CURR_LOG_FILE

/usr/bin/wget -q --tries=10 --timeout=20 --spider XXX.XXX.167.219

if [[ $? -eq 0 ]]; then
    echo "Online" >> $LOG_FILE_N
    echo "Online" >> $CURR_LOG_FILE
    CURRENT_RUN_OP=$(python imd_aws_parse_rc4.py)
    echo $CURRENT_RUN_OP >> $CURR_LOG_FILE
    echo $CURRENT_RUN_OP >> $LOG_FILE_N
    echo "Removing geckodriver.log File" >> $LOG_FILE_N
    echo "Removing geckodriver.log File" >> $CURR_LOG_FILE
    rm /home/pi/CronJob/geckodriver.log
    echo "Removing aws_html.html File" >> $LOG_FILE_N
    echo "Removing aws_html.html File" >> $CURR_LOG_FILE
    rm /home/pi/CronOutput/aws_html.html
    EMAIL_RUN_OP=$(python gmail_run_alert.py)
    echo $EMAIL_RUN_OP >> $CURR_LOG_FILE
    echo $EMAIL_RUN_OP >> $LOG_FILE_N
    echo "Removing Single Run CSV File" >> $LOG_FILE_N
    echo "Removing Single Run CSV File" >> $CURR_LOG_FILE
    rm /home/pi/CronOutput/strip_d_temp.csv
    echo "Script End Success!" >> $CURR_LOG_FILE
    echo "Script End Success!" >> $LOG_FILE_N
    echo -e "\n" >> $LOG_FILE_N
    exit 1
else
    echo "Offline" >> $LOG_FILE_N
    echo "Offline" >> $CURR_LOG_FILE
    EMAIL_RUN_OP=$(python gmail_run_alert.py)
    echo $EMAIL_RUN_OP >> $CURR_LOG_FILE
    echo $EMAIL_RUN_OP >> $LOG_FILE_N
    echo "Script End Unable to Access Website!" >> $CURR_LOG_FILE
    echo "Script End Unable to Access Website!" >> $LOG_FILE_N
    echo -e "\n" >> $LOG_FILE_N
    exit 0
fi

bash: quote your variables -> Security implications of forgetting to quote a variable in bash/POSIX shells — glenn jackman
– glenn jackman, Commented May 1, 2018 at 13:15

Abhijit Pritam Dutta · Accepted Answer · 2018-05-01 12:50:51Z

1

Please provide execute permission to Python script as well. Also check whether your cron tab account has been added to the group which has the ownership of your shell script or not. If not you need to add that account to the proper group. File permission itself it not suffice. Also check whether cron tab account has full excess to the directory path or not. For example suppose your directory path is /home/pi/CronJobs/ now cron tab account should have permission for cd /home/pi/CronJobs/.

answered May 1, 2018 at 12:50

Abhijit Pritam Dutta

5,6112 gold badges13 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Soumil Heble Over a year ago

Is this tutorial Linux / UNIX Restrict at / cron Usage To Authorized Users the right one for adding a user to crontab? The RPi has neither cron.allow or cron.deny files.

glenn jackman · Accepted Answer · 2018-05-01 13:14:29Z

1

You don't provide a full path to the python script. Pick one of

EMAIL_RUN_OP=$(python /home/pi/CronJobs/gmail_run_alert.py)

or

cd /home/pi/CronJobs
EMAIL_RUN_OP=$(python gmail_run_alert.py)

or

cd "$(dirname "$0")"    # the directory of the current script, without hardcoding
EMAIL_RUN_OP=$(python gmail_run_alert.py)

answered May 1, 2018 at 13:14

glenn jackman

249k42 gold badges233 silver badges362 bronze badges

Collectives™ on Stack Overflow

Python Script Not Executing When Run By a Bash Script Called Periodically by Cron

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related