rc.local not able to execute python script with network code

Question

(Disclaimer, this is a crosspost from my Unix & Linux Overflow question, however I got no answers)

To introduce my problem, I'm running Raspbian Stretch on a RPi 3 model B+, and I have two scripts that I run in /etc/rc.local so that the pi can run completely headless and control a GoPro over wifi based on sensor/GPIO input.

The first script is a bash script that simply connects to the GoPro wireless AP using wpa_cli. I've had no problems there (save when the goPro auto turns off for battery...).

The second script is a python script that sends some commands to the goPro over the wifi socket. It uses the urllib.request.urlretrieve(...) method. However, this script fails to execute when running from /etc/rc.local, failing with:

[   50.119390] rc.local[516]: Traceback (most recent call last):
[   50.120173] rc.local[516]:   File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
[   50.124119] rc.local[516]:     encode_chunked=req.has_header('Transfer-encoding'))
[   50.127993] rc.local[516]:   File "/usr/lib/python3.7/http/client.py", line 1229, in request
[   50.132015] rc.local[516]:     self._send_request(method, url, body, headers, encode_chunked)
[   50.136108] rc.local[516]:   File "/usr/lib/python3.7/http/client.py", line 1275, in _send_request
[   50.140314] rc.local[516]:     self.endheaders(body, encode_chunked=encode_chunked)
[   50.144462] rc.local[516]:   File "/usr/lib/python3.7/http/client.py", line 1224, in endheaders
[   50.148440] rc.local[516]:     self._send_output(message_body, encode_chunked=encode_chunked)
[   50.152573] rc.local[516]:   File "/usr/lib/python3.7/http/client.py", line 1016, in _send_output
[   50.156713] rc.local[516]:     self.send(msg)
[   50.160973] rc.local[516]:   File "/usr/lib/python3.7/http/client.py", line 956, in send
[   50.165078] rc.local[516]:     self.connect()
[   50.169443] rc.local[516]:   File "/usr/lib/python3.7/http/client.py", line 928, in connect
[   50.173764] rc.local[516]:     (self.host,self.port), self.timeout, self.source_address)
[   50.177933] rc.local[516]:   File "/usr/lib/python3.7/socket.py", line 727, in create_connection
[   50.182306] [FAILED] Failed to start /etc/rc.local Compatibility.
rc.local[516]:     raise err
See 'systemctl status rc-local.service' for details.
[   50.186404] rc.local[516]:   File "/usr/lib/python3.7/socket.py", line 716, in create_connection
[   50.190904]          Starting Terminate Plymouth Boot Screen...
rc.local[516]:     sock.connect(sa)
[   50.198463]          Starting Hold until boot process finishes up...
rc.local[516]: OSError: [Errno 101] Network is unreachable
[   50.202947] rc.local[516]: During handling of the above exception, another exception occurred:
[   50.208240] rc.local[516]: Traceback (most recent call last):
[   50.212360] rc.local[516]:   File "/home/pi/goPro/gpioControlledGoPro.py", line 154, in <module>
[   50.216155] rc.local[516]:     SendCmd(startRecording)

The script runs perfectly fine once I run as a user in the command line, so I instead ran the script using sudo -H -u pi /usr/bin/python3 [script]. This helped, as the GoPro started the recording, but the second urllib.request.urlretrieve(...) failed, and a very similar error to the one above popped up. I definitely gave the script ample sleep time to make sure there was no interference with the bootup process, and there still was a connection, as I was able to immediately run the script once I logged in as pi with no issues whatsoever

Am I missing something about how the bootup works? Should I follow a different method to autoexecute?

Thanks.

There are hundreds of similar posts. Search these to see why you can't use rc.local — Milliways
– Milliways, Commented Oct 19, 2020 at 9:51
If your using the network, maybe add a 30 second pause to allow the network to start. — CoderMike
– CoderMike, Commented Oct 19, 2020 at 12:17
@Milliways I have looked through a significant amount of posts across a bunch of different stackExchanges, and none really seemed to address the issue that I was specifically facing. And as a hobbyist, I'm not able to understand issue, so I am going to ask as that will be faster. You don't need to open the thread if its a dime a dozen, and I also understand if someone doesn't want to answer since its similar. — robinspi
– robinspi, Commented Oct 20, 2020 at 4:45
@CoderMike I did give that a shot, proceeding on network connectivity too, but things still failed in the end. Thanks for the tip though. — robinspi
– robinspi, Commented Oct 20, 2020 at 4:55

Ingo · Accepted Answer · 2020-10-20 22:21:46Z

Please note that using /etc/rc.local is deprecated since years. It has limitations due to Compatibility with SysV. We have seen many problems here on this site using it. Following the recommendation of the developers from systemd you should avoid using it. Failing functionality of rc.local is expected in respect to the last sentence of the Compatibility document:

Note that there are some areas where systemd currently provides a certain amount of compatibility where we expect this compatibility to be removed eventually.

You see, it's time to use a systemd Unit. Because your script runs from the command line it should not be a big issue to make it running as a service. You have many options to give it the same environment than on the command line. Try the following Unit. Create it with:

rpi ~$ sudo systemctl edit --force --full second-script.service

In the empty editor insert these statements, save them and quit the editor:

[Unit]
Description=Second Script
After=network.target

[Service]
User=pi
WorkingDirectory=/home/pi
#Environment="ENV_VAR_NEEDED=value"
#ExecStartPre=/bin/bash -c `/usr/bin/env`
ExecStart=/usr/bin/python3 /full/path/to/second-script.py

[Install]
WantedBy=multi-user.target

This gives the script the same login with user pi. I think you use it when running it on the command line. It could be that it does not match the environment of the command line. Maybe you have to look for environment variables that second-script.py needs. You can set them as shown with the commented Environment= option. If you like to see the environment of the service at run time just uncomment the #ExecStartPre= line. You will find its output in the journal with:

rpi ~$ journalctl --unit=second-script.service

For all options to configure the needed environment look at man systemd.exec.

A very comprehensive answer, that does explain the things I couldn't find on my own. Thanks — robinspi
– robinspi, Commented Oct 20, 2020 at 19:50
@robinspi: If you've not read it, this article titled "Understanding and Using Systemd" is well worth a few minutes of your time - even if only to learn that the transition to systemd has not been a smooth one, or even a welcome one in some quarters. But Ingo is correct - systemd is here, and rc.local is gone. — Seamus
– Seamus, Commented Oct 21, 2020 at 5:31

Ljm Dullaart · Accepted Answer · 2020-10-19 13:24:32Z

I never understood why Python produces so much useless information when something goes wrong. The key problem here seems:

rc.local[516]: OSError: [Errno 101] Network is unreachable

At the time that rc.local starts, the network is apparently not yet available. Your python code does not handle this well. It should probably test whether the network is available and wait and retest after a few seconds if it is not, instead of just crashing.

There are alternative methods. The "systemd"-way is to create a unit-file to start your services. Systemd-adepts frown upon the use of rc.local. As you seem to have a systemd-distribution, this would be a better choice.

You can also have it run by cron. Put an @reboot line in the crontab.

Either way, you should probably modify your python scripts to handle problem situations better (unless you like the stack dumps).

Ah, I have little familiarity with Debian based/systemd based distros, so maybe this was a contributor to my confusion. I will give the cron method a shot as I think that might be easier than looking into systemd — robinspi
– robinspi, Commented Oct 20, 2020 at 4:54

Stack Exchange Network

rc.local not able to execute python script with network code

2 Answers 2

Your Answer

Hot Network Questions

rc.local not able to execute python script with network code

2 Answers 2

Your Answer

Sign up or log in

Post as a guest

Related

Hot Network Questions