Python modules are not imported when running python script from PHP

Question

I have a Python script, which works fine when I run it in Terminal. Here it is:

import bs4, requests, json, sys
from sys import argv

def getFsmonData(link):
    res = requests.get(link)
    res.raise_for_status()
    soup = bs4.BeautifulSoup(res.text)

    # Get table data
    tds = soup.select('td')
    new_td_list = []
    for i in range(len(tds)):
        inner_td_list = []
        for y in range(len(tds)):
            inner_td_list.append(tds[y].getText())
        new_td_list.append(inner_td_list)
    print(inner_td_list)
    td_list_json = json.dumps(inner_td_list)
    tdf = open("/path/to/file/data/td_data.txt", "w")
    tdf.write(td_list_json)
    tdf.close()

getFsmonData(sys.argv[1])

But now I am trying to run it from PHP and I get the following error:

Traceback (most recent call last): File "/path/to/file/example.py", line 1, in import bs4, requests, json, sys ModuleNotFoundError: No module named 'bs4'

I guess PHP thinks I do not have this module installed, but I am not sure of course. Here is my PHP code:

<?php
    $link = $_POST['dn-link'];

    if (isset($_POST['submit'])) {
        system('/usr/bin/python3 /path/to/file/example.py ' . escapeshellarg($link) . ' 2>&1', $retval);
    }
?>

Who can help solving this issue?

Your script is running as a different user when called from PHP than when called from the command line, so you need to set up that user's Python environment so that it can find the module. — kindall
– kindall, Commented Sep 10, 2018 at 22:28
Why downgrading? People who downgrade this act as if this is a general knowledge question. I am asking because I cannot figure it out on my own probably. — tera_789
– tera_789, Commented Sep 10, 2018 at 22:30
@kindall You want to say that if I run it as sudo, it will work? Well it does not — tera_789
– tera_789, Commented Sep 10, 2018 at 22:32
Write a simple script that just prints sys.path and run it both in your PHP environment and at your usual command line. — kindall
– kindall, Commented Sep 10, 2018 at 22:46
@kindall Output from CLI: ['', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/home/myusername/.local/lib/python3.6/site-packages', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages'] And here is from PHP: ['/dir/with/server/files', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages'] Is it because list from PHP does not include '/home/myusername/.local/lib/python3.6/site-packages'?How can I add it to PHP? — tera_789
– tera_789, Commented Sep 10, 2018 at 23:11

Jamie Marshall · Accepted Answer · 2018-09-10 23:35:36Z

2

I think your misunderstanding the permission issue. It's not that you need to add it PHP. It's that your OS keeps track of permissions, and the permission you have assigned to your PHP interpreter and your Python interpreter are misaligned. There are two permissions that usually people mix up here - USER which is login level access and SYSTEM which is system wide access. If your on your own dev machine then add both to SYSTEM.

We need to know your OS to be able to fully answer, i'm going to guess Mac OS in which case would be this (I think, can't test it right now).

sudo nano "pathOfYourInterpreter"

The other thing could be happening is that the libraries you're tryng to import are somewhere where the current sys path you have setup can't reach them. For example if you have your sys variable for Python set to the directory where the interpretor is but the scripts you're importing are in a different folder. In that case you have several options for accessing those scripts.

Set the current working directory to the folder where they're located before importing with:
```
os.chdir("locationOfScript/")
import script.py
```
Add a sys Path at runtime:
```
 sys.path.append("locationOfScript/)
```

There are more methods of attack, but I think this should get you started.

answered Sep 10, 2018 at 23:35

Jamie Marshall

2,3865 gold badges38 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

tera_789 Over a year ago

Exactly, I just solved this issue with sys.path.append("locationOfScript/"). The path where modules are installed were missing when I ran print(sys.path) from PHP. I have added sys.path.append("locationOfScript/") to the top of my Python script and the issue is resolved. Thank you as well

Dhanu Kanakala Over a year ago

@Jamie Marshall,@tera_789 Can you please explain to me what is this "locationOfScript/", I didn't understand what to mention in place of "locationOfScript/"

Jamie Marshall Over a year ago

@DhanuK - the path (directory) where your script is located needs to go in place of "locationOfScript/"

Dylan Schroder · Accepted Answer · 2018-09-10 23:01:21Z

2

Most likely, when your PHP script executes your Python script, its executing it as a different user (possibly super user). What you can do is check if the packages are installed under the super user by doing

sudo pip list

or if you're using python3

sudo pip3 list

see if the packages you we're wanting to use are listed from this command.

If they aren't listed, you can easily install them via the command:

sudo pip install <pkg name>

or

sudo pip3 install <pkg name>

What you're doing here is installing the packages under the super user rather than you the local user.

answered Sep 10, 2018 at 23:01

Dylan Schroder

417 bronze badges

1 Comment

tera_789 Over a year ago

sudo pip3 list displays this particular module that is causing the error in PHP: beautifulsoup4 (4.6.3)

Dan · Accepted Answer · 2023-09-19 02:04:33Z

To best work around this and ensure you do not compromise the PHP profile, setup a virtual environment for you python installation/application ensuring to pip install all required modules.

Now this is done we can call this instance directly to execute the script. You will however need to ensure that home is set in PHP so that it you can use the actual file paths.

<?php
// Set the 'HOME' environment variable (replace with the appropriate path)
putenv('HOME=/home/yourusername');

// Execute the Python script
exec("/home/yourusername/virtualenv/application_name/python/3.9/bin/python /home/yourusername/location_of_script/example.py", $output, $return_var);

// Check the output and return status once verified this can be edited to just be the expected output
echo "Output: " . implode("\n", $output) . "<br>";
echo "Return Status: " . $return_var;
?>

I know this is old but I found it after having the same issue and after a lot of debugging found the answer. This is here for anyone like me who is having the same issue.

Collectives™ on Stack Overflow

Python modules are not imported when running python script from PHP

3 Answers 3

3 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related