How can i load Module packages of linux using a python script

Question

I am new to python and am learning things by writing scripts. I have tried the following and none of them seem to be working.

1) commands.getoutput('module load xxx')

2) subprocess.check_output(['module load', xxx'])

None of these change the environment as a side effect of the module call. Can someone tell me what is wrong?

I meant loading/switching of Environment modules . ex: module load gcc/3.1.1, module switch gcc gcc/3.2.0 etc — pupil
– pupil, Commented Feb 18, 2015 at 8:38
I am not familiar with this module. I found a reference for it on modules.sourceforge.net, but seems innaccessible. Can you give some reference for further help? — h7r
– h7r, Commented Feb 18, 2015 at 9:33
From what I read, I assume module just does changes to the shell's environment, so unless you start a subshell you won't have any of its effects. None of the variants you are using fit this need. — h7r
– h7r, Commented Feb 18, 2015 at 9:35
Yes module just changes the shell environment alone. As far as i see, when we change the shell environment, the current shell for which you have modified will have the new environment modules set. Can you tell me which method will work. I think this link will give you more details hpc.ucla.edu/hoffman2/computing/modules.php — pupil
– pupil, Commented Feb 18, 2015 at 10:21

Aditya Kashi · Accepted Answer · 2018-01-11 02:55:43Z

1

In case it helps anyone, I managed to do it by prefixing all commands that depend on the module with module load xxx &&. For example,

module load mpi/mpich && mpirun ./myprogram -n 4 -options

Not an elegant solution, but works for me. There's also this answer but I couldn't make it work in the system I have access to.

answered Jan 11, 2018 at 2:55

Aditya Kashi

3663 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

pupil · Accepted Answer · 2015-02-18 11:59:12Z

0

I found a different approach to solve this problem. I have written a shell script which load s the environment modules and i am calling that in the python script.

Something like this

import subprocess subprocess.call(['./module_load.sh'])

and the script has something like this....

module load ***

This seems to be working.

Have to say.....its pretty simple in Perl. it provides the environment modules package to handle this.

edited Feb 18, 2015 at 11:59

answered Feb 18, 2015 at 11:47

pupil

1851 silver badge10 bronze badges

2 Comments

h7r Over a year ago

Out of curiosity: are you sure that after subprocess.call the environment available to the python process knows of the changes? Do you mind posting the effect of os.getenv(...) before and after this call, for some variable you know to be changed by module?

SummerEla Over a year ago

This does not load the modules outside the scope of the bash script.

volcano · Accepted Answer · 2015-02-18 12:01:23Z

0

Both commands create a subshell where the module is loaded - the problem is that this subshell is destroyed the moment Python function terminates

answered Feb 18, 2015 at 12:01

volcano

3,61223 silver badges28 bronze badges

Collectives™ on Stack Overflow

How can i load Module packages of linux using a python script

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related