3

I am running a python script that may or may not take few hours to complete.

In the beginning of my python script, I want to check if this python script is already running or not.

If it is already running, I want to exit my current python that I just started.

For example:

python started 1AM and keeps on running until 3AM started another one at 2AM without knowing it is already running. I want my 2AM python to check and exit since it is already running.

How can I write this python?


This is what I tried for locking..

try:
    l = lock.lock("/home/auto.py", timeout=600) # wait at most 10 minutes

except error.LockHeld:
    e = sys.exc_info()[0]
    logging.error("Error: " + str(e) + " at main gatering Stats")
    smtpObj.sendmail(sender, receivers, message + "Error: " + str(e) + " at main gatering stats")
    exit("Fail: " + str(e) + " at main gathering Stats")
else:
    l.release()

so I thought this will wait for 10 minutes if it is still running then exit.. if it is not running anymore, then run the current python

19
  • 1
    how is this different to stackoverflow.com/questions/29354841/…? Commented Apr 8, 2015 at 16:48
  • @PadraicCunningham I tried most of the answers but its either not working or I can't get it to work Commented Apr 8, 2015 at 16:51
  • So a lockfile did not work? Commented Apr 8, 2015 at 16:54
  • I tried lockfile as will since it was most voted.. I am not sure if it was my coding that didnt work or not.. I couldn't get it to work Commented Apr 8, 2015 at 16:56
  • You should create a temp lock file for that. Check for the lock file then exit or whatever. Commented Apr 8, 2015 at 16:56

1 Answer 1

2

You can try using the lockfile-create command with the r flag to retry a specified amount of times catching a CalledProcessError and exiting, the -p flag will store the pid of the process :

import os
import sys
from time import sleep

from subprocess import check_call, CalledProcessError

try:
    check_call(["lockfile-create", "-q","-p", "-r", "0", "-l", "my.lock"])
except CalledProcessError as e:
    print("{} is already running".format(sys.argv[0]))
    print(e.returncode)
    exit(1)


# main body

for i in range(10):
    sleep(2)
    print(1)

check_call(["rm","-f","my.lock"])

Running a test.py script with the code above while one is already running outputs the following:

$ python  lock.py 
lock.py is already running
4

Options

-q, --quiet

Suppress any output. Success or failure will only be indicated by the exit status.

-v, --verbose

Enable diagnostic output.

-l, --lock-name

Do not append .lock to the filename. This option applies to lockfile-create, lockfile-remove, lockfile-touch, or lockfile-check.

-p, --use-pid

Write the current process id (PID) to the lockfile whenever a lockfile is created, and use that pid when checking a lock's validity. See the lockfile_create(3) manpage for more information. This option applies to lockfile-create, lockfile-remove, lockfile-touch, and lockfile-check.

-o, --oneshot

Touch the lock and exit immediately. This option applies to lockfile-touch and mail-touchlock. When not provided, these commands will run forever, touching the lock once every minute until killed.

-r retry-count, --retry retry-count

Try to lock filename retry-count times before giving up. Each attempt will be delayed a bit longer than the last (in 5 second increments) until reaching a maximum delay of one minute between retries. If retry-count is unspecified, the default is 9 which will give up after 180 seconds (3 minutes) if all 9 lock attempts fail.

Description

The lockfile_create function creates a lockfile in an NFS safe way.

If flags is set to L_PID then lockfile_create will not only check for an existing lockfile, but it will read the contents as well to see if it contains a process id in ASCII. If so, the lockfile is only valid if that process still exists.

If the lockfile is on a shared filesystem, it might have been created by a process on a remote host. Thus the process-id checking is useless and the L_PID flag should not be set. In this case, there is no good way to see if a lockfile is stale. Therefore if the lockfile is older then 5 minutes, it will be removed. That is why the lockfile_touch function is provided: while holding the lock, it needs to be refreshed regularly (every minute or so) by calling lockfile_touch ().

The lockfile_check function checks if a valid lockfile is already present without trying to create a new lockfile.

Finally the lockfile_remove function removes the lockfile.

The Algorithm

The algorithm that is used to create a lockfile in an atomic way, even over NFS, is as follows:

1

A unique file is created. In printf format, the name of the file is .lk%05d%x%s. The first argument (%05d) is the current process id. The second argument (%x) consists of the 4 minor bits of the value returned by time(2). The last argument is the system hostname.

2

Then the lockfile is created using link(2). The return value of link is ignored.

3

Now the lockfile is stat()ed. If the stat fails, we go to step 6.

4

The stat value of the lockfile is compared with that of the temporary file. If they are the same, we have the lock. The temporary file is deleted and a value of 0 (success) is returned to the caller.

5

A check is made to see if the existing lockfile is a valid one. If it isn't valid, the stale lockfile is deleted.

6

Before retrying, we sleep for n seconds. n is initially 5 seconds, but after every retry 5 extra seconds is added up to a maximum of 60 seconds (an incremental backoff). Then we go to step 2 up to retries times.

There seems to be an equivalent package called lockfile-progs on redhat.

On mac you could use lockfile and do something like:

import os
import sys
from time import sleep
import os
from subprocess import Popen, CalledProcessError, check_call


p = Popen(["lockfile", "-r", "0", "my.lock"])
p.wait()
if p.returncode == 0:
    with open("my.pid", "w") as f:
        f.write(str(os.getpid()))
else:
    try:
        with open("my.pid") as f:
            # see if process is still running or lockfile
            # is left over from previous run.
            r = f.read()
            check_call(["kill", "-0", "{}".format(r)])
    except CalledProcessError:
        # remove old lock file and create new
        check_call(["rm", "-f", "my.lock"])
        check_call(["lockfile", "-r", "0", "my.lock"])
        # update pid
        with open("my.pid", "w") as out:
            out.write(str(os.getpid()))
        print("Deleted stale lockfile.")
    else:
        print("{} is already running".format(sys.argv[0]))
        print(p.returncode)
        exit(1)
# main body

for i in range(10):
    sleep(1)
    print(1)
check_call(["rm", "-f", "my.lock"])

In your case maybe using a socket would work:

from socket import socket, gethostname, error, SO_REUSEADDR, SOL_SOCKET
from sys import argv
import  errno



sock = socket()

# Create a socket object
host = gethostname()  
# /proc/sys/net/ipv4/ip_local_port_range is  32768  61000 on my Ubuntu Machine
port = 60001  
# allow connection in TIME_WAIT
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)

try:
    sock.bind((host, port))
    sock.connect((host, port))
except error as e:
    # [Errno 99] Cannot assign requested address
    if e.errno == errno.EADDRNOTAVAIL:
        print("{} is already running".format(argv[0]))
        exit(1)
    # else raise the error
    else:
        raise e

# main body
from time import sleep

while True:
    print(1)
    sleep(2)

sock.close()
Sign up to request clarification or add additional context in comments.

20 Comments

so this will run for the first python and on the scond python it will exit out? also at heck_call(["lockfile-create","-r", "3", "--lock-name","my.lock"]) what is --lock-name and my.lock? is this just any name I can create?
@Tim, the name can be anything, it is created when the script is run. Try it with the example I provided
Traceback (most recent call last): File "./automate.py", line 430, in <module> check_call(["lockfile-create","-r", "3", "--lock-name","my.lock"]) File "/usr/lib64/python2.7/subprocess.py", line 537, in check_call retcode = call(*popenargs, **kwargs) File "/usr/lib64/python2.7/subprocess.py", line 524, in call return Popen(*popenargs, **kwargs).wait() File "/usr/lib64/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1308, in _execute_child raise child_exception`
Can you run lockfile-create from bash?
you mean just lockfile-create ? no -bash: lockfile-create: command not found
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.