Twitter Scraping Repeated Execution Code(python)

Question

This is a Twitter scraping code that extracts tweets which contain famous keywords.

I want to repeat the entire code below every 12 hours. (Or 12 hours + 10 minutes breaks). Can you give me advice on repeating phrases?

import tweepy
import time
import os
import json
import simplejson

search_term = 'word1'
search_term2= 'word2'
search_term3='word3'

lat = "xxxx"
lon = "xxxx"
radius = "xxxx"
location = "%s,%s,%s" % (lat, lon, radius)

API_key = "xxxx"
API_secret = "xxxx"
Access_token = "xxxx"
Access_token_secret = "xxxx"

auth = tweepy.OAuthHandler(API_key, API_secret)
auth.set_access_token(Access_token, Access_token_secret)
api = tweepy.API(auth)

c=tweepy.Cursor(api.search,
                 q="{}+OR+{}".format(search_term, search_term2, search_term3),
                rpp=1000,
                geocode=location,
                include_entities=True)

data = {}
i = 1
for tweet in c.items():
    data['text'] = tweet.text
    print(i, ":", data)
    i += 1
    time.sleep(1)

wfile = open(os.getcwd()+"/workk2.txt", mode='w')   
data = {}   
i = 0       

for tweet in c.items():
    data['text'] = tweet.text   
    wfile.write(data['text']+'\n')  
    i += 1

wfile.close()

Community · Accepted Answer · 2017-05-23 12:18:03Z

1

You could set a Cron job that executes your script every 12 hours. To do so you should save your script with .py extension and make it executable. Then add it to your crontab:

0 0 0/12 * * ? /usr/bin/python yourscript.py

For more detail have a look at this question. Alternatively there are packages in python (e.g. APScheduler) that help you achieve this. In APScheduler you can define a job like this:

from apscheduler.schedulers.blocking import BlockingScheduler

sched = BlockingScheduler()

@sched.scheduled_job('interval', hours=12)
def timed_job():
    print('This job is run every 12 hours.')

sched.configure(options_from_ini_file)
sched.start()

edited May 23, 2017 at 12:18

CommunityBot

11 silver badge

answered Apr 29, 2017 at 9:07

hatef

6,29930 gold badges47 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Twitter Scraping Repeated Execution Code(python)

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related