Python Asyncio - does not seem to be working in asynchronous mode

Question

I have written below code but it does not seem to be giving the true asynchronous result.

code:

   async def getOHLC(kc, symbol):
        print("before .... " + symbol)
        ohlc_val = kc.ohlc(symbol)
        print(ohlc_val)
        print("after .... " + symbol)
    
    
    async def trackPrice(kc):
        stocks_array = []
        with open('../config/stocks.txt') as f:
            for line in f:
                stocks_array.append(line.rstrip('\n'))
    
        await asyncio.gather(*[getOHLC(kc, stock) for stock in stocks_array])
    
    if __name__ == "__main__":
        import time
        s = time.perf_counter()
        asyncio.run(trackPrice(connect()))
        elapsed = time.perf_counter() - s
        print(f"{__file__} executed in {elapsed:0.2f} seconds.")

Result:

 2020-12-13 21:37:21: Establishing connection ....
before .... NSE:TCS
{'NSE:TCS': {'instrument_token': 2953217, 'last_price': 2783.6, 'ohlc': {'open': 2792.7, 'high': 2807.7, 'low': 2764.55, 'close': 2784.3}}}
after .... NSE:TCS
before .... NSE:INFY
{'NSE:INFY': {'instrument_token': 408065, 'last_price': 1163.2, 'ohlc': {'open': 1159.7, 'high': 1171.95, 'low': 1155.25, 'close': 1167.75}}}
after .... NSE:INFY
before .... NSE:WIPRO
{'NSE:WIPRO': {'instrument_token': 969473, 'last_price': 353.5, 'ohlc': {'open': 357.4, 'high': 360, 'low': 352.65, 'close': 355.9}}}
after .... NSE:WIPRO
before .... NSE:ITC
{'NSE:ITC': {'instrument_token': 424961, 'last_price': 216.3, 'ohlc': {'open': 214.45, 'high': 217.95, 'low': 213.35, 'close': 212.7}}}
after .... NSE:ITC
before .... NSE:TINPLATE
{'NSE:TINPLATE': {'instrument_token': 894209, 'last_price': 145.25, 'ohlc': {'open': 146.3, 'high': 150.4, 'low': 143.6, 'close': 145.35}}}
after .... NSE:TINPLATE
before .... NSE:ALLCARGO
{'NSE:ALLCARGO': {'instrument_token': 3456257, 'last_price': 140.9, 'ohlc': {'open': 154, 'high': 155, 'low': 138, 'close': 139.1}}}
after .... NSE:ALLCARGO
before .... NSE:GDL
{'NSE:GDL': {'instrument_token': 3004161, 'last_price': 115.05, 'ohlc': {'open': 117, 'high': 118.4, 'low': 114.7, 'close': 114.85}}}
after .... NSE:GDL
before .... NSE:RELIANCE
{'NSE:RELIANCE': {'instrument_token': 738561, 'last_price': 2005.8, 'ohlc': {'open': 2013, 'high': 2038, 'low': 1974.25, 'close': 2007}}}
after .... NSE:RELIANCE
before .... NSE:TRF
{'NSE:TRF': {'instrument_token': 4604673, 'last_price': 112.5, 'ohlc': {'open': 112.9, 'high': 116.9, 'low': 110, 'close': 106.3}}}
after .... NSE:TRF
before .... NSE:PNCINFRA
{'NSE:PNCINFRA': {'instrument_token': 2402561, 'last_price': 179.5, 'ohlc': {'open': 189.7, 'high': 189.7, 'low': 177.85, 'close': 175.15}}}
after .... NSE:PNCINFRA
before .... NSE:DMART
{'NSE:DMART': {'instrument_token': 5097729, 'last_price': 2685.1, 'ohlc': {'open': 2647, 'high': 2715, 'low': 2632, 'close': 2622.05}}}
after .... NSE:DMART
before .... NSE:HEROMOTOCO
{'NSE:HEROMOTOCO': {'instrument_token': 345089, 'last_price': 3185.3, 'ohlc': {'open': 3200, 'high': 3239.5, 'low': 3169, 'close': 3194.3}}}
after .... NSE:HEROMOTOCO
before .... NSE:MAHLOG
{'NSE:MAHLOG': {'instrument_token': 98561, 'last_price': 406.1, 'ohlc': {'open': 405, 'high': 427.95, 'low': 402.1, 'close': 390.25}}}
after .... NSE:MAHLOG
before .... NSE:IRCON
{'NSE:IRCON': {'instrument_token': 1276417, 'last_price': 90.85, 'ohlc': {'open': 96, 'high': 96, 'low': 90, 'close': 88}}}
after .... NSE:IRCON
before .... NSE:VOLTAS
{'NSE:VOLTAS': {'instrument_token': 951809, 'last_price': 804.9, 'ohlc': {'open': 814, 'high': 820.9, 'low': 800.55, 'close': 814.25}}}
after .... NSE:VOLTAS
/Users/manjulamkumar/Desktop/myRepo/DEV/autoTrade/batch/access_token.py executed in 19.84 seconds.

can you please help me identify if I am missing something ? It is taking almost 20 seconds to fetch the data for 15 stocks. While I was expecting it to take max of 5 secs.

Unlike multithreading, you cannot apply asyncio to existing code and expect it to become parallel, async code must use async functionality from the ground up. As Tomalak points out, getOHLC doesn't await anything, which is a good indicator that it is "async" in name only. Executing a non-awaiting function will fail to suspend when the function needs to wait for a result, and therefore asyncio.gather() will execute such functions in sequence. To fix the issue, you need to either switch to asyncio-aware library, or use threads instead of asyncio. — user4815162342
– user4815162342, Commented Dec 13, 2020 at 16:40
Reading file is an expensive operation. It should await there I think. — Tarique
– Tarique, Commented Dec 14, 2020 at 15:01
@user4815162342 BTW thank you for your posts on SO, you are great asyncio teacher :) — madzohan
– madzohan, Commented Dec 15, 2020 at 14:30

Andrew Svetlov · Accepted Answer · 2020-12-16 13:23:02Z

2

getOHLC method has no yield point (await something expression).

That's why asyncio has no chance to switch between coroutines.

answered Dec 16, 2020 at 13:23

Andrew Svetlov

17.5k8 gold badges70 silver badges72 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Manjulam Kumar Over a year ago

kc.ohlc(symbol) is an API call and it does not seem to be async function and hence if I try to await it errors.

Andrew Svetlov Over a year ago

If it is sunchronous -- you cannot get any benefit from asyncio, sorry.

madzohan · Accepted Answer · 2020-12-15 14:27:05Z

0

Try to open and read the file with aiofile library, and remove second redundant for-loop and async-gather, and yes should rewrite all underneath calls to async behavior (await kc.ohlc(symbol) # ... etc)

    # ...
    async with AIOFile('../config/stocks.txt', 'r') as afp:
        async for line in LineReader(afp):
            await getOHLC(kc, line.rstrip('\n'))
    # ...

edited Dec 15, 2020 at 14:27

answered Dec 15, 2020 at 14:19

madzohan

11.8k9 gold badges43 silver badges68 bronze badges

1 Comment

valentinmk Over a year ago

As far as I can understand '../config/stocks.txt' used for config list of symbols at startup time. The main problem is misunderstanding hole design of this code. The getOHLC function must be async by nature, but not just wrapped to async def

Collectives™ on Stack Overflow

Python Asyncio - does not seem to be working in asynchronous mode

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related