1

How would I be able to start a multiprocess when one of the tasks uses Asyncio and another task uses a subprocess to run?

Here is the main function:

import asyncio
import aioprocessing
import multiprocessing
import subprocess

# local imports
import steam_bot

async def open_auth():
    _ = await asyncio.create_subprocess_exec('wine', '/Users/vortex/Desktop/Steam Project/SDA/Steam Desktop Authenticator.exe')

async def start_bot(q):
    p = aioprocessing.AioProcess(target=steam_bot.main)
    p.start()
    await p.join()

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    tasks = [ 
        asyncio.ensure_future(open_auth()),
        asyncio.ensure_future(start_bot()),
    ]
    loop.run_until_complete(asyncio.wait(tasks))
    loop.close()

Steam Desktop Authenticator.exe is utilizing https://github.com/Jessecar96/SteamDesktopAuthenticator, and is a process that generates a GUI, and doesn't have a defined stop sequence. (note the command is to run the exe on a mac using wine, the command works properly).

Here is steam_bot using the steamio package :

import steam


class MyClient(steam.Client):
    async def on_ready(self):
        print("------------")
        print("Logged in as")
        print("Username:", self.user)
        print("ID:", self.user.id64)
        print("Friends:", len(await self.user.friends()))
        print("------------")

    async def on_message(self, message: steam.Message):
        # we do not want the bot to reply to itself
        if message.author == self.user:
            return

        if message.content.startswith("!hello"):
            await message.channel.send(f"Hello {message.author}")

def main():
    client = MyClient()
    client.run("username", "password")

The errors give usually include object nonetype cannot be used in await occuring with the .create_subprocess_exec() or an issue stating that the steam_bot doesn't contain variables, which are defined in the class initialization (so that cannot be the issue).

There may well be an issue with how I am starting up the program with aioprocess. I have hit a roadblock in trying to resolve these types of errors. I would like assistance in trying to run these two types of programs within a single main program.

Update based on @dano 's suggestion

import sys
import asyncio
import aioprocessing
import multiprocessing
from concurrent.futures import ProcessPoolExecutor
#from multiprocessing import Process, Queue
import subprocess

# local imports
import api
import steam_bot

def open_auth():
    subprocess.run(['wine', '/Users/vortex/Desktop/Steam Project/SDA/Steam Desktop Authenticator.exe'])

async def run_api(q):
    p = aioprocessing.AioProcess(target=api.main, args=(q, ))
    p.start()
    await p.join()

def start_bot(q):
    steam_bot.main(q)
    
async def main():
    loop = asyncio.get_running_loop()
    queue = aioprocessing.AioQueue()

    with ProcessPoolExecutor() as pool:
        open_auth_result = await loop.run_in_executor(pool, open_auth)
        bot_result = await loop.run_in_executor(pool, start_bot, queue)
        #api_result = await loop.run_in_executor(pool, run_api, queue)
        
    
if __name__ == "__main__":
    asyncio.run(main())

Current issue is that it looks like the queue is causing errors, but disregarding that, I still have the following issue: (which seems to be an issue with the startup of the bot, but it seems like all of the errors aren't correct)

concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 415, in runner
    await self.login(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 563, in login
    await self._login(SteamWebSocket.from_client, refresh_token=refresh_token or self.refresh_token)
                                                                                 ^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 241, in refresh_token
    return None if self.ws is None else self.ws.refresh_token
                   ^^^^^^^
AttributeError: 'MyClient' object has no attribute 'ws'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/process.py", line 263, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vortex/Developer/tf2/tf2mart/test_temp2.py", line 37, in start_bot
    steam_bot.main(q)
  File "/Users/vortex/Developer/tf2/steam_bot.py", line 220, in main
    client.run("Username", "Password")
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 418, in run
    asyncio.run(runner(), debug=debug)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 414, in runner
    async with self:
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 362, in __aexit__
    if not self.is_closed():
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/steam/client.py", line 266, in is_closed
    return self._closed
           ^^^^^^^^^^^^
AttributeError: 'MyClient' object has no attribute '_closed'. Did you mean: 'close'?
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/vortex/Developer/tf2/tf2mart/test_temp2.py", line 50, in <module>
    asyncio.run(main())
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/vortex/Developer/tf2/tf2mart/test_temp2.py", line 44, in main
    bot_result = await loop.run_in_executor(pool, start_bot)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MyClient' object has no attribute '_closed'
3
  • You're going to need to include the exact error message you're seeing, with stack traces. You're saying that you have seen two different errors trying to run the program. Under what conditions do you see each? Are you making code changes that result in the different errors, or does re-running the exact same program give you different results? Commented Nov 14 at 17:25
  • Also for this simple use-case, you can just use loop.run_in_executor with a ProcessPoolExecutor instead of aioprocessing (full disclosure: I am the author of aioprocessing). Commented Nov 14 at 17:30
  • @dano would your answer change if I was trying to use a queue as well? (also updated my question with edits based on your suggestion) Commented Nov 17 at 3:05

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.