How to interact with a window's GUI with Python?

Question

Let's say you want to open myapp.exe, open the 3rd menu, then choose the 2nd menu item (i.e. like a user would do with the keyboard or mouse), and then in the dialog window, choose the 2nd button.

pyahk and pyautogui seem to offer this, but in a rather "low-level" way, by simulating clicks:

pyautogui.click(x=moveToX, y=moveToY, clicks=num_of_clicks, interval=secs_between_clicks, button='left')

How to do interact with a Windows GUI in a higher-level way with Python?

Example:

window = gui.open('myapp.exe')
window.menu_open(3).choose_item(2)
child_window = window.wait_for_dialog()
child_window.buttons[1].click()

I do not believe such a method exists... You would have to attach a program to another program and be able to read and interpret what a "button" or "item" is despite the host program being written in an arbitrary language. As well many programs do not allow other programs to be attached to them because that is how you exploit and hack vulnerabilities in the host program. — Reedinationer
– Reedinationer, Commented Feb 22, 2019 at 18:43
@Reedinationer isn't there a unified way to browse through windows (list all the top-level hWnd? <- old memories from Winapi / C) — Basj
– Basj, Commented Feb 22, 2019 at 18:47
Does pywinauto work how you want it? It seems like it's more high-level than the other libraries you mentioned. — Random Davis
– Random Davis, Commented Feb 22, 2019 at 18:47
@RandomDavis Seems great! I think you can post it as an answer, or do you prefer I do it? — Basj
– Basj, Commented Feb 22, 2019 at 18:49

Basj · Accepted Answer · 2019-02-22 18:50:42Z

12

pywinauto seems to be much more in-line with what you want - it utilizes the Win32 API and MS UI Automation among other things.

Here is an example of automation of the notepad application:

from pywinauto.application import Application
app = Application().start("notepad.exe")

app.UntitledNotepad.menu_select("Help->About Notepad")
app.AboutNotepad.OK.click()
app.UntitledNotepad.Edit.type_keys("pywinauto Works!", with_spaces = True)

edited Feb 22, 2019 at 18:50

Basj

47.5k113 gold badges467 silver badges819 bronze badges

answered Feb 22, 2019 at 18:49

Random Davis

6,8974 gold badges17 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Basj Over a year ago

It's exactly what I was looking for, amazing!

jksevend Over a year ago

Can i also modify a currently running application rather than starting it?

Random Davis Over a year ago

@jksevend absolutely. here is how.

Collectives™ on Stack Overflow

How to interact with a window's GUI with Python?

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related