0

We have an existing python application (let's call it control app) that does operation data logging as well as smaller controlling tasks on a machine. We want to extend this application with a web interface, which is based on flask (let's call it web app). Both parts, the control app as well as the web app, are already present, however, the setup feels somehow fishy. In the process of rethinking the setup, I'm somehow undecided on how to structure those two parts.

At the moment, the control app gathers machine data and stores it in a postgres database. Based on several machine states, additional operations are performed that provide new input for the PLCs that control the machine.

The web app currently polls the database to react to machine states to e.g. update visualisation data, change some (state representing) images and such things.

The web app polling the database is the part that somehow smells. So my idea was to unify both apps into one to have the web app tightly coupled to the control app to be able to react on machine state changes instead of polling the database for those state changes.

Based on that idea, I'm wondering how to add a flask app to an existing python app. When I'm not mistaken, the flask app consumes the application's main thread, which would break to already existing logic. Thus I would need to have one of the two parts running on another thread. Thinking about this problem, I'm further wondering whether this merging is a good idea at all.

So, the questions are: Is it a good idea to merge both applications? If yes, how to merge them without breaking one of them? If not, how else should I try to get rid of the database polling (how to synchronize and also move some data from the web app to the control app)?

Improve this question

1 Answer 1

Reset to default
3

It's not a good idea to merge them per se -- problems in one part will affect the other, and this sort of tight coupling is a bad idea both because you can't run the two parts of the program on separate machines and because if one crashes, so does the other one. It's better to have them communicating over some sort of protocol.

If I were designing this, I would probably do the same thing as you did, except that instead of using an SQL database for this, I would use something like Redis which stores its data in memory. Redis allows you to subscribe to events rather than poll for updates, and polling for updates is cheaper because it's in memory.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thanks for your answer. Just two remarks (that don't make it less reasonable): 1. The chance that both apps will run on different machines is more or less negligible. 2. One reason (among others) to use a sqlish database is to lower the hurdle for some partners to use the database. I'm still open to overthink this decision, but Redis doesn't seem to fit our need to persist operational data over longer time periods (days, weeks, months). Also, with update frequencies around 0.1-1Hz, I have the feeling that an in-memory data-store/cache is a little over the top!?
If you need to persist logs, then a proper database is a good idea -- there you can represent your events as a time series and do advanced queries on them and whatnot. I thought you needed just the instantaneous data to show on a dashboard of sorts. If you want to be hip and trendy, you can use something like ElasticSearch which will allow you to build a dashboard in it.
If you really don't want to poll the database for events (although with a web-based dashboard you'll probably have to unless you use some Websocket trickery), then you can use some sort of message protocol like MQTT to send events in real time as opposed to waiting to pick them up at the next update. Although again, if you aren't polling too frequently and you include a clause like WHERE event_id > [latest received event id] then the database will optimize for that query and very little traffic will go through the network or to the disk until an event row is inserted.
Thanks again for the input. Not quite the definite answer I hoped for, but some very valuable input to take into consideration. As always: "It depends" ;)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.