Python called via C# not returning value when in infinite loop

Question

I'm currently implementing a .Net app, which connects to a Raspberry SenseHat. To do so, I'm using the Python implementation https://pythonhosted.org/sense-hat/ and call the python scripts via Processes to be as loosely coupled as possible. Everything works fine, but I have some problems with the joystick: The example uses an infinite loop in the Python script. My "Joystock.py" script is currently looking like this:

import sys
try:
   import queue
except ImportError:
   import Queue as queue
import threading
import requests

from sense_hat import SenseHat
sense = SenseHat()

# Taken from https://stackoverflow.com/questions/48429653/python-returning-values-from-infinite-loop-thread
def _listen(queue):
  while True:
    event = sense.stick.wait_for_event(emptybuffer=True)
    val = event.action + ":" + event.direction
    queue.put(val)

def listen(params):
  q = queue.Queue()
  t1 = threading.Thread(target=_listen, name=_listen, args=(q,))
  t1.start()

  while True:
    value = q.get()
    print(value)

if __name__ == '__main__':
  args = sys.argv
  args.pop(0) # Remove file path
  methodName = args.pop(0) # Pop method name

  globals()[methodName](args)

The bottom part is to pass the method name and the parameters I'd like to call via arguments. My C# call is looking like this:

public void Listen(PythonListeningRequest request)
{
    var startInfo = _startInfoFactory.CreateForListening(request);

    var process = Process.Start(startInfo);
    process.BeginErrorReadLine();
    process.BeginOutputReadLine();
    process.EnableRaisingEvents = true;
    process.OutputDataReceived += (object sender, DataReceivedEventArgs e) =>
    {
        Console.WriteLine("Input: " + e.Data);
    };

    process.ErrorDataReceived += (object sender, DataReceivedEventArgs e) =>
    {
        Console.WriteLine("Error: " + e.Data);
    };
}

And the definition of the ProcessStartInfo:

public ProcessStartInfo CreateForListening(PythonRequest request)
{
    return new ProcessStartInfo
    {
        FileName = FindPythonExeFilePath(),
        Arguments = CreateArgumentsString(request),
        UseShellExecute = false,
        RedirectStandardInput = true,
        RedirectStandardOutput = true,
        RedirectStandardError = true,
        CreateNoWindow = true,
        WindowStyle = ProcessWindowStyle.Hidden
    };
}

private static string CreateArgumentsString(PythonRequest request)
{
    var sb = new StringBuilder();
    sb.Append(request.FilePath);
    sb.Append(" ");

    sb.Append(request.MethodName);
    sb.Append(" ");

    foreach (var arg in request.Arguments)
    {
        sb.Append(arg.AsString());
        sb.Append(" ");
    }

    var result = sb.ToString();
    return result;
}

private string FindPythonExeFilePath()
{
    var possibleFilePaths = new string[]
    {
        @"C:\Users\mlm\AppData\Local\Programs\Python\Python37-32\python.exe",
        @"C:\WINDOWS\py.exe",
        "/usr/bin/python"
    };

    var existingPythonPath = possibleFilePaths.FirstOrDefault(fp => _fileSystem.File.Exists(fp));
    Guard.That(() => existingPythonPath != null, "No python path found.");

    return existingPythonPath;
}

As you can see in the python part, there is a queue used, which I've got from another SO question. Unfortunately, it still doesn't work, as soon as "t1.start()" is in the code, I never get a return value.

Trying the python script manually works fine, so I guess the problem is the Process connection to C#? Unfortuntely, I didn't find anything related to this behavior, has therefore anyone any idea, what could cause this issue?

PilouPili · Accepted Answer · 2019-09-18 06:28:03Z

1

Bottom line : use sys.stdout and sys.stderr followed by flush() on either stream and avoid print

Since I am not able to have SenseHat, I downsized your example to:

try:
    import queue
except ImportError:
    import Queue as queue
import threading
import time
import sys

# Taken from https://stackoverflow.com/questions/48429653/python-returning-values-from-infinite-loop-thread
def _listen(queue):
    val =0
    while True:
        time.sleep(1)
        val = val+1
        queue.put(val)


def listen(params):
    q = queue.Queue()
    t1 = threading.Thread(target=_listen, name=_listen, args=(q,))
    t1.start()

    while True:
        value = q.get()
        sys.stdout.write(str(value) + '\n')
        sys.stdout.flush()


if __name__ == '__main__':
    args = sys.argv
    args.pop(0)  # Remove file path
    methodName = args.pop(0)  # Pop method name

    globals()[methodName](args)

as for the C# part I didn't change a thing just got rid of the class PythonRequest

This seems to work. Whereas with print(value) instead of sys.stdout.write(str(value) + '\n') sys.stdout.flush() I was not getting any return value from the callback OutputDataReceived

So I believe you have to write on sys.stdout and sys.stderr then force flush to write on a stream piped to your C#. Otherwise using print fills the stdout buffer and does not necessarily flush.

edited Sep 18, 2019 at 6:28

answered Sep 17, 2019 at 19:40

PilouPili

2,7192 gold badges20 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Matthias Müller Over a year ago

Works like a charm, thank you very much. One question: Should I alwaays use stdout instead of print? Are there any major differences except the threading issues?

Matthias Müller Over a year ago

Interesting, I'm currently testing on the Raspberry PI, the solution works only without loop and has a huge delay. With a loop I don't get any response again.

PilouPili Over a year ago

All things considered I think the issue is really flushing. I have tired with print str(value) sys.stdout.flush() and it also works for me. So you can use print it's just that since the flush method is only available on stream I think it is clearer to use sys.stdout, moreover you have control over whether what you send to the C# has a '\n' or not.

PilouPili Over a year ago

I have tired on my raspberry. I am not seeing any problem on my part. But again I miss the part concerning SenseHat...

Collectives™ on Stack Overflow

Python called via C# not returning value when in infinite loop

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related