Underneath your code snippet demonstrating PythonInterpreter('some string...') you say
This seems to "compile" the script.
No, this executes the script. Which is why you haven't been able to reuse the instance.
Per the docs:
PythonInterpreter.compile('some string...')
Compiles a string of Python source as either an expression (if possible) or a module.
Your question has two parts.
Focusing on the easier question, without getting too into the specifics of your setup:
The script contains a function that I then call many times [...] Is there a way to save some kind of object (PyCode?) so when another "host" comes along I do not have to call exec(script) again?
Yes. Based on the Jython docs, you should be first compiling the string into your PyCode object via: PyCode my_func= new PythonInterpreter.compile('some string expression or module') that you can then pass around to local threads. Which brings us to the second and slightly more difficult question
This entire process may be repeated for other hosts. Multiple hosts may need to execute the script concurrently. [...] Due to concurrency, I cannot simply hold on to the PythonInterpreter instance and reuse it. I would like to create a new instance of PythonInterpeter and give it some object rather than having it parse the script again.
You will need to manage local threads for each host via threadLocalStateInterpreter -- then simply replace your python.set('env_var',env_var) with a thread-safe python.setLocals(). Something like
import org.python.core.*;
import org.python.util.PythonInterpreter;
import java.util.concurrent.*;
public class JythonRemoteExecutor {
// shared, thread-safe, compiled Python function
private static PyCode compiledFunction;
// thread-local interpreter for each connecting host
private static ThreadLocal<PythonInterpreter> threadLocalStateInterpreter =
ThreadLocal.withInitial(() -> {
PythonInterpreter interp = new PythonInterpreter();
return interp;
});
// compile your Python function once at startup
static {
String pythonCode = "def your_function():...";
PythonInterpreter compiler = new PythonInterpreter();
compiledFunction = compiler.compile(pythonCode);
compiler.close();
}
/* execute the shared function for a specific host */
public static PyObject executeForHost(String hostId) {
// this is where you begin to take advantage of that precompiled
// code from earlier
PythonInterpreter interp = threadLocalStateInterpreter.get();
// execute the compiled function in this thread's namespace,
// this will allow you to use `your_function()` locally,
// independent of other hosts who may be executing their
// instance of your function
interp.exec(compiledFunction);
// set local variables specific to this host using setLocals()
interp.setLocals(new PyStringMap() {{
__setitem__("host_id", new PyString(hostId));
__setitem__("host", host);
__setitem__("tableName", tableName);
__setitem__("state", state);
__setitem__("index", index);
__setitem__("values", values);
}});
interp.exec("row(host, tableName, state, index, values)");
}
/**
* example driver program simulating multiple hosts connecting and executing
*/
public static void main(String[] args) throws InterruptedException {
ExecutorService executor = Executors.newFixedThreadPool(3); // set as needed
Callable<String> host1Task = () -> {
PyObject result = executeForHost("Host-1");
return "Host-1 finished executing."
};
Callable<String> host2Task = () -> {
PyObject result = executeForHost("Host-2");
return "Host-2 finished executing."
};
Callable<String> host3Task = () -> {
PyObject result = executeForHost("Host-3");
return "Host-3 finished executing."
};
// execute all host connections concurrently
Future<String> future1 = executor.submit(host1Task);
Future<String> future2 = executor.submit(host2Task);
Future<String> future3 = executor.submit(host3Task);
// print results -- should see all hosts finish executing
try {
System.out.println(future1.get());
System.out.println(future2.get());
System.out.println(future3.get());
} catch (ExecutionException e) {
e.printStackTrace();
}
executor.shutdown();
}
}
javatag as it seems appropriate.host,tableNameetc.)? i.e. if you are sharing the function does it need to be called before you set locals?