MemoryError when cursor.execute() is called with a return value when outputtypehandler is set in cx_Oracle 7.2 on Python 3.7.x 64 bit

[AlexanderRavenheart]

1. Python 3.7.x 64-bit (tested with 3.7.0 and 3.7.4). Issue is not present on 3.6.8.

2. cx_Oracle 7.2. Issue is not present on cx_Oracle 7.0 and 7.1.

3. cursor.execute("SELECT 1 FROM DUAL WHERE DUMMY LIKE TRANSLATE(:arg0 USING NCHAR_CS) ESCAPE TRANSLATE('\' USING NCHAR_CS)", {':arg0' = 'X'})

4. MemoryError

5. Windows 10 v1809 64-bit.

6. Oracle Instant Client Windows Base 12.1.0.2.0 64-bit. Zip format extracted at D:\Rudi\Libs\Oracle\instantclient_12_1.

7. Database information:

• Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
• PL/SQL Release 12.1.0.2.0 - Production
• "CORE 12.1.0.2.0 Production"
• TNS for Linux: Version 12.1.0.2.0 - Production
• NLSRTL Version 12.1.0.2.0 - Production

8. PATH environment variable contains D:\Rudi\Libs\Oracle\instantclient_12_1.

9. No specific oracle environment variables set.

10. No. This issue was discovered during execution of inspectdb from django as reported here: https://code.djangoproject.com/ticket/30630.

[cjbj]

Please give us a runnable Python script and the SQL to create the schema.

Thanks.

[AlexanderRavenheart]

Script: https://gist.github.com/AlexanderRavenheart/ba0fb7a284dee0f0d635d300eca8867f
Schema: https://gist.github.com/AlexanderRavenheart/671c24b8afcb9a70f7a5af408ec05a77

[cjbj]

Can you cut down the schema to the necessary bits?! 4744 lines seems a bit longer than really needed!

[AlexanderRavenheart]

I don't know what are the necessary bits. Can you be more specific? What should be left out? What if the issue does not trigger because something is removed from the schema?

[anthony-tuininga]

What Chris is asking for is a smaller, reproducible test case. I tried with the statement you specified in your question (after I fixed the syntax error) and it works just fine with cx_Oracle 7.2 on Windows 10 64-bit, Python 3.7.3 and Oracle 12.1 instant client -- exactly as you have specified above. I'll try a few more things and see if I can discover the source of the issue -- but you need to help out here, too! If I can't discover the source and you aren't willing to reduce your test case we'll just have to wait until someone else comes along with the same issue who can provide a test case we can work with. :-)

[anthony-tuininga]

Good news! I think I may have discovered the source of the issue. In the most recent release there was a change to indicate that cx_Oracle is Py_SSIZE_T_CLEAN -- meaning that all sizes are Py_ssize_t instead of int. One of the locations (accessed when using an output type handler) that used a length was not clearly marked as using Py_ssize_t and on Windows where sizeof(long) != sizeof(int) this was a problem. I think this will resolve your issue, but confirmation would be appreciated. I've examined the other locations and they all use Py_ssize_t the way they should.

[cjbj]

[ @AlexanderRavenheart about testcases, see https://stackoverflow.com/help/minimal-reproducible-example ]

[AlexanderRavenheart]

[ @AlexanderRavenheart about testcases, see https://stackoverflow.com/help/minimal-reproducible-example ]

Thank you for that link.

What Chris is asking for is a smaller, reproducible test case. I tried with the statement you specified in your question (after I fixed the syntax error) and it works just fine with cx_Oracle 7.2 on Windows 10 64-bit, Python 3.7.3 and Oracle 12.1 instant client -- exactly as you have specified above. I'll try a few more things and see if I can discover the source of the issue -- but you need to help out here, too! If I can't discover the source and you aren't willing to reduce your test case we'll just have to wait until someone else comes along with the same issue who can provide a test case we can work with. :-)

I'm not sure what else I could do better seeing as I provided a standalone script that triggers the issue with the complete schema for the database I'm using. I always want to help and be proactive in fixing issues, which is why I also spent about 2 days investigating the issue with the help of the django developers on their issue tracker and their irc support channel. If I make a mistake in providing the necessary aid please guide me to what I can do better.

Good news! I think I may have discovered the source of the issue. In the most recent release there was a change to indicate that cx_Oracle is Py_SSIZE_T_CLEAN -- meaning that all sizes are Py_ssize_t instead of int. One of the locations (accessed when using an output type handler) that used a length was not clearly marked as using Py_ssize_t and on Windows where sizeof(long) != sizeof(int) this was a problem. I think this will resolve your issue, but confirmation would be appreciated. I've examined the other locations and they all use Py_ssize_t the way they should.

Can you inform me how I can test this patch?

[cjbj]

@AlexanderRavenheart I appreciate your effort and detail very much. What I had noticed was that the .py file was 50 lines so it didn't seem like 4K lines of SQL was all being used! I was hoping to make it easy to reproduce your case and avoid any issues if it didn't all install.

Anyway, a patch was merged to master branch by @anthony-tuininga and you can build following https://cx-oracle.readthedocs.io/en/latest/installation.html#install-using-github. Installing a C compiler may be slightly more complex on Windows than on Linux, but if you can test, let us know.

[AlexanderRavenheart]

Tested the build (7.2.1) using the instructions provided and the issue is gone.
Thank you for your support and quick responses.

[cjbj]

Thank you.