1

I'm working on a research project and need to generate multiple temporary tables for my analysis. The query to create those tables looks like this:

DECLARE crsr CURSOR FORWARD_ONLY FOR
  SELECT DISTINCT 
    YEAR(s.date) AS Jahr, 
    MONTH(s.date) AS Monat
  FROM ##preliminary_data___id__ AS s;

OPEN crsr;
FETCH NEXT FROM crsr
INTO
  @year, @month;
WHILE @@fetch_Status = 0
BEGIN
  PRINT 'Next Date -------------------------------------'
  DECLARE @date DATE = DATEFROMPARTS(@year, @month, 1);
  PRINT @date;

  INSERT INTO ##final_data___id__(--snip--)
  SELECT 
    --snip--

  FETCH NEXT FROM crsr
  INTO @year, @month;
 
END;

CLOSE crsr;
DEALLOCATE crsr;

The query, executed from MS SSMS works as expected. I have about 15k subjects, so I would like to generate these table automatically, ideally using python. When I use sqlalchemy, this is my code:

engine = sqlalchemy.create_engine(cxn_str)
with engine.raw_connection().cursor() as cursor:
    cursor.execute(query)
    cursor.commit()

This runs successfully, but the resulting ##final_data___id__ only contains the data for one month. It also runs very quickly, which leads me to assume that FETCH NEXT FROM crsr is never actually performed.

How would I correctly execute this query from python, such that the created table contains the data for all available months?

This is a research project that runs against a corporate database, therefore I have no rights to create stored procedures or tables. ##tables are the longest lasting objects that are possible, and in this setup this is acceptable.

Thanks!

4
  • 1
    Two initial comments: (1) Start your anonymous code block with SET NOCOUNT ON; (2) Remove or comment out the PRINT statements when running the code from Python. Commented Mar 20, 2022 at 15:00
  • I am unable to reproduce your issue. This code works fine for me. Commented Mar 20, 2022 at 15:30
  • The first comment actually did the trick. The PRINTs apparently interrupt the execution. I'm not sure why this is, but I have to admit I now recall being aware of something like that. Post the comment as answer and I'll mark it! The separation into several statements (like in the pastebin) seems to be not necessesary. Thanks a lot! Commented Mar 20, 2022 at 18:55
  • I guess the real question is why you are using a cursor and temp tables in the first place. What's wrong with just SELECT DISTINCT DATEFROMPARTS(YEAR(s.date), MONTH(s.date), 1) FROM ##preliminary_data___id__ AS s Why do you need multiple temp tables to hold the same data? Commented Mar 20, 2022 at 21:43

1 Answer 1

2

Two of the most common causes of stored procedures and anonymous code blocks "working when executed in SSMS" but "not working when executed from a pyodbc (Python) connection" are:

  1. the code not beginning with SET NOCOUNT ON;, and
  2. PRINT statements.

Both of those can cause the server to return "results" that might mask the intended results from the code, and in some cases (like this one, apparently) can actually cause the code to fail if those extra "results" are numerous enough to overflow the buffer that the server has allocated for communication with the client.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.