0

I have the following code

logger = get_console_logging_object()
def get_db_path() -> str:
    """ get the path to the database file"""
    script_path = Path(__file__).parents[1]
    db_path = script_path.joinpath("data","verySensetivedata.db")
    logger.info(f"Saving data to {db_path}")
    return db_path

def get_db_connection():
    """ returns the db connections"""
    return sqlite3.connect(get_db_path())

def init_sqlite():
    """ creates the table xxxx"""
    conn = get_db_connection()
    cur = conn.cursor()
    try:
        cur.execute("create table if not exists sfdata (...)")
        conn.commit()
        cur.close()
        conn.close()
    except sqlite3.OperationalError as err:
        logger.error(f"Error in creating the sfdata table with error: {err}")
        cur.close()
        conn.rollback()
        conn.close()
        raise err

def write_data_to_db(data: list):
    """ writes all the data from openai to the database
    data-> list   (account_name, opportunity, created_date, techs, roles)"""

    insert_sql = "insert into sfdata (....... ) values(?,?,?,?,?)"
    conn = get_db_connection()
    cur = conn.cursor()
    try:
        cur.executemany(insert_sql, data)
        conn.commit()
        cur.close()
        conn.close()
    except sqlite3.OperationalError as err:
        logger.error(f"Error in inserting values into sfdata table with error: {err}")
        cur.close()
        conn.rollback()
        conn.close()
        raise err

when I execute the write_data_to_db() I get the log output "Saving the data to ..." for each element in the list, is the the correct behaviour? Is there something wrong with the code?

thanks,

es

3
  • 1
    You need to provide the code calling write_data_to_db. This code is deterministic and explicit, it can't call get_db_connection/get_db_path more than once per call to write_data_to_db, so you're the one calling it more than once. Commented Sep 16 at 12:42
  • 1
    Also, as a side-note, you probably want to use a bare raise, not raise err. The latter raises the exception anew, so the traceback points to the raise statement, while bare raise inside an except block reraises the exception in question as if it wasn't caught in the first place (so the traceback points to the originally failing line, rather than chaining the exception to itself and having two tracebacks, one largely useless, one nested with the original information). Commented Sep 16 at 12:44
  • @shadowranger, your guess is 100% correct. the call to the function is in the wrong place. And thanks for the tip. Commented Sep 16 at 12:47

1 Answer 1

1

Based on the manifestation of the problem (and your later confirmation in comments), the code calling write_data_to_db must be incorrectly calling it many times, rather than just once per list as expected. As written, it's impossible for this code to be reconnecting more than once per call to write_data_to_db.

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.