1

I'm using Qt 5.14.2 with SQLite3. In the function below, I attempt to drop two temporary tables: "COPYPL" and "COPYPL1". Most of the time this works fine, but with some SQLite databases (seemingly at random), I get a "database is locked" error when calling:

DROP TABLE COPYPL
DROP TABLE COPYPL1

This behavior is inconsistent across different databases, but consistent within a specific one (i.e., once a DB exhibits this issue, it always does).

I am not an SQLite expert, so I may be missing something obvious here. Why would this happen? What can I do to ensure the tables are dropped successfully every time?

Here's the relevant Qt/C++ code:

void RenumberDbase::start()
{
    long weldnum = 1000000;
    long pitnum = 2000000;
    long clusternum = 3000000;
    long pipenum = 4000000;
    long dentnum = 5000000;
    long othernum = 6000000;

    QProgressDialog progress;
    progress.setLabelText("Renumbering Database");
    progress.setModal(true);
    progress.setCancelButton(0);
    progress.setValue(0);
    progress.setMinimumDuration(0);

    QSqlQuery initQuery(db);
    initQuery.setForwardOnly(false);
    initQuery.exec("SELECT ID, TYPE FROM PIPELINE ORDER BY ID ASC");
    initQuery.last();
    long long HighID = initQuery.value(0).toLongLong();
    int rowcount = initQuery.at();

    if (HighID < 9000000)
        HighID = 9000000;
    progress.setRange(0, rowcount * 2);
    db.transaction();

    QSqlQuery query(db);
    query.exec("CREATE TABLE COPYPL AS SELECT ID, TYPE, START_DIST_FEET FROM PIPELINE");

    // --- FIRST PASS: move IDs above HighID ---
    {
        QSqlQuery readQuery(db);
        readQuery.setForwardOnly(true);
        readQuery.exec("SELECT * FROM COPYPL ORDER BY START_DIST_FEET");
        QSqlRecord rec1 = readQuery.record();
        int IDindex = rec1.indexOf("ID");
        int TYPEindex = rec1.indexOf("TYPE");
        long whereweat = 0;

        QSqlQuery PipeLineQuery(db);
        PipeLineQuery.prepare("UPDATE PIPELINE SET ID = :idd WHERE ID = :id");

        QSqlQuery PipeQuery(db);
        PipeQuery.prepare("UPDATE PIPE SET ID = :idd WHERE ID = :id");

        QSqlQuery PitQuery(db);
        PitQuery.prepare("UPDATE PIT SET ID = :idd WHERE ID = :id");

        QSqlQuery ClusterQuery(db);
        ClusterQuery.prepare("UPDATE CLUSTER SET ID = :idd WHERE ID = :id");

        QSqlQuery DentQuery(db);
        DentQuery.prepare("UPDATE DENT SET ID = :idd WHERE ID = :id");

        while (readQuery.next()) {
            progress.setValue(whereweat);
            long long id = readQuery.value(IDindex).toLongLong();
            QString TYPE = readQuery.value(TYPEindex).toString();

            if (TYPE == "PIT") {
                PitQuery.bindValue(":id", id);
                PitQuery.bindValue(":idd", HighID);
                PitQuery.exec();
            } else if (TYPE == "CLUSTER") {
                ClusterQuery.bindValue(":id", id);
                ClusterQuery.bindValue(":idd", HighID);
                ClusterQuery.exec();
            } else if (TYPE == "PIPE") {
                PipeQuery.bindValue(":id", id);
                PipeQuery.bindValue(":idd", HighID);
                PipeQuery.exec();
            } else if (TYPE == "DENT") {
                DentQuery.bindValue(":id", id);
                DentQuery.bindValue(":idd", HighID);
                DentQuery.exec();
            }

            PipeLineQuery.bindValue(":id", id);
            PipeLineQuery.bindValue(":idd", HighID);
            PipeLineQuery.exec();

            ++HighID;
            ++whereweat;
        }
        readQuery.finish();
        readQuery.clear();

        if (!db.commit()) {
            qDebug() << "Commit failed (first pass):" << db.lastError().text();
            db.rollback();
            return;
        }
    }

    // Try to VACUUM and drop COPYPL
    QSqlQuery vacuumQuery1(db);
    vacuumQuery1.exec("VACUUM");

    int retryCount = 0;
    while (retryCount < 3) {
        QSqlQuery dropQuery1(db);
        if (dropQuery1.exec("DROP TABLE COPYPL")) {
            break;
        } else {
            qDebug() << "Failed to drop COPYPL, retrying... (" << retryCount + 1 << ")";
            retryCount++;
            QThread::msleep(500);
        }
    }

    // Second pass, with COPYPL1
    query.exec("CREATE TABLE COPYPL1 AS SELECT ID, TYPE, START_DIST_FEET,STATIONNUMBER FROM PIPELINE");
    db.transaction();

    {
        QSqlQuery readQuery(db);
        readQuery.setForwardOnly(true);
        readQuery.exec("SELECT * FROM COPYPL1 ORDER BY START_DIST_FEET");
        QSqlRecord rec1 = readQuery.record();
        int IDindex = rec1.indexOf("ID");
        int TYPEindex = rec1.indexOf("TYPE");
        long whereweat = rowcount;

        // ... (similar update logic as above) ...

        readQuery.finish();
        readQuery.clear();

        if (!db.commit()) {
            qDebug() << "Commit failed (second pass):" << db.lastError().text();
            db.rollback();
            return;
        }
    }

    QSqlQuery vacuumQuery2(db);
    vacuumQuery2.exec("VACUUM");

    retryCount = 0;
    while (retryCount < 3) {
        QSqlQuery dropQuery2(db);
        if (dropQuery2.exec("DROP TABLE COPYPL1")) {
            break;
        } else {
            qDebug() << "Failed to drop COPYPL1, retrying... (" << retryCount + 1 << ")";
            retryCount++;
            QThread::msleep(500);
        }
    }

    progress.setValue(rowcount * 2 + 1);
}

I've already tried:

Calling .finish() and .clear() on the queries.

Explicitly scoping them so they are destroyed.

Using VACUUM before dropping.

Retrying with a delay.

Still, some databases remain locked. What else can I try to ensure these tables can be dropped safely?

1
  • I'd start checking all those exec() return values Commented Jun 5 at 14:01

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.