0

This was the result of my experiments with bytea with postgreSQL and libpq. The result file I am creating with the recieved data is double sized than the original uploaded plus 2 bytes (picture.png). I can not understand exactly how many operations I am doing incorrectly because the procedure is quite confusing for my critter brain. Any help or advice will be a big help for me, thank you in advance.

<edited 27/5/2021> you can find the solution a little below

#include <string>
#include <fstream>
#include <iostream>

#include "libpq/libpq-fs.h"
#include "libpq-fe.h"

#pragma comment(lib, "libpq.lib")   /*!< Only for windows compilation */

int main(int argc, char* argv[])
{
    //************************ SAVING FILE TO DB AS BYTEA **********
    manager.conn = manager.ConnectDB();  // my manager, working fine    

    FILE* file = fopen("powerup.png", "rb");
    if (file == NULL) cout << endl << "FILE WAS UNABLE TO BE READED" << endl;
    fseek(file, 0, SEEK_END);
    long int fileSize = ftell(file);

    rewind(file);

    unsigned char* buffRead = (unsigned char*)malloc(fileSize);
    size_t bytes_read = fread(buffRead, 1, fileSize, file);
    if (bytes_read != fileSize) cout << endl << "fread reading Error";
    fclose(file);

    const char* paramValues[3];
    paramValues[0] = "1";
    paramValues[1] = "powerup.png";
    paramValues[2] = reinterpret_cast<const char*>(buffRead);       //._.' type cast required to PQexeParams?

    const int paramFormats[3]{ 0,0,1 };
    const int paramLenghts[3]{ strlen(paramValues[0]), strlen(paramValues[1]), fileSize};

    PGresult *res = PQexecParams(manager.conn, "INSERT INTO filebyte (id, filename, file) VALUES($1::text, $2::text, $3::bytea)",
        3,              /* params */
        NULL,           /* let the backend deduce param type */
        paramValues,
        paramLenghts,   /* don't need param lengths since text */
        paramFormats,   /* default to all text params */
        1);

    if (res && PQresultStatus(res) == PGRES_COMMAND_OK) cout << endl << "Inserted data to filebyte OK";

    PQfreemem(res);

    //********************** PICKING FILE FROM DB AS BYTEA **********
    
    const char* bytesFromDB = new const char[fileSize];
    int sizeR = -1;
    const char *sentenceEx = "SELECT file FROM filebyte WHERE id='1'";

    res = PQexec(manager.conn, sentenceEx);

    if (res && PQresultStatus(res) == PGRES_TUPLES_OK) 
    {
        sizeR = PQgetlength(res, 0, 0);
        bytesFromDB = PQgetvalue(res, 0, 0);
    }

    ofstream myFile("picture.png", ios::out | ios::binary);

    myFile.write(bytesFromDB, sizeR);
    myFile.close();

    PQfreemem(res);
    free(buffRead);
    delete sizeP;

    manager.CloseConn(manager.conn);

    return true;
}

1
  • Good afternoon. @Laurenz Albe: following your recomendation I have posted my new try using Bytea, I feel I am close, but unsuccesfull. I haven't lost my hope yet :) Commented May 24, 2021 at 14:22

2 Answers 2

3

This is the result of the code working fine. I have some time to post it now in case it could help somebody. It was a little hard for me to understand how bytea works but finally it's solved. Kind regards to the comunity.

enter image description here

#include <string>
#include <fstream>
#include <iostream>
#include <sstream>

#include "libpq/libpq-fs.h"
#include "libpq-fe.h"

#pragma comment(lib, "libpq.lib")   /*!< Only for windows compilation */

int main(int argc, char* argv[])
{
    ManageDB manager;       

    manager.conn = manager.ConnectDB();  // My manager, connects ok

    const char* fileName = {"test.png"};
    const char* fileNameOut = { "testOut.png" };

    FILE* file = fopen(fileName, "rb");
    if (file == NULL) cout << endl << "FILE WAS UNABLE TO BE READED" << endl;
    fseek(file, 0, SEEK_END);
    long int fileSize = ftell(file);

    rewind(file);

    unsigned char* buffRead = (unsigned char*)malloc(fileSize);
    size_t bytes_read = fread(buffRead, 1, fileSize, file);
    if (bytes_read != fileSize) cout << endl << "fread reading Error";
    fclose(file);
    
    size_t* sizeP = new size_t(fileSize);

    const char* paramValues[3];
    paramValues[0] = "1";
    paramValues[1] = fileName;
    paramValues[2] = reinterpret_cast<const char*>(buffRead);   // type cast required to PQexeParams

    const int paramFormats[3]{ 0,0,1 };   //0 = text, 1 = binary
    const int paramLenghts[3]{ strlen(paramValues[0]), strlen(paramValues[1]), fileSize};

    PGresult *res = PQexecParams(manager.conn, "INSERT INTO filebyte (id, filename, file) VALUES($1::text, $2::text, $3::bytea)",
        3,              /* params num */
        NULL,           /* let the backend deduce param type */
        paramValues,
        paramLenghts,
        paramFormats,
        1);

    if (res && PQresultStatus(res) == PGRES_COMMAND_OK) cout << endl << "Inserted data to filebyte OK";

    PQfreemem(res);

    //************************download from DB*************************
    
    const char* bytesFromDB = new const char[fileSize];
    int sizeR;
    // Table columns = id(text) | filename(text) | file(bytea)
    const char* sentenceEx = "SELECT file FROM filebyte WHERE id='1'";

    res = PQexec(manager.conn, sentenceEx);

    if (res && PQresultStatus(res) == PGRES_TUPLES_OK)
    {
        sizeR = PQgetlength(res, 0, 0);
        bytesFromDB = PQgetvalue(res, 0, 0);
    }
    else cout << endl << "Error at inserting file data in filebyte table";

    string hex(bytesFromDB, 2, sizeR-2);  //removing the first '\x' characters of the result.

    // vars for converting to real bytes process
    std::basic_string<uint8_t> bytes;
    bytes.clear();
    std::string nextbyte; 
    nextbyte.clear();
    uint16_t byte;

    // Iterate over every pair of hex values in the input string (e.g. "18", "0f", ...)
    for (size_t i = 0; i < hex.length(); i += 2)
    {
        // Get current pair and store in nextbyte
        nextbyte = hex.substr(i, 2);

        // Put the pair into an istringstream and stream it through std::hex for
        // conversion into an integer value.
        // This will calculate the byte value of your string-represented hex value.
        std::istringstream(nextbyte) >> std::hex >> byte;

        // As the stream above does not work with uint8 directly, we have to cast it now.
        // As every pair can have a maximum value of "ff",
        // which is "11111111" (8 bits), we will not lose any information during this cast.
        bytes.push_back(static_cast<uint8_t>(byte));
    }

    // string obj from bytes-"array"
    std::string result(begin(bytes), end(bytes));

    ofstream myFile(fileNameOut, ios::out | ios::ios_base::binary);
    
    if (myFile.is_open())
    {
        myFile << result;
        myFile.close();
    }
    else cout << endl << "Impossible to writte the file " << fileNameOut;
    
    manager.CloseConn(manager.conn);   //closes connection with database internally ok

    PQfreemem(res);
    free(buffRead);
    delete sizeP;
    free((char*)fileName[8]);
    free((char*)fileNameOut[11]);
    
    return true;
}

Sign up to request clarification or add additional context in comments.

1 Comment

I would like to thank @ElectRocnic for his answer at this [link]stackoverflow.com/questions/21819782/writing-hex-to-a-file from where I have taken a little piece of code (and comments) to convert string representation of hex to real binary. I just declared the variables inside the loop outside to have a better process performance. Thank you :)
0

I feel I am close, I am mising something about data type treatment/conversion. I think it could help to understand what's happening. Still working on it. I show only little fragments of the observed data in the following picture.

Data Analisys with VSCode binary viewer

It seems the result of the query is a character string representing hexadecimal. For the fact I am writting the output file as binary it's converting figurative asci hex representation to real hexadecimal again, destroying the real content of the file and increasing the size. I just added a visual explanation about what is happening with the result file content.

1 Comment

Removing the "\x" from the Hex raw string (the widest one from the image), I have tested the data and is correct. It can be writen as an operative .png file. I will post as soon I find the solution to do it in C/C++

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.