0

I am logging binary data to my SD. I log a datetime object of type 'tm', which is 36 bytes on ESP32. When I create a datetime of type 'tm' on linux it is 56 bytes.

ESP32 write tm to SD: [tm datetime = 36 bytes]

time_t t = std::time(nullptr);
tm datetime = *std::localtime(&t);
uint16_t t = file.write((const uint8_t *)&datetime, sizeof(datetime)); // ==36 bytes.

PC [tm datetime = 56 bytes] (compiled using g++ on linux 64)

tm datetime;
std::ifstream fileStream(filename, std::ios::binary);
fileStream.read((char *)&datetime, sizeof(datetime));

tm is defined in time.h:

struct tm
{
  int   tm_sec;
  int   tm_min;
  int   tm_hour;
  int   tm_mday;
  int   tm_mon;
  int   tm_year;
  int   tm_wday;
  int   tm_yday;
  int   tm_isdst;
#ifdef __TM_GMTOFF
  long  __TM_GMTOFF;
#endif
#ifdef __TM_ZONE
  const char *__TM_ZONE;
#endif
};

My tm struct doesn't have the 'tm_gmtoff' or 'tm_zone', even if it had it wont add up to 56 bytes.

How can my tm be 56 bytes on my PC? If someone has a better way to log a datetime object, without this issue, let me know.

Improve this question
7
  • sizeof(int) on arduino (AVR) is 2, on most other platforms its 4. Commented Nov 17, 2021 at 10:51
  • 1
    Why log in binary? Log as text, in the format of date +'%FT%T%:z'. If you really need binary, log time_t rather than strcut tm. Commented Nov 17, 2021 at 10:54
  • @PMF I'm using ESP32, didn't specify, my bad! Commented Nov 17, 2021 at 11:04
  • @EdgarBonet logging to text is too slow, I log at 100hz using sd.h lib. why is logging time_t better in binary? Commented Nov 17, 2021 at 11:04
  • 3
    The layout of strcut tm is implementation dependent. In contrast, time_t is usually a simple integer, with very little variations across implementations: either 32 or 64 bits, and maybe a different epoch (which is fixed by simply adding a constant). Commented Nov 17, 2021 at 11:18

1 Answer 1

Reset to default
2

You can not trust system-level structs for data storage or transfer between different systems. The main reasons are:

  1. Different data type sizes (16 / 32 / 64 bit ints, floats vs doubles, etc)
  2. Struct element alignment differences owing to underlying architecture optimizations and limitations

Unless you are working with a system struct that is specifically designed for cross-system communication you must define your own and pay special attention to the layout.

  1. Always use explicitly sized variable types (uint8_t, int32_t, etc for example) to ensure both ends always agree on data storage sizes
  2. Always "pack" your struct (__attribute__((packed))) to collapse any specially aligned variables into just the room needed to store those variables

Point 2 requires special attention if you're doing much more than filling the struct and saving / transmitting it. It's advisable to never do more than create the struct instance, fill it, save it, then throw it away. If you start doing advanced operations with pointers to the struct you risk corrupting the packing code if the alignment changes. Also packed structs are often slightly slower to access owing to more complex code usually needed to be used to read / write the individual fields within them (especially if there are no byte-wise operations in the underlying architecture).

Improve this answer
1
  • Thank you! multiple people commented solutions, but you gave the reason why. Also thanks for your blogpost, very helpful! Commented Nov 19, 2021 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.