Python: Reading binary file with date/time

Question

I have part of a binary file that represents a start time.

I am trying to read a time/date value in the file in Python but having trouble.

>>>file = open('file1',"rb")
>>>data = file.read()
>>>data[16:24]
b'Q\xca\rk\x9c\xc6\xd7\x88'
>>>unpacked, = unpack('<Q', data[16:24])
9860568284264254033

I already know that 16-24 location contains Int64 of 8 byte size.

in C#, I can successfully get the value {3/12/2020 11:45:40 AM} by doing

var value = _reader.ReadInt64();
sdatetime = DateTime.FromBinary(value);

Does anyone know how to read it correctly in Python?

Also if it helps,

>>> unpack('q',data[16:24])
(-8586175789445297583,)
>>> unpack('Q',data[16:24])
(9860568284264254033,)

I tried doing:

>>> unpacked
9860568284264254033
>>> secs = unpacked / 10.0 **7
>>> secs
986056828426.4253
>>> delta = datetime.timedelta(seconds = secs)
>>> delta
datetime.timedelta(days=11412694, seconds=66826, microseconds=425293)
>>> ts = datetime.datetime(1,1,1) + delta

which just resulted in OverflowError: date value out of range

Could you please edit the question and post the complete code and show how you're reading the file? — razieh babaee
– razieh babaee, Commented Jun 23, 2020 at 18:44
Well from looking at the C# documentation on the DateTime.FromBinary() method It looks like two bits are used to store the locality (Kind property) and the remaining 62 bits are the number of ticks from the beginning of the 21st century. Extract these number of ticks and perform some weird python datetime.timedelta math and you can figure out your answer I believe. — plum 0
– plum 0, Commented Jun 23, 2020 at 18:54
I'll post the math I attempted, but it popped with error message saying OverflowError: date value out of range — akb515
– akb515, Commented Jun 23, 2020 at 18:55
@akb515 That timedelta is incorrect, a quick sanity check shows that you are looking at a time delta of roughly 31,267 years. Figure out which 62 bits are used to store the number of ticks, determine the time conversion between ticks and seconds, then perform a timedelta on a datetime object representing January 1, 2001. — plum 0
– plum 0, Commented Jun 23, 2020 at 18:59

Dmitrii Altukhov · Accepted Answer · 2020-06-23 20:10:39Z

2

I think I've figured it out

from datetime import datetime, timedelta

time_bytes = b'Q\xca\rk\x9c\xc6\xd7\x88'
time_int = int.from_bytes(time_bytes, 'little')
time_bin = bin(time_int)

# remove two unrelated bits and convert to integer number of ticks
n_ticks = int(time_bin[:2] + time_bin[4:], 2)
secs = n_ticks / 1e7

d1 = datetime(1, 1, 1)
t1 = timedelta(seconds=secs)
print(d1 + t1)

outputs:

2020-03-12 15:45:40.947823

So it's correct up to the time zone

EDIT:

To get an actual timezone, use timedelta once more.

zone_delta = timedelta(hours=-4)
print(d1 + t1 + zone_delta)

gives

2020-03-12 11:45:40.947830

edited Jun 23, 2020 at 20:10

answered Jun 23, 2020 at 19:38

Dmitrii Altukhov

916 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Dmitrii Altukhov Over a year ago

I just saw the comment that @akb515 came up with the same solution. Anyways, maybe someone will use it later.

akb515 Over a year ago

For timezone, since I know it should be 11:45AM, how would I change the last three lines . its a 4 hr difference so where would include this change?

Dmitrii Altukhov Over a year ago

just added it to the solution

chepner Over a year ago

A little simpler: n_ticks = time_int & 0x3fffffffffffffff, using bitwise & to clear the top two bits. The hex value is (1<<63) - 1. (Too bad there isn't a nicer way to specify a 64-bit mask, though.)

Dmitrii Altukhov Over a year ago

I agree this is a bit cleaner.

plum 0 · Accepted Answer · 2020-06-23 19:57:28Z

We ran through a lot of this information in the comments but here is an answer in case comments are removed/deleted.

DateTime.FromBinary() converts C# datetime objects into binary by encoding 62 lower bits as ticks since January 1, 2001 (start of 21st century) and the 2 upper bits as locality/Kind information. This means that we need to extract the 62 lower bits before we can use them. Then we simply convert these seconds into a timedelta and perform the arithmetic to retrieve our python datetime object.

Now there will be a margin of error here because the Kind information that we are ignoring is used to determine timezone. You could dive deeper and look into what the Kind bits are and see if you can retrieve the timezone to be more consistent in your result, but for this answer I have not.

unpacked = unpack('<Q', data[16:24])
# 64 bits with two most significant bits as 0 is decimal: 4611686018427387903
# Here I am performing a bitwise mask to keep only the 62 least significant bits.
# I prefer this method in stead of string manipulation from bin() for potential memory efficiency reasons
ticks = unpacked & 4611686018427387903
# There are 10,000,000 ticks in a second
seconds = ticks / 10000000
td = datetime.timedelta(seconds=seconds)
epoch = datetime.datetime(year=1, month=1, day=1)

resultDate = td + epoch

Collectives™ on Stack Overflow

Python: Reading binary file with date/time

2 Answers 2

EDIT:

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

EDIT:

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related