9

I usually perform things like this in C++, but I'm using python to write a quick script and I've run into a wall.

If I have a binary list (or whatever python stores the result of an "fread" in). I can access the individual bytes in it with: buffer[0], buffer[1], etc.

I need to change the bytes [8-11] to hold a new 32-bit file-size (read: there's already a filesize there, I need to update it). In C++ I would just get a pointer to the location and cast it to store the integer, but with python I suddenly realized I have no idea how to do something like this.

How can I update 4 bytes in my buffer at a specific location to hold the value of an integer in python?

EDIT

I'm going to add more because I can't seem to figure it out from the solutions (though I can see they're on the right track).

First of all, I'm on python 2.4 (and can't upgrade, big corporation servers) - so that apparently limits my options. Sorry for not mentioning that earlier, I wasn't aware it had so many less features.

Secondly, let's make this ultra-simple.

Lets say I have a binary file named 'myfile.binary' with the five-byte contents '4C53535353' in hex - this equates to the ascii representations for letters "L and 4xS" being alone in the file.

If I do:

f = open('myfile.binary', 'rb')
contents = f.read(5)

contents should (from Sven Marnach's answer) hold a five-byte immutable string.

Using Python 2.4 facilities only, how could I change the 4 S's held in 'contents' to an arbitrary integer value? I.e. give me a line of code that can make byte indices contents [1-4] contain the 32-bit integer 'myint' with value 12345678910.

Improve this question
3
  • You cannot change the contents since the string is immutable ... you can create a new string, or use a mutable container like bytearray. Commented Oct 27, 2011 at 21:13
  • Thanks, I'm happy to create a new string or byte array if that's the case. How could I create a modifiable byte array with the same contnets, and update the range [1-4] of that to equal the binary representation of my integer? Commented Oct 27, 2011 at 21:14
  • I extended my posting below and included a working example for Python 2.4 :-) Commented Oct 27, 2011 at 21:23

5 Answers 5

Reset to default
11

What you need is this function:

struct.pack_into(fmt, buffer, offset, v1, v2, ...)

It's documented at http://docs.python.org/library/struct.html near the top.

Example code:

import struct
import ctypes

data=ctypes.create_string_buffer(10)
struct.pack_into(">i", data, 5, 0x12345678)
print list(data)

Similar posting: Python: How to pack different types of data into a string buffer using struct.pack_into

EDIT: Added a Python 2.4 compatible example:

import struct

f=open('myfile.binary', 'rb')
contents=f.read(5)
data=list(contents)
data[0:4]=struct.pack(">i", 0x12345678)
print data
Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

+1. I'd suggest to use the built-in bytearray to create the mutable buffer instead of ctypes.create_string_buffer.
For everyone I tried working it out by reading the pack page and the solutions provided. I seem to be missing alot since I have python 2.4 instead of 2.5. I provided a greatly simplified version of my problem in an edit to my post above - if you can answer that with a line of code, I'll be completely happy :)
I'm afraid your 2.4 compatible version can't use bytearray as it was introduced in 2.6.
Hmmm. That's true, I only avoided the pack_into function ... embarassing. Let's see what was there in 2.4 :)
Ok I think now it works, tried it on a VM with Python 2.4. Other objections?
4

Have a look at Struct module. You need pack function.

EDIT:

The code:

import struct

s = "LSSSS" # your string
s = s[0] + struct.pack('<I', 1234567891) # note "shorter" constant than in your example
print s

Output:

L╙☻ЦI

struct.pack should be available in Python2.4.

Your number "12345678910" cannot be packed into 4 bytes, I shortened it a bit.

Improve this answer

1 Comment

see comment on hochl's answer
2

The result of file.read() is a string in Python, and it is immutable. Depending on the context of the task you are trying to accomplish, there are different solutions to the problem.

One is using the array module: Read the file directly as an array of 32-bit integers. You can modify this array and write it back to the file.

with open("filename") as f:
    f.seek(0, 2)
    size = f.tell()
    f.seek(0)
    data = array.array("i")
    assert data.itemsize == 4
    data.fromfile(f, size // 4)
data[2] = new_value
# use data.tofile(g) to write the data back to a new file g
Improve this answer

1 Comment

see comment on hochl's answer
2

You could install the numpy module, which is often used for scientific computing.

read_data = numpy.fromfile(file=id, dtype=numpy.uint32)

Then access the data at the desired location and make your changes.

Improve this answer

1 Comment

see comment on hochl's answer
1

The following is just a demonstration for you to understand what really happens when the four bytes are converted into an integer. Suppose you have a number: 15213

Decimal: 15213
Binary: 0011 1011 0110 1101
Hex: 3 B 6 D

On little-endian systems (i.e x86 machines), this number can be represented using a length-4 bytearray as: b"\x6d\x3b\x00\x00" or b"m;\x00\x00" when you print it on the screen, to convert the four bytes into an integer, we simply do a bit of base conversion, which in this case, is:

sum(n*(256**i) for i,n in enumerate(b"\x6d\x3b\x00\x00"))

This gives you the result: 15213

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.