Python gdal open layer from bytes

Question

I need to perform a conversion from a FlatGeoBuf file that I read as bytes from an http request to a GeoJSON file.

I know that I could save the .fgb on disk, open it with ogr, and then I have a function to export an OGR layer to GeoJSON.

But I would like to avoid writing on disk. I know of the VSIMEM mechanism, but I'm struggling to make it work... What I came up with looks like this:

def write_vsimem(mem_path: str, data: bytes) -> None:
    vsi_file = gdal.VSIFOpenL(mem_path, "w")
    size = len(data)
    try:
        gdal.VSIFWriteL(data, 1, size, vsi_file)
        if gdal.VSIFCloseL(vsi_file) != 0:
            raise RuntimeError(
                f"Failed to close VSIMEM file '{mem_path}': {gdal.GetLastErrorMsg()}"
            )
    except Exception as e:
        raise RuntimeError(f"Error writing to VSIMEM file '{mem_path}': {e}") from e

def convert_flatgeobuf_to_geojson(flatgeobuf_data: bytes, name: str) -> bytes:
    driver = ogr.GetDriverByName("FlatGeobuf")
    mem_path = f"/vsimem/{name}.fgb"
    fgb_data = driver.CreateDataSource(mem_path)
    if not fgb_data:
        raise RuntimeError(f"Failed to open FlatGeobuf file '{name}' with VSIMEM.")
    write_vsimem(mem_path, flatgeobuf_data)
    # Check if the layer exists
    if fgb_data.GetLayerCount() != 1:
        raise ValueError(
            f"Expected 1 layer in FlatGeobuf file '{name}', found {fgb_data.GetLayerCount()} layers."
        )
    # Get the layer and extract geojson
    layer = fgb_data.GetLayer(0)
    return layer_to_geojson(layer=layer)

When I run this, I get the following logs that seem to say the write_vsimem part works:

Converting FlatGeoBuf of size: 5570849
Write VSIMEM: 5570816

But then I get the following error, which mean the ogr datasource I created is still empty:

Failed to hydrate data: Expected 1 layer in FlatGeobuf file 'my_layer', found 0 layers.

Is my goal even possible to achieve ? Then what am I doing wrong ? Or should I give up and write on disk ?

Any answer to those question is welcome :)

EDIT:

As indicated by Priyanka Konduru, I was misunderstanding how the VSIMEM works: by writing bytes in memory you create the in-memory space (so, no CreateDataSource) and then you can read from memory:

def convert_flatgeobuf_to_geojson(flatgeobuf_data: bytes, name: str) -> bytes:
    mem_path = f"/vsimem/{name}.fgb"
    write_vsimem(mem_path, flatgeobuf_data)
    fgb_data = ogr.Open(mem_path)
    if not fgb_data:
        raise RuntimeError(f"Failed to open FlatGeobuf file '{name}' with VSIMEM.")
    # Check if the layer exists
    if fgb_data.GetLayerCount() != 1:
        raise ValueError(
            f"Expected 1 layer in FlatGeobuf file '{name}', found {fgb_data.GetLayerCount()} layers."
        )
    # Get the layer and extract geojson
    layer = fgb_data.GetLayer(0)
    return layer_to_geojson(layer=layer, layer_name=name)

I don't know anything about these specific libraries, but if you have code that works for a file on-disk then maybe you can use BytesIO or StringIO from the io module documented here: docs.python.org/3/library/io.html - these are generic ways to create a file-like object similar to what's returned by open from in-memory data. — l4mpi
– l4mpi, Commented Oct 1 at 8:28

Priyanka Konduru · Accepted Answer · 2025-10-01 08:11:12Z

1

Problem in your code

The main issue is order of operations with VSIMEM and OGR:

You are calling driver.CreateDataSource(mem_path) before writing bytes into VSIMEM.
- At that point, the file doesn’t exist in memory yet, so OGR sees it as empty — hence GetLayerCount() == 0.
CreateDataSource is for creating new files, not opening existing ones. For reading an existing VSIMEM file, you need Open.

answered Oct 1 at 8:11

Priyanka Konduru

362 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

marting Oct 1 at 9:46

Thanks, I indeed misunderstood how those worked ! I thought I needed CreateDataSource to create the space in memory, and then I could write in it.

Collectives™ on Stack Overflow

Python gdal open layer from bytes

1 Answer 1

Problem in your code

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Problem in your code

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related