EncodingError: Decoder error. (Unable to determine size of bitstream buffer.) when using Webcodec API

I'm using Webcodecs since it now globally supported (except for mozilla mobile). My goal is to create a streaming plateform for cameras, I have a camera that sends it's datas to a server. This server has a websocket where i can connect my client. This websocket sends x264 NALUs and I want the client to decode this video and to display it on a canvas. To create the best UX, I choose to use it inside a worker.

This is the worker i made:

class Decoder {
    private decoder!: VideoDecoder;
    private sps!: Uint8Array;
    private pps!: Uint8Array;
    private isConfigured = false;
    private nalBuffer: Uint8Array[] = [];
    private canvas: OffscreenCanvas | null = null;

    set Canvas(canvas: OffscreenCanvas) {
        this.canvas = canvas;
    }

    constructor() {
        this.decoder = new VideoDecoder({
            output: (frame) => this.displayFrame(frame),
            error: (err) => {
                console.error('Decoder error:', err);
                // Attempt to reset decoder
                this.decoder.close();
                this.isConfigured = false;
                this.nalBuffer = [];
            }
        });
    }

    async decodeNAL(nal: Uint8Array) {
        if (this.decoder.state === "closed") return;

        // Suppression start code
        while (nal.length > 4 && nal[0] === 0x00) {
            nal = nal.slice(1);
        }
        nal = nal.slice(1);

        const nalType = nal[0] & 0x1F;

        switch (nalType) {
            case 7: // SPS
                if (!this.isConfigured) {
                    this.sps = nal;
                    this.nalBuffer.push(nal);
                }
                break;
            case 8: // PPS
                if (!this.isConfigured) {
                    this.pps = nal;
                    this.nalBuffer.push(nal);
                }
                break;
            case 5: // I-Frame
            case 1: // P-Frame
                if (!this.sps && !this.pps) {
                    console.warn("Skipping frame, waiting for configuration frames")

                    return;
                }

                if (!this.isConfigured && this.sps && this.pps) {
                    await this.configureDecoder();
                }
                this.processNal(nal);
                break;
        }
    }

    private processNal(nal: Uint8Array) {
        // Filter out unwanted NAL types if needed
        const acceptedNalTypes = [1, 5]; // Include more types if necessary
        if (!acceptedNalTypes.includes(nal[0] & 0x1F)) return;

        if (nal[0] & 0x1F) {
            this.nalBuffer = this.nalBuffer.slice(0, 3);
        } else {
            this.nalBuffer = this.nalBuffer.slice(0, 4);
        }

        this.nalBuffer.push(nal);

        this.decodeFrame();

    }

    private async configureDecoder() {
        try {
            const codecString = this.getAvcCodecString(this.sps);
            const description = this.convertToAvcC(this.sps, this.pps);

            const config: VideoDecoderConfig = {
                codec: codecString,
                description: description,
                hardwareAcceleration: "no-preference",
            };

            const isSupported = await VideoDecoder.isConfigSupported(config);

            if (!isSupported.supported) {
                console.error('Unsupported decoder config:', isSupported.config);
                throw new Error('No supported codec configuration found');
            }

            this.decoder.configure(config);
            this.isConfigured = true;
        } catch (error) {
            console.error('Decoder configuration failed:', error);
        }
    }

    private decodeFrame() {
        if (!this.isConfigured) {
            console.warn('Decoder not configured');
            return;
        }

        for (const nal of this.nalBuffer) {
            try {
                const chunkType = (nal[0] & 0x1F) === 5 ? 'key' : 'delta';

                const chunk = new EncodedVideoChunk({
                    type: chunkType,
                    data: nal,
                    timestamp: Date.now() * 1000
                });

                console.log(chunk.byteLength);

                this.decoder.decode(chunk);
            } catch (error) {
                console.error('Decoding error:', error);
            }
        }

        // Clear the buffer after processing
        this.nalBuffer = [];
    }

    private getAvcCodecString(sps: Uint8Array): string {
        if (sps.length < 4) return 'avc1.42E01E';

        const profile_idc = sps[1].toString(16).padStart(2, '0');
        const constraint_set = sps[2].toString(16).padStart(2, '0');
        const level_idc = sps[3].toString(16).padStart(2, '0');

        return `avc1.${profile_idc}${constraint_set}${level_idc}`;
    }

    private convertToAvcC(sps: Uint8Array, pps: Uint8Array): Uint8Array {
        const avcCLength = 12 + sps.length + pps.length;
        const avcC = new Uint8Array(avcCLength);

        avcC.set([1, sps[1], sps[2], sps[3], 0xff], 0);

        avcC.set([0xe1, (sps.length >> 8) & 0xff, sps.length & 0xff], 5);

        avcC.set(sps, 8);

        avcC.set([1, (pps.length >> 8) & 0xff, pps.length & 0xff], 9 + sps.length);

        avcC.set(pps, 12 + sps.length);

        return avcC;
    }

    private displayFrame(frame: VideoFrame) {
        if (!this.canvas) throw new Error('Canvas not configured');
        if ("getContext" in this.canvas) {
            // @ts-ignore
            this.canvas.getContext("2d").drawImage(frame, 0, 0, frame.displayWidth, frame.displayHeight);

        }

        frame.close();
    }
}

let decoder: Decoder = new Decoder();

self.addEventListener("message", async (e) => {
    switch (e.data.type) {
        case "buffer":
            await decoder.decodeNAL(new Uint8Array(e.data.data));
            break;
        case "canvas":
            decoder.Canvas = e.data.data as OffscreenCanvas;
    }
})

With this, I receive some I/P NALs that I ignore and then SPS and PPS NALs, with this NALs I configure the decoder using configureDecoder(). When this function ends, the decoder should be usable and create VideoFrame when running decode. But when I receive my first I frame after the decoder was configured I get this error: EncodingError: Decoder error. (Unable to determine size of bitstream buffer.). I don't really understand why this exception is thrown. I did not found any questions about this on the site, and no AI can help since it's niche and a little young