I am getting :
(index):60 Error during fetch or decode process: DataError: Failed to execute 'decode' on 'VideoDecoder': A key frame is required after configure() or flush(). If you're using AVC formatted H.264 you must fill out the description field in the VideoDecoderConfig. at fetchAndDecodeJson ((index):55:24)"
when trying to decode the below h264 data in a json format via webcodec:
[
{
"type": "key",
"pts": 0,
"data": "Z0LgK28gAflBAQAAAwAAAPoAAIIAAO1BAQAAAwAAAPoAAOIwFA=="
},
{
"type": "delta",
"pts": 1,
"data": "aMuHsyAABfsAAABAqAAABfsBAwABxAAAAgAAABfsBAwABxAAAgA=="
},
{
"type": "delta",
"pts": 2,
"data": "aMuHsyAABfsAAABAqAAABfsBAwABxAAAAgAAABfsBAwABxAAAgA=="
},
{
"type": "delta",
"pts": 3,
"data": "aMuHsyAABfsAAABAqAAABfsBAwABxAAAAgAAABfsBAwABxAAAgA=="
}
]
My index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>decode H.264 to Canvas</title>
</head>
<body>
<canvas id="output" style="border: 1px solid black;"></canvas>
<script>
async function fetchAndDecodeJson() {
try {
const response = await fetch("http://127.0.0.1:8000/datatest.json");
const jsonData = await response.json();
console.log("Fetched JSON Data:", jsonData);
const canvas = document.getElementById("output");
const ctx = canvas.getContext("2d");
const videoDecoder = new VideoDecoder({
output: (frame) => {
canvas.width = frame.displayWidth;
canvas.height = frame.displayHeight;
ctx.drawImage(frame, 0, 0);
frame.close();
},
error: (e) => console.error("decode error:", e),
});
const codecSupport = await VideoDecoder.isConfigSupported({
codec: "avc1.42E01E",
});
if (!codecSupport.supported) {
throw new Error("codec not supported");
}
videoDecoder.configure({ codec: "avc1.42E01E" });
//decode each chunk from the JSON data
for (const item of jsonData) {
const { type, pts, data } = item;
//base64 video data
const rawData = Uint8Array.from(atob(data), (c) => c.charCodeAt(0));
//decode the video chunk
const chunk = new EncodedVideoChunk({
timestamp: pts * 1000,
type: type,
data: rawData,
});
videoDecoder.decode(chunk);
}
console.log("decode success");
} catch (error) {
console.error("decode or fetch error:", error);
}
}
fetchAndDecodeJson().catch(console.error);
</script>
</body>
</html>
the test frames are fetched but not decoded enter image description here
i tried fiddling with sps and pps in the decoder config without success
"data":parts are holding some H264 metadata called SPS and PPS (isZ0LgK...andaMuHs...respectively). They are useful for setting up the decoder, so now you need another next "data" but one that makes a keyframe... Attach all 3 things as long one array then send to decode with Webcodecs.videoDecoder.configureis using 42E01E (they are close, but might not decode)... Do you know which one is correct? Also tell the encoding side to first convert bytes to hex string, then do the base64 encode after (ie: decode gives hex string likeFF A8 45 D9) ... I'll try to write an Answer when time allows.