Abstract
A conventional codec stores a video as compressed pixel data. Instead, we store the video along with its audio track as the weights of a single Sinusoidal Representation Network (SIREN) that maps space-time coordinates to RGB values and audio amplitudes.
The network utilizes separate audio and video initialization layers, a stack of shared fully connected hidden layers, and three output branches: one for video and two Siamese audio branches whose disagreement is used to estimate and subtract residual noise. The overfitted teacher network is then compressed by response-based knowledge distillation into a smaller student network, followed by 16-bit symmetric weight quantization and lossless LZMA2 (xz) encoding.
On a 6.08 MiB test video, the quantized student achieves a video PSNR of 28.72 dB with an SSIM of 0.75, and an audio PSNR of 24.18 dB with a log spectral distance of 10.69 dB, while the pipeline reduces the representation from 9.05 MiB to 2.33 MiB, yielding an overall compression ratio of 2.61.
A bit-width sweep from 1-bit to 32-bit quantization shows that reconstruction quality saturates at 16 bits. We compare against H.264, HEVC, and MP3, report where the approach falls short, and describe a browser-based prototype that trains, transfers, and decodes these models over WebRTC.
Blogger's Review: This paper presents an innovative video coding scheme that leverages implicit neural representations to significantly enhance compression efficiency. Through knowledge distillation and quantization techniques, it effectively reduces storage requirements, making it suitable for modern network transmission needs. While the method's video and audio quality still need improvement, it lays the groundwork for future multimedia encoding advancements.