Researchers have introduced LAION-BVD, a massive-scale open-access video dataset designed to facilitate multimodal pre-training across video, audio, and image modalities. The dataset was constructed by harvesting 1.3 billion platform-specific video URLs from CommonCrawl, from which 80 million videos were successfully downloaded, totaling 10 million hours of content. To prepare the data for training, the authors employed content-aware scene detection to segment the footage into discrete clips, followed by the synthetic generation of both video and audio captions. Empirical evaluations demonstrate that models trained on LAION-BVD achieve competitive results on standard video-text and audio-text benchmarks, showing consistent performance gains as both the model and training scale increase. Furthermore, the researchers investigated the utility of extracting scene-changing frames to serve as an alternative source for image-text training. These extracted frames possess a visual distribution that differs significantly from conventional web-based image corpora. Models leveraging these frames demonstrated robust performance in image-text retrieval tasks. By releasing this dataset, the authors provide the research community with an unprecedented volume of multimodal video data to advance large-scale foundation model development.

Read original