表 9 在 HTM-Align 数据集上针对噪声关联的有效性分析总结与展望本文是噪声关联学习 [3][4]—— 数据错配 / 错误关联的深入延续，研究多模态视频 – 文本预训练面临的多粒度噪声关联问题，所提出的长视频学习方法能够以较低资源开销扩展到更广泛的视频数据中。展望未来，研究者可进一步探讨多种模态间的关联问题，例如视频往往包含视觉、文本及音频信号；可尝试结合外部大语言模型（LLM）或多模态模型（BLIP-2）来清洗和重组织文本语料；以及探索将噪声作为模型训练正激励的可能性，而非仅仅抑制噪声的负面影响。参考文献：1. 本站，“Yann LeCun：生成模型不适合处理视频，AI 得在抽象空间中进行预测”，2024-01-23.2.Sun, Y., Xue, H., Song, R., Liu, B., Yang, H., & Fu, J. (2022). Long-form video-language pre-training with multimodal temporal contrastive learning. Advances in neural information processing systems, 35, 38032-38045.3.Huang, Z., Niu, G., Liu, X., Ding, W., Xiao, X., Wu, H., & Peng, X. (2021). Learning with noisy correspondence for cross-modal matching. Advances in Neural Information Processing Systems, 34, 29406-29419.4.Lin, Y., Yang, M., Yu, J., Hu, P., Zhang, C., & Peng, X. (2023). Graph matching with bi-level noisy correspondence. In Proceedings of the IEEE/CVF international conference on computer vision.5.Han, T., Xie, W., & Zisserman, A. (2022). Temporal alignment networks for long-term video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2906-2916).6.Sarlin, P. E., DeTone, D., Malisiewicz, T., & Rabinovich, A. (2020). Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4938-4947).
微信扫一扫 
支付宝扫一扫 
