My research interests mainly lie in combating misinformation, multimedia content analysis, and social media mining. Recently, I focus more on misinformation detection and intervene in the era of LLM.
Focusing on the innovative research perspective of explainable out-of-context misinformation detection,
this paper proposes a new multimodal large language model, SNIFFER, designed to offer both accurate detection and persuasive explanations simultaneously. Enhanced by two-stage instruction tuning and retrieval-enhancement techniques, SNIFFER effectively models both internal image-text inconsistency and external claim-evidence relationships.
This paper proposes that current LLMs may not substitute fine-tuned SLMs in fake news detection but can be a good advisor for SLMs by providing instructive multi-perspective rationales. Based on this assumption, we design an adaptive rationale guidance network ARG in which SLMs selectively acquire insights from LLM's rationales and further derive a rationale-free version by distillation.
This paper is the first comprehensive survey targeted at combating online misinformation videos.
Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors
Peng Qi,
Yuyang Zhao,
Yufeng Shen,
Wei Ji,
Juan Cao,
Tat-Seng Chua ACL Findings, 2023    
PDF
This paper proposes a model-agnosticcross-sample framework NEED for detecting fake news videos,
which utilizes the neighbor relationship
between the related news videos in the same event and thus can largely improve the performance
of existing single-sample detectors. This paper is a successful attempt to combine automatic
fake news detection and fact-checking, and firstly proves the effectiveness of debunking videos in detection.
We construct the largest Chinese fake news short video dataset, namely FakeSV.
This dataset contains complete news contents and rich social context under different events, and thus can support a wide range of research tasks related to fake news.
It firstly provides abundant debunking videos accompanying the fake and real news videos.
We provide exploratory multi-perspective data analysis,
which reveals some interesting and insightful phenomenons.
Moreover, we design a new multimodal detection model as the SOTA method in this benchmark.
We also re-implement 11 single-modal and 4 multimodal methods as baseline methods.
Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues
Peng Qi,
Juan Cao,
Xirong Li,
Huan Liu,
Qiang Sheng,
Xiaoyue Mi,
Qin He,
Yongbiao Lv,
Chenyang Guo,
Yingchao Yu
MM, 2021     PDF     Citation: 50+
This paper explores three valuable text-image correlations in multimodal fake news: entity inconsistency, mutual enhancement and text complementation.
For better multimodal reasoning, this paper firstly import the visual news entities into multimodal fake news detection, which helps to understand the news-related high-level semantics of images and bridge the high-level semantic correlations of news text and images.
Exploring the Role of Visual Content in Fake News Detection
Juan Cao,
Peng Qi,
Qiang Sheng,
Tianyun Yang,
Junbo Guo,
Jintao Li Disinformation, Misinformation, and Fake News in Social Media: Emerging Research Challenges and Opportunities (Book), 2020     PDF     Citation: 100+
This chapter presents a comprehensive review of the visual content in fake news, including the basic concepts, effective visual features, representative detection methods and challenging issues of multimedia fake news detection.
Exploiting Multi-domain Visual Information for Fake News Detection
Peng Qi,
Juan Cao,
Tianyun Yang,
Junbo Guo,
Jintao Li ICDM, 2019     PDF     Citation: 200+
This paper firstly analyze the images in fake news from both the physical and semantic levels, and thus propose a two-stream neural network which fuses the frequency and pixel domains information.
Professional Service
Program Comittee / Conference Reviewer: AAAI 2023, AAAI 2024, CVPR 2024, MM 2024, ECCV 2024
Journal Reviewer: ACM Transactions on Information Systems, Multimedia System
Awards
President Award, Institute of Computing Technology, 2020,2022
Merit Student, University of Chinese Academy of Sciences, 2018,2020
First-level Academic Scholarship, University of Chinese Academy of Sciences, 2019
National Scholarship, Ministry of Education of China, 2014