Compression-Resilient Deepfake Detection for AI-Assisted Social Media Content Monitoring: Feature-Level Fusion of ResNet50 and EfficientNet-B0 with a Learnable Visibility Matrix
Keywords:
deepfake detection; social media content monitoring; content moderation; misinformation; digital trust; compression robustness; ResNet50; EfficientNet-B0; human-in-the-loop AIAbstract
Deepfakes circulating through social media create a combined technical, behavioral, and governance challenge because manipulated videos may be redistributed after resizing, re-encoding, and compression. This study evaluated a frame-level detector combining ResNet50 and EfficientNet-B0 through feature-level fusion and an iterative Visibility Matrix procedure. Frames were obtained from FaceForensics++ c23 and Celeb-DF v2 and resized to 128 × 128 pixels. The fused model produced the highest reported point estimates, including 83% validation accuracy, an AUC-ROC of 0.89, balanced accuracy of 0.81, and approximately 18 ms inference time per image. Reported accuracy was approximately 79% on an additional unseen evaluation and approximately 78–80% under a severe-compression condition, compared with approximately 65–67% for ResNet50. These results are descriptive because the available experiment did not report repeated runs, confidence intervals, formal statistical comparisons, or a fully specified external-testing protocol. The detector should therefore be interpreted as a candidate frame-level triage component requiring video-disjoint validation, formal ablation, calibrated thresholds, and human review before operational use.
Downloads
References
1. Mirsky, Y., & Lee, W. (2021). The creation and detection of deepfakes: A survey. ACM Computing Surveys, 54(1), Article 7, 1–41. https://doi.org/10.1145/3425780
2. Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64, 131–148. https://doi.org/10.1016/j.inffus.2020.06.014
3. Verdoliva, L. (2020). Media forensics and deepfakes: An overview. IEEE Journal of Selected Topics in Signal Processing, 14(5), 910–932. https://doi.org/10.1109/JSTSP.2020.3002101
4. Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2019). FaceForensics++: Learning to detect manipulated facial images. Proceedings of the IEEE/CVF International Conference on Computer Vision, 1–11.
5. Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020). Celeb-DF: A large-scale challenging dataset for DeepFake forensics. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3207–3216.
6. Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Canton Ferrer, C. (2020). The Deepfake Detection Challenge dataset. arXiv:2006.07397.
7. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778. https://doi.org/10.1109/CVPR.2016.90
8. Tan, M., & Le, Q. V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. Proceedings of the 36th International Conference on Machine Learning, 97, 6105–6114.
9. Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7132–7141.
10. Li, Y., Bian, S., Wang, C., Polat, K., Alhudhaif, A., & Alenezi, F. (2023). Exposing low-quality deepfake videos of social network services using a spatial restored detection framework. Expert Systems with Applications, 231, 120646.
11. Woo, S. S. (2022). ADD: Frequency attention and multi-view based knowledge distillation to detect low-quality compressed deepfake images. Proceedings of the AAAI Conference on Artificial Intelligence, 36(1), 122–130.
12. Le, B. M., & Woo, S. S. (2023). Quality-agnostic deepfake detection with intra-model collaborative learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, 22378–22389.
13. Lee, S., An, J., & Woo, S. S. (2022). BZNet: Unsupervised multi-scale branch zooming network for detecting low-quality deepfake videos. Proceedings of the ACM Web Conference, 3500–3510.
14. Gao, J., Xia, Z., Marcialis, G. L., Dang, C., Dai, J., & Feng, X. (2024). DeepFake detection based on a high-frequency enhancement network for highly compressed content. Expert Systems with Applications, 249, 123732.
15. Han, B., Han, X., Zhang, H., Li, J., & Cao, X. (2021). Fighting fake news: Two-stream network for deepfake detection via learnable SRM. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(3), 320–331.
16. Kim, M., Tariq, S., & Woo, S. S. (2021). FReTAL: Generalizing deepfake detection using knowledge distillation and representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1001–1012.
17. Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., & Yu, N. (2021). Multi-attentional deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2185–2194.
18. Cozzolino, D., Rössler, A., Thies, J., Nießner, M., & Verdoliva, L. (2021). ID-Reveal: Identity-aware DeepFake video detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, 15108–15117.
19. Zhao, T., Xu, X., Xu, M., Ding, H., Xiong, Y., & Xia, W. (2021). Learning self-consistency for deepfake detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, 15023–15033.
20. Sun, Z., Han, Y., Hua, Z., Ruan, N., & Jia, W. (2021). Improving the efficiency and robustness of deepfakes detection through precise geometric features. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3609–3618.
21. Ke, J., & Wang, L. (2023). DF-UDetector: An effective method towards robust deepfake detection via feature restoration. Neural Networks, 160, 216–226.
22. Lin, H., Huang, W., Luo, W., & Lu, W. (2023). DeepFake detection with multi-scale convolution and vision transformer. Digital Signal Processing, 134, 103895.
23. Cozzolino, D., Pianese, A., Nießner, M., & Verdoliva, L. (2023). Audio-visual person-of-interest deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 943–952.
24. Wang, B., Wu, X., Tang, Y., Ma, Y., Shan, Z., & Wei, F. (2023). Frequency domain filtered residual network for deepfake detection. Mathematics, 11(4), 816. https://doi.org/10.3390/math11040816
25. Yu, P., Xia, Z., Fei, J., & Lu, Y. (2021). A survey on deepfake video detection. IET Biometrics, 10(6), 607–624.
26. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, 618–626. https://doi.org/10.1109/ICCV.2017.74
27. Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., & McAuley, J. (2021). Adversarial Deepfakes: Evaluating vulnerability of deepfake detectors to adversarial examples. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 3348–3357.
28. Neekhara, P., Dolhansky, B., Bitton, J., & Canton Ferrer, C. (2021). Adversarial threats to deepfake detection: A practical perspective. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 923–932.
29. Perera, A. S., Atukorale, A. S., & Kumarasinghe, P. (2022). Employing super resolution to improve low-quality deepfake detection. Proceedings of the 22nd International Conference on Advances in ICT for Emerging Regions, 13–18.
30. Mi, Z., Jiang, X., Sun, T., Xu, K., Xu, Q., & Meng, L. (2024). Low-quality deepfake video detection targeting compression-degraded spatiotemporal inconsistencies. In International Conference on Intelligent Computing (pp. 267–280). Springer.
31. Vaccari, C., & Chadwick, A. (2020). Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media + Society, 6(1). https://doi.org/10.1177/2056305120903408
32. Chesney, R., & Citron, D. K. (2019). Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review, 107(6), 1753–1820. https://doi.org/10.15779/Z38RV0D15J
33. Groh, M., Epstein, Z., Firestone, C., & Picard, R. (2022). Deepfake detection by human crowds, machines, and machine-informed crowds. Proceedings of the National Academy of Sciences, 119(1), e2110013119. https://doi.org/10.1073/pnas.2110013119
34. Nightingale, S. J., & Farid, H. (2022). AI-synthesized faces are indistinguishable from real faces and more trustworthy. Proceedings of the National Academy of Sciences, 119(8), e2120481119. https://doi.org/10.1073/pnas.2120481119
35. Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., Schudson, M., Sloman, S. A., Sunstein, C. R., Thorson, E. A., Watts, D. J., & Zittrain, J. L. (2018). The science of fake news. Science, 359(6380), 1094–1096. https://doi.org/10.1126/science.aao2998
36. UNESCO. (2021). Recommendation on the Ethics of Artificial Intelligence. United Nations Educational, Scientific and Cultural Organization.
37. Gillespie, T. (2018). Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press.
38. Aliabadian, A. (2025). Melanoma skin cancer detection using deep learning and the ant colony optimization algorithm. Transactions on Machine Intelligence, 8(4), 181–190. https://doi.org/10.22034/tmi.2025.244866
39. Torabi, T., Esmaeili, K., Omran, M., & Anisheh, S. M. (2024). Skin melanoma cancer detection using particle swarm optimization algorithm and deep learning. Transactions on Machine Intelligence, 7(3), 170–178. https://doi.org/10.47176/TMI.2024.170
40. Mohammadi, S., & Anisheh, S. M. (2024). Human activity recognition based on deep learning using sensor data. Transactions on Data Analysis in Social Science, 6(4), 222–230. https://doi.org/10.47176/TDASS.2024.222
41. Karimi Dastgerdi, A., & Zamani Boroujeni, F. (2020). A review of deep learning methods for financial market prediction. Transactions on Data Analysis in Social Science, 2(3), 164–172. https://doi.org/10.47176/TDASS.2020.164
