Performance evaluation of deep learning models for detecting deep fakes

IJOSI

International Journal of Systematic Innovation

2077-79732077-8767

AccScience Publishing

10.6977//IJoSI.202403_8(1).0005

Article

Performance evaluation of deep learning models for detecting deep fakeshttps://artdesignp.com/journal/IJOSI/8/1/10.6977//IJoSI.202403_8(1).0005RajeevAishwarya,PRaviraj

2024

2024-02-22

The proliferation of deep fake content in multimedia has necessitated the development of robust detection mech-anisms. In this study, a comparative analysis of four state-of-the-art deep learning models for detecting deep fakes is conducted: CNN+RNN, DAFDN,Hybrid Inception ResNet v2, and Xception. The evaluation focuses on their perfor-mance metrics, emphasizing accuracy as a primary measure. Through extensive experimentation and evaluation on a comprehensive dataset, the findings reveal notable distinctionsamong these models. The CNN+RNN architecture demonstrates a commendable accuracy of 94.8%, providing a solid baseline for comparison. Surpassing this, the DAFDN model achieves an accuracy of 97.8%, showcasing superior discriminatory capabilities in identifying manip-ulated content. Furthermore, the CNN model stands out with an accuracy of 98%, exhibiting remarkable effectiveness in distinguishing between genuine and deep fake media. The comparative analysis delves into the strengths and weak-nesses of each model, shedding light on their respective performance levels in detecting sophisticated deep fake con-tent. The observed accuracies underscore the nuanced differences in their architectures and training methodologies, offering insights crucial for selecting appropriate models based on specific detection requirements.Face Forensics, Convolutional neural network, recurrent neural network, DAFDN, Resnet v2, Xceptio

Shad, H. S., Rizvee, M. M., Roza, N. T., Hoq, S. M., Monirujjaman Khan, M., Singh, A. &amp; Bourouis, S. (2021). Comparative analysis of deepfake image detection method using convolutional neural network.Computational Intelligence and Neuroscience,2021. Rahman, A., Islam, M. M., Moon, M. J., Tasnim, T., Siddique, N., Shahiduzzaman, M. & Ahmed, S. (2022). A qualitative survey on deep learning based deep fake video creation and detection method, Aust. J. Eng. Innov. Technol,4(1), 13-26. Nguyen, T. T., Nguyen, C. M., Nguyen, D. T., Nguyen, D. T. & Nahavandi, S. (2019). Deep learning for deepfakes creation and detection, arXiv preprint arXiv:1909.11573,1(2), 2. Bode, L. (2021). Deepfaking Keanu: YouTube deepfakes, platform visual effects, and the complexity of reception, Convergence,27(4), 919-934. Verdoliva, L. (2020). Media forensics and deepfakes: an overview, IEEE Journal of Selected Topics in Signal Processing,14(5), 910-932. Kousik, N., Natarajan, Y., Raja, R. A., Kallam, S., Patan, R., & Gandomi, A. H. (2021). Improved salient object detection using hybrid Convolution Recurrent Neural Network.Expert Systems with Applications,166, 114064. Kamaleldin, M. G. M., Abu-Bakar, S. A. & Sheikh, U. U. (2023). Transfer Learning Models for CNN Fusion with Fisher Vector for Codebook Optimization of Foreground Features, IEEE Access. Guo, Z., Yang, G., Chen, J. & Sun, X. (2021). Fake face detection via adaptive manipulation traces extraction network, Computer Vision and Image Understanding,204, 103170. George, A. S. & George, A. H. (2023). Deepfakes: The Evolution of Hyper realistic Media Manipulation, Partners Universal Innovative Research Publication,1(2), 58-74. Wang, T., Zhang, Y., Qi, S., Zhao, R., Xia, Z. & Weng, J. (2023). Security and privacy on generative data in aigc: A survey, arXiv preprint arXiv:2309.09435. Ding, F., Zhu, G., Alazab, M., Li, X. & Yu, K. (2020). Deep-learning-empowered digital forensics for edge consumer electronics in 5G HetNets, IEEE consumer electronics magazine,11(2), 42-50. Karie, N. M., Kebande, V. R., & Venter, H. S. (2019). Diverging deep learning cognitive computing techniques into cyber forensics, Forensic Science International: Synergy,1, 61-67. Chintha, A., Thai, B., Sohrawardi, S. J., Bhatt, K., Hickerson, A., Wright, M., & Ptucha, R. (2020). Recurrent convolutional structures for audio spoof and video deepfake detection, IEEE Journal of Selected Topics in Signal Processing,14(5), 1024-1037. Wu, B., Su, L., Chen, D., & Cheng, Y. (2023). FPC‐Net: Learning to detect face forgery by adaptive feature fusion of patch correlation with CG‐Loss, IET Computer Vision,17(3), 330-340. Sedik, A., Faragallah, O. S., El-sayed, H. S., El-Banby, G. M., El-Samie, F. E. A., Khalaf, A. A. & El-Shafai, W. (2022). An efficient cybersecurity framework for facial video forensics detection based on multimodal deep learning, Neural Computing and Applications, 1-18. Wu, J., Zhu, Y., Jiang, X., Liu, Y., & Lin, J. (2023). Local attention and long-distance interaction of rPPG for deepfake detection, The Visual Computer, 1-12. Coccomini, D. A., Messina, N., Gennaro, C. & Falchi, F. (2022, May). Combining efficientnet and vision transformers for video deepfake detection, InImage Analysis and Processing–ICIAP 2022: 21st International Conference, Lecce, Italy, May 23–27, 2022, Proceedings, Part III, Cham: Springer International Publishing, 219-229. Aishwarya Rajeev, A., & Raviraj, P. (2023). An Insightful Analysis of Digital Forensics Effects on Networks and Multimedia Applications, SN Computer Science,4(2), 186. Xiao, J., Li, S., & Xu, Q. (2019). Video-based evidence analysis and extraction in digital forensic investigation, IEEE Access,7, 55432-55442. Ahmadi, F., Gupta, G., Zahra, S. R., Baglat, P. & Thakur, P. (2021, March). Multi-factor biometric authentication approach for fog computing to ensure security perspective, In2021 8th international conference on computing for sustainable global development (INDIACom), IEEE, 172-176. Ross, A., Banerjee, S., & Chowdhury, A. (2020). Security in smart cities: A brief review of digital forensic schemes for biometric data, Pattern Recognition Letters,138, 346-354. Chandaliya, P. K., & Nain, N. (2022). ChildGAN: Face aging and rejuvenation to find missing children, Pattern Recognition,129, 108761. Ivanova, E., & Borzunov, G. (2020). Optimization of machine learning algorithm of emotion recognition in terms of human facial expressions, Procedia Computer Science,169, 244-248. Sikkandar, H., & Thiyagarajan, R. (2020). Soft biometrics‐based face image retrieval using improved grey wolf optimisation, IET Image Processing,14(3), 451-461. Keshari, T., & Palaniswamy, S. (2019, July). Emotion recognition using feature-level fusion of facial expressions and body gestures, In2019 international conference on communication and electronics systems (ICCES), IEEE, 1184-1189. Hussain, S. A. & Al Balushi, A. S. A. (2020). A real time face emotion classification and recognition using deep learning model, InJournal of physics: Conference series, IOP Publishing, 1432(1), 012087. Ashwin, T. S. & Guddeti, R. M. R. (2019). Unobtrusive behavioral analysis of students in classroom environment using non-verbal cues, IEEE Access,7, 150693-150709. Agbolade, O., Nazri, A., Yaakob, R., Ghani, A. A. & Cheah, Y. K. (2019). 3-Dimensional facial expression recognition in human using multi-points warping, BMC bioinformatics,20(1), 1-15. Bozkir, E., Özdel, S., Wang, M., David-John, B., Gao, H., Butler, K. & Kasneci, E. (2023). Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges.arXiv preprint arXiv:2305.14080. Dargan, S. & Kumar, M. (2020). A comprehensive survey on the biometric recognition systems based on physiological and behavioral modalities, Expert Systems with Applications,143, 113114. Verdoliva, L. (2020). Media forensics and deepfakes: an overview, IEEE Journal of Selected Topics in Signal Processing,14(5), 910-932. Hashmi, M. F., Ashish, B. K. K., Keskar, A. G., Bokde, N. D., Yoon, J. H. & Geem, Z. W. (2020). An exploratory analysis on visual counterfeits using conv-lstm hybrid architecture, IEEE Access,8, 101293-101308. Deshmukh, A. & Wankhade, S. B. (2020). Deepfake Detection Approaches Using Deep Learning: A Systematic Review, Intelligent Computing and Networking: Proceedings of IC-ICN 2020, 293-302. Awotunde, J. B., Jimoh, R. G., Imoize, A. L., Abdulrazaq, A. T., Li, C. T. & Lee, C. C. (2022). An Enhanced Deep Learning-Based DeepFake Video Detection and Classification System, Electronics,12(1), 87. Byrnes, O., La, W., Wang, H., Ma, C., Xue, M., & Wu, Q. (2021). Data hiding with deep learning: A survey unifying digital watermarking and steganography, arXiv preprint arXiv:2107.09287. Chen, L., Zhang, Y., Song, Y., Liu, L. & Wang, J. (2022). Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection, InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 18710-18719. Xi, Z., Niu, Y., Chen, J., Kan, X. & Liu, H. (2020). Facial expression recognition of industrial internet of things by parallel neural networks ecombining texture features, IEEE Transactions on Industrial Informatics,17(4), 2784-2793. Jeong, D., Kim, B. G., & Dong, S. Y. (2020). Deep joint spatiotemporal network (DJSTN) for efficient facial expression recognition, Sensors,20(7), 1936. Zhu, M., Shi, D., Zheng, M., & Sadiq, M. (2019). Robust facial landmark detection via occlusion-adaptive deep networks, InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3486-3496. Zhang, W., & Zhao, C. (2019, November). Exposing face-swap images based on deep learning and ELA detection, InProceedings, MDPI, 46(1), 29. Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., & Yu, N. (2021). Multi-attentional deepfake detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2185-2194. Bonomi, M., Pasquini, C., & Boato, G. (2021). Dynamic texture analysis for detecting fake faces in video sequences, Journal of Visual Communication and Image Representation,79, 103239. Rahman, A. (2022).Deepfake Video Detection Using CNN and RCNN(Doctoral dissertation, Bangladesh University of Business and Technology). Luo, Y. X. & Chen, J. L. (2022). Dual Attention Network Approaches to Face Forgery Video Detection. IEEE Access,10, 110754-110760. Ikram, S. T., Chambial, S., & Sood, D. (2023). A performance enhancement of deep fake video detection through the use of a hybrid CNN Deep learning model. International journal of electrical and computer engineering systems,14(2), 169-178.