PERTANIKA JOURNAL OF SCIENCE AND TECHNOLOGY

Nowadays, person recognition has received significant attention due to broad applications in the security system. However, most person recognition systems are implemented based on unimodal biometrics such as face recognition or voice recognition. Biometric systems that adopted unimodal have limitations, mainly when the data contains outliers and corrupted datasets. Multimodal biometric systems grab researchers’ consideration due to their superiority, such as better security than the unimodal biometric system and outstanding recognition efficiency. Therefore, the multimodal biometric system based on face and fingerprint recognition is developed in this paper. First, the multimodal biometric person recognition system is developed based on Convolutional Neural Network (CNN) and ORB (Oriented FAST and Rotated BRIEF) algorithm. Next, two features are fused by using match score level fusion based on Weighted Sum-Rule. The verification process is matched if the fusion score is greater than the pre-set threshold. The algorithm is extensively evaluated on UCI Machine Learning Repository Database datasets, including one real dataset with state-of-the-art approaches. The proposed method achieves a promising result in the person recognition system.

• Alay, N., & Al-Baity, H. H. (2020). Deep learning approach for multimodal biometric recognition system based on fusion of iris, face, and finger vein traits. Sensors, 20(19), Article 5523. https://doi.org/10.3390/s20195523

• Alazawi, S. A., Abdulaaliabdulbaqi, H., & Mohialden, Y. M. (2019). New authentication model for multimodal biometrics based on shape features vectors. Journal of Southwest Jiaotong University, 54(6). https://doi.org/10.35741/issn.0258-2724.54.6.18

• Aleem, S., Yang, P., Masood, S., Li, P., & Sheng, B. (2020). An accurate multi-modal biometric identification system for person identification via fusion of face and finger print. Springer, 23(6), 1299-1317. https://doi.org/10.1007/s11280-019-00698-6

• Ammour, B., Boubchir, L., Bouden, T., & Ramdani, M. (2020). Face–iris multimodal biometric identification system. Electronics, 9(1), Article 85. https://doi.org/10.3390/electronics9010085

• Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010). BRIEF: Binary robust independent elementary features. In Lecture Notes in Artificial Intelligence Lecture Notes Bioinformatics (Vol. 6314, No. 4, pp.778-792). Springer. https://doi.org/10.1007/978-3-642-15561-1_56

• Chanklan, R., Chaiyakhan, K., Hirunyawanakul, A., Kerdprasop, K., & Kerdprasop, N. (2015). Fingerprint recognition with edge detection and dimensionality reduction techniques. In Proceedings of the 3rd International Conference on Industrial Application Engineering (pp. 569-574). The Institute of Industrial Applications Engineers. https://doi.org/10.12792/ICIAE2015.098

• Devi, R., & Sujatha, P. (2017). A study on biometric and multi-modal biometric system modules, applications, techniques and challenges. In Proceedings of Conference on Emerging Devices and Smart Systems (ICEDSS) (pp. 267-271). IEEE Publishing. https://doi.org/10.1109/ICEDSS.2017.8073691

• Fatt, W. K., Kushsairy, A. K., Nasir, H., Safie, S. I., & Noor, N. M. (2017). Fingerprint and face recognition: Application to multimodal biometrics system. Journal of Telecommunication, Electronic and Computer Engineering, 9(2-2), 81-85.

• Gavisiddappa, C. M. P., Shivakumar, M., & Pramod, K. S. (2019). Multi-modal biometric: Bi-directional empirical mode decomposition with hilbert-hung transformation. International Journal of Advanced Computer Science and Applications (IJACSA), 10(6), 528-537. http://dx.doi.org/10.14569/IJACSA.2019.0100669

• Guo, Z., Li, X., Huang, H., Guo, N., & Li, Q. (2019). Deep learning-based image segmentation on multimodal medical imaging. IEEE Transactions on Radiation and Plasma Medical Sciences, 3(2), 162-169. https://doi.org/10.1109/TRPMS.2018.2890359

• Huang, L., Yu, C., & Cao, X. (2018). Bimodal biometric person recognition by score fusion. In Proceedings of 5th International Conference on Information Science and Control Engineering (ICISCE) (pp. 1093-1097). IEEE Publishing. https://doi.org/10.1109/ICISCE.2018.00225

• Ismail, A., Ahmad, S. A., Soh, A. C., Hassan, M. K., & Harith, H. H. (2020). Deep learning object detector using a combination of convolutional neural network (CNN) architecture (minivggnet) and classic object detection algorithm. Pertanika Journal of Social Science and Humanities, 28(S2), 161-171. https://doi.org/10.47836/pjst.28.s2.13

• Kumar, T., Bhushan, S., & Jangra, S. (2021). An improved biometric fusion system of fingerprint and face using whale optimization. International Journal of Advanced Computer Science and Applications (IJACSA), 12(1), 664-671. http://dx.doi.org/10.14569/IJACSA.2021.0120176

• Kurban, O. C., Yildirim, T., & Bilgic, A. (2017). A multi-biometric recognition system based on deep features of face and gesture energy image. In Proceedings of IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA) (pp. 361-364). IEEE Publishing. https://doi.org/10.1109/INISTA.2017.8001186

• Liu, G. F., Bao, H. Q., & Han, B. K. (2018). A stacked autoencoder-based deep neural network for achieving gearbox fault diagnosis. Mathematical Problems in Engineering, 5, 1-10. https://doi.org/10.1155/2018/5105709

• Luo, C., Yang, W., Huang, P., & Zhou, J. (2019). Overview of image matching based on ORB algorithm. Journal of Physics Conference Series, 1237(3), Article 032020. https://doi.org/10.1088/1742-6596/1237/3/032020

• Lv, G. L., Shen, L., Yao, Y. D., Wang, H. X., & Zhao, G. D. (2020). Feature-level fusion of finger vein and fingerprint based on a single finger image: The use of incompletely closed near-infrared equipment. Symmetry, 12(5), Article 709. https://doi.org/10.3390/sym12050709

• Mwaura, G., Mwangi, W., & Otieno, C. (2017). Multimodal biometric system: Fusion of face and fingerprint biometrics at match score fusion level. International Journal of Scientific & Technology Research, 6(4), 41-49.

• Parkavi, R., Babu, K. R. C., & Kumar, J. A. (2017). Multimodal biometrics for user authentication. In Proceedings of 11th International Conference on Intelligent Systems and Control (ISCO) (pp. 501-505). IEEE Publishing. https://doi.org/10.1109/ISCO.2017.7856044

• Pawar, M. D., Kokate, R. D., & Gosavi, V. R. (2021, June 28-29) An optimize multimodal biometric authentication system for low classification error rates using face and fingerprint. In Proceedings of the International Conference on IoT Based Control Networks & Intelligent Systems. Kottayam, India. http://dx.doi.org/10.2139/ssrn.3883852

• Peng, J., Wu, J. Y., & Chen, Y. (2020). A score level fusion method on fingerprint and finger vein. E3S Web of Conferences, 185(14), Article 03035. https://doi.org/10.1051/e3sconf/202018503035

• Rosten, E., Porter, R., & Drummond, T. (2010). Faster and better: A machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 105-119. https://doi.org/10.1109/TPAMI.2008.275

• Somashekhar, B. M., & Nijagunarya, Y. S. (2018). Face and fingerprint fusion system for identity authentication using fusion classifiers. International Journal of Computer Science & Engineering Survey, 9(1/2/3), 1-12. https://doi.org/10.5121/IJCSES.2018.9301

• Song, S. H., & Kim, D. K. (2017). Development of a stress classification model using deep belief networks for stress monitoring. Healthcare Informatics Research, 23(4), 285-292. https://doi.org/10.4258/hir.2017.23.4.285

• Xin, Y., Kong, L. S., Liu, Z., Wang, C. H., Zhu, H. L., Gao, M. C., Zhao, C. S., & Xu, X. K. (2018). Multimodal feature-level fusion for biometrics identification system on IOMT platform. IEEE Access, 6, 21418-21426. https://doi.org/10.1109/ACCESS.2018.2815540

• Yang, W., Wang, S., Hu, J., Zheng, G., Chaudhry, J., Erwin, A., & Craig, V. (2018). Securing mobile healthcare data: A smart card based cancelable Finger-Vein Bio-Cryptosystem. IEEE Access, 6, 36939-36947. https://doi.org/10.1109/ACCESS.2018.2844182

• Zhao, J., Yan, S., & Feng, J. (2020). Towards age-invariant face recognition. In IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE Publishing. https://doi.org/10.1109/TPAMI.2020.3011426

• Zhu, N., Yu, Z., & Kou, C. (2020). A new deep neural architecture search pipeline for face recognition. IEEE Access, 8, 91303-91310. https://doi.org/10.1109/ACCESS.2020.2994207