Human Activity Recognition Based on Webcam Footage Using a Convolutional Neural Network (CNN) Method with a MobileNet Architecture
Article Sidebar
Main Article Content
Abstract
Human beings are inseparable from their daily activities, which form an integral part of human life. "Human activity recognition" is currently a highly researched topic, driven by rapid advancements in technology. The COVID-19 pandemic has impacted nearly every sector, restricting human activities. Education, in particular, has been significantly affected; as universities shifted to online learning, it became increasingly difficult for lecturers to monitor students during lessons and examinations, as they could no longer observe student activities firsthand. This research aims to develop a model capable of recognizing an individual's activities during online examinations based on webcam footage, utilizing a deep learning approach—specifically a Convolutional Neural Network (CNN) with the MobileNetV2 architecture. Hyperparameter tuning was conducted to identify an optimal model, testing batch sizes of 16, 32, and 64, alongside 1, 3, 5, and 7 dense layers. The process yielded an optimal model with the following hyperparameters: a maximum of 20 epochs, early stopping with a patience of 10, a learning rate of 0.0001, a batch size of 16, and 5 dense layers. The model was evaluated using cross-validation and a confusion matrix, achieving a final F1-score of 84.52%.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
Sharma, N. K. et al. (2021) ‘CNN Implementation for Detect Cheating in Online Exams During COVID-19 Pandemic: A CVRU Perspective’, Materials Today: Proceedings. Elsevier Ltd, (xxxx). doi:
1016/j.matpr.2021.05.490.
Rukajat, A. (2018) Teknik Evaluasi Pembelajaran. Deepublish.
Ehatisham, M. et al. (2020) ‘C2FHAR: Coarse-to-Fine Human Activity Recognition with Behavioral Context Modeling Using Smart Inertial Sensors’, IEEE Access, 8, pp. 7731–7747. doi:
1109/ACCESS.2020.2964237.
Nurhikmat, T. (2018) Implementasi Deep Learning Untuk Image Classification Menggunakan Algoritma Convolutional Neural Network (CNN) Pada Citra Wayang Golek. doi: 10.13140/RG.2.2.10880.53768.
Budiman, B. (2021) ‘Pendeteksian Penggunaan Masker Wajah Dengan Metode Convolutional Neural Network’, Jurnal Ilmu Komputer dan Sistem Informasi, Vol.9 No.1.
Ferrari, A. et al. (2021) ‘Trends in Human Activity Recognition using Smartphones’, Journal of Reliable Intelligent Environments. Springer International Publishing, 7(3), pp. 189–213. doi: 10.1007/s40860-021-
-0.
Sanhudo, L. et al. (2021) ‘Activity Classification using Accelerometers and Machine Learning for Complex Construction Worker Activities’, Journal of Building Engineering, 35(December). doi:
1016/j.jobe.2020.102001.
San-Segundo, R. et al. (2018) ‘Robust Human Activity Recognition Using Smartwatches and Smartphones’, Engineering Applications of Artificial Intelligence, 72(October 2017), pp. 190–202. doi:
1016/j.engappai.2018.04.002.
Zhang, S. et al. (2017) ‘A Review on Human Activity Recognition
Using Vision-Based Method’, Journal of Healthcare Engineering,
doi: 10.1155/2017/3090343.
Sutojo, T. (2017) Pengolahan Citra Digital. Penerbit Andi.
Deng, L. and Yu, D. (2013) ‘Deep Learning: Methods and Applications’, Foundations and Trends in Signal Processing, 7(3–4), pp. 197–387. doi: 10.1561/2000000039.
Géron, A. (2019) Hands-on Machine Learning whith Scikit-Learing, Keras and Tensorfow, O’Reilly Media, Inc.
Wei, L. (2017) ‘Human Activity Recognition Using Deep Neural Network With Contextual Information’, VISIGRAPP 2017 - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 5(Visigrapp), pp. 34–43. doi: 10.5220/0006099500340043.
Kim, H. (2019) ‘Human Activity Recognition By Using Convolutional Neural Network’, International Journal of Electrical and Computer Engineering, 9(6), pp. 5270–5276. doi: 10.11591/ijece.v9i6.pp5270-5276.
Ignatov, A. (2018) ‘Real-Time Human Activity Recognition from Accelerometer Data using Convolutional Neural Networks’, Applied Soft Computing Journal. Elsevier B.V., 62, pp. 915–922. doi: 10.1016/j.asoc.2017.09.027.
Yeole, C. et al. (2021) ‘Deep Neural Network Approaches for Video Based Human Activity Recognition’, 6(6), pp. 1586–1589.
Patterson, J. and Gibson, A. (2017) Deep Learning: A Practitioner’s Approach, O’Reilly.
Rahmaniar, W. and Hernawan, A. (2021) ‘Real-Time Human Detection Using Deep Learning On Embedded Platforms: A Review’, Journal of Robotics and Control (JRC), 2(6), pp. 462-468Y. doi: 10.18196/jrc.26123.
Nagrath, P. et al. (2021) ‘SSDMNV2: A Real Time DNN-Based Face Mask Detection System Using Single Shot Multibox Detector and MobileNetV2’, Sustainable Cities and Society. Elsevier Ltd, 66(December 2020), p. 102692. doi: 10.1016/j.scs.2020.102692.
Sandler, M. et al. (2018) ‘MobileNetV2: Inverted Residuals and Linear Bottlenecks’, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 4510–4520.
Burkov, A. (2019) ‘The Hundred-Page Machine Learning Book’, Expert Systems, 5(2), pp. 132–150. doi: 10.1111/j.1468- 0394.1988.tb00341.x.
Zhang, F. et al. (2021) ‘Accelerating Hyperparameter Tuning in Machine Learning for Alzheimer’s Disease With High Performance Computing’, Frontiers in Artificial Intelligence, 4(December), pp. 1–9. doi: 10.3389/frai.2021.798962.
Molokwu, B. C. et al. (2020) Node Classification in complex Social Graphs via Knowledge-Graph Embeddings and Convolutional Neural Network, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer International Publishing. doi: 10.1007/978-3-030-50433-5_15.
Goodfellow, I., Bengio, Y. and Courville, A. (2016) Deep Learning. Available at: http://www.deeplearningbook.org/front_matter.pdf.
Müller, A. and Guido, S. (2017) Introduction to Machine Learning with Python. doi: 10.1007/978-3-030-36826-5_10.
Kuhn, M. and Johnson, K. (2013) Applied Predictive Modeling, Applied Predictive Modeling. doi: 10.1007/978-1-4614-6849-3.
Jukes, E. (2017) Encyclopedia of Machine Learning and Data Mining, Encyclopedia of Machine Learning and Data Mining. doi: 10.1007/978-1-4899-7687-1.
Atoum, Y. et al. (2017) ‘Automated Online Exam Proctoring’, IEEE Transactions on Multimedia, 19(7), pp. 1609–1624. doi: 10.1109/TMM.2017.2656064.
Khan, S. et al. (2018) ‘A Guide to Convolutional Neural Networks for Computer Vision’, Synthesis Lectures on Computer Vision, 8(1), pp. 1–207. doi: 10.2200/s00822ed1v01y201712cov015.