Implementasi Deteksi Gerakan Tangan untuk Sistem Interaktif Kios menggunakan Metode Long Short-Term Memory (LSTM)
DOI:
https://doi.org/10.34010/y895rc89Abstract
Deaf individuals in Indonesia face challenges in using voice-based technology. This study aims to develop an interactive kiosk system utilizing hand gesture detection based on Long Short-Term Memory (LSTM) to provide a more inclusive solution. The research process includes collecting hand gesture datasets using MediaPipe, splitting the dataset into training and testing data with a 75:25 ratio, and training the model using a Learning Rate Scheduler. The model architecture is designed to capture patterns from keypoint data by optimizing the use of dropout layers and the softmax activation function. The evaluation shows that the model achieves an accuracy of 90.22% on the test data, with an average precision of 91%, recall of 89%, and F1-score of 90%. The trial results also demonstrate consistent performance for simple gestures, while accuracy decreases for complex gestures and greater distances. This research provides a significant contribution to enabling voice-free interaction, particularly for deaf individuals, by integrating LSTM technology into interactive kiosk systems.
References
[1] M. Imran, “Peningkatan Pemberdayaan Penyandang Tunanetra melalui Perancangan Social Media Newsletter di Yayasan Sosial Tunanetra,” J. Komunitas J. Pengabdi. Kpd. Masy., vol. 6, no. 2, pp. 229–239, 2024, doi: 10.31334/jks.v6i2.3587.
[2] I. M. Esa Darmayasa Adi Putra, I. M. Putra Arya Winata, I. Fauzi, K. S. Krishna Prasad, and I. W. Widhiada, “Model Object Detection Neural Network Berbasis Hand Gesture Recognition Sebagai Kontrol Prostesis Tangan,” Maj. Ilm. Teknol. Elektro, vol. 22, no. 2, p. 185, 2023, doi: 10.24843/mite.2023.v22i02.p05.
[3] I. Sari, N. Fivrenodi, E. Altiarika, and N. Sarwindah, “Sistem Pengembangan Bahasa Isyarat Untuk Berkomunikasi Dengan Penyandang Disabilitas (Tunarungu),” vol. 1, no. 1, pp. 20–25, 2023, doi: 10.35438/jits.v1i1.21.
[4] S. Abdullah and S. Rahmadinni, “Deteksi Gestur Tangan Berbasis Pengolahan Citra,” J. Rekayasa Elektr., vol. 18, no. 2, 2022, doi: 10.17529/jre.v18i2.25147.
[5] E. Altiarika and W. P. Sari, “Pengembangan Deteksi Realtime Untuk Bahasa Isyarat Indonesia Dengan Menggunakan Metode Deep Learning Long Short Term Memory Dan Convolutional Neural Network,” J. Teknol. Inform. Dan Komput., vol. 9, no. 1, pp. 1–13, 2023, doi: 10.37012/jtik.v9i1.1272.
[6] S. Nurhaeni, R. Faurina, F. P. Utama, and K. Anggriani, “Sentiment Analysis Komentar Berbahasa Bengkulu Menggunakan Long Short-Term Memory (LSTM),” Pseudocode, vol. 10, no. 2, pp. 117–125, 2023, doi: 10.33369/pseudocode.10.2.117-125.
[7] M. I. Awaludin, “Pengenalan Gerakan Tangan Menggunakan Fuzzy-Logic Dengan Algoritma Jaringan Syaraf Tiruan Backpropagation,” Fidel. J. Tek. Elektro, vol. 2, no. 3, pp. 67–71, 2022, doi: 10.52005/fidelity.v2i3.118.
[8] A. Sani and S. Rahmadinni, “Deteksi Gestur Tangan Berbasis Pengolahan Citra,” J. Rekayasa Elektr., vol. 18, no. 2, 2022, doi: 10.17529/jre.v18i2.25147.
[9] H. S. Izzadiana, H. Napitupulu, and F. Firdaniza, “Peramalan Data Univariat Menggunakan Metode Long Short Term Memory,” Sisinfo J. Sist. Inf. Dan Inform., vol. 5, no. 2, pp. 29–39, 2023, doi: 10.37278/sisinfo.v5i2.669.
[10] A. Graves, A. Mohamed, and G. E. Hinton, “Speech Recognition With Deep Recurrent Neural Networks,” pp. 6645–6649, 2013, doi: 10.1109/icassp.2013.6638947.
[11] M. A. Satriaman and I. Atastina, “Analyze News Effect on Trend Stock Price in Indonesia Based on Bidirectional-Long Short Term Memory,” Build. Informatics Technol. Sci., vol. 5, no. 1, 2023, doi: 10.47065/bits.v5i1.3631.
[12] R. S. Pontoh et al., “Jakarta Pandemic to Endemic Transition: Forecasting COVID-19 Using NNAR and LSTM,” Appl. Sci., vol. 12, no. 12, p. 5771, 2022, doi: 10.3390/app12125771.
[13] G. Budiprasetyo, M. Hani’ah, and D. Z. Aflah, “Prediksi Harga Saham Syariah Menggunakan Algoritma Long Short-Term Memory (LSTM),” J. Nas. Teknol. Dan Sist. Inf., vol. 8, no. 3, pp. 164–172, 2023, doi: 10.25077/teknosi.v8i3.2022.164-172.
[14] T. A. Faisal Muhammad and M. I. Irawan, “Implementasi Long Short-Term Memory (LSTM) Untuk Prediksi Intensitas Curah Hujan (Studi Kasus: Kabupaten Malang),” J. Sains Dan Seni Its, vol. 12, no. 1, 2023, doi: 10.12962/j23373520.v12i1.106892.
[15] R. Kanani, J. Fiaidhi, and V. Patel, “Depression Sentiment Analysis Based on Social Media Content like Twitter,” TechRxiv, pp. 1–10, 2022, doi: 10.36227/techrxiv.21694676.v1.
[16] N. None, A. A. Lydia*, F. S. Francis, and N. None, “Learning Rate Scheduling Policies,” Int. J. Innov. Technol. Explor. Eng., vol. 9, no. 1, pp. 3641–3644, 2019, doi: 10.35940/ijitee.a4648.119119.
[17] I. Al-Nader, A. Lasebae, R. Raheem, and A. Khoshkholghi, “A Novel Scheduling Algorithm for Improved Performance of Multi-Objective Safety-Critical Wireless Sensor Networks Using Long Short-Term Memory,” Electronics, vol. 12, no. 23, p. 4766, 2023, doi: 10.3390/electronics12234766.
[18] J. Tang, R. Yang, G. Yuan, and Y. Mao, “Time-Series Deep Learning Models for Reservoir Scheduling Problems Based on LSTM and Wavelet Transformation,” Electronics, vol. 11, no. 19, p. 3222, 2022, doi: 10.3390/electronics11193222.
[19] A. Alfando and R. Hayami, “Klasifikasi Teks Berita Berbahasa Indonesia Menggunakan Machine Learning Dan Deep Learning: Studi Literatur,” Jati (Jurnal Mhs. Tek. Inform., vol. 7, no. 1, pp. 681–686, 2023, doi: 10.36040/jati.v7i1.6486.
[20] A. Hanifa, S. A. Fauzan, M. Hikal, and M. B. Ashfiya, “Perbandingan Metode LSTM Dan GRU (RNN) Untuk Klasifikasi Berita Palsu Berbahasa Indonesia,” Din. Rekayasa, vol. 17, no. 1, p. 33, 2021, doi: 10.20884/1.dr.2021.17.1.436.
[21] S. Lestari and M. F. Pratama, “Penerapan Metode Long Short-Term Memory Pada Pendataan Warga Berbasis Android,” J. Comput. Syst. Informatics, vol. 3, no. 4, pp. 156–161, 2022, doi: 10.47065/josyc.v3i4.1951.
[2] I. M. Esa Darmayasa Adi Putra, I. M. Putra Arya Winata, I. Fauzi, K. S. Krishna Prasad, and I. W. Widhiada, “Model Object Detection Neural Network Berbasis Hand Gesture Recognition Sebagai Kontrol Prostesis Tangan,” Maj. Ilm. Teknol. Elektro, vol. 22, no. 2, p. 185, 2023, doi: 10.24843/mite.2023.v22i02.p05.
[3] I. Sari, N. Fivrenodi, E. Altiarika, and N. Sarwindah, “Sistem Pengembangan Bahasa Isyarat Untuk Berkomunikasi Dengan Penyandang Disabilitas (Tunarungu),” vol. 1, no. 1, pp. 20–25, 2023, doi: 10.35438/jits.v1i1.21.
[4] S. Abdullah and S. Rahmadinni, “Deteksi Gestur Tangan Berbasis Pengolahan Citra,” J. Rekayasa Elektr., vol. 18, no. 2, 2022, doi: 10.17529/jre.v18i2.25147.
[5] E. Altiarika and W. P. Sari, “Pengembangan Deteksi Realtime Untuk Bahasa Isyarat Indonesia Dengan Menggunakan Metode Deep Learning Long Short Term Memory Dan Convolutional Neural Network,” J. Teknol. Inform. Dan Komput., vol. 9, no. 1, pp. 1–13, 2023, doi: 10.37012/jtik.v9i1.1272.
[6] S. Nurhaeni, R. Faurina, F. P. Utama, and K. Anggriani, “Sentiment Analysis Komentar Berbahasa Bengkulu Menggunakan Long Short-Term Memory (LSTM),” Pseudocode, vol. 10, no. 2, pp. 117–125, 2023, doi: 10.33369/pseudocode.10.2.117-125.
[7] M. I. Awaludin, “Pengenalan Gerakan Tangan Menggunakan Fuzzy-Logic Dengan Algoritma Jaringan Syaraf Tiruan Backpropagation,” Fidel. J. Tek. Elektro, vol. 2, no. 3, pp. 67–71, 2022, doi: 10.52005/fidelity.v2i3.118.
[8] A. Sani and S. Rahmadinni, “Deteksi Gestur Tangan Berbasis Pengolahan Citra,” J. Rekayasa Elektr., vol. 18, no. 2, 2022, doi: 10.17529/jre.v18i2.25147.
[9] H. S. Izzadiana, H. Napitupulu, and F. Firdaniza, “Peramalan Data Univariat Menggunakan Metode Long Short Term Memory,” Sisinfo J. Sist. Inf. Dan Inform., vol. 5, no. 2, pp. 29–39, 2023, doi: 10.37278/sisinfo.v5i2.669.
[10] A. Graves, A. Mohamed, and G. E. Hinton, “Speech Recognition With Deep Recurrent Neural Networks,” pp. 6645–6649, 2013, doi: 10.1109/icassp.2013.6638947.
[11] M. A. Satriaman and I. Atastina, “Analyze News Effect on Trend Stock Price in Indonesia Based on Bidirectional-Long Short Term Memory,” Build. Informatics Technol. Sci., vol. 5, no. 1, 2023, doi: 10.47065/bits.v5i1.3631.
[12] R. S. Pontoh et al., “Jakarta Pandemic to Endemic Transition: Forecasting COVID-19 Using NNAR and LSTM,” Appl. Sci., vol. 12, no. 12, p. 5771, 2022, doi: 10.3390/app12125771.
[13] G. Budiprasetyo, M. Hani’ah, and D. Z. Aflah, “Prediksi Harga Saham Syariah Menggunakan Algoritma Long Short-Term Memory (LSTM),” J. Nas. Teknol. Dan Sist. Inf., vol. 8, no. 3, pp. 164–172, 2023, doi: 10.25077/teknosi.v8i3.2022.164-172.
[14] T. A. Faisal Muhammad and M. I. Irawan, “Implementasi Long Short-Term Memory (LSTM) Untuk Prediksi Intensitas Curah Hujan (Studi Kasus: Kabupaten Malang),” J. Sains Dan Seni Its, vol. 12, no. 1, 2023, doi: 10.12962/j23373520.v12i1.106892.
[15] R. Kanani, J. Fiaidhi, and V. Patel, “Depression Sentiment Analysis Based on Social Media Content like Twitter,” TechRxiv, pp. 1–10, 2022, doi: 10.36227/techrxiv.21694676.v1.
[16] N. None, A. A. Lydia*, F. S. Francis, and N. None, “Learning Rate Scheduling Policies,” Int. J. Innov. Technol. Explor. Eng., vol. 9, no. 1, pp. 3641–3644, 2019, doi: 10.35940/ijitee.a4648.119119.
[17] I. Al-Nader, A. Lasebae, R. Raheem, and A. Khoshkholghi, “A Novel Scheduling Algorithm for Improved Performance of Multi-Objective Safety-Critical Wireless Sensor Networks Using Long Short-Term Memory,” Electronics, vol. 12, no. 23, p. 4766, 2023, doi: 10.3390/electronics12234766.
[18] J. Tang, R. Yang, G. Yuan, and Y. Mao, “Time-Series Deep Learning Models for Reservoir Scheduling Problems Based on LSTM and Wavelet Transformation,” Electronics, vol. 11, no. 19, p. 3222, 2022, doi: 10.3390/electronics11193222.
[19] A. Alfando and R. Hayami, “Klasifikasi Teks Berita Berbahasa Indonesia Menggunakan Machine Learning Dan Deep Learning: Studi Literatur,” Jati (Jurnal Mhs. Tek. Inform., vol. 7, no. 1, pp. 681–686, 2023, doi: 10.36040/jati.v7i1.6486.
[20] A. Hanifa, S. A. Fauzan, M. Hikal, and M. B. Ashfiya, “Perbandingan Metode LSTM Dan GRU (RNN) Untuk Klasifikasi Berita Palsu Berbahasa Indonesia,” Din. Rekayasa, vol. 17, no. 1, p. 33, 2021, doi: 10.20884/1.dr.2021.17.1.436.
[21] S. Lestari and M. F. Pratama, “Penerapan Metode Long Short-Term Memory Pada Pendataan Warga Berbasis Android,” J. Comput. Syst. Informatics, vol. 3, no. 4, pp. 156–161, 2022, doi: 10.47065/josyc.v3i4.1951.
Downloads
Published
2025-05-03
Issue
Section
Articles
How to Cite
[1]
“Implementasi Deteksi Gerakan Tangan untuk Sistem Interaktif Kios menggunakan Metode Long Short-Term Memory (LSTM)”, Komputika, vol. 14, no. 1, pp. 31–39, May 2025, doi: 10.34010/y895rc89.
