Perbandingan Metode Evaluasi Usability Antara Heuristic Evaluation dan Cognitive Walkthrough

Lit Malem Ginting; Grady Sianturi; Christina Vitaloka Panjaitan

doi:10.34010/jamika.v11i2.5480

Lit Malem Ginting
Grady Sianturi
Christina Vitaloka Panjaitan Institut Teknologi Del

DOI: https://doi.org/10.34010/jamika.v11i2.5480

Keywords: Heuristic Evaluation, Cognitive Walkthrough, Usability Testing, Prototype

Abstract

Usability evaluation is needed to identify and analyze usability problems in an application. This study will compare the results of usability evaluation with Heuristic Evaluation and Cognitive Walkthrough methods on the SIMRS Del Egov Center web from the aspects of usability problems found, the level of usability problems, and end user responses that will be evaluated using usability testing to find a more effective method of finding usability problems. Heuristic Evaluation will refer to the 10 heuristic principles proposed by Jacob Nielsen, while Cognitive Walkthrough, the expert will follow the task provided by the researcher. The results showed that the results of the evaluation conducted by Heuristic Evaluation found more usability problems in aspects: efficiency, memorability and satisfaction, while Cognitive Walktrough found more usability problems in aspects: learnability and error. In the severity rating aspect, Cognitive Walktrough is more effective in finding usability problems with a higher severity level, with an average of 3, while heuristic evaluation with an average of 2. In the aspect of end user responses to websites based on usability testing, Heuristic Evaluation has a SUS score which is higher at 57, while Cognitive Walktrough has an SUS score of 54.5. This shows that based on these three aspects, the Heuristic Evaluation method is better at finding usability problems in the SIMRS Del Egov Center application study object.

References

J. Nielsen, “Usability engineering,” Comput. Sci. Handbook, Second Ed., pp. 45-1-45–21, 2004, doi: 10.1201/b16768-38.

M. Farzandipour, E. Nabovati, G. H. Zaeimi, and R. Khajouei, “Usability evaluation of three admission and medical records subsystems integrated into nationwide hospital information systems: Heuristic evaluation,” Acta Inform. Medica, vol. 26, no. 2, pp. 133–138, 2018, doi: 10.5455/aim.2018.26.133-138.

Z. Ebnehoseini, M. Tara, M. Meraji, K. Deldar, F. Khoshronezhad, and S. Khoshronezhad, “Usability evaluation of an admission, discharge, and transfer information system: A heuristic evaluation,” Open Access Maced. J. Med. Sci., vol. 6, no. 11, pp. 1941–1945, 2018, doi: 10.3889/oamjms.2018.392.

A. Holzinger, “Usability engineering methods for software developers,” Commun. ACM, vol. 48, no. 1, pp. 71–74, 2005, doi: 10.1145/1039539.1039541.

A. Fernandez, E. Insfran, and S. Abrahão, “Usability evaluation methods for the web: A systematic mapping study,” Inf. Softw. Technol., vol. 53, no. 8, pp. 789–817, 2011, doi: 10.1016/j.infsof.2011.02.007.

T. Mahatody, M. Sagar, and C. Kolski, “Cognitive Walkthrough for HCI evaluation: basic concepts, evolutions and variants, research issues,” EAM’07, Eur. Annu. Conf. Human-Decision Mak. Man. Control, no. May 2014, p. 12, 2007, [Online]. Available: https://www.researchgate.net/publication/228938202_Cognitive_Walkthrough_for_HCI_evaluation_basic_concepts_evolutions_and_variants_research_issues.

R. Farrahi, F. Rangraz Jeddi, E. Nabovati, M. Sadeqi Jabali, and R. Khajouei, “The relationship between user interface problems of an admission, discharge and transfer module and usability features: A usability testing method,” BMC Med. Inform. Decis. Mak., vol. 19, no. 1, pp. 1–8, 2019, doi: 10.1186/s12911-019-0893-x.

B. Murillo, J. Pow Sang, and F. Paz, “Heuristic evaluation and usability testing as complementary methods: A case study,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 10918 LNCS, pp. 470–478, 2018, doi: 10.1007/978-3-319-91797-9_34.

R. Khajouei, M. Zahiri Esfahani, and Y. Jahani, “Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems,” J. Am. Med. Informatics Assoc., vol. 24, no. e1, pp. e55–e60, 2017, doi: 10.1093/jamia/ocw100.

H. Joo, “A study on understanding of UI and UX, and understanding of design according to user interface change,” Int. J. Appl. Eng. Res., 2017.

S. Saeedbakhsh, S. Isfahani, M. Saber, and G. Yadegarfar, “Hospital information system usability of educational hospitals in Isfahan using heuristic evaluation method,” Int. J. Educ. Psychol. Res., 2016, doi: 10.4103/2395-2296.180308.

A. Atashi, R. Khajouei, A. Azizi, and A. Dadashi, “User interface problems of a nation-wide inpatient information system: A heuristic evaluation,” Appl. Clin. Inform., 2016, doi: 10.4338/ACI-2015-07-RA-0086.

C. Lewis, P. Poison, C. Wharton, and J. Rieman, “Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces,” Conf. Hum. Factors Comput. Syst. - Proc., pp. 235–242, 1990, doi: 10.1145/97243.97279.

M. Georgsson, N. Staggers, E. Årsand, and A. Kushniruk, “Employing a user-centered cognitive walkthrough to evaluate a mHealth diabetes self-management application: A case study and beginning method validation,” J. Biomed. Inform., 2019, doi: 10.1016/j.jbi.2019.103110.

C. M. Barnum, Praise for Usability Testing Essentials. 2011.