Usability of QiraahBot for extensive Arabic reading activities

Today's changing learning landscape urges the integration of mobile phone technology and Arabic learning as a foreign language. Learning through mobile phones is seen as a potential teaching aids which demonstrate effectiveness in teaching and learning activities. A QiraahBot prototype was built to aid the implementation of extensive Arabic reading activities. The user's perspective on QiraahBot's usability is essential for the effectiveness of learning through mobile phone applications. This study aims to identify the level of usability of the QiraahBot application prototype and to explore the deeper experience of users using the application. Therefore, a group of 30 student users was involved in the study. They were assigned tasks to be completed within the specified periods via the application, and then given an open-ended questionnaire adapted from the System Usability Scale. The findings show that the QiraahBot usability level is moderate. The study also revealed some of the advantages of QiraahBot as easy to use and the ability to help the learning process. In addition, there are weaknesses in applications, especially aspects of application usability, which needs to be improved in the next version. The researcher has provided some suggestions for further studies that are more comprehensive in the future.


Introduction
Reading skills are an important skill for second language learners including Arabic.Having these skills help students to master Arabic better.Students who master the reading skills will acquire rich language information and also assist in the acquisition of the language (Krashen, 2004).Additionally, enhanced reading skills also help to improve other language skills such as listening, speaking and writing (Cummins, 2003).This situation leads to the mastery of the language as a whole.Extensive reading (ER) is one of the ways to improve reading efficiency.The concept of ER is someone reading a lot of preferred text (Jeon & Day, 2016;Krashen, 2004).ER aims to develop the ability to read and to acquire as much knowledge as possible about the language and subjects being read.Studies have shown that ER has helped students, as in vocabulary, motivation, textual comprehension, reading fluency and reading efficiency.The meta-analysis of ER's effectiveness on reading efficiency by Jeon and Day (2016) shows that ER affects reading efficiency starts with small to medium size.According to both researchers, the results of the study are consistent with other meta-analysis.Hence, previous studies suggest that ER should be included in the curriculum besides the Intensive Reading tradition (Jeon & Day, 2016;Lee & Mallinder, 2017).
Today, times have changed.Students do not have to read text on paper anymore.Now, students can read a text on the Internet that can be accessed through the mobile phone screen, respectively.In Malaysia, internet usage is widespread with the percentage of consumers aged 15 years and above at 71.1 per cent in 2015.Interestingly, 97.5 per cent of users use mobile phones to access the Internet (Department of Statistics Malaysia, 2016).Therefore, mobile phones have changed the landscape of learning in a much different fashion because of the extensive access to the cheap and sophisticated devices (Nezarat & Mosavi Miangah, 2012).In Malaysia, the use of mobile phones in all sectors of education is increasingly acceptable and enters the era of mobile learning research extension (Nordin, Embi, Norman, & Panah, 2017).This situation requires changes in terms of ER's implementation in Arabic language teaching as a second language in line with the current technology development of students.Teaching and learning that still use traditional methods in reading classes have negative effects on students such as lack of motivation to learn, relying entirely on teachers, coming to class without preparation and weak competition (Hazaea & Alzubi, 2016).
ER activities in Arabic language can be carried out through the Mobile assisted language-learning concept (MALL).MALL refers to integrating mobile phones and their applications to help and improve language learning inside or outside the classroom.MALL is considered as a tool in the field of language acquisition (Chinnery, 2006;Kukulska-Hulme & Shield, 2008).Researchers believe that mobile phone integration or applications can improve the learning effectiveness and ability of students.Chen and Hsu (2008) claim that mobile phone applications can help students to improve their vocabulary information as well as help them to improve reading comprehension.Chang and Hsu (2011) also point out that the second language student considers translation applications, PDA-based annotations or the web useful and easy to use for individual reading activities.Supporting the use of MALL in reading activities as in the study by Huang and Lin (2011) which emphasizes that reading on mobile phones is more popular than the traditional reading activity.

Literature Review
The development of mobile phone technology and its applications in the education world cannot be denied.Mobile phone applications are getting the attention of educational researchers, including instant messaging and bot applications.
With IM's popularity among users, instant messaging applications in 2014 introduced a third-party service support known as a bot.Today, popular IM applications such as Telegram, We Chat, Kik, Facebook Messenger and Skype support the bot function, which is a communication application system that interacts with users where it works in the messaging application itself.More interestingly, this bot is located in a very common place for the user, namely in the communication thread.Bots are placed together in a list of personal conversations with friends and relatives.Most users actually use IM several times a day.They already have good experience and understanding of the interface and how the IM application works (Klopfenstein et al., 2017).Hence, installation and use of bots in messaging applications is not a new thing.
According to Cuno Klopfenstein and Bogliolo (2017) and Klopfenstein et al. (2017), this bot works as an automated agent that communicates with users via simple text communication.At this time, the bot application has mastered most of IM.It is increasingly used as a platform to access certain information or to use the services offered in that way.Due to the widespread popularity of IM through the mastery of the use of smart phones, bots offer attractive platforms, direct access to a large number of users and low value access (Cuno Klopfenstein & Bogliolo, 2017;Klopfenstein et al., 2017).In other words, bot offers the same advantages as IM to developers and users.In addition, bot users are reportedly inclined to use bot due to productivity factors.Bot efficiently makes it easier for them to get help or timely information (Brandtzaeg, 2017) Bot can be considered as an autonomous application that ride on IM.Bot can be built with the Application Programming Interface that is available on common applications such as messaging, payments, bot directories, authentication and others.Bot also accepts UI such as buttons, locations, images and others.This ability provides space for developers to provide innovative services through conventional consumer experience (Klopfenstein et al., 2017).In addition, the Bot application does not need to be downloaded and installed.It is automatically installed when called through a unique and functional code once the conversation is started in the messaging application.Device storage capacity is almost unnecessary.Installation is fast and easy.Accordingly, complex configuration steps are not required.This makes bot as a fast and light application when compared to traditional mobile applications.
As discussed earlier, many studies confirm the importance of bringing today's technology, particularly in relation to IM in teaching and learning.The use of IM can stimulate better learning through communication and collaboration between student groups or between students and teachers.With the support of IM to bot, there is a new learning and teaching space to be captured through this development.A study by Cuno Klopfenstein and Bogliolo (2017) has introduced a bot system that contains educational quizzes in the field of online programming.This quiz bot works in IM Telegram.The study explains the detailed system of bots developed including functional programs, application strengths and weaknesses.The findings suggest the effectiveness of coding quiz operated by the bot and its implementation program.The study also noted the ability of bot to be used more effectively in many educational activities.
A study by Fryer and Carpenter (2006) was conducted to identify the potential of bot in foreign language learning.211 student respondents were asked to use two bots for foreign language learning in the classroom and their feedback was recorded in writing.The results showed that most students like using the bot.They are also generally found to feel more comfortable communicating with bot than friends or teachers.This shows that there is a tendency for students to use bot for foreign language learning and some of them prefer to learn the language or practice it through bot than their own teachers or friends.
Supported by literature review, this study builds a bot-based application as a tool for ER activity.Teachers in the process of teaching Arabic reading skills as a second language use this bot.The bot named QiraahBot works on the IM Telegram platform.The main objective of the study is to determine the extent of usability of these applications in terms of acceptance of students who use it.

Construction of QiraahBot
QiraahBot application is a bot-based automatic system that can interact with users individually or in groups.An interaction occurs via an interface based on the given option button.See figure 1.This means that user interaction and QiraahBot application are guided and strict.Based on the nature of extensive reading beyond the classroom, the QiraahBot system works based on the 'send and monitor' concept.Thus, there are two main functions of the application.1) Supplying extensive texts, and 2) monitoring the progress of the student reading process.The 'send' function built into QiraahBot is supplied with the following features: a. Existing reading materials uploaded into the application.This material can be accessed through the buttons set on the interface by category.Users can browse this material using persistent buttons.Material selection based on Narrow Reading approach recommended by Krashen (2004).b.Automatically post from selected service accounts to QiraahBot users.Some outside services such as Facebook, Twitter and web pages have been linked to the application.Based on the Narrow Reading approach, several Facebook accounts, Twitter and certain web pages have been selected.Any post in the selected account will be automatically displayed as a message in the application.This message is additional texts besides existing materials.As a message, it is assisted by the mobile notification system.New message alert will appear and disappear after reading like regular message in IM. c. Builder or QiraahBot admin has a button to send the reading material.This material works as a Message.The message will be sent to the application user immediately or at the time specified by the admin in the application.The 'monitor' function on QiraahBot was built to facilitate the teacher as an admin.Monitoring through instant messaging conversations is in text form.Conversations occur according to the strict "questions and answers" format.Admin assigns a question through a special button where the user initiates the interaction by touching the button.The user typed the response and sent it.Then, the system will collect the local response based on the question.Admin can click the entire response at any time.(Lella & Lipsman, 2014).Therefore, using existing platforms is a good choice.In addition to the above factors, bot is easily installed and functioning virtually without the use of telephone storage, which is the driver of bot system selection as the application medium.

Methodology
QiraahBot's usability study uses qualitative and quantitative methodologies.A set of questionnaires containing the System Usability Scale (SUS) instrument and the open-ended question was used.30 student respondents were given a QiraahBot boat application for a week and a brief description of the installation and use.An Extensive Reading assignment was also given for a week.ER monitoring was also conducted through QiraahBot once.The QiraahBot application provided for the study is prototype version.After 7 days of use, respondents were given an online questionnaire built on the Microsoft Form platform.
SUS Instruments developed by Brooke (1996) contains 10 items using 5 scales.SUS is designed to get an instant overview of the applicability of an application.According to Lin et al. (2011), SUS is a questionnaire used to assess the user's views and the level of satisfaction with particular systems or products.SUS is usually given after the user uses a system or product that is evaluated and before they are involved with any orientation or discussion (Suominen, 2013).Item statements on odd numbers are framed positively, and items on even numbers are negative.For research purposes, the word 'system' of the original SUS instrument was changed to QiraahBot.At the end of the questionnaire, there are open-ended questions.It aims to get respondents qualitative inputs about the strengths, weaknesses and scope of improvements that can be made.
The data obtained were analyzed using SPSS statistical software.Descriptive analysis and frequency applied to the SUS instrument response.Qualitative data results from openended questions were also discussed at the end of the findings based on quantitative data results.

Total SUS score
SUS is built to get usability scores quickly and easily.Researchers can know the usability of a system or product through the overall SUS score ranging from 0 to 100.SUS maker, Brooke (1996) has provided a standard scoring formula for calculating the overall score of SUS.Each SUS score obtained from the respondents was assessed using the mean to obtain the overall score of SUS.
The overall SUS score for QiraahBot application is 65.5.The descriptive analysis for the SUS score in this study is shown in table 1.The SUS mean score for QiraahBot usability is 65.5, the minimum score is 55 and the maximum score is 85.Bangor, Kortum and Miller (2009) suggests that good score is between 70 and 80.For scores value of 90 and above, it indicates a high usability.If the overall score of SUS is between 50 and 70, the system or product analyzed is considered marginally acceptable.However, the score below 50 is considered unacceptable.Based on this guide, SUS score of 65.5 indicates that the QiraahBot application assessed by respondents can generally be marginal.
Although the QiraahBot application is deemed to be used according to Bangor et al. (2009), a study by Sauro (2011) on 500 previous usability studies shows a more stringent read score.According to him, the average SUS score is 68.The score above 68 is considered above average, and the score below 68 is considered below average.This score reading illustrates the usability of the QiraahBot application that is slightly below the score SHS Web of Conferences 53, 04005 (2018) https://doi.org/10.1051/shsconf/20185304005ICHSS 2018 rated as average usability by respondent's experience as a user.It also reveals that the application still has a room that can be improved before it is officially adopted.The score for this item ranges from 1.90 to 2.87.Low values on negative statements suggest that most respondents disagree with the negative aspects of the QiraahBot application.Almost all respondents disagree that they need more learning to continue using this application (mean = 1.90).Most respondents also did not consider it difficult to use (mean = 2.87) and they did not need technical assistance for it (mean = 2.13).However, most respondents agreed that the QiraahBot application was still unstable for use (mean = 3.32).Hence, some respondents are uncomfortable using this application (mean = 3.57).
In conclusion, the analysis of each item indicates that the QiraahBot application is generally acceptable.However, there are significant weaknesses that cause users to feel uncomfortable.

Qualitative results
This study instrument besides containing SUS also includes open-ended questions to respondents.This question asks respondents to reflect on the experience of using QiraahBot and suggestions for improvement.According to Albert and Tullis (2013), the use of openended questions in usability studies can help identify areas for improving a product or system.
Generally, most respondents provide a good reflection throughout the experience using the QiraahBot application.According to respondents, QiraahBot is a good application (P06, P09, P11, P13, P15, P24, P29 & P43).The qualitative data also recorded some positive features such as easy-to-use the application of QiraahBot (P03, P14, P23, P26, P28 & P38).In the context of the use of QiraahBot as a learning tool for Arabic Language Extensive Reading, some respondents agreed that this application helped them in the learning process.
Behind the positive elements of QiraahBot, there are comments and suggestions by respondents who make room for improvement of this application system.The bot application, which is still at the prototype level and in the beginning of the test, certainly requires constructive comments and comments before formally and widely used.
The most significant weakness in this application is technically viewed.The respondents reported that they were easily caught (P05) and slightly complex (P04 & P01).Quantitative data reported that respondents did not agree that they needed to learn more before using them.However, qualitative comments suggest that learning is still needed.This application system is found to have a slightly complex system or setting.This situation requires users to seek help or try several times before using it.Some respondents also say something about IM Telegram platform that supports bot applications.They '... feel a little confused ...' (P44) and '... lack of understanding ...' (P45) QiraahBot application because '... less skilled ...' (P45) and '... rarely use ... ' (P44) IM Telegram platform.
The respondents' comments throughout QiraahBot's experience provide useful inputs and suggestions, especially in subsequent versions of improvements.Improvements are necessary to make QiraahBot more user-friendly and effective in the user's perspective.

Discussion
According to descriptive analysis, QiraahBot is considered to be used at a moderate level by respondents (mean = 65.5).If the value 68 is placed as usability measure as in the study by Sauro (2011), QiraahBot is slightly below average.However, according to Bangor et al. (2009), QiraahBot's reading position indicates that it is still OK with some aspects of the application requiring improvements before it can be fully used.
SHS Web of Conferences 53, 04005 (2018) https://doi.org/10.1051/shsconf/20185304005ICHSS 2018 A fairly low usability score may be due to respondents' immature usage experience.One task within a week can be considered brief so students are still unable to familiarize themselves with the app.Sauro (2011a) and Orfanou, Tselios and Katsanos (2015) emphasis that there are significant differences between first-time users and experienced users support this assumption.The study also revealed that the SUS score would be high if the students were more frequently using technology or learning-related systems.Hence, there is a higher probability of SUS QiraahBot being valued if more experience and time is given.From observation by researchers, students have been using the last-minute times to finish the work from a weeklong period.
However, for newly built applications and still considered prototypes, the level of "OKAY" usability based on the given period may be considered a good start (Ganapathy, Shuib, & Azizan, 2016).Tanja (2011) in a study of usability personal video recording that earned a score of 64 states that it be considered a candidate for continued improvement.Study by Ganapathy et al. (2016) has scored 64.17 for the MyGraTe prototype.For them, the modest usability value is acceptable for applications that are still at the prototype level.In addition, Nieminen, Runonen, Tyllinen and Nieminen (2014) also agreed that a prototype system with some technical problems and lack of features, but obtaining a marginal SUS value is acceptable.This is acceptable because the preliminary test of a prototype is the first step for further action.Therefore, the lack of early application construction is normal.
Some respondents expressed support for using this application as it is easy to use and does not require technical assistance from other parties.However, details on quantitative and qualitative descriptive analysis reveal a lack of usability when using QiraahBot.This can be considered normal because usability is typically considered from the point of view of issues or problems encountered by users (Kukulska-Hulme, 2007).Bangor et al. (2009) when commenting on the score at OKAY level (52 to 72) asserted that the system is okay, but still clearly deficient in terms of perceived usability.
According to Mostakhdemin-Hosseini (2009), the usability of mobile learning applications should consider several aspects, including device usability, application usability and content usability.This study did not find any deficiencies in terms of device usability.This is likely due to the lightweight, easy-to-install feature of the bot application, which does not require complex storage and configuration capacity and is supported by IM applications (Brandtzaeg, 2017;Klopfenstein et al., 2017).Similarly, in the context of content usability.The study did not receive any negative comments on the content.In fact, the respondents were praised for the content of the application that was considered productive (P15 & P18).The lack of QiraahBot reported is on application usability.Respondents noted the issue of time lagging and the need to facilitate system travel.Although a SUS descriptively detailed score implies the QiraahBot app is generally rated as easy to use, there is still a complaint about inconsistency and complexity when the application works.This condition is also normal when there are other studies that are almost identical to the findings of study by Al-Khalifa (2010) and Ganapathy et al. (2016).Their study also reported a survey system or product that was seen to be user friendly through the SUS scores but faced with application usability issues such as inconsistency.
Additionally, there are respondents who still need time and guidance to use QiraahBot skillfully.Student needs for help and clarification of usage are supported by a study conducted by Chen and Denoyelles (2013).They pointed out that students still need technical assistance to use mobile phone technology for learning needs.Technical training and skills development are an important factor in the advancement of learning technology.This is because students have considered both of them more important than the technology itself.Therefore, Ganapathy et al. (2016) recommends that developers for mobile-based SHS Web of Conferences 53, 04005 (2018) https://doi.org/10.1051/shsconf/20185304005ICHSS 2018 learning applications take into account the need to provide support when developing an effective mobile learning environment.

Conclusions
The education world is now witnessing the rapid adoption of mobile phone application technology in learning sessions, including Arabic language learning as a foreign language.This technology changes the learning environment to be more dynamic and open.The QiraahBot application was developed to embellish these changes and bring Arabic students to the latest in the education world.It serves as a tool to facilitate the implementation of extensive Arabic reading activities.However, it is important for us to realize that learning technology through mobile phones depends on the acceptance of the students involved.Any new technology including QiraahBot needs to be 'adopted' by the student as a user so that it remains and is effective as expected.Studies on the usability of the QiraahBot prototype reported moderate acceptance by respondents.The findings also reveal some aspects of usability that need to be improved for the next step.
This study was conducted with certain limitations.The study involved only respondents using the QiraahBot prototype at a relatively short time, as this study is an initial study of newly built applications.It may yield more in-depth and rich insights when involving more respondents, longer usage times and covering multiple tasks.This study involves only one task that needs to be resolved by the respondents.The study also focuses on quantitative on SUS instruments.In the future, it is proposed that the usability study using the mix method; qualitative and quantitative as proposed by Albert and Tullis (2013).Qualitative data such as focus group interviews, observations and usage analysis can provide in-depth information for developers or researchers for further action.Therefore, studies on the application of QiraahBot in the future should involve more extensive and deep usability ratings as well as beyond the limitations of previous studies.

Fig 1 .
Fig 1. QiraahBot Interface This application is implemented as a bot using IM Telegram platform.The action does not develop its own application based on the current trends of mobile phone users.Research shows that most smart phone owners barely install new apps on their devices every month

Table 1 .
SUS Descriptive StatisticsDescriptive statistics on each item in the SUS give a clearer picture of the respondent's assessment.Based on Table2below, item number 5 reaches the highest mean value, namely 4.17 (mean).The mean value of item 5 indicates that most respondents agree that the diversity of functions in QiraahBot is well integrated.Item number 1, 3 and 7 also got a good score above 4.00.Item number 1 (mean = 4.10) represents the desire of most users of QiraahBot to continue using it.Possibly because most respondents consider that QiraahBot is easy to use as interpretation of score 3 (mean = 40).Not only QiraahBot's respondents are able to use it, most of them also think that others are also learning how to use this application quickly as shown by item number 7 (mean = 4.00).In addition, descriptive analysis of all positive SUS items showed mean values between 3.63 to 4.17.In general, respondents gave a simple agreement on the various aspects of the application of QiraahBot.

Table 2 .
Descriptive Statistics of Each SUS Items