Factors in the Acceptance of Computer Based Assessment by Men and Women

June 29, 2014 / / No Comments

Back for a third time today, folks. As I said before, it’s crunch time and there are many readings to be reflected upon in these last few days before the final deadline for this class. Once this outpouring of blog posts ends, I will be attempting to post regularly on topics of my own choosing. Some will come from my next class in the masters program, some will be topics and articles that I just genuinely wish to spend time discussing. No matter what this blog turns its attention to, I hope that it continues to be interesting and thought-provoking to the readers who are kind enough to spend their time reading and considering my thoughts and those of the authors whose articles I present here. Now, let us move on to the topic of this blog post.

Today I will be discussing cyber assessment. Cyber assessment is a broad term and can be used to describe anything from using technology to assess whether writing is the result of plagiarism to the more literal usage of e-learning and digital tests/quizzes. In my current course, our dear instructor allows us to submit our quizzes for the class in a typed format. This is a fine example of the more literal definition of cyber assessment. While we did read articles about data mining being used to detect plagiarism, as well as a comparison of the response rates between online and paper-based surveys, these two topics were not particularly interesting. The article that I did find intriguing was the one by Terzis and Economides, which explores the differences between men and women in their perceptions and acceptance of computer based assessment (CBA).

Terzis and Economides (2011) had previously created a Computer Based Assessment Acceptance Model (CBAAM) that uses eight factors in order to define behavioral intention (BI) to use a CBA. These factors are Perceived Playfulness (PP), Perceived Usefulness (PU), Perceived Ease of Use (PEOU), Computer Self Efficacy (CSE), Social Influence (SI), Facilitating Conditions (FC), Goal Expectancy (GE) and Content (pg. 2110). For those that read my previous post and remember Davis’ research, you know that PEOU and PU were the main factors that he said determined acceptance of any type of information technology by a user. Here, Terzis and Economides state that their model shows that behavioral intention to use a CBA is strongly correlated to PP and PEOU. They also state which of the other aforementioned factors significantly explains Perceived Usefulness and Perceived Playfulness. According to the research, PU is significantly explained by Goal Expectancy, Content, Social Influence and Perceived Ease of Use while Perceived Playfulness is defined by Usefulness, Content, Ease of Use and Goal Expectancy. The authors then add that Computer Self Efficacy and Facilitating Conditions also help to determine Perceived Ease of Use (p. 2110).

In this study, Terzis and Economides (2011) wanted to determine the differences between men and women in the factors that affected their perception and acceptance of a CBA. They formulated hypotheses based on a vast amount of research in a variety of areas. I do not want to elaborate on all of it, but one example is their hypothesis that PP influences behavioral intention to use a CBA more in men than in women. They based this on research that determined that men expressed more positive feelings toward multiple choice assessments than females and also because previous studies found that men enjoy playing computer games more than women, meaning that men’s characteristics are more aligned with CBA’s game orientation (p. 2112).

Ultimately, the research done by Terzis and Economides showed some very interesting things. Before we talk about gender differences, I would like to highlight some of the general observations. First, the authors state that Social Influence is a strong determinant of PU. Remember when I previously talked about those who might use Facebook just because everyone else is? As it turns out, there is correlations with Davis’ PU factor after all. In other words, even if a user would not necessarily perceive a site as useful on their own, they may already have that perception in their mind upon their first visit if they know that a lot of other people are using the same system (p. 2116).

The authors also noted that Goal Expectancy shows that students who are prepared and expect to be successful in a course are more likely to find a CBA useful as well as playful. They also noted that while Content has no direct effect on BI, it does influence PP, PU, and GE. This indirect effect on BI means that a CBA that contains content that is clear and interesting is more likely to be used. Finally, the research did not show any correlation between BI and PU like previous research, but an indirect effect exists here as the correlation between PP and PU was high. As mentioned above, this research showed that BI is most strongly correlated to PEOU and PP, not PU like with Davis’ study. Since PU has a high correlation with PP though, it must be considered when designing the CBA (p. 2216).

As for the gender differences, the authors found that the male students are more influenced to use a CBA through Playfulness, Usefulness, Content and Social Influence. This means that if the creators of a CBA want to influence male students to use the CBA, they have to make the CBA playful, they must make it useful for enhancing male students’ knowledge and performance, they must ensure that it delivers both appropriate and clear content, and they must make the CBA something that will be recommended and suggested by their peers and their teachers (p. 2119).

The female students are mainly influenced to use a CBA through Playfulness, Ease of Use, Content and Goal Expectancy. This means that Playfulness and Content are important for both males and females, but the authors note that these factors do not matter to the same degree as they do for males. On the other hand, female students are influenced by Ease of Use and Goal Expectancy, unlike the male students, and not by Usefulness and Social Influence as the male students were. This means that creators of a CBA who want to influence female students to use it must create a CBA that is easy to use with a simple design. The CBA also has to pique the interest of the female students in a way that maximizes their preparation and raises their Goal Expectancy (p. 2119).

Using this research, educators who wish to create and utilize CBA’s in their classrooms can understand the factors that affect students as a whole, as well as the individual factors that matter more to men than women and vice versa. This understanding can lead them to create better CBA’s that all of their students want to use. Without this knowledge, utilization of a CBA might be fractured amongst a given student population, resulting in less than optimal outcomes and perhaps even a reduction in willingness to use a CBA in the future by the instructor themselves.

 
References

Terzis, V. & Economides, A. A. (2011). Computer based assessment: Gender differences in perceptions and acceptance. Computers in Human Behavior, 27, 2108–2122.

Using Interface and Screen Layouts to Increase User Base

June 29, 2014 / / 2 Comments

Hello, readers. Long time, no see. Crunch time at the end of the term both at work and at grad school had severely limited my blogging time, but you will have a number of posts to look through as I focus my efforts on meeting deadlines for my class. As usual, I will not rush anything to publication so if any of the topics are covered briefly, it is only from my own lack of interest in fully engaging with it. Today’s post will certainly not fall under that category as I am extremely interested in screen and interface layout as a web designer. Why should you care about this topic if you are an educator with no professional-level foray into game creation or web design? Well, if you are following this blog and want to be a teacher that incorporates as much technology into their teaching style as possible, then you will most likely create a web page, however simplistic in design, at some point. That or you will be advising students who are creating a web page for a project that they are doing for your class. You may never develop a game for educational use, although I will attempt to convince you to try in an upcoming post, but having working knowledge of what makes for a good screen/interface layout is helpful. It is certainly one of the first things that you must consider before creating a web page, and even more important if you are crafting an entire website.

I will look at a number of articles that I found interesting on this topic, probably covering about three of the eight that were read for the class. Each article focuses on slightly different elements or has differing results from what the researchers expected based on previous research done in that area. The oldest of the articles is from back at the end of the 1980’s. Davis (1989) wrote back then about how the perceived usefulness and the perceived ease of use of any kind of information technology was a critical factor in that technology being accepted and widely used. Keep in mind that we are talking about perceptions here. Regardless of the truth of the matter, Davis’ study focused on whether a user believed a given system enhanced their job performance as well as whether they believed using a system to be effortless (p. 320). Davis is even careful to warn readers at the end of his article that the results are not based on objective reality, only the subjective appraisal of the participants (p. 335).

The fact that this study was done on electronic mail is, in itself, interesting, but let us take a moment to think of the real implications here. Even if an interface’s or a website’s layout is complex beneath the surface, with numerous functions and capabilities, if it looks easy to use and users can see a real purpose for its existence, they will become regular users. For us educators, we may want our students to use a webpage or website that we construct to go along with our class, or we may have designed a simple game to help them study something like vocabulary. From Davis’ research, an educator looking to do something along these lines must, at the onset of the design phase, start considering how they will construct something that their students will instantly see as beneficial while also not requiring any sort of a learning curve.

Consider Facebook for a moment. I have not yet begun to cover social media, but nevertheless, it can be used for the purpose of our discussion here. At its core, Facebook has a great number of features that have to be learned if you really want to master it, but at first glance, it looks surprisingly easy. I find my friend, I click on a button to “add” them, and when they accept my request, I begin seeing all of their posts, pictures, etc. on my News Feed as soon as I visit the site. I can easily see how useful Facebook is for keeping up with people that I might not have many chances to meet in person so that drives me to continue using it. Of course, I may find out later that I need to learn how to upload photos, create albums, and play games with friends (which I personally avoid like the plague), but I became a user because it all seemed simple enough and I saw value in the system. Now I cannot claim that it is all due to the two factors Davis found, as many people are likely to do something just because everyone else is, but I am sure that these factors still contribute, particularly for those who may not be as comfortable with computers and the Internet, yet who still use Facebook.

Altaboli and Lin (2011) focused more on screen aesthetics in their research and how these affect a user’s perception of what they are being presented with. In the article, they mention previous research done by Ngo et. al. that proposed fourteen measures of screen aesthetics. These measures were balance, symmetry, equilibrium, unity, sequence, density, proportions, cohesion, simplicity, regularity, economy, homogeneity, rhythm, and order (p. 2). Altaboli and Lin, however, decided to focus only on balance, unity, and sequence based on the findings of earlier research by Ngo and Byrne. Completely subjective as my opinion is, I agree that these elements are factors that are present on websites that are aesthetically pleasing, whereas the other elements such as economy and proportions do not seem as crucial.

What I took issue with, as did those classmates with whom I discussed this article, is the experimental design used by Altaboli and Lin. In fact, much of it has to do with the participants, of whom there were only 13 (p. 4). That alone is not enough to obtain statistically valid results, although Altaboli and Lin claim otherwise (p. 5). Among some of the other grievances is that the participants included only three women, the age range of the participants was limited, with a median age of 29.3 years of age and a standard deviation of only 6.1 years, and finally, all of the participants were graduate students of engineering (p. 4). I also took issue with the fact that one of the screens that should have scored low because it did not possess any of the characteristics mentioned above actually scored quite high, yet Altaboli and Lin eventually write it off as experimental error, saying that effects of some of the other fourteen elements should later be investigated just to be sure that those are not causing the result (p. 6).

Still, I did like that Altaboli and Lin created a regression model and utilized it along with already existing survey data for 42 websites, despite their formulas originally being designed for data entry screens. The result was that they found correlation between the values computed by the model and the scores on the website questionnaires for items that dealt with the visual layout of the site (pg. 7). I would like to see them redo their experiment with a larger variety of participants and then use that data along with the website questionnaires, which did have an adequate number of participants as well as adequate variety. Until then, I can only continue to subjectively look at websites that I find aesthetically appealing and check if the three elements singled out in this study are present or not.

Finally, we have the study done by Salimun et. al. (2010). This paper was interesting to me because another paper coauthored by Salimun (in the same year, no less) had indicated that users actually preferred a medium level of aesthetics for an interface, yet this paper indicates that the higher the aesthetic level of an interface, the better the response time a user would have with a searching activity (Conclusion section, ¶2). In practical application, this would correspond to how fast a user could find what they are looking for on a web page, for instance. In this study, six elements were looked at compared to the three from Altaboli and Lin’s study. The six elements included cohesion, economy, regularity, sequence, symmetry, and unity (Methodology section, ¶1). Thus, two of the three elements from Altaboli and Lin’s study are present.

Again, I found that the number of participants was quite low, at only 22, but they were split evenly between males and females and they all came from different backgrounds of study. Unfortunately, they were also all computer literate and used computers daily, whereas I would have liked to have some participants who rarely used computers participate (Participant section, ¶1). Nevertheless, we can gather from this study that a website or interface with a high level of aesthetics as measured from the usage of the aforementioned six elements would have a greater chance of increasing user response time than one that does not. If we indeed believe in the study’s results and design our project using these measures, assuming it produces the intended results, then it would also likely increase the perceived ease of use. Users would quickly find what they are looking for rather than getting lost during navigation, which generally causes frustration. As I mentioned with Davis’ study at the very beginning, perceived ease of use is what draws users in and keeps them coming back. While the use of aesthetics may not cover the perceived usefulness aspect of Davis’ study, it does cover the other half and is thus something that must be considered when designing any site, application, or game if it is to succeed.

 
References

Altaboli, A. & Lin, Y. (2011). Investigating effects of screen layout elements on interface and screen design aesthetics. Advances in Human-Computer Interaction, 2011, 1-10.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 318-340.

Salimun, C., Purchase, H.C., Simmons, D., Brewster, S. (2010). The effect of aesthetically pleasing composition on
visual search performance. Paper presented at NordiCHI 2010, New York, New York: ACM.