Factors in the Acceptance of Computer Based Assessment by Men and Women

June 29, 2014 / / No Comments

Back for a third time today, folks. As I said before, it’s crunch time and there are many readings to be reflected upon in these last few days before the final deadline for this class. Once this outpouring of blog posts ends, I will be attempting to post regularly on topics of my own choosing. Some will come from my next class in the masters program, some will be topics and articles that I just genuinely wish to spend time discussing. No matter what this blog turns its attention to, I hope that it continues to be interesting and thought-provoking to the readers who are kind enough to spend their time reading and considering my thoughts and those of the authors whose articles I present here. Now, let us move on to the topic of this blog post.

Today I will be discussing cyber assessment. Cyber assessment is a broad term and can be used to describe anything from using technology to assess whether writing is the result of plagiarism to the more literal usage of e-learning and digital tests/quizzes. In my current course, our dear instructor allows us to submit our quizzes for the class in a typed format. This is a fine example of the more literal definition of cyber assessment. While we did read articles about data mining being used to detect plagiarism, as well as a comparison of the response rates between online and paper-based surveys, these two topics were not particularly interesting. The article that I did find intriguing was the one by Terzis and Economides, which explores the differences between men and women in their perceptions and acceptance of computer based assessment (CBA).

Terzis and Economides (2011) had previously created a Computer Based Assessment Acceptance Model (CBAAM) that uses eight factors in order to define behavioral intention (BI) to use a CBA. These factors are Perceived Playfulness (PP), Perceived Usefulness (PU), Perceived Ease of Use (PEOU), Computer Self Efficacy (CSE), Social Influence (SI), Facilitating Conditions (FC), Goal Expectancy (GE) and Content (pg. 2110). For those that read my previous post and remember Davis’ research, you know that PEOU and PU were the main factors that he said determined acceptance of any type of information technology by a user. Here, Terzis and Economides state that their model shows that behavioral intention to use a CBA is strongly correlated to PP and PEOU. They also state which of the other aforementioned factors significantly explains Perceived Usefulness and Perceived Playfulness. According to the research, PU is significantly explained by Goal Expectancy, Content, Social Influence and Perceived Ease of Use while Perceived Playfulness is defined by Usefulness, Content, Ease of Use and Goal Expectancy. The authors then add that Computer Self Efficacy and Facilitating Conditions also help to determine Perceived Ease of Use (p. 2110).

In this study, Terzis and Economides (2011) wanted to determine the differences between men and women in the factors that affected their perception and acceptance of a CBA. They formulated hypotheses based on a vast amount of research in a variety of areas. I do not want to elaborate on all of it, but one example is their hypothesis that PP influences behavioral intention to use a CBA more in men than in women. They based this on research that determined that men expressed more positive feelings toward multiple choice assessments than females and also because previous studies found that men enjoy playing computer games more than women, meaning that men’s characteristics are more aligned with CBA’s game orientation (p. 2112).

Ultimately, the research done by Terzis and Economides showed some very interesting things. Before we talk about gender differences, I would like to highlight some of the general observations. First, the authors state that Social Influence is a strong determinant of PU. Remember when I previously talked about those who might use Facebook just because everyone else is? As it turns out, there is correlations with Davis’ PU factor after all. In other words, even if a user would not necessarily perceive a site as useful on their own, they may already have that perception in their mind upon their first visit if they know that a lot of other people are using the same system (p. 2116).

The authors also noted that Goal Expectancy shows that students who are prepared and expect to be successful in a course are more likely to find a CBA useful as well as playful. They also noted that while Content has no direct effect on BI, it does influence PP, PU, and GE. This indirect effect on BI means that a CBA that contains content that is clear and interesting is more likely to be used. Finally, the research did not show any correlation between BI and PU like previous research, but an indirect effect exists here as the correlation between PP and PU was high. As mentioned above, this research showed that BI is most strongly correlated to PEOU and PP, not PU like with Davis’ study. Since PU has a high correlation with PP though, it must be considered when designing the CBA (p. 2216).

As for the gender differences, the authors found that the male students are more influenced to use a CBA through Playfulness, Usefulness, Content and Social Influence. This means that if the creators of a CBA want to influence male students to use the CBA, they have to make the CBA playful, they must make it useful for enhancing male students’ knowledge and performance, they must ensure that it delivers both appropriate and clear content, and they must make the CBA something that will be recommended and suggested by their peers and their teachers (p. 2119).

The female students are mainly influenced to use a CBA through Playfulness, Ease of Use, Content and Goal Expectancy. This means that Playfulness and Content are important for both males and females, but the authors note that these factors do not matter to the same degree as they do for males. On the other hand, female students are influenced by Ease of Use and Goal Expectancy, unlike the male students, and not by Usefulness and Social Influence as the male students were. This means that creators of a CBA who want to influence female students to use it must create a CBA that is easy to use with a simple design. The CBA also has to pique the interest of the female students in a way that maximizes their preparation and raises their Goal Expectancy (p. 2119).

Using this research, educators who wish to create and utilize CBA’s in their classrooms can understand the factors that affect students as a whole, as well as the individual factors that matter more to men than women and vice versa. This understanding can lead them to create better CBA’s that all of their students want to use. Without this knowledge, utilization of a CBA might be fractured amongst a given student population, resulting in less than optimal outcomes and perhaps even a reduction in willingness to use a CBA in the future by the instructor themselves.

 
References

Terzis, V. & Economides, A. A. (2011). Computer based assessment: Gender differences in perceptions and acceptance. Computers in Human Behavior, 27, 2108–2122.

Thumbs Up, Thumbs Down: A Siskel and Ebert Approach to Website Reviews

June 29, 2014 / / 2 Comments

Hello again, readers. This second post for today is going to again deal with the screen layout literature that I previously spoke about in the first, but in a much different way. Using this literature as a basis, I have been tasked with reviewing two websites of my own choosing with the caveat being that one must be a website that I approve of while the other should be one that I do not. Naturally my satisfaction or lack thereof is to be based primarily on their layout and design.

The factors that helped me to choose my “thumbs up” website are the factors that are practically nonexistent in my “thumbs down” site. For those that read my previous blog post on the topic, you should already be familiar with these factors, but I explain them here again for the purpose of this assignment, which I am to write in a format resembling an academic paper while being allowed to present it on this blog. I have also cited literature that I did not discuss on the first post from earlier today, so there are fresh elements to keep things interesting.

Thumbs Up Website – Edmodo

I choose Edmodo as my website that I feel exhibits many of the features of good layout and design. I presented this same website in class as part of a show-and-tell of useful technology in which my fellow peers and I took turns introducing and explaining in detail some sort of technology that we felt our peers might not be aware of or, at the very least, not have a deep understanding of.

Edmodo, at its core, is a website designed to help teachers increase the efficiency of their classrooms. Teachers can create groups for each of the classes they teach and then invite their students to join these groups, and the service itself, without students ever having to give their personal email addresses to the teacher. This certainly eliminates any concerns over privacy issues that academic institutions might have, particularly at ones where students are not given a school email address. Once both teacher and students are signed up for Edmodo, the teacher can use the site to give students assignments, send reminders, create quizzes and tests, and much more. Students, in turn, can submit assignments via file upload, takes quizzes and tests, post questions for the teacher, etc. Teachers can also mark submitted work and assessments right on the site and have the results shown to their students immediately. It is this kind of functionality that contributes to the perceived usefulness of the website, which is now being utilized by thousands of teachers around the world (Davis, 1989).

Edmodo is also as simple or as complex as you wish to make it. The decision of which features to use is entirely up to the teacher and nothing will appear under the group for a given class unless the teacher specifically creates and posts it. Until that time, icons for creating things such as assessments, polls, and assignments sit up at the top of a class’ group page, completely out of the way and undistracting as can be seen in Figure 1 below. It is this design choice that also gives Edmodo its perceived ease of use, as teachers can find the tools they want to use quickly, but are not distracted by unused elements taking up screen space (Davis, 1989).

edmodo1
Figure 1: Screenshot of an Example of an Edmodo Group

If we need a reason to further delve into the aesthetic side of Edmodo, we can refer back to the work of Kurosu & Kashimura (1995), who discovered a high correlation between interface aesthetics and perceived usability. It was Altaboli & Lin (2011), however, who defined what makes a website aesthetically pleasing, citing the three elements of unity, balance, and sequence as necessary in a site’s design in order for it to be considered so. Looking at Figure 2 below, Edmodo clearly exhibits these traits. It has uniformity across all pages in terms of fonts, colors, and even icon design, which utilizes an outline-type design with a great amount of empty space in the center and no color fill. Balance is achieved by the use of significant white space on the sides to keep the main elements of each page centered while also using a smaller amount between elements to keep them from blending together. Sequence is present as the page naturally flows from left to right and top to bottom as one would expect. Icons, as previously mentioned, are placed in locations that make sense without being distracting.

edmodo1
Figure 2: Screenshot of Edmodo’s Main Screen After Login

Because Edmodo follows a design style typical of many popular social networking sites such as Twitter and Facebook, it can be said that it has a high level of prototypicality. Everything is where we have come to expect certain items to be, with the features relating to the main site itself, such as the home button and search field, located in the upper left and continuing along the top, while navigational menus that are found once you are on a group page are located to the left. Even the icons for creating the various content items are set up much like the status update field of Facebook, located directly under the main site icons at the top and complete with tabs to change the type of content we are creating. This high level of prototypicality keeps users from wasting time getting their bearings on a page and allows them to begin adding or editing the content they are interested in immediately (Tuch, Presslaber, Stocklin, Opwis, & Bargas-Avila, 2012; Roth, Tuch, Mekler, Bargas-Avila, & Opwis, 2013). In my own usage, I have found that I can easily navigate to the group for the class I want upon logging in and from there immediately find the assignment submissions I wish to view or create the content that I need to send out to my students.

Overall, I believe this combination of perceived usability, perceived usefulness, and the aesthetic features that help to bolster these things are a driving force behind Edmodo’s consistently growing number of users. The fact that Edmodo is also fully customizable so that teachers can tailor it to their needs while also being a completely free service simply increases the overall satisfaction that this site gives its user base from the very first time they sign on.

Thumbs Down Website – Yahoo Japan

I chose Yahoo Japan as my “thumbs down” site as we had discussed it briefly in my class and the discussion we had about it was interesting so I wanted to bring that discussion here to my blog as well. Yahoo Japan seems to suffer from, quite simply, being a Japanese website. In fact, I, and many of my classmates, are convinced that a majority of Japanese websites are built on the premise of trying to fit as much content onto the screen as possible at any given time. Figure 3, which follows this paragraph, clearly demonstrates this apparent design choice. The American version of Yahoo is certainly not this cluttered.

yahoo1
Figure 3: Screenshot of the Yahoo Japan Homepage

The only saving grace of Yahoo Japan is in the fact that it uses white space on the sides to center content, much like Edmodo. Unfortunately, I cannot say anything positive aside from this in terms of aesthetics. The abundance of icons makes finding the content that I want akin to a word search. The small font size also greatly reduces the readability of the website and I am a younger man viewing the site on a Macbook Pro that has a high resolution Retina display. I can only imagine how horrible it must be for someone older viewing the site on a low resolution monitor. The content that the home page displays from the beginning is also seemingly random, with no clear sense behind why the many boxes are in the order in which they appear. At the very least, the search bar is where one would expect, but aside from this, Yahoo Japan has an incredibly low level of prototypicality.

If Yahoo Japan wants to turn things around, they may want to first mirror their American sibling, which has considerably less items in its navigational menus and employs much larger font sizes for headlines and other text. I think that even the U.S. site could use more white space between articles on its main page, but I understand that it is a news site and it wants to still keep at least 4 or 5 articles on the screen at any one time. It is still leaps and bounds above Yahoo Japan in terms of allowing me to find content that I am interested in and navigating back and forth between various pages on its site.

It would be easy to condemn the designers of Yahoo Japan if it were not for the fact that popular Japanese sites such as mixi are also cluttered messes. It makes me wonder what those that pursue web design as a career in Japan are taught inside the classrooms of their vocational schools. As someone who is well-versed in web design, I am sure that the answer might horrify me, but I would love to hear back from you and get your opinions on the matter, dear readers. Is this an actual phenomenon in Japanese design or am I just perceiving it as such because the two sites I mentioned just happen to be some of the most popular in Japan in recent history? I would also appreciate links to some well designed Japanese websites. While I do not believe all Japanese websites suffer from this problem, having actual evidence of such would be nice as my own web searches are seeing my inner web designer sink into depression.

 
References

Altaboli, A., & Lin, Y. (2011). Investigating effects of screen layout elements on interface and screen design aesthetics. Advances in Human-Computer Interaction, 2011(5), 1-10.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319-340.

Kurosu, M, & Kashimura, K. (1995). Apparent usability vs. inherent usability: experimental analysis on the determinants of the apparent usability. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM Press. http://portal.acm.org/citation.cfm?id=223355

Roth, S. P., Tuch, A. N., Mekler, E. D., Bargas-Avila, J. A., & Opwis, K. (2013). Location matters, especially for non-salient features–An eye-tracking study on the effects of web object placement on different types of websites. International Journal of Human- Computer Studies, 71(3), 228-235.

Tuch, A. N., Presslaber, E. E., Stocklin, M., Opwis, K., & Bargas-Avila, J. A. (2012). The Role of Visual Complexity and Prototypicality Regarding First Impression of Websites: Working Towards Understanding Aesthetic Judgments. International Journal of Human- Computer Studies, 70, 794-811.

Using Interface and Screen Layouts to Increase User Base

June 29, 2014 / / 2 Comments

Hello, readers. Long time, no see. Crunch time at the end of the term both at work and at grad school had severely limited my blogging time, but you will have a number of posts to look through as I focus my efforts on meeting deadlines for my class. As usual, I will not rush anything to publication so if any of the topics are covered briefly, it is only from my own lack of interest in fully engaging with it. Today’s post will certainly not fall under that category as I am extremely interested in screen and interface layout as a web designer. Why should you care about this topic if you are an educator with no professional-level foray into game creation or web design? Well, if you are following this blog and want to be a teacher that incorporates as much technology into their teaching style as possible, then you will most likely create a web page, however simplistic in design, at some point. That or you will be advising students who are creating a web page for a project that they are doing for your class. You may never develop a game for educational use, although I will attempt to convince you to try in an upcoming post, but having working knowledge of what makes for a good screen/interface layout is helpful. It is certainly one of the first things that you must consider before creating a web page, and even more important if you are crafting an entire website.

I will look at a number of articles that I found interesting on this topic, probably covering about three of the eight that were read for the class. Each article focuses on slightly different elements or has differing results from what the researchers expected based on previous research done in that area. The oldest of the articles is from back at the end of the 1980’s. Davis (1989) wrote back then about how the perceived usefulness and the perceived ease of use of any kind of information technology was a critical factor in that technology being accepted and widely used. Keep in mind that we are talking about perceptions here. Regardless of the truth of the matter, Davis’ study focused on whether a user believed a given system enhanced their job performance as well as whether they believed using a system to be effortless (p. 320). Davis is even careful to warn readers at the end of his article that the results are not based on objective reality, only the subjective appraisal of the participants (p. 335).

The fact that this study was done on electronic mail is, in itself, interesting, but let us take a moment to think of the real implications here. Even if an interface’s or a website’s layout is complex beneath the surface, with numerous functions and capabilities, if it looks easy to use and users can see a real purpose for its existence, they will become regular users. For us educators, we may want our students to use a webpage or website that we construct to go along with our class, or we may have designed a simple game to help them study something like vocabulary. From Davis’ research, an educator looking to do something along these lines must, at the onset of the design phase, start considering how they will construct something that their students will instantly see as beneficial while also not requiring any sort of a learning curve.

Consider Facebook for a moment. I have not yet begun to cover social media, but nevertheless, it can be used for the purpose of our discussion here. At its core, Facebook has a great number of features that have to be learned if you really want to master it, but at first glance, it looks surprisingly easy. I find my friend, I click on a button to “add” them, and when they accept my request, I begin seeing all of their posts, pictures, etc. on my News Feed as soon as I visit the site. I can easily see how useful Facebook is for keeping up with people that I might not have many chances to meet in person so that drives me to continue using it. Of course, I may find out later that I need to learn how to upload photos, create albums, and play games with friends (which I personally avoid like the plague), but I became a user because it all seemed simple enough and I saw value in the system. Now I cannot claim that it is all due to the two factors Davis found, as many people are likely to do something just because everyone else is, but I am sure that these factors still contribute, particularly for those who may not be as comfortable with computers and the Internet, yet who still use Facebook.

Altaboli and Lin (2011) focused more on screen aesthetics in their research and how these affect a user’s perception of what they are being presented with. In the article, they mention previous research done by Ngo et. al. that proposed fourteen measures of screen aesthetics. These measures were balance, symmetry, equilibrium, unity, sequence, density, proportions, cohesion, simplicity, regularity, economy, homogeneity, rhythm, and order (p. 2). Altaboli and Lin, however, decided to focus only on balance, unity, and sequence based on the findings of earlier research by Ngo and Byrne. Completely subjective as my opinion is, I agree that these elements are factors that are present on websites that are aesthetically pleasing, whereas the other elements such as economy and proportions do not seem as crucial.

What I took issue with, as did those classmates with whom I discussed this article, is the experimental design used by Altaboli and Lin. In fact, much of it has to do with the participants, of whom there were only 13 (p. 4). That alone is not enough to obtain statistically valid results, although Altaboli and Lin claim otherwise (p. 5). Among some of the other grievances is that the participants included only three women, the age range of the participants was limited, with a median age of 29.3 years of age and a standard deviation of only 6.1 years, and finally, all of the participants were graduate students of engineering (p. 4). I also took issue with the fact that one of the screens that should have scored low because it did not possess any of the characteristics mentioned above actually scored quite high, yet Altaboli and Lin eventually write it off as experimental error, saying that effects of some of the other fourteen elements should later be investigated just to be sure that those are not causing the result (p. 6).

Still, I did like that Altaboli and Lin created a regression model and utilized it along with already existing survey data for 42 websites, despite their formulas originally being designed for data entry screens. The result was that they found correlation between the values computed by the model and the scores on the website questionnaires for items that dealt with the visual layout of the site (pg. 7). I would like to see them redo their experiment with a larger variety of participants and then use that data along with the website questionnaires, which did have an adequate number of participants as well as adequate variety. Until then, I can only continue to subjectively look at websites that I find aesthetically appealing and check if the three elements singled out in this study are present or not.

Finally, we have the study done by Salimun et. al. (2010). This paper was interesting to me because another paper coauthored by Salimun (in the same year, no less) had indicated that users actually preferred a medium level of aesthetics for an interface, yet this paper indicates that the higher the aesthetic level of an interface, the better the response time a user would have with a searching activity (Conclusion section, ¶2). In practical application, this would correspond to how fast a user could find what they are looking for on a web page, for instance. In this study, six elements were looked at compared to the three from Altaboli and Lin’s study. The six elements included cohesion, economy, regularity, sequence, symmetry, and unity (Methodology section, ¶1). Thus, two of the three elements from Altaboli and Lin’s study are present.

Again, I found that the number of participants was quite low, at only 22, but they were split evenly between males and females and they all came from different backgrounds of study. Unfortunately, they were also all computer literate and used computers daily, whereas I would have liked to have some participants who rarely used computers participate (Participant section, ¶1). Nevertheless, we can gather from this study that a website or interface with a high level of aesthetics as measured from the usage of the aforementioned six elements would have a greater chance of increasing user response time than one that does not. If we indeed believe in the study’s results and design our project using these measures, assuming it produces the intended results, then it would also likely increase the perceived ease of use. Users would quickly find what they are looking for rather than getting lost during navigation, which generally causes frustration. As I mentioned with Davis’ study at the very beginning, perceived ease of use is what draws users in and keeps them coming back. While the use of aesthetics may not cover the perceived usefulness aspect of Davis’ study, it does cover the other half and is thus something that must be considered when designing any site, application, or game if it is to succeed.

 
References

Altaboli, A. & Lin, Y. (2011). Investigating effects of screen layout elements on interface and screen design aesthetics. Advances in Human-Computer Interaction, 2011, 1-10.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 318-340.

Salimun, C., Purchase, H.C., Simmons, D., Brewster, S. (2010). The effect of aesthetically pleasing composition on
visual search performance. Paper presented at NordiCHI 2010, New York, New York: ACM.