Usability Test: are students able to do their tasks in LMS’s?

At the Usability test conducted last week and previously posted yesterday, I’ve got an opportunity to look at the data and

some stats and for the ratings variable these are the numbers I got from 4 tested students.

Task 1 [Logging in] (measures if users were able to log into the system and access the proper course by with provided

instructions)

Blackboard [ 94 % ]

Angel [ 100 % ]

Moodle [ 100 % ]

eCollege [ 100 % ]

Task 2 [Content Access]  (measures if users were able to access syllabus, welcome letter and learning unit)

Blackboard [ 88 % ]

Angel [ 94 % ]

Moodle [ 94 % ]

eCollege [ 88 % ]

Task 3 [Discussion]  (measures if users were able to access discussion and reply to a topic)

Blackboard [ 100 % ]

Angel [ 100 % ]

Moodle [ 94 % ]

eCollege [ 94 % ]

Task [4] Assignment (measures if users were able to access assignment area and submit it with an attachment)

Blackboard [ 42 % ]

Angel [ 92 % ]

Moodle [ 79 % ]

eCollege [ 46 % ]

Task [5]  Quiz (measures if users were able to access quiz area and take it)

Blackboard [ 75  %]

Angel [ 94 % ]

Moodle [ 94 %]

eCollege [ 94 % ]

Task [6] Grades (measures if users were able to access and understand their grades)

Blackboard [ 75 % ]

Angel [ 100 % ]

Moodle [ 100 % ]

eCollege [ 100 % ]

Task [7] eMail (measures if users were able to access the email area, compose a message, add recipients and send it)

Blackboard [ 75 % ]

Angel [ 90 % ]

Moodle [ 100 % ]

eCollege [ 100 % ]

Task [8] Help (measures if users were able to access a help topic)

Blackboard [ 100 % ]

Angel [ 100 % ]

Moodle [ 25 % ]

eCollege [ 88 % ]

In continuing the  Usability test results series one of the variables looked at measuring if students were successful (able to) completing daily student-tasks. 4 students were tested.

Thank you Larry and Ray for your comments on previous postings. We are looking to usability-test more students. It was quite of a challenge to have students to participate in this tests because in community college reality we couldn’t offer much rewards for their participation. They would say “I don’t want to participate because your rewards is too low”.

I’m also looking forward to our accessibility test in a few weeks. The plan is to have special needs users to test each LMS – Angel, Moodle, Blackboard and eCollege. And again… I’ll be postings each result in here.

In the usability test, we gave students a set of 8 tasks listed below and measured their success of completing each task. The numbers point the percentage rate they could complete a given task in each LMS.

Task 1 [Logging in] (measures if users were able to log into the system and access the proper course by with provided  instructions)

Blackboard [ 94 % ]

Angel [ 100 % ]

Moodle [ 100 % ]

eCollege [ 100 % ]

Task 2 [Content Access]  (measures if users were able to access syllabus, welcome letter and learning unit)

Blackboard [ 88 % ]

Angel [ 94 % ]

Moodle [ 94 % ]

eCollege [ 88 % ]

Task 3 [Discussion]  (measures if users were able to access discussion and reply to a topic)

Blackboard [ 100 % ]

Angel [ 100 % ]

Moodle [ 94 % ]

eCollege [ 94 % ]

Task [4] Assignment (measures if users were able to access assignment area and submit it with an attachment)

Blackboard [ 42 % ]

Angel [ 92 % ]

Moodle [ 79 % ]

eCollege [ 46 % ]

Task [5]  Quiz (measures if users were able to access quiz area and take it)

Blackboard [ 75  %]

Angel [ 94 % ]

Moodle [ 94 %]

eCollege [ 94 % ]

Task [6] Grades (measures if users were able to access and understand their grades)

Blackboard [ 75 % ]

Angel [ 100 % ]

Moodle [ 100 % ]

eCollege [ 100 % ]

Task [7] eMail (measures if users were able to access the email area, compose a message, add recipients and send it)

Blackboard [ 75 % ]

Angel [ 90 % ]

Moodle [ 100 % ]

eCollege [ 100 % ]

Task [8] Help (measures if users were able to access a help topic)

Blackboard [ 100 % ]

Angel [ 100 % ]

Moodle [ 25 % ]

eCollege [ 88 % ]

Re-evaluating the LMS Market

A number of educators, instructional designers, course developers and other Angelusers have expressed theur apprehension or skepticism in regards to the Blackboard merger. In  It seems Blackboard is working at its best PR to convince Angel clients according to The Chronicle’s  Blackboard and Angel Learning Officials Try to Reassure Skeptical Clients. eSchoolNews also had previously published an article highlighting some concerns from educators about the BB-Angel mergehttp://www.eschoolnews.com/news/top-news/news-by-subject/technologies/?i=58675

Jared Stein publishes in his post … a very interesting note:

What once looked like good news for Angel has become a nightmare for an CMS consumer market in need of diverse and innovative choices, and a looming shadow for other CMS producers, as Blackboard continues to thwart and denigrate healthy competition in the field of e-learning.

I’ve been sharing some thoughts with peers and colleagues, additionally to the college institution I work with. Many who were WebCT clients bear a remembrance of what Blackboard-acquisition and migration processes meant to them. System support, customer service, program conflicts…and so on.

Blackboard has made a number of past acquisitions of which most of these previous tools are no longer supported after about 3 years – in average. The predatory strategy has reached Angel’s board committee where an apparently “successful” negotiation has led the Indianapolis-based company to decide for selling its rapidly-growing business for $95 mi. Angel must’ve been somehwat a “burden” for its committee board [in debt or its profit was not meeting its expectations - or any other deep reason]. The real reason behind the scenes we will never know. But for a greater audience the acquisition was a disappointment.

Some say that if your institution is undecisive about a new LMS purchase due to Angel purchase by Blackboard, they will go to Moodle. However, how would institution that needs support would move with the open source tool? They could hire a Moodle partner for servicing.

What are other LMS’s around? eCollege has a different focus from Angel and Blackboard. Its focus is in large, massive online course distribution as opposed to faculty-student personalized interaction. It doesn’t have as many features as Angel,  Blackboard and Moodle.  I have not had the opportunity to try D2L yet. Can’t really say much about it. Ideas anyone? Is this the end of competitive market? Is it monopoly?

Some folks suggest that as opposed to improving their own systems or innovating with their own and proper ideas, they are trying to get it from somewhere else, like Angel. it could be. But Blackboard is not the only character in this plot. Angel “agreed” with the acquisition.

http://intelligentdistancelearning.wordpress.com/2009/05/13/blackboard-buys-angel-competition-what-competition

http://economicalthoughts.wordpress.com/2009/05/07/the-death-of-competition-in-learning-management-systems/

http://centraliacollege.wordpress.com/2009/05/13/blackboardangel-update/

http://rmostell.wordpress.com/2009/05/07/blackboard-buys-angel-learning/

http://randallcase.wordpress.com/2009/05/10/blackboard-buys-angel/