Just finished the last OOo Impress prototype test, I have now done eight of them in total with different subjects from four different educational institutions. I will publish the results on Friday (and boy, is there a lot to talk about or what).
Tomorrow I will try to code together the first bits of actual code, in order to put them into the tracker, in order to gain Moodle CVS access, in order to start seriously developing :).
The discussion about scenarios has gotten into a nice-ish start. I also published some details about how the scenarios interviews were done. Although I will try to take into account all the feedback from that background data, it does seem quite far from concrete use cases or the UI. The work I am doing just on two - although quite complex - screens, still mostly seems too simple to be affected much by the high-level differences between the personas, though some changes have been made based on them, too.
Nevertheless, I have been working on the prototype again, for the most part making fixes based on the usability tests done on May 28th and 29th. I will probably do some rather quick&spontaneous prototype testing with basically anybody I can get my hands on, not to prove anymore the UI suits the tasks of teachers, but more trying to make sure the functionality is understandable, in accordance with the ideas presented about discount usability testing in Don’t make me think by Steve Krug. After that I will publish the final OOo Impress prototype.
The different areas of focus in the actual implementation and coding work are envisioned to be as follows. Note that most functionality is already present in the current Quiz - this is UI work, after all.
(What was here was moved to the Implementation plan page.)
Published this on Saturday in the Quiz forum.
For each task identified (or major tasks, or particularly special tasks if many tasks are defined), write a description of how that user would accomplish the task independent of how they would complete it within the application.
Still working on the scenarios. I would not have thought it can be such a handful.
However: slowly, but surely, I am starting to get them pretty simple and easy-to-read. Maybe someone else will bother to read them and perhaps, just perhaps even understand them enough to discuss them in the Moodle forums!
From the interviews, I have at least some data about the entire quiz creating process. However, I am going to try to just concentrate on the parts of Quiz that are relevant to creating quizzes, and then move on. That is the focus of this project, after all, plus extracting data into scenarios is a lot of work, and time is sparse.
Yesterday, I started to grasp more comprehensively the relationship between personas, use cases and scenarios.
While trying to piece this together, I drew what used to be a simple diagram. But then it got a bit out of hand:
Download diagram: PDF, Openoffice ODG
First off, there’s reality. That is to say, this is not scientific research. In a small scale guerilla usability project, we do not pretend it is, much. Still, we observe reality and gather knowledge about the users. Iteration in testing and also in interviews must serve as our backup against which we check that our conclusions are hitting their target: we take the knowledge we find to the user interface and to our generalized conclusions, and then verify with our users.
In a sense, modeling use cases equals doing actual UI design: the main difference is in the abstraction level. Use cases will be the actual hard ground when drawing prototypes, representing the idea or ideal about what the UI should be like. That is, what should be possible to accomplish with it, and how.
When designing the actual user interface, the designer takes all the use cases (=the goals), and merges them using his/her creativity and experience. As the use cases may conflict, the resulting user interface is usually some sort of a compromise between the different use cases. Also platform restrictions pose threats to the ideals: technology requirements (no dependency on Javascript in this case) and needing to conform to existing UI conventions would be examples of this.
In the case of this project, I have so far been thinking about the word “use cases” mainly from the point of view of the old user interface - I have understood it roughly as the functionality of the UI, which is to be transferred to the new UI, just better enabling the users’ scenarios (better enabling users to do their tasks). However, this is actually an overly computer-centric approach to use cases, and basically incorrect. More importantly, as is obvious in terms of the definition, use cases take the functionality of the program and model the user in interaction with that functionality.
There are several ways to see use cases. First, when we look at the current UI, you can see three different sets of use cases, from ideals to reality:
When we just have the use cases of existing software, we do not get very far. At best, with testing, we see the problems users are having with the current software. What we do not see, is whether or not we are even trying to solve the right problems.
Enter scenarios. That is: independent of the application and its UI, what is it that the user aims to achieve, and what is the way s/he does it best?
How do we find out? Talk to the users. It may be scary at first. They are people, after all! But go ahead and try - turns out that people are actually glad if someone wants to learn about what they need to make their lives easier.
Whenever you talk with actual users about their actual tasks and goals, you are in the sphere of scenarios. But the hard part is taking the scenarios and making them somehow digestible pieces. That is: you look at what the users tell you, and tear it in pieces until they are atomary, that is, single actions (you might want to observe this in terms of GTD’s Next Actions: visible, physical actions). It is not just the tasks you need to find out about, but what affects accomplishing those tasks: patterns, goals, skills, attitudes, and environment (as stated in Cooper’s article about personas, link below).
So now we have a set of use cases from the old software. Then we have a set of scenarios describing the actual environment, tasks and goals of the user. Now, we try to match them: scenario 1a fits with use case 1c, and actually, also with use case 1d (that is, the user can do the same thing in more than one way in the application). But then with scenario 1b, the user has to do a task in an unnatural order of subtasks or can’t do it at all because it is too hard for him/her or because it cannot be done at all.
And once we get that far, we can take those differences, and start writing new use cases. That is, designing the interaction that the application should facilitate. Once sufficient iteration has been achieved or the deadlines are too close, we can state: We know - well enough - the interaction and whatever affects what that interaction should be like - now let’s get to business, the actual UI.
Oh yeah, almost forgot - something brilliant: Perfecting Your Personas
This week, I have been working on the scenarios. Today, added a new project page to Docs and created new pages there for each of the major scenarios, filling in details of the scenarios according to the different personas. The pages are still far from complete but can soon serve as basis for hopefully rich community discussion.
Last Thursday and Friday, I have been to Helsinki University of Technology and University of Turku and done five wireframe/prototype tests. The results were, as always when it comes to usability tests, at the same time surprising and not at all surprising. I am indeed grateful for finding teachers and support staff from these schools to take part in interviews and testing.
The biggest surprise was that actually, adding random questions is not such a hard task - at least not if users first read a bit of introductory text (3 paragraphs) explaining the three core concepts (quiz/exam, question bank, random question). That is, in the test the setting was somewhat artificial since users were told to first familiarize themselves with the conceptual model of the application (actual usage of the application was not explained though). Clarification: Novice users did without trouble create single questions to the quiz before having read the text, though.
So, the tests gave a preliminary promise that if users understand the underlying concepts, with a reasonable UI no further documentation is needed for a simple task such as creating an exam. This was the real challenge I thought we were facing with the UI, but it seems we are getting there, though slowly. As usual, we discovered several issues, too, but more about them as soon as I get the results into a reasonable format.
It is June 2nd, the first official day of Summer Code. As I have already done a great part of the actual usability work and there still is a substantial amount of usability research to do before implementing the UI as PHP, I will publish a new project plan today. (Update: I did.) I am also planning to do my masters thesis about this project, but I have not gotten very far with that, yet.
I am currently designing semifunctional prototypes using OOo Impress, and found myself in with the challenge that while in full screen (in a slide show), I need Impress to react to only mouse clicks that hit a button or some other element.
The normal behaviour, which is taken for granted of course is that when you click on a slide you go to the next one. There is no setting for this that I could find, but it is possible. I am using Impress 2.4:
I do admit this is an overkill, and OOo really should have the feature to just disable advancing the slide show for the entire show at once. However, it works. Do report to me if it doesn’t work for you, please. :)
P.S. Once I get the prototype tested, I will publish it here next week, along with the test questions. It is, however, in Finnish (since my test persons are Finnish).
Someone working with KDE has a great open source usability blog, one of the things found was this bit about use cases and scenarios. Incidentally, this is just what I am currently working on.
Also there: The #1 Problem in OSS Usability and What I’m Going to Do About It, which Antti Kaijanmäki, another Kesäkoder, told me about some weeks ago. Though it is sad, seeing that other open source projects suffer similar problems helps keep me from going nuts :). Slowly I am becoming convinced about the critical need we have for clearly documented knowledge about our users.
Still, I guess the greatest challenge is to actually get the community thinking about users also from some other point of view than “I am the user so you should be listening to me about feature X”.
The discussion about the upcoming second prototype is heating up in the forums (alright alright, not that much yet :D). Otherwise, I have been contacting teachers, currently mostly from Haaga-Helia since they have been the most active, and trying to determine what to ask them once we get there next week.
Yes, the first interviews have been scheduled plus a usability teacher from my university, Saila Ovaska, agreed to review my plans before that. Last week’s chat with Ivar Ekman, a fellow CS student at the uni, has proved fruitful, opening my eyes to a set of issues I will still need to address.
I will probably post a new project plan at some point soon, too, taking into account that I have begun working on this already, contrary to the original plans.
Somehow I wish the process was even more transparent than it is now. I am not sure how to do that. I was inspired by Nielsen’s Guerilla HCI. How low could we take the barrier to usability work?
I feel that for me, learning about what to do in a project like this has been hard enough and I am still trying to learn more: having a concrete example about how a project like this should proceed, I dream, might help others to get a grip of practical open source usability. I would like to publish the interview material here, but the problem is that if my interviewees see the material beforehand, will it affect the actual interviews?
UPDATE: I am excited about the other discussion I found just now that is gaining momentum in the forums! This will be good :).
