Prototype Review Questionnaire

You are participating in an initial review that is focused on identifying any usability problems with the prototype design and user interface. The results from this review will be used to revise the prototype before being subject to a final expert evaluation. Please be as detailed as possible in your responses.

The following list is a set of ten heuristic rules that are commonly followed in website design to support the goals of the user and to create a user-friendly interface. A well-designed website will adhere to these rules. Reading this set of rules before answering the questions on this questionnaire will help you to effectively identify the positive and negative aspects of the prototype design and assess the usability.

1. The website should always inform the user about what is going on through appropriate feedback within a reasonable time.

2. The website should speak the user's language with common words, phrases, and concepts with real-world conventions and information appearing in a natural, logical order.

3. Users should have control of the website and be free to navigate around. If a function is selected by mistake, there should be a clear and logical way of exiting the situation.

4. The website should use standards and be consistent. User's should not have to wonder if different words, actions, or situations mean the same thing.

5. Careful design will minimize errors and problems. The website should give enough instructions and information to the user so that desired operations and pathways can be found and executed.

6. Good labels and descriptive links will prevent the user from getting lost. Objects, actions, and options should be clearly visible. The website should not rely on users to remember information and instructions.

7. The website should be flexible and efficient to cater to both the novice and expert. The links should take users where they expect to go. Images and data should not take too long to load.

8. The website should be aesthetically pleasing and minimalistic in design. Dialogue shouldn't contain information that is irrelevant. Colors used should be appropriate and design should be elegant.

9. Error messages should be in plain language and indicate the problem precisely with suggested solutions.

10. Although it is best to design a website that can be used without help and documentation, links to help and documentation should be available to the user.

From: Nielsen, J. and Mack, R., 1994. Usability Inspection Methods.
John Wiley and Sons, Inc., New York, pg 30.

Section #1 Interface Design Characteristics Please rate the characteristics of the prototype interface on the left with the appropriate scale to the right:
PROTOTYPE INTERFACE CHARACTERISTICS	Strongly Agree	Agree	Neither Agree Nor Disagree	Disagree	Strongly Disagree
1. Size of text is appropriate:
2. Buttons are reasonably easy to click on:
3. User has good control of interface:
4. The font used is easy to read:
5. Common words are used that are understandable:
6. Information is presented in logical order without surprises:
7. It is easy to navigate around:
8. When actions are performed, appropriate and logical feedback is given:
9. The operations, actions, and responses are consistent:
10. It is designed to minimize user error:
11. The design is comprehendible:
12. The colors used are appropriate:
13. The layout is user friendly:
14. The speed of response to actions performed is appropriate:
15. The hyperlinks to other websites are relevant and useful:
16. Help and documentation are easily accessible:
17. The design is aesthetically pleasing:
18. The design is accommodating for both the novice and expert:
19. Objects, buttons, actions, and options are clearly visible:
20. The amount of information provided is adequate:

Section #2 Prototype Tools Please rate the characteristics of the prototype tools on the left with the appropriate scale to the right:
PROTOTYPE TOOLS	Strongly Agree	Agree	Neither Agree Nor Disagree	Disagree	Strongly Disagree
21. The tools are useful when working with the data:
22. The tools are recognizable:
23. It is easy to understand what action each tool will perform:
24. If I don't know what a particular tool will do, I can easily find out:
25. The text labels on each tool button are appropriate:
26. The graphics used on each tool button are logical:
27. The tools are easy to employ:
28. I can always tell what tool is selected at any given time:
29. If I make a mistake while using a tool, a rational error message is given:
30. The level of technicality that the tools provide is appropriate:

Section #3 Data Layers and Map Display Please rate the characteristics of the prototype data and map display on the left with the appropriate scale to the right:
DATA LAYERS AND MAP DISPLAY CHARACTERISTICS	Strongly Agree	Agree	Neither Agree Nor Disagree	Disagree	Strongly Disagree
31. The type of data is appropriate:
32. The amount of data is adequate:
33. It is easy to select and display a particular data layer:
34. The symbols used to represent the data are logical:
35. The colors used to differentiate data are appropriate:
36. I can easily read and understand the information displayed about the data:
37. The data display is not too cluttered:
38. The legend is legible and understandable:
39. The Overview Map is the appropriate size:
40. When the website is opened, the initial map extent (scale) is acceptable:

Section #4 Prototype Tutorial Please rate the characteristics of the prototype tutorial on the left with the appropriate scale to the right:
TUTORIAL CHARACTERISTICS	Strongly Agree	Agree	Neither Agree Nor Disagree	Disagree	Strongly Disagree
41. The tutorial is logical and understandable:
42. Instructions are clear and consistent:
43. Figures, demonstrations, and graphics are helpful:
44. I learned a lot about the tools and capabilities of the prototype while working through the spatial temporal analysis:
45. The tutorial is accommodating for both the novice and expert:
46. The amount of information provided is appropriate:
47. I feel well-informed about the tools, functions, capabilities, and usefulness of the prototype after completing the tutorial:
48. The layout of text, graphics, figures, and links is user-friendly:
49. The colors used in the tutorial are appropriate:
50. The text size and font are legible:

Section #5 Short Answer Questions Type your responses to the questions on the left in the corresponding box to the right. Please be as detailed as possible in your response:
SHORT ANSWER QUESTIONS	YOUR RESPONSES
51. What is your overall impression of the prototype design, information content, and performance?
52. Are the prototype tools user-friendly? Which tools are the most valuable for this prototype? Which are the least valuable? Why? What limitations or improvements need to be made with the tools?
53. Is the amount and type of data in this prototype sufficient? Is there any data that should be added to make the prototype more useful? Is there any data that should be removed? If so, why?
54. Does the prototype provide an efficient and usable means of accessing, querying, analyzing, and interpreting data? Are there any problems with the Data Layers section that compromise the usefulness of the prototype? What are the advantages and limitations of this prototype design with respect to data access, analysis, and interpretation?
55. What is your overall impression of the main map interface in the center of the prototype? Is the map legible at all times, understandable, easy to navigate through, and easy to analyze? Did you ever feel lost while using the map interface?
56. How easy or difficult is the task of accessing real-time USGS stream flow data using this prototype? Is the process of finding a stream gage station, river, or group of stations efficient and reasonable? Does the hyperlink take you to the appropriate USGS webpage?
57. In the second section of the tutorial, you were asked to freely explore the prototype. After this exploration, do you think that this prototype would be useful to you? Why or why not?
58. The Query, Find, and Identify tools produce information results in the gray box below the main map. Is the information displayed useful, appropriate, legible, and understandable? Is this a good location for displaying this information? Would you recommend changing any aspect of this information box?
59. In the third section of the tutorial, you were asked to compare the existing USGS California NWIS homepage with this prototype. What are the main advantages and disadvantages of each? What changes, if any, are needed to make this prototype a substantial improvement over the existing USGS California NWIS homepage? Does this prototype have the potential to replace the existing website?
60. How useful is the Internet as a means for interaction with geospatial information? Does this prototype facilitate the search, retrieval, sharing, analysis, and visualization of hydrologic data? Would this prototype be useful in decision-making within water resource management?
Please provide any additional comments. Was there anything on your mind while working with the prototype, or through the tutorial, or through this questionnaire that you have not communicated?

Housing

Prototype Review Questionnaire