Visit the SALMON Bookshop for recommended books on this topic
SALMON won the 1998 Universities and Colleges Information Systems Association (UCISA) Teaching and Learning Award for demonstrating innovative or exemplary use of the Web
Rationale for Delivering Questionnaires via the Internet
The purpose of this web page is to describe how to go about putting a questionnaire on the Internet. Although this is a relatively straightforward process it may be worth answering the question "Why bother?".
Until recently most psychological questionnaires were delivered to users on paper, and the experimenter was faced with the daunting task of manually entering each user's replies into a computer prior to data analysis. This is a tedious and error-prone exercise that we have all done and hated! One advantage of computer based questionnaires is that the data can be collected and stored with a minimum of intervention by the experimenter thus reducing the risk of 'transposition errors'. This type of computer-based questionnaire has been used previously but mounting questionnaires on the Internet offers a further advantage.
Before the advent of Internet-based questionnaires the experimenter was restricted to collecting data from subjects that they actually came into contact with. Usually this meant recruiting subjects on your campus, close neighbourhood, or asking a colleague at a remote site to recruit subjects for you. Internet-based questionnaires overcome this problem by allowing people from all over the world that you have never met to participate as subjects in your study.
But like everything in life there are snags: In order to participate in your study respondents must have access to a computer with a high speed link to the Internet. This is a problem at the moment - most suitable computers are located in universities or owned by a small number of relatively wealthy people. Hopefully this will change. In the early days of any new technology, the number of users is relatively small, but useful things like telephones, motor cars and TV sets have a habit of being widely adopted across the world.
Nevertheless you should be aware that Internet-users are NOT a random sample of humanity - undergrad students aren't either, but that hasn't stopped their use in a lot of psychology experiments. But more seriously, you don't have the control over who fills in your questionnaire when you put it on the Internet. It is possible to restrict access to your questionnaire using passwords etc. but that is beyond the scope of this document.
Before
converting your questionnaire into a form that can be viewed by users
on the
Internet, you need to have the basic understanding of HyperText
Markup
Language (HTML)
Here is a template for a generic html document:
<html>
<head>
<title>Generic_Page</title>
</head>
<body>
<h1>Generic_Header</h1>
My first web page
</body>
</html>
All of this should make sense to you, if not you need to read the guides to HTML before you go any further. Visit the SALMON Bookshop for recommended
books on this topic
In order to make your web page visible to other users of the internet, you must place it on a special computer called a web server.
Launch the application WS_FTP
to communicate with the web
server used in this module.
Hit the 'Connect' button to bring up the dialogue that allows you to set various ftp session properties
| Click
'New' to create a new profile.
Type in these properties if you are taking PSY224 The password will be provided in the workshop session |
Click
'New' to create a new profile.
Type in these properties if you are taking PSY537 The password will be provided in the workshop session |
 |
 |
Below is a screen shot showing the contents of
arrow to
transfer your HTML file from your Local System to your folder on the
Remote System |
You must check that the file you have transferred in available on the web. Using any computer open the file in a web browser. The URL for your file is:
The values in bold will vary between students
Quick start: Template Questionnaire
Now that you have read how to do it you may want to download and save a template questionnaire with blanks for you to fill in. The template contains several examples of radio buttons and free text entry boxes.
 |
Open the template in an HTML editor and examine the relationship between the underlying HTML and the browser display.
 |
Feel free to modify and extend the HTML to meet your specific needs.
Open the template in an HTML editor and examine the FORM tag
 |
The FORM tag instructs the page to execute a PolyForm script (set of instructions) when the user clicks the Submit button on the web page. In this example the script has the name 'pktest'. The instructions executed by the script are stored on a computer called 'salmon.psy.plym.ac.uk'. PolyForm is the name of the programme on salmon.psy.plym.ac.uk that executes the commands contained in a script.
Scripts are used to capture the entries made by the user of a web page and store them on a computer. For example, the script 'pktest' captures the users: nationality, age and sex.
I have set up individual scripts for each student taking the module. The script name corresponds to your student number. For example, if your student number is student1, your script is student1, if your student number is student20, your script name is student20.
Retrieving results saved by scripts
I have set up your scripts to save your results in a 'comma separated value' file on salmon.psy.plym.ac.uk.
Comma separated value files are files with the extension .csv
In a csv file, a comma separates each data field.
Your results are saved in a file named studentn.csv where the value of n corresponds to your student number. If your student number is 1 your results are stored in student1.csv, if your student number is 20 your results are stored in student20.csv
In order to view your results you must use WS_FTP to connect to salmon.psy.plym.ac.uk which is the computer which contains all the results files. Note that results files and HTML files are stored on different remote computers.
Launch WS_FTP and click the
Connect button.
| Click
'New' to create a new profile.
Type in these properties if you are taking PSY224 The password will be provided in the workshop session |
Click
'New' to create a new profile.
Type in these properties if you are taking PSY537 The password will be provided in the workshop session |
 |
 |
Locate you results (studentn.csv) file on the Remote System; highlight the file, and click the View button to see its contents
 |
So far you have learnt how to:
You will use these skills many times during this module as you refine your web questionnaire.
Once you have reached this stage of the module you may wish to work at home between weekly classes. This is possible if you have:
You can obtain the necessary software from a number of sources e.g.
Please note that I cannot give you advice on setting up modems, ISP accounts, or installing software on your home computer.
Using INPUT to capture research data
The <INPUT
...> tag is
used to set up elements on a web page which capture data from the user.
The template questionnaire uses two type of input fields:
 |
In our example the code:
<<NPUT TYPE="radio" NAME="02sex"
VALUE="female"
>
means that the value of the variable "02sex" is set to" female" if the
user selects this radio button.
If you want, you can have a CLEAR button that allows users to discard their answers and start over again. I don't use this facility on long questionnaires because if users hit it by mistake they would loose a lot of work. But it is useful on shorter questionnaires.
A modified INPUT element is used to create "submit" and "clear" buttons. It is pretty much the same old INPUT, but with a different set of TYPE attributes which control what kind of a field it is.
| <input type="submit" value="Send In This Form"> | Use the submit type to submit the form. |
| <input type="reset" value="Clear"> | Use the reset type to clear the form and reset all fields to their default values. |
The clear button doesn't actually clear the form, it merely resets all the fields to their default values. If the field started out as empty it will be returned to the empty state. However fields starting out with text in them, will have that text restored to them if it has been modified.
The importance of variable names
You may be wondering why each name in our example is in the form
You MUST place two digit numbers at the start of variable names so that your data is correctly stored in studentn.csv by your script. The number must be in the form of two digits i.e. 01,02,.....98,99.
Using HIDDEN fields to collect session information
Hidden fields, as their name suggests, are hidden from users of your form. Use hidden fields to record when your questionnaire was filled in. Hidden fields also allow you to record the unique identification (RemoteAddress) of the number of the computer that submitted data. This allows you to check if a machine has been used to submit multiple replies to your questionnaire. This may be useful if you suspect that someone has tried to submit multiple replies to your questionnaire.
<INPUT
TYPE="hidden"
NAME="51date" VALUE="pfDate" >
<INPUT TYPE="hidden" NAME="52longdate"
VALUE="pfLongDate" >
<INPUT TYPE="hidden" NAME="53time" VALUE="pfTime" >
<INPUT TYPE="hidden" NAME="54TransactionID"
VALUE="pfTransactionID" >
<INPUT TYPE="hidden" NAME="55RemoteHost"
VALUE="pfRemoteHost" >
<INPUT TYPE="hidden" NAME="56RemoteAddress"
VALUE="pfRemoteAddress" >
<INPUT TYPE="hidden" NAME="57Referer"
VALUE="pfReferer" >.
You can check this out
Using Excel to check result files
It is absolutely vital that you check that your studentn.csv file contains the replies entered by users of your web questionnaire.
Here is a screenshot of users' responses stored in a CSV file that has been imported into an Excel spreadsheet.
The variable names given to each question have been added along the top of the spreadsheet: Nationality, sex, age, A1,A2 etc. Note the "hidden" variables reported by PolyForm.
 |
Note that you have to insert variable names into the .csv file - they are not automatically included by the PolyForm script.
You can use this data to do whatever statistical calculations you want: average age, age range, number of male and female respondents, mean response to each question etc.
Checking the Validity of Data Returned by PolyForm
You
must take steps to ensure that the replies to your questionnaire are
accurately
captured by your PolyForm script.
The following screenshot shows one way of doing this.
 |
| The Excel function
RANDBETWEEN(bottom,top) returns a random number between the numbers you
specify.
A new random number is returned every time the worksheet is calculated. If this function is not available, run the Setup program to install the Analysis ToolPak. After you install the Analysis ToolPak, you must enable it by using the Add-Ins command on the Tools menu. RANDBETWEEN(bottom,top) Bottom is the smallest integer RANDBETWEEN will return. Top is the largest integer RANDBETWEEN will return. |
| The Excel function IF
returns one value if a condition you specify evaluates to TRUE and
another value if it evaluates to FALSE.
Use IF to conduct conditional tests on values and formulas. Syntax 1 IF(logical_test,value_if_true,value_if_false) Logical_test is any value or expression that can be evaluated to TRUE or FALSE. Value_if_true is the value that is returned if logical_test is TRUE. Value_if_false is the value that is returned if logical_test is FALSE. |
The screenshot shows you how you can use a formula that compares the two sets of responses to automatically highlight discrepancies.
The formula used is IF(randomly generated value = returned value,"OK","error")
If you
have problems consider:
Sorting results in Excel
You can use Excel to sort data in studentn.csv prior to statistical analysis.
|
 |
 |
 |
Excel's Chart wizard to create
a chart of your data| Syntax:
AVERAGE(number1,number2, ...) Number1, number2, ... are 1 to 30 numeric arguments for which you want the average. Examples If A1:A5 contains the numbers 10, 7, 9, 27, and 2, then: AVERAGE(A1:A5) equals 11 |
 |
Using JavaScript to verify FORM input
We can use JavaScript to verify the data entered into HTML Forms.
For example, imagine a
situation in which you
wanted to check that participants were over 18 years of age before they
proceeded with a
questionnaire.
The file event_handler.html contains a JavaScript function which is called when a user enters their age. If they are over 18 they are allowed to proceed. Under 18 year olds can be warned not to participate in the experiment.
This diagram shows the code in event_handler.html that controls this process.
The participant is asked to type in their age. As soon as they hit the 'return' key the value is checked. If it is 18 or more, an alert window pops up to confirm that they can proceed. Then the browser displays the file proceed.html
If the value entered is less than 18, a message pops up to warn them that they are too young to participate, and the file terminate.html is displayed.
You can suppress the pop-up window by inserting the characters // before alert(You ...
Modify the contents of these files to meet your needs.
Using JavaScript to check a password
You may want to restrict
entry to your experiment by
distributing a password to selected participants.
For example, you might advertise the availability of your questionnaire on the web and invite interested people to contact you so that you can send them a password. The file password_handler.htm contains is a simple way to check that the participant has the correct password.
It is important to emphasize that this is a very crude method of password protection: examining the source code reveals the password!
Secure methods of password protection are possible. These techniques are not covered in this module.
Using JavaScript to program a conditional branch in a questionnaire
Imagine a situation in
which you
want to control the questions presented to participants on the basis of
their answer to a
particular question. For example, you may want to ask female
participants a
particular question(s), and male participants
another question(s), before asking
the same question(s) of both sexes. This is an example of conditional
branching which can
be programmed in JavaScript using the IF function
Put simply:
| Examine the web page jump_marker.htm to see how
this is done
The function check_sex() is passed a value (male or female) depending on which radio button is selected. This value is stored in the variable 'gender'.
Apply these principles to ensure that the appropriate question(s) are displayed after the gender-specific question(s) have been completed. |
 |
Sending Form data to an email address
It is possible to send the data from a Form to an email address by using the following HTML tag:
<FORM ACTION=mailto:your email address ENCTYPE="text/plain">
You may find it useful. For example, this technique avoids using PolyForm to set up a script. However you do need to check the order that variables are listed in the email.
When you click on the submit button a thank you page is returned to the user. You can download this and customize it for your needs.
This page can contain debriefing information for participants, as well as links to web sites that provide further information about the issues raised by your study.
In order to send this page to users after they have clicked the 'submit' button on your questionnaire, you need to contact me so that I can modify your PolyForm script.
In this module I anticipate that you will only need to produce a small number of pages that:
Whole books have been written about how to design a web site. Many of these are aimed at people who want to design large sites for commercial purposes where it is important to address the issues of information management and site navigation. Nevertheless you are encouraged to explore web site design as this will:
Here are some design issues raised by Pearrow (Chapter 5) for you to consider:
Site design: references:
There is a vast range of books that focus on how to design attractive and functional web sites. You do not need to use any of the following books (which may not be available in the library) to perform well on this module, but I include them here for people who are interested in this aspect of the web.
So far this module has stressed the importance of ensuring that the data entered by the user into your web questionnaire is accurately recorded. Once you are satisfied that the HTML code contains no errors that compromise the collection of accurate responses from the participants in your experiment, you can move on to check the usability of your web pages.
Usability testing involves checking that your web pages "perform the function for which they were designed, with the minimum amount of user frustration, time and effort." (Pearrow, 2000).
Usability testing can be a complex affair involving special laboratories equipped with two way mirrors and video cameras to observe and record users behaviour as they work through a series of tasks designed to seek 'bugs' in page design (Pearrow, 2000). It is not unusual to find psychologists employed to design, analyse and carry out usability tests. Resist the temptation to go down this route during this module. One of the leading figures in usability, Jakob Nielsen, has argued that testing five users will reveal 85% of a site's usability problems. You will get a lot of valuable information and experience running usability studies if you ask just one or two people to use your web pages.
Most books on usability testing are concerned with designing attractive and effective commercial sites.Commercial imperatives are very different from your needs in this module which is to design a web site to collect reliable data from participants in an experiment. Therefore I suggest you concentrate on key task testing (Krug, page 153) which involves asking the user to do something and observing how they manage the task.
During usability tests:
I suggest you ask a workshop colleague to do your initial usability tests. They will help identify the most glaring problems in your pages, then try out the tests on 'naive' users.
Be sensitive to the feelings of site testers:
Observe how people react to the briefing page your questionnaire. Remember that people read books but scan web pages. Consequently do not be surprised if people scan the briefing page, find the link to the questionnaire, and start answering questions without giving you fully-informed consent to participate in the study. It is your responsibility to take reasonable steps to ensure that participants are fully informed before they participate. For example you might force the user to click on a button to confirm that their participation in the study is based on informed consent.
| You should design web pages so that participants: | For example, ask the usability tester: |
| give you informed consent before they participate in your study. |
|
| understand to what extent their confidentiality is protected if they participate in your study |
|
| understand their right to withdraw at any stage of the study. |
|
| are fully debriefed. |
|
| Keep a record of responses you collect during usability tests |
Usability tests: references:
How do I recruit subjects on the Internet?
Take care when announcing
your questionnaire on the Internet. Please bear in mind that you are
about to present your
work to a potential world-wide audience of 30 million or so people.
Consider the following
questions
If in doubt about the last question, ask yourself "Would I be happy for
my relatives,
the Head of Department and the Vice-chancellor to see the
questionnaire?" If you
score less than two 'Yes' on this criterion - think again.
There are a number of ways that you can ask people to fill in your questionnaire, but before you jump in imagine that you are about to enter a downtown bar in a foreign city as a lone stranger......that's the right frame of mind to be in.
Here is
a local copy of the FAQ document How
To Announce Your
New Web Site (Other Places)
Here is a link to Boutell's document which describes how to announce
your website on the
newsgroup:
comp.infosystems.www.announce
This link takes you to a local copy
of Boutell's document. It's
faster to load but may be out of date
What other experiments are there on the Internet?
(c) Dr Paul Kenyon, 1994-2007
Further reading