CTE - Assessment Workbook

One of the most common tools for measuring student learning outcomes indirectly are surveys.  A survey or a tool for gathering information about beliefs, values, or attitudes towards a particular topic can be used as an indirect measurement of student learning. 

You will want to use a survey as part of your assessment process in order to:

• Gather information from and about a certain population
• Explore beliefs, values, and attitudes about a certain population
• Make comparisons between populations
• Make comparisons about populations longitudinally

Surveys should not be used without a clear, stated purpose for both the facilitators and the participants or with a small number of participants.  

For any survey, it is essential that the response rate or the number of people who participate in the survey vis-à-vis the number of people within the population to be statistically significant.  The response rate will depend on the population, the margin of error you are willing to accept, and how confident you are in this margin of error. 

There are some ways to help improve your response rate:

• Tailor the length and complexity of the survey for your target population. 
• Use a clear, simple design will entice the target population to complete it.
• Use the best method for reaching your population:
  • web-based survey if the population is not immediately accessible
  • paper survey if the population is accessible
• Determine the best time to administer the target population.
• Provide an introduction or cover sheet that
  • explains the purpose of the survey and its intended uses. 
  • introduces the surveys facilitators and their interest in the data.
  • reinforces the privacy and confidentiality of the data.
  • provides an estimated completion time.
  • discusses any potential benefits or risks
  • reinforce the value and importance of their responses
• Provide an appropriate closing date to allow the target population an opportunity to complete.
• Make ‘check-in’ points such as calendar, email, or telephone reminders

First, the survey facilitators should determine if they are going to use:

Open-ended questions or questions that allow participants to provide their own response

Advantages

• provide valuable qualitative data that allow the facilitators to analyze the reasoning and logic behind a participant’s answer

• reveal responses that were not immediately considered by the facilitators

Disadvantages

• take more time to complete, which could result in a lowered response rate
• take more time for facilitators to analyze

Closed questions or questions where the facilitators provide responses

Advantages

• provide valuable quantitative data that allow facilitators to easily analyze trends and patterns in the participants’ answers
• offer pointed feedback for areas the facilitators are specifically trying to assess

Disadvantages

• do not offer participants the opportunity to present their own responses to the issue
• can be too narrowly focused on a specific topic

Next, the survey facilitators will need to determine if the survey will be anonymous or named. 

Anonymous surveys

• protect the privacy of the participant
• may encourage more honest, candid responses
• may result in higher response rates

Named surveys

• direct information to specific participants
• allow for matching responses with institutional data
• make tracking for responses easier
• easier for longitudinal study

Finally, when designing a survey, it’s important to carefully construct survey items.  Survey items should be clearly worded with no ambiguous language.  Use direct wording that your participants will easily understand.  

Some suggestions to keep in mind:

• If using a Likert-type scale, use five response points (strongly agree, agree, undecided, disagree, strongly disagree).
• If using anonymous surveying, make sure to ascertain any important demographic information such as age, gender, class level, major, degree program and the like.
• Refrain from overly personal questions or questions that your participants may see as intrusive.
• Refrain from using generalizations and universals (all, none, always, never, just, only).
• Only ask one question for each response opportunity to avoid confusion.

When analyzing the results of an open-end question survey:

• Organize and categorize similar responses
• Summarize these categories and choose representative examples
• Note patterns and trends within these categories and choose representative examples

When analyzing the results of a closed question survey:

• Note frequencies or the number of participants who responded in a similar way
• Note statistically relevant information such as
  • Maximum and minimum
  • Mean
  • Mode
  • Median
  • Standard deviation
  • Variance
  • Skewness