ACET, Inc. Blog

The concept of race and the associated terminology has altered over time. Dr. Michael Rodriguez, associate professor of Educational Psychology at the University of Minnesota, sent me a very intriguing example of the evolution of the race item on the U.S. Census. The census is administered every 10 years, and almost every administration seems to contain a variation in the language surrounding race. The full pictorial history of its evolution can be found here: http://racebox.org/

Some interesting things of note:

1. “Colored persons” started to be counted in 1830, and then only if they were classified as “free” (although the three-fifths compromise, allowing African slaves to count as three-fifths of a person for representation and distribution of taxes, began in 1787).
2. The distinction of counting only “free” people disappears after 1850 (the Civil War began in 1861).
3. The item specifically concerning “…of Spanish/Hispanic origin or descent” begins in 1980.
4. Also beginning in 1980, “color” (the color of a person’s skin) is no longer affiliated with race in the item stem.

Anything interesting you see? Leave us a comment and let us know.

Joseph

We often get requests to provide feedback on surveys. As we review the surveys, we tend to see three challenges occur quite frequently. I’ve written this blog to offer suggestions for addressing each of the challenges.

1. Double-Barreled Questions. In a double-barreled question, two concepts or ideas are included in one survey question. For example, the three survey items below are all double-barreled questions:
○        “How much of your monthly income is spent on restaurant dining and entertainment?”
○        “How often do you volunteer your time or make charitable donations?”
○        “How often do you consume beer and smoke cigars?”

But how does a respondent answer a double-barreled question? For example, if a family spent no money on restaurant dining but 10% of their income on entertainment, how should they respond? 0%? 10%? Average the two and respond 5%? Usually survey respondents do provide an answer, but they often utilize an individualized strategy for finding a response, which can be challenging for program staff to interpret.

Double-barreled questions can be addressed by limiting each survey item to one – and only one – concept. Below is a double-barreled question followed by revisions that eliminate the double-barrel:

2. Imbalanced Response Options. Imbalanced response options refer to a set of response choices that do not cover the range of possible choices. An imbalanced response scale is problematic because it limits respondents’ responses to only one end of a scale and may produce inaccurate results. For example, if a respondent wants to answer “strongly disagree” to a question but that option is not available, the survey is not capturing the respondent’s perspective. Using a set of balanced response options ensures that respondents’ answers can be accurately captured. A very good resource for a variety of balanced response scales can be found here: http://www.gifted.uconn.edu/siegle/research/Instrument%20Reliability%20and%20Validity/Likert.html

Below are some examples of imbalanced response options with improvements to the scale:

3. Overlapping Response Options. Overlapping response options simply means that there is some overlap in the response choices available to the respondent. For example, the following survey item has overlapping response options:

How many books did you read in the last month?
0 or 1 book
1 or 2 books
2 or 3 books
3 or more books

Overlapping response options are problematic because they can confuse the survey respondent. If you read two books in the last month, which option would you select: “1 or 2 books” or “2 or 3 books”? In addition, questions with overlapping response options are a challenge to interpret. How do you know how many books were read if the response options overlap? Overlapping response options can be addressed very easily by carefully constructing the response option list. Simply revise and edit the list so that there is no overlap between the response choices.

If you have any questions about the above challenges to designing a survey, please feel free to leave a comment or email me at Kirsten@acetinc.com. Please also contact any of us at ACET if you have questions about survey design in general.

Kirsten

ClearWay Minnesota℠ and the Minnesota Department of Health recently released results of the 2010 Minnesota Adult Tobacco Survey (MATS). Overall, news about tobacco use appears positive: adult smoking rates have continued to decline from 22.1% in 1999 to 16.1% as reported in the latest survey; exposure to secondhand smoke has decreased from 56.6% to 45.6% since 2007; and 87.2% of homes have gone smoke-free – up from 83.2% in 2007.

The results of the survey also pointed to areas of concern, such as a drastic increase in the use of smokeless tobacco products in both cigarette smokers and non-cigarette smokers. In fact, smokeless tobacco use for cigarette smokers has more than doubled since 2007, rising from 4.4% to 9.6%.

To view the full report and a fact sheet, click here: http://www.mnadulttobaccosurvey.org/

Cassie

Every three years, several Minnesota state departments (Education, Employment and Economic Development, Health, Human Services, and Public Safety) collaborate to administer the Minnesota Student Survey. The survey was last administered in 2010 to primary and secondary students (grades 6, 9, and 12) to assess their attitudes, beliefs, perceptions, and behaviors on a variety of topics. The survey covers areas such as students’ background, school, activities, heath, and behavior. A total of 335 school districts participated in the survey with a total of 130,908 students taking the survey across the state.

Results are reported by grade and the corresponding trends between grades are very similar; students appear to be changing attitudes, perceptions, beliefs, and behaviors nearly simultaneously. Many positive changes can be seen between recent years. For example, when comparing twelfth graders’ answers on the 2010 administration with the 2007 responses, several positive differences can be seen:

• More twelfth graders reported they were planning to attend college (88.9% in 2010 compared to 85.8% in 2007)
• More twelfth graders reported being physically active on five or more days in the past week (43.4% compared to 41.4%)
• Less twelfth graders reported alcohol usage in the past year (55.3% compared to 62.5%)
• Less twelfth graders reported binge drinking (five or more drinks in a row in the past two weeks; 23.4% compared to 29.0%)
• Less twelfth graders reported driving a motor vehicle after drug or alcohol usage (18.0% compared to 23.9%)
• Less twelfth graders reported smoking in the past 30 days (19.2% compared to 22.8%)

For more information follow the links below.

Select trends: http://education.state.mn.us/mdeprod/idcplg?IdcService=GET_FILE&dDocName=019017&RevisionSelectionMethod=latestReleased&Rendition=primary

2010 complete data tables: http://education.state.mn.us/mdeprod/idcplg?IdcService=GET_FILE&dDocName=019009&RevisionSelectionMethod=latestReleased&Rendition=primary

Joseph

Last August Heather and Kirsten offered a free workshop on survey design, which covered how to write useful items for your survey and common pitfalls to avoid. The feedback on the workshop was very positive and all attendees seemed to benefit from the material. ACET also offered a complementary review of an attendees’ existing survey as a means of following-up for the event.

We are pleased to announce that our second workshop has been scheduled! The Results are In: Analyzing and Reporting Survey Data for Stakeholders will cover common options for analysis of your survey data, guide you through the analysis process, and offer you suggestions for sharing your information with stakeholders in a way that is useful and meaningful.

The Results are In: Analyzing and Reporting Survey Data for Stakeholders will be held on:

Tuesday, February 23rd, 2010 from 1:00 to 3:00 pm at the
Neighborhood House Wellstone Center
179 Robie Street East, Saint Paul, MN 55107
Wellstone Center phone: (651) 789-2542

Seats are limited to this FREE event so please RSVP to Heather Scholz via email at heather@acetinc.com or via phone at the number below by Thursday, February 18th, 2010. For more information about ACET please visit our website at www.acetinc.com or contact our office at 952-922-1811.