Responses to UBE survey raise concerns about validity of conclusions

I am interested in responding to your Oct. 15 article “Survey Says…” in which you published some of the results of an Undergraduates for Better Education survey about the satisfaction levels of the Syracuse University undergraduate student body. I have some concerns about the methodology used in the survey in question.

The survey made use of a voluntary sample, which introduces voluntary response bias. In this case, it is likely to manifest itself as an unrepresentative high proportion of extreme respondents.

Consider the following: Upon receipt of the UBE survey, those with strong opinions on the matter were far more likely to make the investment of time and effort to respond than those who could not have cared less. Given that of the 13,943 students surveyed, only 778 – around 6 percent – responded, and it is likely that this 6 percent holds more extreme viewpoints than the general student body, it’s hard to justify implying that the results are representative of all students.

Further, it is hard to say what the response of the missing 94 percent of the student body would have been. We assume that it would have been weighted more toward the middle of the spectrum than the survey results, but where is that point?

An argument could be made that these uncaptured students would mostly respond neutrally, but I think it’s more likely that they would respond as “satisfied.” Why? Because as a student, you are exchanging your money for your education. Were the average student not at least satisfied with this exchange, they would not choose to participate and would instead seek education elsewhere. In fact, one must question how dissatisfied the survey respondents that chose “very dissatisfied” can really be if they are still students and willingly exchanging their money for Syracuse’s education.

Finally, we must question why the survey results were interpreted as they were. Almost universally in every response category, the “satisfied” and “very satisfied” results outweighed the “dissatisfied” and “very dissatisfied” results. By choosing to group the neutral results in with the two dissatisfied options, the study presents a picture of a highly dissatisfied student body, where another potential, and perhaps more accurate presentation, is a highly satisfied one.

I have posed these questions to the UBE Facebook page, as well, and I suspect they will respond in due time. I would like to hear their decision-making process behind their choice to present their study as they did and if any steps were taken to mitigate the methodological concerns above.

I will also go on the record as saying that my experience at SU has been a very satisfying one, but of course that evidence is merely anecdotal, so please do not use it to make judgments. I could be an outlier. Besides, as a former grad student, I’m not in the study’s target population anyway.

In closing, I would like to say that I applaud the UBE for their student advocacy and I hope they continue their mission. Regardless of questioning this study, I feel there is great value in what they are attempting to do and admire their ambition and attitude.

Matthew Neagley
Master’s student in statistics