This all started with a simple question I asked on my Twitter:
With such a small number of responses the results were hardly definitive, But there was something of a clear trend. Reader Dawnstorm tells me that the results may just be random, I’ll take his word for it. That lead me to post the same poll here at the Lounge last week.
I half expected the trend to continue… But I didn’t expect this:
Even though it’s still a small number of respondents (14, and thus still suspect), the trend is even more clear. Readers like a blog that at least contains some things other than anime. This never was a referendum on the future of the Lounge, but it’s nice to see what I’ve been doing is liked. (On the other hand, since the poll was of my readers there’s no doubt a bit of bias and self selection going on…)
Thanks to those who took part! Drop a line down in the comments and tell me what you think!
Apprentice Mages Lounge. 
I wasn’t actually saying that the results “may just be random”. In your post you said:
My response was to that. “Surprisingly close” is different from “surprisingly, it was close”. But the category of “close” contains only one ratio with 5 responses, and that is 60:40 whichever way it leans. The more responses you get the more ratios are potentially within a range one might call “close” (and the more likely we are to have different instincts). Basically, “surprisingly close” here means little more than that you have gotten so few responses. If it hadn’t been surprisingly close, it’d have been “surprisingly decisive” (unless you actually expected an 80:20 distribution or higher).
When you get more responses, the range of possible ratios increases, and not-too-close/not-too-decisive results become possible. That’s not yet a question of empirical distributions; it’s a question of systematic probabilistic behaviour. That is: with only 5 responses the surprise only looks like it comes from how close the results are; it’s really only surprising because your expectations didn’t take in account the number of replies you’d receive (again, I’m wrong if a result of 80:20 or above is what you’d have actually expected).
Percentages have a sort “sacred” status these days, representing scientific rationality. But percentages of a small sample don’t mean the same thing as percentages of a large sample, and I’m not even talking about representativeness yet. Percentages are a measure we use to compare samples of different sizes; but with very low sample sizes percentages stop making sense, because the range of possible ratios is so low. If only two people had replied, you have two possibilities: unanimosity or a perfect split. There’s no way that’s usefully expressed as a percentage. It’s possible that it’s perfectly representative though: that would be the case, for example, if by chance the results match, or if two people isn’t actually a sample but all the replies you need.
I wasn’t saying the result was random (though that’s part of it). 1 % is one in a hundred. If you have a smaple of five, and you chose to use these results as an estimate for, say, a popluation of 100, then you’re making a hidden assumption that people will vote in sync in batches of 20 – which is absurd.
Differently put: if you’re in a room with 5 people and you ask them all, would you be saying that 3:2 is surprisingly close? As it is, you’re not in a room with anyone relevant, and you’re asking the question of an indeterminant number of people (with the max being all the people who read your twitter account). You may well have an intuitive estimate in mind, and you’ll then see the poll result and look at the percentage and think that’s surprisingly close, because you had some (unspecified) larger difference in mind. But with only five replies, the percentage number isn’t really meaningful.
If you took all that in account and the results still surprised, I’m sorry for assuming stuff. It’s just that I see people abusing percentages to make a rhetorical point so much, that I sort of tend to jump to conclusions. In any case, it’s a really minor point, and it’s just my geekery (as I said in the other post, too).
LikeLike
My apologies for misunderstanding your meaning.
I didn’t take anything of that into account… My reactions were emotional, subjective, and mostly unscientific. Heck, it took me a couple of readings just to figure out what you said. (I’m a smart guy, but sociology and statistics are so not my thing.)
Anyhow, once I worked it out, I see what you’re saying and you make a good point. I was trying to be careful in what I read into the results, I guess I wasn’t careful enough.
Always feel free to share your geekery! As I’ve said before, the comments section belongs to the readers.
LikeLike