• Welcome to Smashboards, the world's largest Super Smash Brothers community! Over 250,000 Smash Bros. fans from around the world have come to discuss these great games in over 19 million posts!

    You are currently viewing our boards as a visitor. Click here to sign up right now and start on your path in the Smash community!

URC Analysis - Voluntary Response Polling and the 75% Myth

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
@tuen...
you really make me want to marry you LMAO
Heh, thanks for the offer, but I'm happily taken!

---

Ooh, I forgot something related to single tournament polling. Those results are valid! The problem they pose is national generalization. If memory serves, that in-person poll showed mostly SoCal representation, so you might say something like... "SoCal opposes the MK ban". That would be pretty valid, notwithstanding any real analysis... I'm just gut-shotting this.

The problem we're facing is the implementation of a test which can generalize on a larger scale. We could try a series of in-person polls for various regions, if we're looking to constrain the polling to tourney-goers. Then we'd get a variety of results for different regions. Maybe we'd see PNW and So/NorCal opposed to the ban... and maybe we'd see Texas/NJ for the ban. The results could be interpreted for their parent regions and compared to the number of active events... if we identify 15 active regions and poll 10, we could say we were approaching a valid result.

It'd still present the same issues though. If we forced it for tournament entrance, there would be a call against that. If we left it voluntary, it would be another voluntary response poll with a different population (all online users vs the reached tournament subset). We'd even ADD the problem of full coverage. Say everyone would participate if they were asked in person... who's to say we'd actually get polling slips to everyone that attends events this month?

It'd be kinda fun to manage a "tournament poll initiative". I'd be lazy and just manage though. I'm too busy IRL to actually poll people :-p.

Anyways. More food for thought, since it seems people like it.
 

Cassio

Smash Master
Joined
Jul 1, 2011
Messages
3,185
Have only read most of the OP and the SP.

So you say we have a poor cross-section of the community, and then you proceed to post results from polls from a single large local tournament? Doesn't add up.
These are two different topics. Blue was my criticism of the URC, but it only makes sense with the full context of the statement it was with. Since its in the OP I wont elaborate to save text, but will do so on request.

Red is a criticism against the census I did at E4H that has been covered in the thread. Its true that it cannot say more about the community than its scope, but I never claimed that it could and stated what it was and what its uses were on the same post.
What AZ said about the poll is correct. It is not perfect. However for a sample size that large, not to mention the trends seen going across each public poll, it's very telling. Also the public poll was not the end-all-be-all decision factor of the ban, many things came into play as we have said on numerous occasions, though the public poll was clearly an important thing, in my opinion at least.
Tuen said everything I wouldve and more in regards to the poll. Gathering data is very difficult and its limitations need to be considered. You can't simply say 'close enough' and call it a day. And in the cases where correct procedure is not used some data needs to be tossed out altogether. For those reasons analyzing data isnt very simple either, and requires a certain level of knowledge.

I dont fault individual members since you cant expect any particular person to be aware of these problems, but its a systemic issue when no one in the URC can analyze the data correctly and it goes ignored and unnoticed for this long; even worse because there was not a single person on the opposite end of the vote to be able to say "hey, maybe we should be more critical of this", which is why its often important that a diverse set of views is well represented (with few exceptions). Its the primary reason Im able to accuse the URC collectively of confirmation bias.

And yes Im aware that the poll was not the only thing, however its also not the only reasonable criticism thats been made in regards to the URC and its ability to make ruleset decisions. Together they bring up concerns of how well equipped the URC is to handle this issue and others, and tend to confirm the suspicion that the URCs only strength is its stated origin, the ability to enforce. Again this isnt the fault of its individual members so much as the structure of the URC itself that doesnt seek to include members that are capable of vetting ruleset decisions, but rather those that can push their decisions onto the community.
Regardless on whether or not you feel we have a poor cross-section on the community, you realize that applications are open for the URC right? Any well known TO is welcome to apply, and we're fairly accepting. If a region feels under-represented, reach out to your community to get their voice in there.
This is another part of the systemic issue. First, the fact that its expected the community will solve the URCs issues for them is troubling. Thats a terrible quality for a decision making body. Secondly, the community does not consist entirely of TOs, and since the URC only accepts TOs that means its incapable of representing a cross-section of the community.

Thanks for posting, I wasnt expecting genuine engagement from a URC member that disagreed with the OP but I appreciate it.
 

#HBC | Red Ryu

Red Fox Warrior
Joined
Jun 15, 2008
Messages
27,486
Location
Milwaukee, Wisconsin
NNID
RedRyu_Smash
3DS FC
0344-9312-3352
The top 100 poll might be considered very telling as it got a very high response rate, but the public poll is not so much. That poll got approximately 900 responses, and there are over 100,000 names registered here at Smashboards. Even if our representative community was half that number, to account for multiple accounts, dummy accounts and abandoned accounts, that'd still be 1.8% community representation.
Considering this is also for a website almost 12 years old and that not every account is made for Smash or Brawl, that isn't nearly as bad of a response rate.

Plus the fact steps were taken to stop alts and other raids from getting votes in from 4chan or something.
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
Considering this is also for a website almost 12 years old and that not every account is made for Smash or Brawl, that isn't nearly as bad of a response rate.

Plus the fact steps were taken to stop alts and other raids from getting votes in from 4chan or something.
Yes, I've heard. The steps involved an age cutoff. The search function on this site has allowed me to check out what our population looks like at various age cut offs. Here are the steps I've taken to make this a conservative estimate:

1. All population numbers are rounded down
2. All population numbers are cut in half (as I did above)
3. All percentage response numbers are rounded up

I took user counts from January 1, 2011 and back (2010, 2009, and 2008). The estimated response percentages follow:

2011
Population: 85,000
Halved Population: 42,500
Halved Percent Response: 2.1%

2010
Population: 69,000
Halved Population: 34,500
Halved Percent Response: 2.6%

2009
Population: 59,000
Halved Population: 29,500
Halved Percent Response: 3.1%

2008
Population: 38,000
Halved Population: 19,000
Halved Percent Response: 4.7%

===============================
===============================

I'll revise my statement from above: It is very unlikely that even 10% of our total community responded to the poll.

If we had a larger percent response, we could get closer to solving some of the problems that come with using a voluntary response poll. However, with the response cross section so small, there is nothing we can really say about the community at large... no generalizations can be made.
 

DeLux

Player that used to be Lux
Joined
Jun 3, 2010
Messages
9,302
The Brawl tourney attending population for North America from 2010-Aug 2011 was closer to 8000 based on Rajam's spread sheet. A majority of the 8000 only were reported in results for one tournament.

I think a more accurate population total would be 4000. That would however be cause to cut out the votes of people not in the population from that list.
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
The Brawl tourney attending population for North America from 2010-Aug 2011 was closer to 8000 based on Rajam's spread sheet. A majority of the 8000 only were reported in results for one tournament.

I think a more accurate population total would be 4000. That would however be cause to cut out the votes of people not in the population from that list.
I really don't mean to attack work... I swear.

But I don't think that Rajam's data is the best cross section of below mid-level players. This then brings up the discussion of "who is this decision really for?". If it is indeed for everyone to have their say in, then we have a problem. If it is geared towards those that have been identified in that list... then send a poll to them. We know who they are.
 

DeLux

Player that used to be Lux
Joined
Jun 3, 2010
Messages
9,302
I thought people tied for 4608?

I haven't looked at the sheet since like, I did the examination on the top 100, so I don't remember the numbers off the top of my head.

But yeah, the point Tuen was more that your population stats were really really high in comparison to the people that would go to a Unity event where the URS applies.
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
I thought people tied for 4608?

I haven't looked at the sheet since like, I did the examination on the top 100, so I don't remember the numbers off the top of my head.

But yeah, the point Tuen was more that your population stats were really really high in comparison to the people that would go to a Unity event where the URS applies.
Woah woah, slow down for one moment. Does Rajam's data exclude non URS events?
 

DeLux

Player that used to be Lux
Joined
Jun 3, 2010
Messages
9,302
It's catered towards US/CA tournaments that reported results which is the primary demographic of players that would go to a Unity tournament
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
It's catered towards US/CA tournaments that reported results which is the primary demographic of players that would go to a Unity tournament
Oh... OK. I'd need a bit more time to check out the distribution of recorded events versus the number of observable events to make a more accurate statement about that.

We should keep the population of the boards in the back of our heads though... They are the "potential" of our tournament future.

Also just for funsies:

Population: 4608
Percent Response: 19.6%
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
Just curious why did we cut off at 08?

I'm pretty sure users from before 08 could vote on the poll as well.
If you're talking about the numbers I put up, they were XXXX year and older. So the 2008 was all users that joined then or before then, that's why the numbers grow as the years go up.
 

Hippieslayer

Smash Ace
Joined
Aug 12, 2008
Messages
953
Location
Azeroth
I really dig how the OP first brings up confirmation bias then quotes two randoms that just so happen to strenghten his case.
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
I really dig how the OP first brings up confirmation bias then quotes two randoms that just so happen to strenghten his case.
Oho, are you going to start using appeal to authority to strengthen your case? What authority are you appealing to here? Good players? That doesn't work... good players aren't necessarily fit to carry out proper statistical analysis.
 

Cassio

Smash Master
Joined
Jul 1, 2011
Messages
3,185
So did you.

Edit: Aight, Ill go through this. I find it interesting that youd choose to pick out two quotes in the intro of my post that I did not explicitly use in my reasoning, over the textbook on statistics I quoted and did explicitly use, and which contained multiple definitions and examples from professional staticians.

Truth be told while I could have probably used the quotes as examples, I picked and placed them where they were as attention getters. Using quotes as attention getters is very common in persuasive writing and something Ive learned to use in writing courses.
 

SombreroJon

Smash Cadet
Joined
Oct 5, 2011
Messages
35
Location
Nor Cal
As a new-ish smash player - but stats major in college, I disagree with much of what the OP suggests. I'll say this now, I'm nowhere near close enough to the metagame to be able to have an opinion on whether or not MK should be legal.

HOWEVER, I can say that the sample size of the poll referenced creates a far more accurate depiction of the smash community than the OP suggests. While it is true that both confirmation bias and skewed polls do exist, the large sample size gathered almost completely eliminates any possibility of a completely wrong poll. I would say that the margin of error is no larger than 6%, and more likely closer to 3%.

edit: In response to your literary digest example, I've taken elementary stats (obviously) just as you have, and let me tell you that you are cherry picking pretty hard there. While it is true that voluntary polls are not the optimal method of polling, the book uses that specific case in order to prove a point, not necessarily to be accurate. The poll you mentioned is mentioned in every single stats book in the country because it is one of a kind. While there is no concrete evidence as to WHY the poll failed, most statisticians conclude that the poll was given to a biased survey base - possibly in an attempt to change the votes of their readers. Literary Digest is an isolated incident that textbooks use to further their point that voluntary polls are not optimal - however most statisticians would agree that the poll's sample size should have given them the right prediction, and therefore one of two things happened. They either got corrupted results due to bad sampling (ie. asking a majorly democrat California who they would vote for) or a one in a million occurrence that will never happen again. If you're using that example to defend your point, I would hope that you're also arguing for bad sampling area at smashboards (even though we ARE the smash community) or perhaps a one in a million thing (not likely as most/all polls ever taken with a decent sample size suggest a desire to ban MK).

The OP also suggests that players may have voted for a ban "for the lulz" - but makes no mention of the possibility going the other way. There are many people who may have voted against the ban just to piss people off, just as there may have been people who did the same for the other side. To assume on is to assume the other - but regardless they are both speculation and most likely inconsequential. Furthermore, if the results were ever mentioned before a person cast their vote, a human trait is to try to balance things they see or do (often what people call OCD). When a player who otherwise does not have an opinion on a poll sees that it is skewed in one sides' favor, it is in their human nature to try to balance the poll. Combined with the people who may have voted to keep MK legal in order to piss people off, I don't see an argument on trolls legitimately effecting the vote.

I also saw mentioned several times in this thread the idea that "well you don't know how it would have turned out if EVERYONE voted - who knows they could mostly be against the MK ban". That is logically AND statistically wrong. Once a large enough sample size is collected, and say... 60% of people vote for the ban in the sample. It can be assumed (and in all likelihood correctly assumed) that roughly 60% of the people who did not vote would ALSO be for the ban. The only time a situation like this would occur would be in a situation where those who are AGAINST the ban are placed in a situation where they are less likely to vote. Considering that there are no outside factors acting on the people AGAINST the ban, it is logical to assume that the sampled people are valid.

Lastly, as a stats major it is frustrating to see something like this (posted by Tuen) -

"I'll revise my statement from above: It is very unlikely that even 10% of our total community responded to the poll.

If we had a larger percent response, we could get closer to solving some of the problems that come with using a voluntary response poll. However, with the response cross section so small, there is nothing we can really say about the community at large... no generalizations can be made."

This only shows that you are like a normal person who does not quite understand stats. 10%, sheesh only 5% of a population is most definitely a big enough sample size! Have you ever heard of the gallup poll? These people poll less than .01% of the US population (usually around 1500 people), and ONLY take answers from people who choose to respond (for those of you voluntary poll people), and they have a margin of error less than 3%! Realistically even about 150 people would make an accurate representation of the smash communities' opinion. The fact is, whenever we see these polls come up, they always tend come out in favor of the MK ban - by a fairly significant majority.

As I said, I'm not educated enough myself to know whether MK is actually banworthy. However it is obvious to see that the solid majority of smash players DO believe that he should be banned, and that much you really cannot argue.
 

Cassio

Smash Master
Joined
Jul 1, 2011
Messages
3,185
They either got corrupted results due to bad sampling (ie. asking a majorly democrat California who they would vote for) or a one in a million occurrence that will never happen again.
Bolded. It was only stated in the example itself :p. Of course if youre a stats major Im 99% sure that you knew it would come down to whether or not those who voted represented a good sample or not, and yet in your post you dedicated less than one sentence to the matter:
even though we ARE the smash community
convinving point, lol. Now care to explain how a self-selected sample is capable of representing a random sample?

Tuen can correct me if Im wrong but Im fairly certain he meant at some higher margin of participation even poor sampling can be better overcome (Id wait for him to respond). That was also my intention when I took the census for my regions opinion.

But honestly, we arent unaware that smaller but well done samples lead to better results. Not only was that the point of the thread, but it was stated directly in the OP:
The lesson here is that it is not necessarily the size of the sample that makes it effective, but it is the sampling method.
The rest of your post makes silly assumptions that I wont dwell on. It feels like you were reading what you wanted to read instead of what was on the page :(, I made none of the claims you said I did.

Edit: One more thing. I thought it was odd that youd hint Gallup employs voluntary response polling, but before I responsed I decided to check for myself. Gallup uses population proportions when it can and otherwise uses a simple random sample. It does not use a self-selected sample from what I can see so its a really odd comparison. Their methodology is really strict, uses lots of population information, and likely what allows them to have such accurate results with such small samples. But you can compare for yourself, heres a link on their methods.
https://worldview.gallup.com/content/methodology.aspx
 

SombreroJon

Smash Cadet
Joined
Oct 5, 2011
Messages
35
Location
Nor Cal
I'm aware of the way gallup polls people lol. They are random at two different points in the process, which is awesome. However, gallup (just like the smash poll), runs into some of the same problems. You said in the OP that some people may have voted for the ban just to cause an uproar, did you not? Gallup experiences the same things when they ask questions about Obama's approval.

As for the bolded part - I see your point, I just wish you elaborated more in the OP as to WHY it is a corrupt sample. I really don't believe that it was... As far as I know the poll was open. In order for the sample to be corrupted, it would have to have been administered to something like "only MK mains" or "only those who do not use MK". THAT would be an obviously corrupt poll, but having it open to smashboards does nothing to limit or skew public opinion. And as I said, it being on this site and Allisbrawl IS the community opinion as a whole.... not the opinion of only one side of the argument. I know you think that a "self-selected" sample is a load of crap and this should have been random... Sure, random is nice, but this "self-selected" sample isn't as bad as you think. It was open to people who play Brawl, and wasn't posted on other random sites or anything. It may not be capable of the accuracy of a random sample, but that is accounted for by a larger margin of error.

Lastly, I know you said "The lesson here is that it is not necessarily the size of the sample that makes it effective, but it is the sampling method." in the OP, and I'm sorry if I got away from my central theme in my last post. However I would argue that in this situation you make no real case. The method used, while not ideal, did it's job - and under all statistical analysis can be proven to be pretty darn close to what the public actually feels. Unless you have a convincing/true reason as to why this poll would not reflect the public opinion (ie. those against the ban were influenced to not vote, or were more unaware of the vote than those FOR the ban), the poll seems legit. I mentioned that I would say it has about a fairly large 6% margin of error because of the polling method - despite the huge sample size. I don't really see how any argument that doesn't involve major polling bias will be able to make the poll get anywhere near 50-50.
 

SombreroJon

Smash Cadet
Joined
Oct 5, 2011
Messages
35
Location
Nor Cal
Reading your OP one more time leads me to believe that you have the wrong idea of a "self selected sample". You say that because the poll was self selected - it is wrong. However, what your book means by a self selected sample is only opening the poll to MK players or something of the sort. This poll was not self selected (AFAIK), which is what confuses me so much about your OP. This wasn't a poll in people magazine asking if guys are evil and getting 75% answers yes because of their primarily female reader base lol. Anyways, sorry if any of this has come across condescendingly - I don't mean to be a doucher. I'm going off of what I read/know :)
 

Cassio

Smash Master
Joined
Jul 1, 2011
Messages
3,185
Hmm, I dont think so. I'm fairly certain Im using the term I want from what I recall and from reading the text. It could be that were using the same term to mean different things. Actually I think I see where our disagreement is. Its with the population.
Unless you have a convincing/true reason as to why this poll would not reflect the public opinion (ie. those against the ban were influenced to not vote, or were more unaware of the vote than those FOR the ban), the poll seems legit.
Italicized. Well thats a big concern with VSP from what I know. The point is we dont know. Could everything have come together and potential issues unintentionally avoided to create at an accurate representation of the population? Yes. Could every potential issue go wrong and create an inaccurate representation of what opinion may be? Yes. Unless polling is done so that these concerns are mitigated its exceptionally difficult to say how big a roll they may have played and its unfair to anyone who may be affected by its results.
You said in the OP that some people may have voted for the ban just to cause an uproar, did you not?
Not exactly. I completely understand that issue goes both ways. But in that case it was that there are people who legitimately dont and are not interested in playing this game that are still capable of voting in the poll. Theres discussion about what population is being polled, but I believe the general consensus is that its the tournament entrants at offline events in Canada + the US, so I hope you can see why thatd be an issue. I also think theres significantly better ways of polling this population. Theres a few other potential issues here and there in regards to passion and advertisement that are worth being concerned about, but I think time is better spent gauging or approaching the issue in a better way than debating issues that are likely unprovable one way or another. I said that somewhere in the thread but not in the OP.
 

SombreroJon

Smash Cadet
Joined
Oct 5, 2011
Messages
35
Location
Nor Cal
You're a smart guy. I agree that arguing over semantics is not solving anything. The one thing I think we can agree to disagree on is this...

Could every potential issue go wrong and create an inaccurate representation of what opinion may be? Yes. Unless polling is done so that these concerns are mitigate its exceptionally difficult to say how big a roll they may have played
All I'm saying is looking at the sample size and the non-biased questioning or posting, I don't see any possible way for this to be a largely inaccurate representation =/. I know in the OP you said as a summary "In short, the poll requesting MK's ban in the URS is a flawed and useless measurement." I have to respectfully disagree. I see the logic you are using to come to this conclusion, but from a purely stats perspective on the poling method and results... I just don't see the poll being somehow the wrong when gauging the communities' opinion. And seeing how you have used stats (to the point where you quoted your stats book) as your method to defend your claim... well, I would start moving my argument towards a "most people are not properly educated on MK enough to vote" movement.
 

Cassio

Smash Master
Joined
Jul 1, 2011
Messages
3,185
I guess it depends on what you mean by gauging the communities opinion. Even assuming that, in September, a majority of players felt MK should be banned in the URC ruleset; 75% is different from 85% is different from 65% is different from 55%. Especially as fickle as our community is. From a policy standpoint a certain level of certainty is necessary for coming to a good decision, especially when multiple options and multiple criteria are taken into consideration. Its especially concerning when individual regions arent in agreement with each other, because a closer vote mightve indicated how divisive a nationally enforced ban mightve been and resulted in a better compromise (which is the reason I advocate allowing locals and regionals outside URC scope).

Also Im really not arguing for or against the ban, as Ive stated before Im in favor of encouraging locals and regionals to do as they please to help tournament attendance and please their local constitients that are the heart of our community. This thread was in regards to the poor work of the URC.

Lastly just to put some numbers to one of the concerns:
People capable of voting: 38,000 - 85,000
-the amount of Smashboards users capable of voting.
Target Population: 4608
-the best guess on the number of tournament entrants. Target population was stated earlier in this thread in regards to this specific poll, but could be reasonably argued to include more people.
Not that I outright diregard your assessment, it simply conflicts with what Ive learned thus far, that polls which dont meet a certain standard are generally disregarded as opposed to simply increasing its margin of error. In fact the point of procedure was hammered home very hard and is generally true for a wide range of disciplines, which is what makes me wary. I try to take little for granted so Im more than willing to acknowledge potential issues, but at the same time any source or slightly more detailed explanation would be very helpful.

edit: also dont be too concerned with the OP, I know its not perfect and doesn't include everything. Writing means you need to be capable of writing to your audience, and I wasnt writing to an audience of stats majors, lol. I prefer addressing concerns from individuals more personally and specific, else I end up with an incomprehensible 15 page long thread trying to address everything while doing the opposite.
 

Naridax

Smash Journeyman
Joined
Dec 3, 2011
Messages
223
Location
Charlottesville, VA (UVA) / Virginia Beach, VA
Large sample sizes are indeed preferable, as a larger sample is more likely than a smaller sample to better represent the population. However, increasing sample size does not reduce bias. At all.

Bias is the result of some systematic error. In other words, something is wrong with the sampling method. For this reason, biased samples are likely to yield similar results. If you increase the sample size, you will simply be making the same error more times. If you repeat the procedure, you will again be making the same error more times. In order to reduce bias in a sample, the sampling method is what must be altered.

I do not remember for sure, but I believe someone claimed that the individuals that did not vote in the polls are simply stupid or indifferent. Unless the polls were conducted only to compare how many individuals strongly supported banning Meta Knight to how many individuals strongly opposed banning Meta Knight, then those that are indifferent or have weaker opinions do matter. The underrepresentation of individuals with weak opinions and overrepresentation of individuals with strong opinions is actually the problem that arises from voluntary response bias, and what this quote is referring to:

With such voluntary response sample, we can only make valid conclusions about the specific group of people who chose to participate, but a common practice is to incorrectly state or imply conclusions about a larger population. From a statistical viewpoint, such a sample is fundamentally flawed and should not be used for making general statements about a larger population.
Also, due to biased samples, the URC polls are all, as Cassio said, useless. Conclusions cannot be drawn from them. Future randomized polls (which I hope will be taken) may still yield the similar results. Unfortunately, however, the samples were biased, and, as far as I know, biased samples are irrecoverable.
 

Ghostbone

Smash Master
Joined
Sep 20, 2010
Messages
4,665
Location
Australia
People not voting isn't a reason to oppose something, do you think someone would be taken seriously if they didn't vote during the election and then started protesting afterwards that another ballot should be taken as the original one was biased?

With a randomised poll, you still have the exact same "problem" of people possibly not responding.
 

Dr. Tuen

Smash Lord
Joined
Apr 26, 2009
Messages
1,396
3DS FC
0559-7294-8323
Oi. A lot of stuff happened and I didn't really notice. I think my name showed up too. Beh~

I would like to emphasize a fact about voluntary response polls though. They are inherently biased, but not necessarily in ways we can detect. It could be biased towards people who care enough to vote. I know for a fact there are people from my region with opinions about the poll that did not vote. It could be biased towards SWF users, though there was an AiB poll too (which would be biased towards people who saw it on their "Me" page).

The point is, a voluntary response poll is never better than a random sampling. The other point brought up is the huge difference between the approximately 4,700 active tournament goers and the 50,000+ usernames on smashboards. It could indicate a couple of things... we may not have a full recording of the events that happen (indicating the 4,700 isn't a complete figure). The SWF user count could be flawed by old or duplicated accounts (i cut the actual number in half to get 50,000).

One last interesting point to make... if we want to reach the regular in-person tournament community, then why don't we just take a random sample of the 4,700? This, in turn, brings up yet another question. Is that the sample we want to be testing? Do we want to include online players and casual players in this poll, since they may very well be the future of in-person events?

Just some food for thought. Remember to have fun!
 

Naridax

Smash Journeyman
Joined
Dec 3, 2011
Messages
223
Location
Charlottesville, VA (UVA) / Virginia Beach, VA
People not voting isn't a reason to oppose something, do you think someone would be taken seriously if they didn't vote during the election and then started protesting afterwards that another ballot should be taken as the original one was biased?
Oh, but, in this situation, it is grounds for opposition.

Correct me if I am wrong, but I believe that the intent of these polls was to gauge the opinions of the community as a whole so a decision could be made.

The flaw in your logic is thinking that an election is a good representation of the US population. All elections in the United States suffer from voluntary response bias, because the responsibility of voting falls on the individual. (In some countries, voting is compulsory.) For this reason, I somewhat disagree with the use of an election as an example in the OP, but it is successful in getting Cassio's point across. Ideally, elections would represent the entire US population, but they don't, and they probably never have and never will. It is common knowledge that much of the population does not vote. (According to "The Myth of the Vanishing Voter" by Michael P. McDonald and Samuel L. Popkin, in 2000, only 51.2% of the voting age population and 55.6% of the voting elligible population actually voted in the presidential elections. In the United States, individuals have the choice of voting by their right. Also, in the past, elections results were often corrupted by fake ballots and coercion.)

The problem that I have with the polls is that their results are being used as if they actually represent the opinions of the entire community. They don't. They are biased.

With a randomised poll, you still have the exact same "problem" of people possibly not responding.
Bias is impossible to completely eliminate. It is possible, however, to reduce bias. This is usually done through randomization. In a randomized sample/survey/experiment, bias does still exist, but it is reduced to an extent that its influence on the results can be treated as being negligible.

Most likely, every individual will not respond. In that situation, the next random individual is polled. The results may vary because individuals among the random sample may choose to refrain from voting, but that is okay. This variation in random samples is not an error. It is known as sampling variability.

One last interesting point to make... if we want to reach the regular in-person tournament community, then why don't we just take a random sample of the 4,700? This, in turn, brings up yet another question. Is that the sample we want to be testing? Do we want to include online players and casual players in this poll, since they may very well be the future of in-person events?
For this sentence: "Is that the sample we want to be testing?" Do you mean "is that the population we want to be sampling?"
 
Top Bottom