erico9001
You must find your own path to the future.
Discussion in the Discord server made me wonder about the ideas behind the scenes in our tier lists.
"Viability" is always the answer. We make tier lists to determine viability. However, here's where things get a little bit funny. What do we mean by that? What is viability? What does viable mean? From the Oxford online dictionary (what Google uses):
Alright. So basically what you should gather from this is we are making a tier list to basically tell somebody what characters could work for them, or the degree to which they would. Great. So... now what if I was to say the word "potential?" Are you finding yourself triggered?
A lot of people are triggered by this. For instance, ZeRo gets a lot of bad rep and dismissal about his potential based tier list. And what actually started this whole conversation in the Discord in the first place was myself saying the word potential, in a situation where I meant to say viability. People were on me really quick for that one, especially since I am one of the leaders of Shulk players, who play a character whose potential has always been a big discussion point. K... What's the definition of "potential?" From Oxford again:
On top of this, both words share the same synonym "possible"
So what's the big deal with this word? Why are people triggered by it? What makes ZeRo's potential based tier list any different from everybody else's? Well, after digging into it with the discussion, here's the answer:
So here's the deal. "The Data" is associated with viability, while "Theorycrafting" is associated with Potential.
Alright, so two new terms. Let's quickly define them...
The Data - In competitive Smash, we have data on all sorts of things. However, the data in this case is our tournament 'top ##' results data. What more specifically? Check this thread out. You can also go onto Smashwiki and search for the placement data on major tournaments, if you want to be more thorough about it. An approach solely from The Data is just copy and pasting it.
Theorycraft - Extrapolating from data such as frame data and parameter data and comparing against results data in order to create a general theory about how viable a character should be.
If you ask most people in the backroom, I would expect they would say you should go mostly from the data. If you want evidence for this, look no farther than the starting post in the last voting thread.
"That is the reality we live in" - The Data
The goal of this approach is to eliminate subjectivity in order to produce more accurate results. The idea is 'it is what it is; look at it. Reality. Behold!' While I appreciate the approach, as an engineer I am obligated to share with you how this approach is flawed. There is a large degree of inaccuracy of interpreting the scores of the data as direct implications of viability. However, don't be mistaken, it seems @Thinkaman does know this, because as he said himself his tier list had many deviations. "I'm dumb and not afraid to be wrong." Although, I would say there's nothing dumb at all about it.
The main flaw with the data is it does not account for representation of the character. We can split that up farther into two main areas - quality of representation and quantity of representation. Quality of representation refers to the top level players, like ZeRo, who either do or do not play the characters. Questions like:
On the other hand, quantity of representation just directly skews the data. Having a high quantity of players of a character gives a higher chance for higher quality too --- picking 4 marbles from the bag rather than 2 gives a higher chance of the marble you want. In the end, if the character is more popular, there ends up being more really good players playing him, and more placements in the top ## of the data. All that the data does is count total quantity of results. Therefore, characters are made higher tiers solely because they are played more or the inverse if they are played less.
A live example of this is very, very easy to see... Quick, where should Dark Pit be placed on the tier list compared to Pit?!
That's the difference between a high low tier and a bottom tier. Does that make sense? In the list Thinkaman posted, this may have been accounted for in the "few factors accounted for." If that's the case, well, say hello to theorycraft. The reason why Dark Pit being far removed from Pit strikes us so bad is it just doesn't make logical sense to us. It goes against our personal theorycraft. It goes against how we make sense of this game. With all stats the same and just slightly modified moves, we figure Dark Pit should not be that much lower. He might even be higher in some people's opinions. This is a very obvious example of representation skewing the results of the data to not accurately depict what we are looking for - viability. What do we do to account for it then? We apply our theorycraft. 'In theory, Dark Pit should not be far removed from Pit.' In adjusting his placement to be higher than what the data says, we argue that Dark Pit could potentially be seen in that higher tier, given better representation. We argue that he is that viable.
Is that bad? The actual goal listed by the creator of the data, on the first post of the data thread says no.
While theorycraft comes in use, its issue is that we as humans get things wrong. We are subject to biases, errors, misjudgments, misunderstandings, misinformation, and much more. The last thing we want is some meme about Charizard sucking to bring him to the last tier placement. A tier list based solely on theorycraft lacks a certain level of order and could even be chaotic. People being susceptible to informational faults is just common knowledge. For an example... why not watch Shulk Commentators Say The Darndest Things.
If this data is what we have to work with, the highest accuracy we're going to be able to achieve is going to have to be some sort of mix. The data should be used but not in raw form. It should be interpreted and adjusted according to our theory of this game. We need to apply logic to it. However, we need to be careful not to take this far. You could see it as a sort of sweetspot or Goldilocks zone. There's some position with just the right amount of balance between personal perspective and the data. That position is where the tier list is going to be the best representation of actual viability.
I encourage people to push their own ideas about how viability in this game is defined. Just prepare for some questions as you do so. 'Why do you think it's so different from the data?' is a question you should be asked if there is a large deviation. If you don't have an answer to that, you probably haven't thought enough about it. On the inverse, if your tier list is almost copy pasted from the data, you might also have not thought enough about it. Theorycraft and potential are an important part of our voting process. If we were just going to go off of what the raw data says, why is a backroom even needed? I would be curious to see more personal ideas put into tier lists. If you're not sure what to take away from all of this, there's one main lesson.
Make your tier list by how you think the metagame really looks, but make sure you understand and can explain why it is different from the data if it is.
What is and ought to be the mental process of creating a tier list?
Question: What are we trying to weigh in our tier lists?
The Golden Word: "Viability"
The Golden Word: "Viability"
"Viability" is always the answer. We make tier lists to determine viability. However, here's where things get a little bit funny. What do we mean by that? What is viability? What does viable mean? From the Oxford online dictionary (what Google uses):
Alright. So basically what you should gather from this is we are making a tier list to basically tell somebody what characters could work for them, or the degree to which they would. Great. So... now what if I was to say the word "potential?" Are you finding yourself triggered?
The Swear Word: "Potential"
A lot of people are triggered by this. For instance, ZeRo gets a lot of bad rep and dismissal about his potential based tier list. And what actually started this whole conversation in the Discord in the first place was myself saying the word potential, in a situation where I meant to say viability. People were on me really quick for that one, especially since I am one of the leaders of Shulk players, who play a character whose potential has always been a big discussion point. K... What's the definition of "potential?" From Oxford again:
Well then, huh? On easy inspection, you can see that the words are very similar, basically synonymous. Crack open your friendly neighborhood thesaurus if you don't believe me. From Oxford again:
On top of this, both words share the same synonym "possible"
So what's the big deal with this word? Why are people triggered by it? What makes ZeRo's potential based tier list any different from everybody else's? Well, after digging into it with the discussion, here's the answer:
The Data vs. Theorycrafting
So here's the deal. "The Data" is associated with viability, while "Theorycrafting" is associated with Potential.
Alright, so two new terms. Let's quickly define them...
Wikipedia said:
To bring this into Smash terms...Oxford said:
The Data - In competitive Smash, we have data on all sorts of things. However, the data in this case is our tournament 'top ##' results data. What more specifically? Check this thread out. You can also go onto Smashwiki and search for the placement data on major tournaments, if you want to be more thorough about it. An approach solely from The Data is just copy and pasting it.
Theorycraft - Extrapolating from data such as frame data and parameter data and comparing against results data in order to create a general theory about how viable a character should be.
Which approach should you take? How should you vote? This is probably something you've asked yourself in some way or another as you're looking at that blank tier list in the tier list maker.If you ask most people in the backroom, I would expect they would say you should go mostly from the data. If you want evidence for this, look no farther than the starting post in the last voting thread.
"That is the reality we live in" - The Data
The goal of this approach is to eliminate subjectivity in order to produce more accurate results. The idea is 'it is what it is; look at it. Reality. Behold!' While I appreciate the approach, as an engineer I am obligated to share with you how this approach is flawed. There is a large degree of inaccuracy of interpreting the scores of the data as direct implications of viability. However, don't be mistaken, it seems @Thinkaman does know this, because as he said himself his tier list had many deviations. "I'm dumb and not afraid to be wrong." Although, I would say there's nothing dumb at all about it.
What's wrong with The Data?
The main flaw with the data is it does not account for representation of the character. We can split that up farther into two main areas - quality of representation and quantity of representation. Quality of representation refers to the top level players, like ZeRo, who either do or do not play the characters. Questions like:
or
...raise valid issues with using uninterpreted data to determine viability. In Smash, there's a constant struggle to figure out how much of what you see in battle is the character and how much of it is the player. Unfortunately, there is no objective system worked out to adjust for this. It comes down to personal preference.
On the other hand, quantity of representation just directly skews the data. Having a high quantity of players of a character gives a higher chance for higher quality too --- picking 4 marbles from the bag rather than 2 gives a higher chance of the marble you want. In the end, if the character is more popular, there ends up being more really good players playing him, and more placements in the top ## of the data. All that the data does is count total quantity of results. Therefore, characters are made higher tiers solely because they are played more or the inverse if they are played less.
A live example of this is very, very easy to see... Quick, where should Dark Pit be placed on the tier list compared to Pit?!
That's the difference between a high low tier and a bottom tier. Does that make sense? In the list Thinkaman posted, this may have been accounted for in the "few factors accounted for." If that's the case, well, say hello to theorycraft. The reason why Dark Pit being far removed from Pit strikes us so bad is it just doesn't make logical sense to us. It goes against our personal theorycraft. It goes against how we make sense of this game. With all stats the same and just slightly modified moves, we figure Dark Pit should not be that much lower. He might even be higher in some people's opinions. This is a very obvious example of representation skewing the results of the data to not accurately depict what we are looking for - viability. What do we do to account for it then? We apply our theorycraft. 'In theory, Dark Pit should not be far removed from Pit.' In adjusting his placement to be higher than what the data says, we argue that Dark Pit could potentially be seen in that higher tier, given better representation. We argue that he is that viable.
Is that bad? The actual goal listed by the creator of the data, on the first post of the data thread says no.
So why not just ignore the data all completely? Well, that has a lot of issues...
Problems with Theorycraft
While theorycraft comes in use, its issue is that we as humans get things wrong. We are subject to biases, errors, misjudgments, misunderstandings, misinformation, and much more. The last thing we want is some meme about Charizard sucking to bring him to the last tier placement. A tier list based solely on theorycraft lacks a certain level of order and could even be chaotic. People being susceptible to informational faults is just common knowledge. For an example... why not watch Shulk Commentators Say The Darndest Things.
If this data is what we have to work with, the highest accuracy we're going to be able to achieve is going to have to be some sort of mix. The data should be used but not in raw form. It should be interpreted and adjusted according to our theory of this game. We need to apply logic to it. However, we need to be careful not to take this far. You could see it as a sort of sweetspot or Goldilocks zone. There's some position with just the right amount of balance between personal perspective and the data. That position is where the tier list is going to be the best representation of actual viability.
Conclusion
I encourage people to push their own ideas about how viability in this game is defined. Just prepare for some questions as you do so. 'Why do you think it's so different from the data?' is a question you should be asked if there is a large deviation. If you don't have an answer to that, you probably haven't thought enough about it. On the inverse, if your tier list is almost copy pasted from the data, you might also have not thought enough about it. Theorycraft and potential are an important part of our voting process. If we were just going to go off of what the raw data says, why is a backroom even needed? I would be curious to see more personal ideas put into tier lists. If you're not sure what to take away from all of this, there's one main lesson.
Make your tier list by how you think the metagame really looks, but make sure you understand and can explain why it is different from the data if it is.
Last edited: