The North Carolina Melee Power Rankings! Updated 8/14/14!

[Dr Peepee]

It matters indirectly in a way since you have set counts to go off of so if you place higher then you're more likely to beat people that cement a spot on the PR, but technically the panel does not go by placings when constructing a PR.

 

[Dark Hart]

Oh...

That's dumb

 

[GofG]

me being too exhausted to even play articanus should be assigned the same value as me losing to him; why wouldn't it? i can see that you might wish to draw a line at some point; exhaustion from drugs would be different from exhaustion from being at your mother's funeral early in the morning which is still different from playing badly because you're really ****ing pissed at something.

the problem is that we haven't all agreed on what objectively the power rankings should represent. my upbringing believing that occam's razor is defined in terms of entropy leads me to believe that we should find a single value clearly objectively obtainable from the data which we collect, and we should collect as much data as possible. this is the unix way: accept as much extra **** as you can parse as your program (brains of the panel) parameters, and give the cleanest and most purpose-intentive **** as your output (always going to simply be a list of 10, although you ****ed up this time with your cluster**** at 10).

i don't like the fuzzy human brain in most situations, but i feel like intuitive thinslicing of a smasher's skill relative to another smasher is something which the brain would excel at; do we accept this data as parameters for the program? no, because people would get pissed off (not necessarily because it wouldn't be fair). do we take matchups into account? no, this rewards playing lowtiers by a nonzero amount at worst, and has no effect at best. do we take into account splitting/bracket rigging between friends? absence due to one reason rather than another reason? prior performance in previous PR periods ppppp?

no, we don't use any of this data, but it isn't because this data isn't correlative to skill. we have already agreed to strip melee down to the game which we play, which does not have hyrule temple or items, and have the winner be decided exactly every time objectively by what character shoes up on the victory screen: the function of your match returns this value to the superfunction of the tournament and this is all that matters. We get people who argue that perhaps we should allow items, to allow hyrule temple, to allow wobbling. We also have people who run tournaments for single player mode, for low-tier characters, and other variables which can be changed. this situation is isomorphic to the problem of what values to accept as meaningful when determining the power rankings.

The parameters of what everyone agrees is the ideal "smash tournament", though, has solidified down to a couple different instantiations of what can be considered the same thing, with only the smallest discrepancy. The parameters of what everyone agrees is the ideal "power ranking list", however, has not, even though all of the data which we have agreed upon, and all of the data we have considered, is objective and binary, much like the actual game of smash.

this is because the question of who is better at a videogame is several orders of magnitude more complex (in the mathematical sense) than the question of who won a particular match at a videogame.

i still think it can be solved with math. i would propose eliminating the idea of the power ranking "period" and instead have a constantly-updating and deterministically-realized PR list with the following parameters:

model the probability that a certain player will beat another player using bayes' law, multiplying the prior by a >1 coefficient based on how long ago the evidence used to make the model was generated. use this model to play a thousand iterations of a tournament with every player being given every possible seed, have a payout system which pays out proportionally to every place instead of top 3, and then use the payout to determine powerranking listings which also include relative gaps between players.

edit2: the benefit of doing this is that we have generalized the problem of "what evidence do we accept as valuable when generating power rankings" to math; we can tweak variables as much as we want until the model can predict old data using doubleold data and then it works.

edit: if this is nonsensical ignore it

 

[*P*L*U*R*]

I ****ing love John Wittle.

also, PRs are super serious business. lol

 

[Redd]

Not enough PR drama imo... ours had sooooo much haha.

Ours was also hard to do though. PRs suck.

 

[Sneak8288]

Hopefully if people feel like its in accurate they will go to more things so we have more to work with

 

[Dr Peepee]

omfg my head

@gofg: you should aim me sometime so I could better understand what you're getting at because it would probably take too long in this thread. I'm not discounting your idea, but I want to be sure I understand it so I can discuss with you and the state/panel the merits of that idea.

edit: wish I knew what bayes' law was that would've saved me =(

 

[Dark Hart]

That's a fairly panelitist comment imo sneak

Mr. HighHorsePanelMemberLOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOL

fucn bnuhbz

REAL TALK THOUGH

Maybe in situations like this, the panel should make the list shorter for the time being, maybe 8 (or even 5) spots instead of 10. This could give room for some real honorablementions/inactiveplayers section.

Maybe we should just have less people on our ranking in general, I don't see what's wrong with 8 people, and then like I said some REAL honorablementions/inactiveplayers section

 

[Dr Peepee]

This PR is gonna suck no matter what this time around lol. It doesn't matter if we do top 2 or top 20.

Why are HMs a big deal? In my opinion if you're good enough to make the list then that's good for you but if not then try again next time. Why do you feel HMs should be legit? How far would they go and what qualifies one for HMs in the first place to you?

 

[GofG]

i'll message you on aim kevin

 

[Dark Hart]

iunno I just feel like HMs are really where you could put some of those difficult choices

help us know who you guys are/were considering

that way people know they're heading in a good direction

but seeing the PRs come out like **** doesn't make me motivated to go to tournaments, since if I do happen to beat anyone that "matters" I'll probably lose to some who "didn't place well" so whatev

 

[lord karn]

me being too exhausted to even play articanus should be assigned the same value as me losing to him; why wouldn't it? i can see that you might wish to draw a line at some point; exhaustion from drugs would be different from exhaustion from being at your mother's funeral early in the morning which is still different from playing badly because you're really ****ing pissed at something.

the problem is that we haven't all agreed on what objectively the power rankings should represent. my upbringing believing that occam's razor is defined in terms of entropy leads me to believe that we should find a single value clearly objectively obtainable from the data which we collect, and we should collect as much data as possible. this is the unix way: accept as much extra **** as you can parse as your program (brains of the panel) parameters, and give the cleanest and most purpose-intentive **** as your output (always going to simply be a list of 10, although you ****ed up this time with your cluster**** at 10).

i don't like the fuzzy human brain in most situations, but i feel like intuitive thinslicing of a smasher's skill relative to another smasher is something which the brain would excel at; do we accept this data as parameters for the program? no, because people would get pissed off (not necessarily because it wouldn't be fair). do we take matchups into account? no, this rewards playing lowtiers by a nonzero amount at worst, and has no effect at best. do we take into account splitting/bracket rigging between friends? absence due to one reason rather than another reason? prior performance in previous PR periods ppppp?

no, we don't use any of this data, but it isn't because this data isn't correlative to skill. we have already agreed to strip melee down to the game which we play, which does not have hyrule temple or items, and have the winner be decided exactly every time objectively by what character shoes up on the victory screen: the function of your match returns this value to the superfunction of the tournament and this is all that matters. We get people who argue that perhaps we should allow items, to allow hyrule temple, to allow wobbling. We also have people who run tournaments for single player mode, for low-tier characters, and other variables which can be changed. this situation is isomorphic to the problem of what values to accept as meaningful when determining the power rankings.

The parameters of what everyone agrees is the ideal "smash tournament", though, has solidified down to a couple different instantiations of what can be considered the same thing, with only the smallest discrepancy. The parameters of what everyone agrees is the ideal "power ranking list", however, has not, even though all of the data which we have agreed upon, and all of the data we have considered, is objective and binary, much like the actual game of smash.

this is because the question of who is better at a videogame is several orders of magnitude more complex (in the mathematical sense) than the question of who won a particular match at a videogame.

i still think it can be solved with math. i would propose eliminating the idea of the power ranking "period" and instead have a constantly-updating and deterministically-realized PR list with the following parameters:

model the probability that a certain player will beat another player using bayes' law, multiplying the prior by a >1 coefficient based on how long ago the evidence used to make the model was generated. use this model to play a thousand iterations of a tournament with every player being given every possible seed, have a payout system which pays out proportionally to every place instead of top 3, and then use the payout to determine powerranking listings which also include relative gaps between players.

edit: if this is nonsensical ignore it

Well, I've always operated under the assumption that the PRs represent how well players did vs. other players. Strictly using tournament placings is arbitrary because it includes random things like where you ended up on the bracket and makes it hard to deal with attendance related issues (is one first better than 4 2nds?, etc).

As it is, we look only at set counts. We consider all sets, and look at who has the best overall record. We try to simultaneously consider multiple data points to get closer to an accurate list. This system automatically accounts for attendance, because if you attend so little that you can't get enough data to be placed, then you won't be. It avoids having to use arbitrary numbers like 'two tournaments,' but considers it all naturally. For instance, lozr went to one tournament but got data good enough to get on. However, none of us will deny that if he had more data he would have been higher. Therefore, he was punished for not attending, but is also at least in a somewhat more accurate position than if we required 2 tournaments. Simply put, based off of his performance vs. other players, he is where he deserves for this period.

There are a few things about the system that I am still unsure of. I've never been able to decide whether or not 2-1 victories should count differently than 2-0, or whether sets at bigger tournaments should count differently, or whether or not pools should count less. Currently, all of these things are counted equally. I'm up for debate on that, but not for using a set-count based system.

The only inherent weakness of our system is that it does slightly skew the data because of the old prs. When a new ranking period starts, players still have seeds from the previous prs. Because of this, already ranked players actually get much less data because they don't play good players as early on as lowly seeded players. This actually only really has an affect on the people on the lower half of the prs. The best players will just get far enough in the tournament to get more data anyways, but for players like me/cam/david/etc, a lot of our decisions come down to one or two results. Sometimes this is remedied by those players having good attendance, but often times not. This is a problem of data, though, not of the way we interpret it. The only way to solve it is to either have random seeds (which is just even worse) or have a progressive system like john was advocating. I'd be up for looking into a progressive system.

And yeah, I'm 100% certain this could be mapped using math. That's kind of the whole point of the way it is designed. David thinks so too, and he was going to make a program to do it a while ago but he got lazy, lol. If you think you could make a program that analyzes the data for us after we put in all data, then that would be awesome.

 

[stingers]

get ur heads in the game guys...this is real life

 

[Dr Peepee]

iunno I just feel like HMs are really where you could put some of those difficult choices

help us know who you guys are/were considering

that way people know they're heading in a good direction

but seeing the PRs come out like **** doesn't make me motivated to go to tournaments, since if I do happen to beat anyone that "matters" I'll probably lose to some who "didn't place well" so whatev

Just put whoever you want in top 10 and the other 2 as HMs this time around with GofG if it helps you feel better about them. <.< Arguably the 10th spot is HMs anyway lol.

The PRs coming out badly isn't related to HMs necessarily, and there was nothing that could be done to prevent a bad turnout of the PRs this time around.

If you are so inconsistent that you beat someone good but also lose to someone bad then why should you represent the state?

 

[[FBC] ESAM]

You guys shouldn't even HAVE HMs. It's like you are creating a top 10, and then 11-25 really innacurately and it gives people, idk, a false sense of feel-good that they were mentioned? If you aren't on the PR you aren't on the PR, and regardless of if you are HM, inactive, or just a scrub, get better and be on the PR. People end up arguing over the HMs saying "Why was this person an HM but I wasn't" when it doesn't really matter because NEITHER of them are on the PR.

 

[Dr Peepee]

I like what Esam said. <3

 

[Dark Hart]

I still don't think we should have PRs

they don't do anything other than ruin our butt****ed community

I'm gonna find a way to hack into smashboards so I can delete this dumb **** once and for all

I honestly can't think right now because this is literally the most ******** thing I've seen on smashboards since virgilijuice

You guys suck

seriouslyt


go die in a hole

or something

GAH!!!

 

[Dr Peepee]

They motivated you enough about smash to post about smash. I think it's going pretty well so far lol.

Calm down man no one's gonna die or anything if we have PRs. <.<

 

[Dark Hart]

lol it's just frustrating, being passionate about something

and people obviously don't take me seriously

brush me off like "oh just it's our good friend ramblin' Dark Hart with his angry opinion LOOOOOOOOL
just pisses me off

oh and I can never get a straight answer

 

[GofG]

And yeah, I'm 100% certain this could be mapped using math. That's kind of the whole point of the way it is designed. David thinks so too, and he was going to make a program to do it a while ago but he got lazy, lol. If you think you could make a program that analyzes the data for us after we put in all data, then that would be awesome.

I mean, it is more a matter of tweaking an already-existing bayesian prediction machine for our data.

I am curious as to what David's methods were going to be.

edit: why does smashboards randomly select whether to include my signature?

 

[Dr Peepee]

lol it's just frustrating, being passionate about something

and people obviously don't take me seriously

brush me off like "oh just it's our good friend ramblin' Dark Hart with his angry opinion LOOOOOOOOL
just pisses me off

oh and I can never get a straight answer

I am taking your concerns seriously. I responded to both you and Cam the same way(and probably Dorsey too but I forget exactly what I said to him).

What is a straight answer to you? A shorter one? I'll make my responses to you shorter if that is better but otherwise I don't know what you're looking for, sorry. =(

edit: why does smashboards randomly select whether to include my signature?

Number of characters in your post, including whatever you quote.

 

[Dark Hart]

lol Kevin you don't take anything I say seriously XD

and neither does anyone else

 

[GofG]

Number of characters in your post, including whatever you quote.

No way. My huge wall of text post up there did not have a signature.

 

[lord karn]

I mean, it is more a matter of tweaking an already-existing bayesian prediction machine for our data.

I am curious as to what David's methods were going to be.

edit: why does smashboards randomly select whether to include my signature?

I don't know anything about programming and kinda suck at math. @__@

I think David was just going to make something where we put in the parameters and the data and it just ranks every player put into it.

 

[GofG]

I don't know anything about programming and kinda suck at math. @__@

I think David was just going to make something where we put in the parameters and the data and it just ranks every player put into it.

Err yes, that is how programs work. I am more interested in what happens in between the input and output phases.

 

[Dark Hart]

Gofg wanna play melee?

 

[Dr Peepee]

lol Kevin you don't take anything I say seriously XD

and neither does anyone else

Lucas I responded to your post seriously. I legitimately don't know what you want from me anymore LOL. I guess you want me to change and do exactly what you said but, like for anyone else, you must convince me first.

I have responded to all of your posts and concerns seriously and I think doing any less is wrong on a few levels.

<3

No way. My huge wall of text post up there did not have a signature.

Oh....yeah sometimes you gotta quote and it'll show up every time LOL. SWF is buggy and confuses me sometimes XD

 

[Dark Hart]

Well if this is you taking me seriously, then you obviously don't take me seriously lol

 

[Lightsyde]

The parameters of what everyone agrees is the ideal "smash tournament", though, has solidified down to a couple different instantiations of what can be considered the same thing, with only the smallest discrepancy. The parameters of what everyone agrees is the ideal "power ranking list", however, has not, even though all of the data which we have agreed upon, and all of the data we have considered, is objective and binary, much like the actual game of smash.

this is because the question of who is better at a videogame is several orders of magnitude more complex (in the mathematical sense) than the question of who won a particular match at a videogame.

Mmmmmmm ****ing love John Wittle. Really interesting explanation. You'll need to define/explain some of the more mathematical stuff towards the end for me in person to better understand the system you're proposing, but I think your analysis of the problem is correct enough.

Also, I kinda like the idea of a top 15. Why?

-It eliminates the need to remove anyone (being that it seems to be consensus that if earned spots are just that regardless of tentative future attendance),

-Resolves the 3/4/5 way tie for 10th...at least far enough to let people who are within the spectrum of contenders for that spot be held above the rest of the unranked who are not. The case has been made that any of those people could be in that last spot, so why not extend the list a little and properly designate the potential 10th's to be PR'd?

-If we have a top 15, by necessity, there will be more ranked matches between ranked players and therefore more data to work with, even with lowered attendance. This lowers the ambiguity problem of determining how bad losses to unranked players are by narrowing the list of unranked players.

Just a thought.

 

[Dr Peepee]

Okay Lucas. I'm sorry you feel that way.



@Josh: It takes away from the prestige of being a PR if we open it up to include everyone(top 15). Sometimes we don't even have 15 man tournaments. <.<

Expanding our PR number size should only happen if our skill levels open up across the board and either cause lots of swapping of data(not inconsistency but so many similarly skilled people) or some other irregularity that makes placings pretty hard to distinguish between.

That 3 way tie and other shenanigans associated with this update are a result of the length of the period and irregular data scattered throughout it(including spotty attendance). One irregular period isn't cause for any kind of overhaul in the PR method because they all would have been screwed up due to how difficult and unusual the data was.

 

[GofG]

That 3 way tie and other shenanigans associated with this update are a result of the length of the period and irregular data scattered throughout it(including spotty attendance). One irregular period isn't cause for any kind of overhaul in the PR method because they all would have been screwed up due to how difficult and unusual the data was.

The model couldn't handle a fringe case and so the model could be improved upon, period.

 

[Dark Hart]

k another day then

 

[Dr Peepee]

I could have just chopped 2 of them off and left one. To some that's an improvement and to others it's not. Every model could be improved upon.

 

[GofG]

if you could have objectively determined which of them was actually better than the other two, you would have done that, so I assume you couldn't.

what would be better than a PR system would be a point system where people were ranked based on numbers of points, which would be the arbitrary unit of value used to determine PR placings anyway. It is perfectly okay for two people to have the same number of points, but they can't both be "5th best" in a community.

since it is necessary to build such a system, even with the value points being extremely abstract and never manifesting in anything other than the minds of the panelists, you might as well have that system be the main thread of the PR program rather than a function called by it.

 

[lord karn]

Mmmmmmm ****ing love John Wittle. Really interesting explanation. You'll need to define/explain some of the more mathematical stuff towards the end for me in person to better understand the system you're proposing, but I think your analysis of the problem is correct enough.

Also, I kinda like the idea of a top 15. Why?

-It eliminates the need to remove anyone (being that it seems to be consensus that if earned spots are just that regardless of tentative future attendance),

-Resolves the 3/4/5 way tie for 10th...at least far enough to let people who are within the spectrum of contenders for that spot be held above the rest of the unranked who are not. The case has been made that any of those people could be in that last spot, so why not extend the list a little and properly designate the potential 10th's to be PR'd?

-If we have a top 15, by necessity, there will be more ranked matches between ranked players and therefore more data to work with, even with lowered attendance. This lowers the ambiguity problem of determining how bad losses to unranked players are by narrowing the list of unranked players.

Just a thought.

Actually, it really wouldn't resolve the tie. The tie is there because we literally couldn't (or at least didn't see a way to) resolve the lack of data between those three players. We know those three players are the ones in contention for the spot, so if we went to 15, it would just be a three way tie for spot 10, and then people after that like normal. Usually when we make PRs we have a pretty good idea of who would fill the next few spots if they were to keep going (and that's usually how I try to seed tournaments when I end up being the person seeding). For instance, Gofg would certainly be next in the list if it kept going. That being said, I'm neutral as to whether or not we extend the PRs.

The problem is that the data is so ambiguous between me/sneak/david, that we as humans can't see whatever little piece of data would make one of us jump ahead of another. It would probably be something that has to do with who the people we beat/lost to beat or lost to. When you have so many variables it's hard to consider that many second order things. Another reason why a program would do this much better. It would be able to consider all of that stuff.

 

[Dr Peepee]

if you could have objectively determined which of them was actually better than the other two, you would have done that, so I assume you couldn't.

what would be better than a PR system would be a point system where people were ranked based on numbers of points, which would be the arbitrary unit of value used to determine PR placings anyway. It is perfectly okay for two people to have the same number of points, but they can't both be "5th best" in a community.

Why would it be better?

 

[bossa nova ♪]

i love john wittle


please just sing to me at all moments your beautiful song that becomes my sense and sustenance

 

[lord karn]

if you could have objectively determined which of them was actually better than the other two, you would have done that, so I assume you couldn't.

what would be better than a PR system would be a point system where people were ranked based on numbers of points, which would be the arbitrary unit of value used to determine PR placings anyway. It is perfectly okay for two people to have the same number of points, but they can't both be "5th best" in a community.

since it is necessary to build such a system, even with the value points being extremely abstract and never manifesting in anything other than the minds of the panelists, you might as well have that system be the main thread of the PR program rather than a function called by it.

In all actuality, one of those three players probably deserves the spot. But it would be decided based off of some small observation that none of the panelists saw. For instance, it could go to the second order of who beat/lost to who, who they beat/lost to, etc. The problem is not with the system but with the panelists inability to consider all the data that relates to the decision, because it is such a close call. A true tie could only exist in a triangle situation (if I beat david, who beat sneak, who beat me) if that was ALL the data that we had. But a case that is actually a true circle is almost impossible to have when we have so many different set counts (even if the decision has to come down to some random not so good player someone beat).

Again, that's why we need a program. @__@

 

[bossa nova ♪]

ranking history is an important thing that i think is sometimes overlooked. at least, the right aspect of it.


if there were to be a point system (which, from a conversation with sneak a long *** time ago, definitely seems like the best way to go) i think what you were on the last ranking should factor into the equation. and if your new only from inactivity, but you've been normally pretty highly ranked, depending on the other players around you, there should be some sort of consideration for that.


points are just solid though. they're whole and understandable. unless you want to keep being bothered about the way you guys go about doing things, SOLIDIFY what it is you do, for EVERYONE to understand. this arbitrary stuff just doesn't seem to work out that well. (please do not say "either way it's arbitrary" or anything of that nature, you will die if you do)

 

[GofG]

The problem is not with the system but with the panelists

the panelists are the entire system from beginning to end