• The new bonus episode of the Smash Brothers Documentary Metagame, Godslayer, is now available on Vimeo

  • Welcome to Smashboards, the world's largest Super Smash Brothers community! Over 250,000 Smash Bros. fans from around the world have come to discuss these great games in over 19 million posts!

    You are currently viewing our boards as a visitor. Click here to sign up right now and start on your path in the Smash community!

CPU's A.I. Explained "The learning process"

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
There Has Been Much discussion on whether or not the CPU learns I have decided to put it to the test Here is what i found

Validated
1.each CPU has been coded separately they do not share a Collective AI
2.ALL The CPU's Have problem solving Skills
3.Some CPU's have been programmed better then others
4.The CPU has a small learning curve but it does exist...Not How you or I would learn but learning none the less
5.Its learning is based solely on Circumstance,Positioning, And Success Rates
6.The CPU does not attack targets at random It selects And Prioritizes them (The basis of which is currently under validation)


Currently Under Validation
The CPU does not attack based on % or whether or not you are human or CPU but instead an oddly setup priority system Which I am Currently Testing(see tests 3&4). the main basis of these test are based on one or many of the following, distance,harm it has taken from the target,It is Locked onto a Random person at start of match this Lock changes at certain intervals or event triggers Locking onto th next closest Target,The CPU is locked onto a Human Player at the start of the match if the human is out of reach to picks a new target,and/or An unknown and/or unforeseen possibility.

Here are some videos Showing tests which I ran. Only Some CPU's won't land on spikes others will. but they all recognize the spikes as a platform not as a danger before they jump. though after trial and error they recognize the location as a Place which isn't safe to go to where as a human would recognize the spikes as a danger not the location itself.


Test#1

Edit: After more tests the CPU has become progressively better at understanding the spikes and is no longer chasing me off the edge it still occasionally jumps down to the spikes but recovers right away 1 of the most peculiar things i noticed is that the CPU ties its own damage into the equation i made the spikes lower in order to make it harder for the CPU to recover and falcon w8'd to use his up B on 3 different occasions till he was at 60% from the spikes. Leading me to believe the CPU is self aware of its own damage I tested to see if it is aware of others and found that the CPU is aware of others damages because they hunt the person with the Highest % most of the time especially after they spawn


So next the test will be overcoming obstacles Test #2

Many People have discussed that The CPU only attacks hum players I decided to launch a test to check this Test#3
It was found to be true initially but as the CPU's realize they cannot attack the human player they will lock onto each other

Which Lead to proving whether the above conclusion is correct or not whether or not this can be changed or modified Test#4
And it was discovered that the CPU attack priorities could in fact be altered and effected by different circumstances

All the Videos posted are not the tests in their entirety they are merely example to demonstrate the process in which a test was carried out. rest assured things like alternate characters,maps,and situations were taken into account


"varios things I have talked about in the "Do CPU's Learn" Thread
Its not complicated the CPU records your most frequent actions and when deciding probabilities I.E. your next action it essentially rolls a dice...if u spam A alot in X location in X circumstance there will be a higher possibility on the dice that that will be your next action so it does X line of code but its not so much learning as it is recording your most frequent attack style. it might seem the computer is learning but its just your being repetitive is increasing the probability that Line X will run. The AI may seem like its reading your mind or actually learning something but its not. use a different style of play for a while and it will change probability X. this can be easily chained into the cpu's AI to based its attacks off the same probability I.E. making the cpu seem like it copies your style of play.

For Comparison Only
NHL 2008 ON THE FLY AI is similar
Different Game but Same Principles
http://www.youtube.com/watch?v=QMqBejbOAEs
Might Help You Understand...

The Game Designers Probably tied the AI for guessing your actions into the same AI that you fight it would make less code to write if they used the same variable assignments I.E. you use hammer spam 50 million times the computer will both tie the probability that u will use hammer spam into fighting you as it will when it fights I.E. it will use hammer spam. the code for guessing your actions is probably tied in the same argument as for guessing its own(yes the computer essentially guesses its every action it has no master plan) which is good on the designers part. When i say probability i say so because the computer needs to not make the perfect decision every time or you wouldn't be able to beat it EVER which is why the CPU sometimes does things that a human would never do. The coders assigned priorities as per when to do certain actions as you lower the CPU's level you decrease the chance that it will choose the most correct choice
For example When lower level AI is in air+in striking distance
50% hammer spam(based off move frequency usage)
20% Fair
10% Nair
20% Do nothing
A higher level they would merely adjust the probability
70% hammer spam
20% Fair
15% Nair
5% do nothing
(these percentages are obviously flawed and are merely to help others to understand don't take them seriously)

It doesn't "Learn" it records the most frequent move usage per character which I assure you takes up almost no memory if i use A alot in X location each map would have assigned areas in order to to notify the cpu of its location the computer can only do about 6-7 actions at any given time. the locations would be measured as instances based on the cpu's distance from map edge's and the center of the map and i can tell you that any given instance say position x,y (the game is 2d for play purposes) would be then measured to the most local location I.E. the closest platform. that is where it would decide its actions from simple lines of code

if on platform then
X code
else measure dist (get x,y)
measure fall rate/Speed
detect enemy/Attack (based on move frequency[See Earlier post] when x dist to x dist from platform and based on the if it has enough time to finish attack before hitting ground the computer will never execute a move it doesn't have time to complete)
Nearest Route To safe landing

(Now that is an example the actual code would be more complicated)

The nearest route to safe landing I Learned by messing around with a cpu doing a series of normal play debugger tests...sad to say this is a little disappointing on the programmers side it allows the cpu to suicide in many ways
(Cruel Brawl and etc is not an example that happens because the computers B-Moves have been disabled and the coders didn't change the code for polygons so the computer thinks it can land safely when it jumps not knowing its B move is disabled) my example is when the area is actually unsafe to land in I.E. there is an attack in progress or Spikes etc
As per why i am disappointed it is because this means the computer will measure to see if there is a place to land before it jumps but not whether or not it is safe For example you can create a custom map

. . . . . . .--------(platform)
. . . . . . .I_I_I_I_I . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . ^^^(spikes). . . . . .
The computer will recognize the spikes as a platform in which it can land before it jumps but not whether it is safe or not (its Not Safe) when you jump off the ledge the computer will follow you and try to attack etc only after said attack will it try to find a safe place to land now this was lazy on the programmers part but the cpu has been told to NEVER touch spikes for any reason So the computer will automatically try to find the next safe location which is the above platform which it is to far to make it to but the computer has been told to use its recovery move any way (this is for ascetic reasons people don't like watching things fall to there death without trying to survive) at which point it will fall to its death I did this little experiment in order to prove the computers AI is only based on measured distances and move frequency
the computer WILL NEVER LEARN to jump on the spikes to survive no matter how many times you do it because falling on spikes is not a move (you don't actually have to land on the spikes you could use pit to fly back etc) So I repeat hopefully for the final time the CPU does not learn it records move frequency and acts accordingly I play pikachu alot and i use Quac+Dair the computer has started to use agility alot more often and down Air but it does not use it out of agility because the move does not actually have enough time to finish the computer will never glide toss or do anything that there technically isn't enough time to do because it doesn't actually copy you it merely records the move frequency
Some of this may seem overly complicated....but anyone with a basic understanding of code should understand

Hopefully You guys find this helpful/insightful
P.S. You Have Been Crit By "wall of text" instant K.O. Sorry...
 

Cease Tick

Smash Ace
Joined
Jun 30, 2008
Messages
579
Location
/tr/
Very fascinating... are there any other kinds of tests that one could perform, do you think?
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
Unsure as of yet usually I play around till i notice a peculiar interaction on the AI's part then see how it can be exploited or understood
 

BanjoKazooiePro

Smash Champion
Joined
Sep 20, 2008
Messages
2,144
Location
Wisconsin
I still think they're ********, but they have shown that they may be smarter than some things, maybe Britney Spears or Paris Hilton.
 

Alus

Smash Champion
Joined
Jul 7, 2008
Messages
2,539
Location
Akorn(Akron) OH
NNID
Starsauce
3DS FC
5327-1023-2754
So if you shieldgrabbed often would that mean that a computer would adapt to attempting to sweepdodge your grab?
 

feardragon64

Smash Champion
Joined
Apr 9, 2008
Messages
2,154
Location
San Francisco
Very interesting indeed.

I'd love to hear the outcome of this. Some questions that come to my mind are:

-Do computers recognize stages in blocks? In other words, if you have two spikes, does the computer see them as different hazards or one? That will obviously be affected by the way in which the perceive a "safe landing spot."

-Is there a priority list in what they deal with? I'm sure there is, but what is it?

-Computer vs. Computer scenarios. I recall someone doing a test to see what happens when a player plays on a stage with 3 computers, but the player puts themselves in an area that the computers cannot reach. The stage looked like this
| . . . . . . . . |
| . . . . . . . . |
|__comps___|
. . . . . . . . . .
___p1____

The player stayed in that area below the computers and they computers....almost outright refused to attack each other. They would all throw out random attacks but would very rarely attack another computer unless it was a long ranged attack(falcon kick). This might be interesting to experiment with.

-So many questions, but most of them take too much time to test. This was a good read. I look forward to any other information you can contribute. =]
 

ZeroFox

Smash Lord
Joined
Sep 6, 2008
Messages
1,048
Location
New Jersey.
Very interesting. I wouldn't have thought that each CPU has a different programming to it. I kind of thought that there WAS a collective AI.
 

MasterWarlord

Smash Champion
Joined
Aug 24, 2008
Messages
2,857
Location
Laughing at you
Luigi has been said to be the smartest computer on many occassions, yet Captain Falcon is the best learner? Perhaps there is more to these bots then we thought. How useful is this learned knoweledge to the computers?
 

Morrigan

/!\<br>\¡/
Joined
Mar 10, 2006
Messages
18,681
My theory is....


Brawl's CPUs weren't programmed to work correctly on custom stages.
Did you try the exact same thing on normal stages?
 

Sigrid Fiinikkusu

Smash Apprentice
Joined
Feb 5, 2008
Messages
144
Location
Gresham, OR
Very interesting indeed.

I'd love to hear the outcome of this. Some questions that come to my mind are:

-Do computers recognize stages in blocks? In other words, if you have two spikes, does the computer see them as different hazards or one? That will obviously be affected by the way in which the perceive a "safe landing spot."

-Is there a priority list in what they deal with? I'm sure there is, but what is it?

-Computer vs. Computer scenarios. I recall someone doing a test to see what happens when a player plays on a stage with 3 computers, but the player puts themselves in an area that the computers cannot reach. The stage looked like this
| . . . . . . . . |
| . . . . . . . . |
|__comps___|
. . . . . . . . . .
___p1____

The player stayed in that area below the computers and they computers....almost outright refused to attack each other. They would all throw out random attacks but would very rarely attack another computer unless it was a long ranged attack(falcon kick). This might be interesting to experiment with.

-So many questions, but most of them take too much time to test. This was a good read. I look forward to any other information you can contribute. =]
I've done that before lol. I left my Wii on for a few hours and when I came back the still hadn't barely touched each other. Then again they were only level 1's lol
 

feardragon64

Smash Champion
Joined
Apr 9, 2008
Messages
2,154
Location
San Francisco
LvL 1's dont use kill moves...
I lol'd
Pretty hard.
But then again level 1's don't do much of anything. 70% of the time they walk around.
10% of the time they stand
10% of the time they just randomly shield or grab
10% of the time they throw out random attacks with no regard to where you are.
 

VideoCupcakes

Smash Rookie
Joined
Nov 11, 2008
Messages
19
Lvl 1s don' t use kill moves? Ya right, your a funny man. They do use final smashes (if they get it of course), smash attacks, special attacks, they don't charge them, but they can kill you. They do use knockout attacks, or kill attacks, as you put it.
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
If the level one CPU is giving you a hard time this may be for you.


(further testing will be required at least 1000 matches to make a more precise % calculation on the LVL 1 CPU's hit rate and chance to use smash attacks but rest assured i shall personally watch each 99 stock lvl 1 vs lvl 1 no items unlimited time match to perfect this)
Until then As a tip 70-90% of the time if you stand still a lvl 1 CPU wont hit you. This might help as a possible tactic the next time you face one also try hiding behind large enclosures like rocks and launching a sneak attack or using a character with lots of range. don't worry they used to give me trouble as well until I learned there secrets always remember surprise is KEY..

I was joking when I said lvl 1 CPU's don't use kill moves....try reading your own writing aloud before you post something if it sounds to you like a whine/flame/rant you probably shouldn't post it...because it No offense makes you sound like an idiot ...if you want to point out a flaw or mistake you think someone has made try phrasing it as a joke(not at the person but the situation)...or at least be polite about it.

Examples "Man LVL 1 CPU's Do so use kill moves! They kill me all the time!"

Or "After vigorous testing I.E. waiting patiently for 30 minutes for the LVL 1 CPU to actually hit me and another 25 for my damage to get high enough i have discovered that LVL 1 CPU's can actually kill!"

Trust me people will be aware that you actually don't lose to level 1 CPU's and everyone will know that it didn't take 55 minutes

but if you don't feel like joking around at least do it in a polite way

Example "Correction LvL 1 CPU's Do use kill moves like smash attacks just not as accurately or proficiently as the other CPU's"
 

13th Nightmare

Smash Cadet
Joined
Nov 12, 2008
Messages
56
Location
East Kilbride, Scotland
I sometimes wonder if the lvl 9 CPU have a "collective mind" A few times when I battle against a group of lvl 9 CPU's in a free-for-all, they have the tendency for forgetting about each other and all go for me. I have done a few battles where I stay away from the battling and see what happens. Again, they forget about each other and all go after me
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
I have no problems with lvl1 cpus, it is just that it is possible for them to kill you.
Ok the fact that ANYONE would feel the need to mention that they have no problems with level 1 cpu's...makes me die a little inside....I am going show show a picture to help illustrate to you what you just said...


anyone with an IQ over lets say 60 the average being much higher...can beat a level 1 cpu upon picking up the game and playing it for the first time... If somehow you got the impression that anyone actually thinks you have trouble with lvl 1 cpu's and felt the need to defend yourself against said allegations...I just don't know...you may in fact have no sense of humor...or your a cyborg

(just in case there is any confusion for future references I DON'T ACTUALLY THINK YOUR A CYBORG)

 

PK-ow!

Smash Lord
Joined
Mar 28, 2008
Messages
1,890
Location
Canada, ON
I sometimes wonder if the lvl 9 CPU have a "collective mind" A few times when I battle against a group of lvl 9 CPU's in a free-for-all, they have the tendency for forgetting about each other and all go for me. I have done a few battles where I stay away from the battling and see what happens. Again, they forget about each other and all go after me
I think that the way they rate their own "success" in the internal metric they check for error (you know, to do 'learning') is just by how much the player characters get hurt and K.O.'d. They don't consider the injury of other CPUs to be a goal. And this is a stable state, since if they're never hurt by other CPUs, then they never 'defend' themselves.
(I wonder what would happen if their initial behaviour was one where they hit each other, though.)

See, they're not trying to win, because then they could only re-evaluate themselves after the match. Instead, they need to notice when they are winning. AND, since you could theoretically screw with them by playing horribly handicapped matches, they need to use a measure of the "slope" of their success value, not its actual value - hence, checking when you are being hurt.

In contrast to Dekuu, I think that custom stages are the main reason we're seeing this jump in programming. An A.I. like this is the only hope the computers had of functioning on custom stages at all. They needed to be able to work (forget about do well, just *work*) on every stage. So their A.I. is like this - they set a goal, and they seek it while avoiding hazards.
 

DJKingDavid

Smash Apprentice
Joined
Nov 5, 2008
Messages
82
Location
DALLAS TX WESTSIDE OG
I have been playing 3 Lv 9's in a custom stage I built called box-o-death. Everytime i play them we all have 99 stock with no items and theres only a very small hole to get KOed through. I noticed that everytime i fight them they are aiming me out of this hole...I.E. usmash through the hole...or they team up to knock me out of the box. Do you think they understand why I am not dying?! Please let me know! :)
 

mc4

Smash Journeyman
Joined
Jul 15, 2008
Messages
283
omg vv2, the second picture is pretty funny. I do actually think that cp's (level 9 being the subject) do learn and adjust to your moves. Also when playing multi they will generally attack either the player in the lead, or the player with the highest percentage.
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
I have been playing 3 Lv 9's in a custom stage I built called box-o-death. Everytime i play them we all have 99 stock with no items and theres only a very small hole to get KOed through. I noticed that everytime i fight them they are aiming me out of this hole...I.E. usmash through the hole...or they team up to knock me out of the box. Do you think they understand why I am not dying?! Please let me know! :)
I doubt the programmers installed trajectory calculations into the AI way to much work(variables like bouncing,move sets,%'s and yes they programmed these things into the game when you get hit but to make it so the CPU can calculate it and act accordingly per move per character...would take awhile and they were busy putting sonic into the game[haha])anyways there would not enough to gain from doing it...So it doesn't know that hitting you will actually get you out of the hole so my thoughts are..

I ll have to test it later to be sure but..
my hypothesis would have to be that the CPU is hitting you with attacks that in the past it has had success with but now its not. so it is adjusting the location,circumstances in which is IS having success
I think you will find that at first the CPU had trouble killing you but the more you play that same map repetitively the more the CPU will adjust where/when/how it should attack in order to kill I.E. if it has had a kill in the past in location X(under the hole) using Uthrow or Usmash then the chance it will use those attacks in that location will be higher.
 

VideoCupcakes

Smash Rookie
Joined
Nov 11, 2008
Messages
19
I like to be nice. So i am not going to say anything about that redundancy post Vv2. And your lucky I'm not. You won't like an angry Cupcake.
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
<3 dekuu that made my day. anyways i should be posting a few new tests today So check the original post
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
Updated 2 NEW tests Added #3 & #4 These Tests pertain to the AI's Targeting system and how it prioritizes targets
 

dumba989

Smash Apprentice
Joined
Sep 29, 2008
Messages
81
Location
Georgia/New York
I thought i noticed that, because the CPU from time to time, god forbid when i play them when no one else is around, will do combos, strats, and techniques that me and other people that have played on my Wii do
 

Birchnatch

Smash Rookie
Joined
Oct 16, 2008
Messages
12
Location
SF, CA
wait im confused, so each cpu charachter keeps its learned knowledge from match to match? or just in the match then forgets it untill the next? and is the knowledge based on ur name, or ur character or on ur system? does it change in special curcomstances such as sudden death or special brawl? and can the player # affect the AI? :confused::confused::confused:
 

HeroMystic

Legacy of the Mario
Premium
Joined
Aug 3, 2008
Messages
6,473
Location
San Antonio, Texas
NNID
HeroineYaoki
3DS FC
2191-8960-7738
Very interesting.

So it seems that each character was specifically given a different intelligence capacity and different tactics. However when it comes to attacking any person, it's generally the same thing.

How odd.
 

Vv2

Smash Apprentice
Joined
Oct 29, 2008
Messages
104
The next test I am going to attempt to try is figuring out whether the CPU bases its moves off success rates I.E. things that get kills / wrack up damage (which will probably will a test in itself come to think on it) or is it based on frequency

in layman's terms is it monkey see monkey do or / is it X% done in 5 seconds best reward ratio / or this move kills X% of the time at percentage X

or possibly a combination of them all

I am thinking using Falcon punches 50 times without hitting something and see how often falcon uses falcon punch in a 1 min match and then 50 times hitting something each time then seeing again how often he uses it in between each test doing a few fights with using F punch at all to lower the rates back down

but with out wiping my save the results might be skewed any input on this would be helpful
 

luvs2pluck

Smash Apprentice
Joined
Sep 7, 2008
Messages
115
Location
College of William and Mary, VA
one interesting targeting thing to test would be a team battle with two humans vs two cpus. see if the cpus start off targeting one player, and if they do, what makes that one player a priority over the other?
 

Sup Dawg

Smash Apprentice
Joined
Aug 29, 2007
Messages
107
Location
Burbank, CA
wait im confused, so each cpu charachter keeps its learned knowledge from match to match? or just in the match then forgets it untill the next? and is the knowledge based on ur name, or ur character or on ur system? does it change in special curcomstances such as sudden death or special brawl? and can the player # affect the AI? :confused::confused::confused:
hey I was wondering the same thing!

can comps "remember" how to fight someone based on the name they have chosen for themselves?

can they "remember" the way you use individual characters, even under individual names?
 

felix0-9

Smash Cadet
Joined
Sep 8, 2008
Messages
28
Location
Yes.
Interesting. Certainly, the CPUs on my Wii have recently learned to gimp and even on one occasion (gasp) Dedede super chain throw. But I think it's more likely that this is a programmed reaction to how well you do things then that they learn with you. I.E. the computer detects that your amount of meteor smash KOs has gone up or your amount of consecutive hits or something like that. Then it accesses proportionately more of its programmed skill.
 

NFS (sonic)

Smash Cadet
Joined
Nov 23, 2008
Messages
28
Location
FL
Its pretty much a matter of 'fitness' levels of a certain situation, they record each match and compare the results with your previous matches, then they basically find moves that would best suit that situation IF it happens to be recreated.
 
Top Bottom