amiibo Raising

[Shiliski]

Regarding the differences between Level 50 Amiibos and Level 9 CPUs, I did a quick experiment to try and discredit the notion that they are in any way the same.

10 Games in a row. Amiibo is level 50 the whole time and so is already trained. CPU is of course level 9. Stages are randomly selected Omega stages. CPU and Amiibo are both Link. 3 Stocks, no time limit, no items.


Game 1: Amiibo wins by 1 Stock

Game 2: CPU wins by 1 Stock

Game 3: Amiibo wins by 2 Stocks

Game 4: Amiibo wins by 2 Stocks

Game 5: Amiibo wins by 3 Stocks.

Game 6: Amiibo wins by 2 Stocks.

Game 7: Amiibo wins by 1 Stock.

Game 8: Amiibo wins by 2 Stocks.

Game 9: Amiibo wins by 2 Stocks.

Game 10: Amiibo wins by 1 Stock.

Amiibo won 90% of the time. 30% of the time it had one stock remaining, meaning a close battle. 50% of the time it had 2 stocks remaining, showing a significant lead. 10% of the time it won by 3 stocks, but considering that's only 1 match we can't really tell if it's really supposed to be 15% or 5% or whathaveyou. Amiibo only lost 10% of the time, and even then it was only by 1 stock, and it comes with the same disclaimer as the 3 stock match

So we have it that 60% of the time, Amiibo wins by a significant margin, barely wins 30% of the time, and only loses 10% of the time. We can be fairly certain that, at the very least, the Amiibo is stronger than the CPU.

It's also worth noting that the Amiibo struggled at first, but after it lost 1 match it started playing significantly better. This coincides with Nintendo saying that the amiibo learns faster when it loses. I also noticed that the amiibo was really strong shortly after that first win, but after the 3 stock it didn't seem to do as well. I expected it to continue 3 stocking the whole time, so the fact that the amiibo did not do this is worthy of note.

Furthermore, there are a number of behaviors that the Amiibo did quite often that the CPU did not.

Example A: The Amiibo would pull out a bomb and hold onto it until it exploded, often performing other moves while seemingly forgetting that the bomb even existed. Out of 10 matches, the Amiibo held onto the bomb too long a total of 8 times. The CPU only held on too long 2 times, but the CPU usually had a reason for holding onto it, such as getting stuck in a combo so that it couldn't throw it. It's also worth nothing that, before these 10 matches, the Amiibo almost NEVER used bombs (less than one time a match), whereas when it faced the CPU mirror it started using the bombs poorly and then eventually started using them correctly.

Example B: The Amiibo would almost always jump before using the bow (at least 5 times a match), while the CPU would almost never do this (only once in all 10 matches).

Example C: The Amiibo would often retreat from the enemy (presumably so that it could start spamming projectiles, which is a pattern that it has had for a while now), while the CPU would never retreat. This often led to the Amiibo running away from the CPU and the CPU chasing it down.

Example D: Related to C, the Amiibo seems to have 2 distinct modes of attack: "Run away and spam projectiles" and "Approach and Attack". The CPU only has 1 mode, which is "Approach and Attack".

Based upon this evidence, I can be very comfortable in the conclusion that the Amiibo AI is very different from the CPU AI, based upon the fact that they have very different flaws and patterns. Secondly, I can say with confidence that the Amiibo is consistently superior to the CPU, not enough to 3 stock it every single game but enough to win out overall. Thirdly, there are several notable changes in the Amiibo's behaviors that happen over time, where as the CPU has displayed no notable change in its behaviors. These changes tend towards an improvement in the Amiibo's performance, which is synonymous with learning.

I think we can say with certainty that the Amiibos actually do learn, that Nintendo didn't outright lie or exaggerate when they said that Amiibos learn, and that the Amiibo AI is not the same as the CPU AI.

The Amiibo is not consistent with the default CPU. It is consistent with an AI that learns.

Also I did not see any significant bonus damage from the Amiibo. If there is any such damage increase, it is well below 5%.


EDIT:

I just did an experiment where I reset the amiibo, took him into a 1v1 match on FD, and didn't put in any inputs. I just set the controller down and did nothing. The amiibo walked right up to me and stared, also doing nothing.

After a while I started spamming taunts to see if I could get it to also spam taunts, but it wouldn't do it. It copied my top-tier strategy of doing nothing and not moving, but it didn't copy the taunts.

After a while I started attacking it. Only then did it start attacking back, instantly leveling to 4. Next match I took on the same strategy and stood still, and the amiibo again walked right up to me and did nothing. This went on until suddenly the amiibo leveled to 6 and started throwing out attacks.

 

[mugwhump]

Dude, what you basically said was "They're not using reinforcement learning, they're using reinforcement learning!"

Oh no, what I meant was that, if they've already got the reward function, they should just go for nothing but reinforcement learning. Using supervised learning when the person you're learning from has a different action set (since they're a different character) is kinda sketchy. There are hybrid supervised reinforcement methods out there, but I don't know of any that match the profile we want, which is:

1. Nonassociative
2. Capable of online updates
3. Incredibly (I mean incredibly) fast, both to run and to update
4. Can store its learned parameters on a < 1KB NFC chip. This is a real sticker and the cause of #1 above. These chips might be in the right price range for amiibos (honestly they might be too expensive), and they can only store 716 bytes. Amiibos also have some other stuff they need to store.

Look I'm an experienced programmer and I have a pretty good idea what is and is not possible. If you don't even know what Big O or O(1) means in terms of measuring program efficiency you're not even anywhere near my level.

I'm hardly an expert, but I'm not pulling stuff out of my butt. I did do a research project on supervised learning in video games, where I used radial basis function networks (with both online and offline updates), decision trees, LWPR, gaussian processes, and neural nets to train an AI that played Asteroids (somewhat) like a human. Didn't have time to implement any reinforcement learning algs, but I did read a book about it. I also did some poking at Brawl's AI (since I wanted to do that as my project but the Wii's memory was too limited and my ASM wasn't strong enough). So my skepticism about amiibos using supervised learning is based on my own experiences.

Status to Response mapping (different people might call it different things) is pretty basic, can be done in constant time (in layman's terms, very quickly), and is exactly the kind of thing I'd expect from Nintendo. People have already used such learning AIs to learn simple games like checkers. (As a side note, most chess AIs use a different method based on completely different properties.)

Status to response mapping? You're just talking about general approaches where game states are mapped to actions, yes? That's extremely broad. Could you link to a paper describing what kind of algorithm you think is being used? That would probably be easier.

The problem with that is storing even a highly-simplified description of the current game state, in a game as complicated as smash, would go waaaay over the memory budget of NFC chips. For example, in my Asteroids project, I simplified the game state down to 22 variables. Things like ship position, ship speed and rotation, some asteroid positions, etc. If you used only a single byte for each variable, you can fit... maaaaybe 20 descriptions of the game state on an NFC chip. Assuming smash is no more complicated than Asteroids. It certainly rules out non-parametric methods. Will you please just believe me when I say that any kind of associative or situationally-aware method is really unlikely?

You clearly didn't see what I saw. It mistook my mistake for an effective tactic and deliberately went from "standing perfectly still and watching me die" to "jumping into the waterfall". There's no question in my mind. Considering some of the other things you've said I'm not really inclined to believe your statistics. You are ignoring the fact that these Amiibo are learning in VERY controlled circumstances where they only have to worry about maybe 20-30 things happening at once. The numbers you give are more believable for AI that have to interact with the real world, where the amount of uncontrolled variables can be ridiculous. Computers can easily deal with a small number of variables in real time.

It saw me do something new and wanted to see if it was effective. It experimented. It learned that it was not effective and went back to the tactics that it already knew.

Smash is simpler than reality, but not simple enough for that. Copying something after seeing it once, immediately after it happens, is just not going to happen in a game as complex as smash. You saw it happen one time and you yourself thought it was a bug in the learning algorithm. Why not just a bug in the regular AI? It's never been very good at navigating terrain.

You clearly don't understand how reinforcement learning works, but we've already established that. It makes perfect sense to limit move learning to chunks. Or rather, said another way, it makes a lot more sense to limit the amiibo to, say, normals at first. By focusing on one thing the amiibo can get a pretty good grasp on it before deciding to expand its horizons.

It's not generally done, at least not in the methods I've seen. Optimistic initial values might be used to make sure the entire action space is explored, but I haven't heard anything beneficial about segregating actions into groups and blocking some of them off. Can you link to an algorithm that does this?

Because you did not do it correctly. If you wanted to train him as a grabber, you should have picked Link, only attacked via grabs, and then try to shield every thing the amiibo does. It would have imitated the successful grab, then found success on its own because you kept shielding. The way you did it, maybe it would grab, or maybe it'd do other things. Even if it found out that grabs were effective it would still experiment on its own until it realized that other tactics failed. Unless it sees you grabbing, and having success with grabbing, it's not going to have any reason to think that grabbing is effective.

Hybrid supervised reinforcement methods seem to work by using an early imitation phase to narrow down the range of possible actions to play with in the reinforcement phase, by eliminating unused actions. I never grabbed once, but grabbing certainly didn't get eliminated from his repertoire. He kept doing it just as much as a regular CPU. I saw zero evidence of any early supervised learning phase, since all I did was shield. Nor with my samus amiibo, who I exclusively short-hop zaired against.

Also I did not see any significant bonus damage from the Amiibo. If there is any such damage increase, it is well below 5%.

Unfortunately, we've confirmed that lv.50 amiibos get a 1.5x damage multiplier. Which both sucks and makes this stuff harder to test. What would be really useful would be if someone had multiples of the same amiibo, and could manually train them in very specific ways.

I'll take you at your word about amiibos exhibiting different behaviors than regular CPUs. That's relatively easy to test just by pitting them against one another and watching. What's hard to test is how they got that way. It could have been learning; or, amiibos might just run on a different set of scripts.


tl;dr Let me summarize what I feel confident of thus far regarding amiibos:

1. At their core, they're still state machines. Just like Brawl, just like... well, the vast majority of video games. Characters have scripts they run, which transition into other scripts. Here's a diagram of one from brawl. Higher-level amiibos get better ones, which accounts for a large part of the changes you see in their behavior as they level up. Amiibos may have different scripts than regular cpus.
2. There's no association or situation-based learning, so probably no copying. It might be possible that they can learn something like "I like to grab." But I'm pretty sure they can't learn "I like to grab when he's shielding and in range." It's both computationally more expensive, and would not fit on an NFC chip. Supervised/imitation learning is most commonly associative, and it wouldn't make much sense for them to rely on supervised learning when people are probably playing as a different character with different actions.
   
3. These state machines may or may not have a handful of variables that are tweaked in some kind of learning/selection process. It's not impossible that the scripts have weights or preference levels. Let's say falcon has a paunch script: if there's reinforcement learning, its weight might go down if you punished falcon every time he paunched. Weights for every script or state transition could conceivably be stored on an NFC chip. I think the best way to discover these variables would be to train your amiibo using extremely repetitive behaviors, then compare it to an identical amiibo trained differently. For example, I shielded constantly against my link in an attempt to reinforce grabbing, to see if there was a "grabbiness" variable. I don't think there is, but I don't have another Link to test against.

 

[cot(θ)]

This whole thing really makes me want to get a Wii U so I can run my own experiments...

Rather than needing two matching figure players, it might suffice to train an Amiibo in one way, take several videos of it's performance, then reset it and try to train it in a completely different way, and compare the fighting styles of the two Amiibos.

Just pointing this out, as I probably wouldn't want to buy two of the same Amiibo to test this out myself.

Also, regarding the cost of NFC chips, buying them as mass production parts is going to be significantly cheaper than buying them individually on Amazon, so I wouldn't be surprised if they had a much better chip.

 

[mugwhump]

Rather than needing two matching figure players, it might suffice to train an Amiibo in one way, take several videos of it's performance, then reset it and try to train it in a completely different way, and compare the fighting styles of the two Amiibos.

It's better than nothing for sure. Or you could just save replays.
Speaking of: replays featuring amiibos would need to save an amiibo's data. We could try peeking at replay files to see if we could get at the amiibo data. If not, then just examining the size of the replay could be pretty helpful.

Maybe we could read the data on the chip with a regular NFC reader? :O
It's worth a shot. If anyone has a phone on this list, see what you can do. Apparently my S1 is too ancient, lol.
Edit: nvm, that's not necessary any more. We know the size and the data's gonna be encrypted anyway.

Also, regarding the cost of NFC chips, buying them as mass production parts is going to be significantly cheaper than buying them individually on Amazon, so I wouldn't be surprised if they had a much better chip.

Certainly, but those Whizchip guys are also mass producing them (though probably not at Nintendo's scale), and they want to make a hefty profit margin on each chip, just like Nintendo does on each amiibo, so you can't get too pricey. That's also around the memory range I heard more knowledgeable people suggesting. This page shows a price of $.53 each for bulk orders of 888 byte chips, which sounds like it's in amiibo range. Chips that store up to 4KB exist, but those would start seriously eating into the amiibo profit margins.

Keep in mind this is a lot of guesswork here. What we really need is for someone to crack open their amiibo and take a peek at the chip model (after we try the less destructive methods above).

 

[Twigler]

I'm sad that Amiibo's are just CPUs. I thought they would be your personal playable character all along... T_T

 

[Shiliski]

While I appreciate you actually discussing your experience (It's hard to tell over the internet if someone actually has a clue or if they're just talking out of their butt), and I can certainly understand why you have your doubts, I also can't really just ignore what I've seen.

I guess I was off about the 1.5x multiplier. I thought my own moves were doing roughly the same damage, but I admit I wasn't paying close attention to that and was focusing on other things. I hate it, though. I wish Sakurai hadn't done that. It's not like we need it. More cheating AI BS.

As for the chip, I think that we can safely assume that most of the computation is being done on the Wii U and not on the chip. The chip is likely being used only for data storage. In fact it's impossible for it to be any other way, considering that the chip isn't used during the match.

Here are the variables that I think the Amiibo is tracking.

(pretty sure)
Relative X distance between itself and another character.
Success rate of moves previously used
Its own level (I'm not sure how XP is gained or what level really relates to)
Relative position to edges, at least for X values.
What action the opponent(s) is(are) currently performing. (Such as: attacking, shielding, etc)

(Not quite sure)
Relative Y distance between itself and another character. (Note: It sometimes does a very poor job of tracking relative Y distance, as mine will often stand far beneath me on stages like Guar Plains and Wrecking Crew and attack. I'm not sure what the limitations here are.)
Absolute position on stage.
Frame data of its own moves.

My theory is that it has some kind of FSM that tracks a number of "common" states, such as "opponent within x distance" or "opponent attacking" or "opponent shielding" and so on, and then has a list of moves that have or have not worked in the past in those situations. In addition to it's "experience" data it probably also has some preloaded data so that the amiibo isn't completely clueless. I also think that if the amiibo doesn't know what to do, or if what it's doing isn't working, it will do moves at random until it finds something does work.

That said, there are still times when the Amiibo is clueless, such as "not knowing that it should throw a bomb after pulling it out", so I'm not sure how deep that goes.

I imagine that, despite tracking relative distance, it has some way of simplifying that to ranges instead of dealing with every single possible relative distance. For example, I don't think that a distance of 2.35827 is treated differently than a distance of 2.4. I think they're treated the same. That's just a theory though, as I think treating them as different states would make things way more complicated than they need to be.

Worst case, it has a series of flags such as "knows how to use items" or "knows how to use <insert move here>". I don't think it's that simple though, because its behavior does seem to change over time, even if the behavior change isn't easy to control.

 

[Darklink401]

I'm sad that Amiibo's are just CPUs. I thought they would be your personal playable character all along... T_T

Every character in the roster is your own personal playable character.

 

[Twigler]

Every character in the roster is your own personal playable character.

But it won't have the sentimental value of having an Amiibo action figure that you can upload into the game through NFC and use to play with.

 

[Darklink401]

But it won't have the sentimental value of having an Amiibo action figure that you can upload into the game through NFC and use to play with.

So you'd pay 13 dollars to do something you can do for free (after purchasing the game) by customizing your character with equipment and custom moves?

I literally don't see your point, I'm sorry.


EDIT:

Seriously, everything an amiibo can do, you can do yourself in the custom characters thing.

 

[Shiliski]

I did another experiment regarding making an amiibo that grabs a lot. A couple of people were stating that they were unsuccessful in doing this. I stated before that it could be done by (A) shielding a lot and (B) Grabbing a lot.

Instead of just speculating and leaving it at that, I did an actual experiment, doing exactly what I said to do.

For the first 35 levels or so, the amiibo just behaved randomly, not really reacting to the fact that I was constantly shield-grabbing at all.
At levels 36-38, the behavior changed significantly, and the amiibo began grabbing a lot more often. I played 2 games, took replays, and took a tally of how often each move was used.

Times each move used-

Game A

Grab = 21
Up smash = 6
Jab = 6
Neutral B = 4
F tilt = 2
Fair = 1
D smash = 1

Game B

Grab = 25
F smash = 13
B = 13
Jab = 6
Up tilt = 4
Dair = 3
F B = 3
Dash Attack = 2
Up smash = 2
D smash = 2

Keep in mind that I was shield grabbing the whole time. A lot of times it would wait for me to grab, then f-smash while I was vulnerable. Also keep in mind these were Link mirror matches on FD, for however much that matters.

For Game C I did not constantly grab-shield and just played normally.

Game C

Grab = 19
F B = 9
B = 8
Up tilt = 6
F smash = 4
Up smash = 4
Jab = 3
Fair = 3
Nair = 1
Bair = 1


In all 3 games it uses grabs more often than it uses any other single move. There is a notable decrease in game C but it's also worth noting that I shielded less in that game.

 

[mugwhump]

^Saving replays and counting move frequencies is a good approach. However, you need a control. Can you erase your amiibo's data and try re-training it to not be a grabber? Then play some matches of similar length, and compare the amount of grabbing.

Edit: it would also be good to control for differences in the way we play, which means playing in a way that's easy to reproduce. So basically the first thing we try should be matches where we just stand still, or only do one action over and over. Also, remember not to save your amiibo's data after any test matches.

 

[Shiliski]

^Saving replays and counting move frequencies is a good approach. However, you need a control. Can you erase your amiibo's data and try re-training it to not be a grabber? Then play some matches of similar length, and compare the amount of grabbing.

Edit: it would also be good to control for differences in the way we play, which means playing in a way that's easy to reproduce. So basically the first thing we try should be matches where we just stand still, or only do one action over and over. Also, remember not to save your amiibo's data after any test matches.

Well, what I've done so far is, after a fresh reset, I've played 2 games that were 2-minute Timed matches. For about 1 minute in thie first match match I stood still (no controller inputs) and the amiibo just walked up to me and did absolutely nothing. Then for 40 seconds I taunted, but the amiibo just continued staring at me. At the end I tried to kill it before it went to sudden death, but it went to SD anyways and the amiibo landed a hit first. The second match I stood still for about a minute and the amiibo also just stood still. When I eventually jabbed it just jumped over the jab. After a certain level it just started attacking me on its own, but before level 6 it just stood still.

After a bit I considered that experiment finished (as much as it was going to be) and reset the amiibo, and then tried the grab experiment. All I did was grab-shield, with some rolling and running around to make sure that the grabs landed. It didn't really start to imitate me until it hit level 37 or so, at which its behavior suddenly changed which makes me wonder what impact the levels have.


I could reset it again and never shield and focus on punishing grabs, but it being Thanksgiving I might not be able to do it today. That should make it less likely to grab, if I'm understanding it correctly. I think it's important to use the same amiibo just to eliminate the possibility of things like... say... different amiibo having different random seeds.

For the control, I thought I could see how often the CPU grabs, but that's not a very good control since I'm already convinced that the AIs are different so I didn't try it.

 

[mugwhump]

Yes, my low-level amiibos were also pretty... pacifistic. Lol. The noticeable changes in behavior that sometimes accompany leveling up are suggestive of AI scripts getting swapped or upgraded.

I think it's important to use the same amiibo just to eliminate the possibility of things like... say... different amiibo having different random seeds.

Oh snap. That's completely possible, would be extremely easy for them, would be portable between Wii Us, and we wouldn't be able to tell what was different from looking at amiibo data on replays or nfc chips. They could just tweak certain behavior variables randomly based on the amiibo's unique ID, to get an easy way to give each amiibo a unique "personality".

I think I'll pick up another link tomorrow.

Edit: Oh **** son! Looks like we know what NFC chip amiibos are using.

NFC tag info:

• RF technology: Type A (ISO/IEC 14443 Type A)
• Tag type: Topaz NFC IC
• Manufacturer: Innovision Research and Technology Plc (UK)
• ATQA: 0c00
• Header ROM: 124c
• Maximum message size: 454 bytes
• Memory size: 512 bytes
• Page size: 8 bytes
• Number of pages: 64

Data pages:

• Data page 0 contains the UID, and is locked.
• Pages 1-12 are unlocked and writable.
• Page 13 is reserved and locked. Page data is always set to: 0x5555aaaa124c0600
• Page 14 contains the lock bits/OTP, and is locked. Page data is always set to: 0x01e0000000000000
• Page 15 is unlocked and writable. Page data is always set to: 0x000000000000ffff
• Pages 16-63 are unlocked and writable.

So, 512 bytes. Bit smaller than I was hoping, but not unexpected. That should give us 496 writable bytes. They're probably using less than 100 bytes to store amiibo names, color, level, customs, and stats. So let's say we have at least 400 bytes to potentially store AI parameters. I think this rules out any kind of situation-based or associative learning, but there's still room for some specialized variables. Now we just need to find them.

 

[daguyontheladder]

you know what would be a good test for the amiibos? have two of the the same figure, one trained by a human and the other by a cpu. then once they both level cap, see which one beats the other.

 

[Darklink401]

you know what would be a good test for the amiibos? have two of the the same figure, one trained by a human and the other by a cpu. then once they both level cap, see which one beats the other.

And make sure they have the same amount of matches

 

[Shiliski]

Turns out I had time to do this after all.

I reset the amiibo and trained it on FD using every move except shielding and grabbing. I trained it using 3 stock untimed matches on FD, no items. At level 37 I saved the amiibo, and then played 3 battles. After each battle I exited smash mode without saving the amiibo, to eliminate the possibility of it learning how to deal with my shield-grabbing.


For control games A and B I went back to Shield Grabbing the entire match, just to match the conditions of the test games A and B. Then I went back to playing normally without using shields or grabbing for control game C.


Control Game A (Shield Grabbing)
F smash – 14
Up smash – 13
B – 10
F tilt - 6
F B – 5
Dair – 4
Grab – 3
Dash Attack – 2
Jab - 2
D B – 1 (once again it didn’t throw it, and instead spammed B while holding the bomb)
Nair - 1
Uair – 1
Bair – 1

Control Game B (Shield Grabbing)
Up Smash – 14
F tilt – 14
F Smash – 8
B – 4
F B – 4
Jab – 3
Fair – 3
Grab – 1
Dair – 1
Dash Attack – 1

Control Game C (Normal Play minus shielding and grabbing)
B - 12
Up smash – 12
F tilt – 6
F B – 3
Up tilt- 3
Jab – 3
Fair – 3
Dair - 2
Nair - 2
F Smash - 2
Grab - 1

So the control amiibo data grabs a lot less than the test amiibo data did. However, both the control amiibo data and the test amiibo data seemed to favor both B and Up smash, for whatever reason.

This is a pretty significant difference and lines up with my own previous observations. While one might argue about whether or not this really counts as "learning", it certainly does change its behaviors according to previous experiences.

The exact specifics of how it learns, as well as the specifics of how good its learning algorithm is, remain to be seen. Nevertheless, how you raise your amiibo certainly appears to change its behavior.

As a side note, I think the amiibo trained through "normal play" was a lot stronger overall than the amiibo trained via constant shield grabbing.

 

[mugwhump]

Oh my, that's definitely a significant difference :O
Looks like something's really going on here. Good work Shiliski!

 

[Logsmash]

I bought a Yoshi amiibo on Day 1, along with Smash4. I started off by playing 100 matches of Yoshi dittos against a level 9 CPU, to get a feel for the character and see how the CPU-9 acted. Then I trained my Yoshi amiibo up to level 50 just by sparring, assorted stages, 3 stocks, 8 minute time-limit. Of note is that early on in the sessions, I would spam egg throws to juggle the amiibo as often as possible whenever it was in the air above me. Now at level 50, I played 100 matches against the amiibo and whenever I was in the air while it was on the ground, it would throw eggs with precision that hit me every time unless I dodged. Seriously, this thing could build up over 60% just by juggling me with egg throws (each egg deals between 5 and 10 percent depending on how it hits) if I didn't try to dodge. I then went back and played another 100 matches against a CPU-9. When I was in the air above the CPU, it would sometimes throw an egg, and although its aim was precise it never juggled me with multiple eggs like the amiibo did. So I reset the amiibo, trained it up to 50 without spamming egg throws, then played against it. It threw eggs about as often as the CPU-9. Also noteworthy, is that both times it was 50 it would grab me if I missed a down-B, which happens often. I rarely grab, and I have yet to see my amiibo use down-B. Instead, it'll use Dairs or Bairs which I have had far more success hitting with.

Take this with a grain of salt, since I'm just one player with one amiibo, but I think the idea of adapting "more effective" tactics that are used against the amiibo may possibly be partially truthful, though replicating this test and others with multiple amiibo and players would be the most ideal.

 

[mugwhump]

Thanks Logsmash, good stuff.

It's looking like the AI is indeed a state machine with variables thrown in for learning. I'm guessing the devs manually specified exactly how each variable gets updated. If each variable's update method is hand-coded, they could be hacky reinforcement or imitation methods for different things. So I guess there isn't much of an "algorithm" to speak of, they just kinda bolted some pseudo-learning onto a state machine, lol. So, probably no generalized learning; all they can learn is things the programmers have specifically designed them to be able to learn. You probably can't teach Shulk to really feel it at the beginning of each match, but you might very well be able to teach him to use backslash more or less often.

I also did some testing vs. my Link, doing the same test matches at lv.37 as Shiliski. However, I trained him by shielding exclusively, with no grabbing. I'm waiting till tomorrow to pick up my 2nd link to go over the replays and train a non-grabber for comparison, but he was definitely grabbing a crapload. So grabbing often might not be necessary to train a grabber.

Also, I must be ********, because apparently I never read the built-in amiibo tutorial, which actually gives an example of how learning works. It says "When an enemy blocks with his or her shield, it mimics the shield, or responds with a grab!" That sounds like they've scripted some pseudo-supervised learning for copying shielding, and some pseudo-reinforcement learning for responding to shielding. Did they write any code for copying your own grabbing, or responding to your own grabbing? Not sure about the former, but it seems like grabbing at least has a variable governing how likely the amiibo is to insta-dodge it, and it goes up when you grab them.

Some things to investigate:

1. Attack-specific dodge rates. During my tests against my amiibo I'd trained by only shielding, I noticed that he didn't really dodge grabs at the beginning, but dodged them all the time by the end. Since this is all input reading rather than complicated prediction, it'd be pretty easy for the programmer to make a variable for dodge rate that increased every time the amiibo got hit by an attack. We should investigate whether all attacks have such a variable (if the variable really exists), whether it carries over between matches, whether it only goes up when you succeed in hitting them, and if increasing the amiibo's dodge rate against Link's dair will also increase it against, say, mario's dair.
2. Attack preferences. Logsmash seems to have trained Yoshi to use more up-Bs via imitation. Does this work for all specials? Does it also work for all normals? Can you train Yoshi to use his up-B by using up-B as a different character? Does playing with them on a team make them copy you as well (the tutorial suggests yes)? Is a teammate or an opponent more influential? Can attack preferences also be adjusted by reinforcement? A good way to test that would be by getting an amiibo with 2 projectiles, one of which is slow. Always block the slow one, but let the other one hit you.
   
3. Shielding. The tutorial suggests shielding can be learned via imitation (I should be able to check this tomorrow). Can it also be encouraged or discouraged by reinforcement? For example, train one amiibo to shield by having it successfully block your attacks, and discourage another from shielding by using shieldbreakers or grab attacks. We should also investigate whether roll-spamming can be imitated or reinforced.
4. Stages. I've heard of amiibos not dealing well with stages they haven't played on before. Best way to test this would be with two amiibos of the same character, one of which has only played on that stage, one of which hasn't.
   
5. Off-stage play. I've heard a few people talking about this, we should look into it. Is this learned, or is it just a feature of high-level amiibo scripts?
6. Really feeling it. I guessed you couldn't train an amiibo to taunt at the start of the match, but we might as well make sure. This would be an example of generalized learning. We're more likely to be able to train them to taunt or not after a kill. They seem to do it by default, as I trained mine to lv.37 without taunting, but the amiibo still taunts after a kill. Maybe with more leveling we could get them to stop? Or maybe you can get them to favor one taunt over another.

Also, instead of doing 3-stock matches, it's probably better for us to do 3 1-stock matches, resetting the amiibo in between. We want to minimize the effect of any learning that occurs during the test matches. Edit: actually, timed matches are probably better than stock, since it makes it easier to compare action frequencies.

 

[Darklink401]

Thanks Logsmash, good stuff.

It's looking like the AI is indeed a state machine with variables thrown in for learning. I'm guessing the devs manually specified exactly how each variable gets updated. If each variable's update method is hand-coded, they could be hacky reinforcement or imitation methods for different things. So I guess there isn't much of an "algorithm" to speak of, they just kinda bolted some pseudo-learning onto a state machine, lol. So, probably no generalized learning; all they can learn is things the programmers have specifically designed them to be able to learn. You probably can't teach Shulk to really feel it at the beginning of each match, but you might very well be able to teach him to use backslash more or less often.

I also did some testing vs. my Link, doing the same test matches at lv.37 as Shiliski. However, I trained him by shielding exclusively, with no grabbing. I'm waiting till tomorrow to pick up my 2nd link to go over the replays and train a non-grabber for comparison, but he was definitely grabbing a crapload. So grabbing often might not be necessary to train a grabber.

Also, I must be ********, because apparently I never read the built-in amiibo tutorial, which actually gives an example of how learning works. It says "When an enemy blocks with his or her shield, it mimics the shield, or responds with a grab!" That sounds like they've scripted some pseudo-supervised learning for copying shielding, and some pseudo-reinforcement learning for responding to shielding. Did they write any code for copying your own grabbing, or responding to your own grabbing? Not sure about the former, but it seems like grabbing at least has a variable governing how likely the amiibo is to insta-dodge it, and it goes up when you grab them.

Some things to investigate:

1. Attack-specific dodge rates. During my tests against my amiibo I'd trained by only shielding, I noticed that he didn't really dodge grabs at the beginning, but dodged them all the time by the end. Since this is all input reading rather than complicated prediction, it'd be pretty easy for the programmer to make a variable for dodge rate that increased every time the amiibo got hit by an attack. We should investigate whether all attacks have such a variable (if the variable really exists), whether it carries over between matches, whether it only goes up when you succeed in hitting them, and if increasing the amiibo's dodge rate against Link's dair will also increase it against, say, mario's dair.
2. Attack preferences. Logsmash seems to have trained Yoshi to use more up-Bs via imitation. Does this work for all specials? Does it also work for all normals? Can you train Yoshi to use his up-B by using up-B as a different character? Does playing with them on a team make them copy you as well (the tutorial suggests yes)? Is a teammate or an opponent more influential? Can attack preferences also be adjusted by reinforcement? A good way to test that would be by getting an amiibo with 2 projectiles, one of which is slow. Always block the slow one, but let the other one hit you.
   
3. Shielding. The tutorial suggests shielding can be learned via imitation (I should be able to check this tomorrow). Can it also be encouraged or discouraged by reinforcement? For example, train one amiibo to shield by having it successfully block your attacks, and discourage another from shielding by using shieldbreakers or grab attacks. We should also investigate whether roll-spamming can be imitated or reinforced.
4. Stages. I've heard of amiibos not dealing well with stages they haven't played on before. Best way to test this would be with two amiibos of the same character, one of which has only played on that stage, one of which hasn't.
   
5. Off-stage play. I've heard a few people talking about this, we should look into it. Is this learned, or is it just a feature of high-level amiibo scripts?
6. Really feeling it. I guessed you couldn't train an amiibo to taunt at the start of the match, but we might as well make sure. This would be an example of generalized learning. We should also test if we can turn them into taunt-spammers, for fun.

Also, instead of doing 3-stock matches, it's probably better for us to do 3 1-stock matches, resetting the amiibo in between. We want to minimize the effect of any learning that occurs during the test matches.

On point 6, I've got some analysis.

My amiibos used to taunt after each kill in the lower levels.

However, I never do against them, and now, at level 50, I'll be liucky to see them taunt at all.

So not sure you can get them to taunt right after the match starts, but you can probably alter the frequency of it.

 

[Shiliski]

Since this is all input reading rather than complicated prediction

While testing grabs yesterday I did notice that it was indeed input reading and not actual prediction. Which is too bad, but realistically what could you expect from something that has to be computed in real time? I bet that Nintendo took a lot of short cuts, and this is one of them.

It also feels like the amiibo "eases up" on you if it's winning a lot.

As for taunting, I think it only taunts after a kill, and at later levels it only taunts a lot if it sees other players taunting. It doesn't seem like you can really teach it new conditions to taunt under. That said, it may be like the grab problem, where it only learns the teacher's patterns after a certain level. Taunting at the start of each match all the way up to level 50 would be time consuming, but it might be the best way to make sure.

I also feel as if the amiibo learns more between matches than it does during a match. As in, after it has finished a match it does a bit of post-battle processing. I say this because its behavior sometimes seems to change significantly from one match to another, usually becoming more precise and aggressive.

 

[mugwhump]

While testing grabs yesterday I did notice that it was indeed input reading and not actual prediction. Which is too bad, but realistically what could you expect from something that has to be computed in real time? I bet that Nintendo took a lot of short cuts, and this is one of them.

Yeah, needing to be able to run 8 amiibos at once on the wii u's weak CPU is a harsh limitation for sure. Have they said whether amiibos were coming to the 3DS version? That would be even harsher.

It also feels like the amiibo "eases up" on you if it's winning a lot.

Maybe you're getting better? Lol. Sm4sh does have some comeback mechanics, but this one could be difficult to test.

I also feel as if the amiibo learns more between matches than it does during a match. As in, after it has finished a match it does a bit of post-battle processing. I say this because its behavior sometimes seems to change significantly from one match to another, usually becoming more precise and aggressive.

Offline/batch updates are the norm for traditional machine learning algs (though amiibos aren't really using those). I kinda felt that way. Reliable way to test would be to level an amiibo up over a single, long match, and then compare it to one leveled up over multiple matches. Doing multiple test runs could be time-consuming though, since you'd need to play that single, long match multiple times.

Where the NEETs at?

 

[Supersmashbrothers.]

I'm here to say I have bought 2 Amiibos (Link and Samus)
I self-trained Link to level 50 then Trained Samus to about level 30 and then Let her and Link battle. And Samus beat Link easily.

Point is : I don't think there's a specific way to make your Amiibo stronger unless it's feeding them.

 

[mugwhump]

^Your link would have been dealing more damage than the Samus and possibly had stronger shielding or something, so... good for Samus? Lol. You should probably have them fight more than one match, though.

Since they're just using scripted variables, they could update pretty quickly. Who knows, you might be able to maximize or minimize a variable in 2 minutes. In which case a lv.50 vs a lv.30 might not be that uneven, learning-wise.

 

[Mazdamaxsti]

What exactly does that mean?

In terms of implementation, how exactly would you make an algorithm that learned a playstyle WITHOUT learning the player's techniques? Supervised learning usually works by generalizing about how the player's likely to do "action A in game state XYZ." If these are detailed enough in terms of game state and are formed based on tons of observations, you might wind up with a halfway-decent approximation of a human. That's definitely not happening with amiibos, and the NFC chips in them wouldn't have nearly enough room to store that kind of AI in any case.

I think the most that could possibly be going on would be a couple of variables governing very general behaviors, like likelihoods for shielding, rolling, grabbing, etc. But I've seen no evidence for these at all. I tried to train my link amiibo as a grabber, by doing nothing but shielding all the time. If there were supervised learning, he'd be shielding a lot; if there were reinforcement learning, he'd be grabbing a lot. But he didn't seem to be doing either of those things more than usual for an AI. My Samus experiments likewise showed no evidence of learning.

Well I was fighting my Kirby amiibo with my amiibo and everytime I would recover I would up B, and now my Kirby does it too. I don't think it learns by you doing complicated techniques, it learns by the basic things. Plus if all you did as Link is grab, most likely it would not grab all the time, it learns from you it doesnt replicate you.

 

[Darklink401]

Well I was fighting my Kirby amiibo with my amiibo and everytime I would recover I would up B, and now my Kirby does it too. I don't think it learns by you doing complicated techniques, it learns by the basic things. Plus if all you did as Link is grab, most likely it would not grab all the time, it learns from you it doesnt replicate you.

IDK about complicated techniques, but my Fox does a dthrow - fair combo that I used to hit it with during its earlier levels.

 

[mugwhump]

I got another link amiibo, will upload some datums later.
If anyone else wants to do testing, you don't need to watch replays and painstakingly count every time an action is performed. You can just let the match play out in the background, then look at the stats on the results screen. For example it's got statistics for number of grabs, throws, ground attacks, air attacks, smashes, and projectiles. Not only is it easier, it's also more accurate. If your amiibo gets interrupted during a startup animation it can be hard to tell what it was doing. Unfortunately it doesn't break it down into specific attacks, but it's good enough for my current experiment, which is grab rate.
Also, if you're doing a test of your amiibo fighting against a "normal" opponent, it would be better to pit the amiibo against a CPU, since it'll be easier to reproduce the results.

Well I was fighting my Kirby amiibo with my amiibo and everytime I would recover I would up B, and now my Kirby does it too.

wut
I'm pretty sure using up B to recover is something every AI does, unless you meant something else.

Plus if all you did as Link is grab, most likely it would not grab all the time, it learns from you it doesnt replicate you.

Our limited data so far, and the amiibo tutorial text, suggests it's a mixture. Sometimes supervised learning/imitation, sometimes reinforcement, depending on the specific thing. We need testan to figure out which things are "learned" which way. I put "learned" in quotes because it's really probably just tweaking variables.

IDK about complicated techniques, but my Fox does a dthrow - fair combo that I used to hit it with during its earlier levels.

That's probably just part of the higher-level amiibo scripts. I really doubt amiibos can learn arbitrary sequences of actions, or new actions to do in situations where those actions weren't part of their scripts. Too much work for the CPU and amiibos just plain don't have room to store that stuff.

 

[Darklink401]

I got another link amiibo, will upload some datums later.
If anyone else wants to do testing, you don't need to watch replays and painstakingly count every time an action is performed. You can just let the match play out in the background, then look at the stats on the results screen. For example it's got statistics for number of grabs, throws, ground attacks, air attacks, smashes, and projectiles. Not only is it easier, it's also more accurate. If your amiibo gets interrupted during a startup animation it can be hard to tell what it was doing. Unfortunately it doesn't break it down into specific attacks, but it's good enough for my current experiment, which is grab rate.
Also, if you're doing a test of your amiibo fighting against a "normal" opponent, it would be better to pit the amiibo against a CPU, since it'll be easier to reproduce the results.


wut
I'm pretty sure using up B to recover is something every AI does, unless you meant something else.

Our limited data so far, and the amiibo tutorial text, suggests it's a mixture. Sometimes supervised learning/imitation, sometimes reinforcement, depending on the specific thing. We need testan to figure out which things are "learned" which way. I put "learned" in quotes because it's really probably just tweaking variables.


That's probably just part of the higher-level amiibo scripts. I really doubt amiibos can learn arbitrary sequences of actions, or new actions to do in situations where those actions weren't part of their scripts. Too much work for the CPU and amiibos just plain don't have room to store that stuff.

But wouldn't it activate easier with exposure to that combo, or does the exposure only seem to play a part?

 

[mugwhump]

OK finally got my data together.

I have 2 Link amiibos, both trained to level 37 against Kirby on FD or omega stages.
Link A was trained by doing nothing but shielding and sometimes turning into a rock when my shield was wearing out.
Link B was trained with no inputs, I just left the controller while he wailed on me.

As mentioned I use the post-match statistics screen to count moves used. Note that when it says "Ground attacks" it means anything used on the ground, whether it be a tilt, smash, projectile, or a grab. "Air attacks" is the same, but for the air. That means grabbing will add one to both the grab count and ground attack count, and using the bow in the air adds one to projectiles and air attacks.
All test matches were played as kirby on FD.

Test 1 - grabbed all match. Sum of two 3-stock matches:
Link A - 143 ground, 58 grabs, 92 air, 37 smashes, 60 projectiles (41% of ground attacks were grabs)
Link B - 239 ground, 43 grabs, 120 air, 94 smashes, 102 projectiles (18% of ground attacks were grabs)

Test 2 - fought "normally," tried to avoid grabbing. Sum of two 3-stock matches:
Link A - 91 ground, 32 grabs, 23 air, 43 smashes, 20 projectiles (35% of ground attacks were grabs)
Link B - 152 ground, 11 grabs, 67 air, 68 smashes, 71 projectiles (7% of ground attacks were grabs)

Test 3 - no inputs. Sum of two 3-stock matches:
Link A - 80 ground, 70 grabs, 16 air, 5 smashes, 12 projectiles (88% of ground attacks were grabs)
Link B - 75 ground, 1 grab, 21 air, 21 smashes, 48 projectiles (1% of ground attacks were grabs)

Test 4 - shielded constantly, like how Link A was trained. Each amiibo tested with one 5-minutes timed match. In hindsight I should have done all of them with timed matches.
Link A - 125 ground, 38 grabs, 6 air, 7 smashes, 25 projectiles (30% of ground attacks were grabs)
Link B - 208 ground, 24 grabs, 31 air, 76 smashes, 103 projectiles (12% of ground attacks were grabs)

Conclusion - Link A, who was trained by an opponent who did nothing but shield, did indeed grab at a much higher rate than the opponent who was trained with no inputs. This suggests that grabbing yourself is not necessary to teach an amiibo to grab, but Test 1 suggests it can help make them grab more. Both amiibos had higher rates in that test than in Test 2, meaning the amiibos may have been copying my grabbing behavior in each of the 3-stock matches.

Link A also used way fewer projectiles than Link B. I guess that Link A's projectiles being blocked lowered his projectile preference (or, Link B's preference was raised by them always hitting).

But wouldn't it activate easier with exposure to that combo, or does the exposure only seem to play a part?

Only if the programmer specifically made the script check whether the human was doing that particular combo, then allocated precious bytes to store the amiibo's familiarity with that combo. So, no, it's not likely.

 

[Mazdamaxsti]

I got another link amiibo, will upload some datums later.
If anyone else wants to do testing, you don't need to watch replays and painstakingly count every time an action is performed. You can just let the match play out in the background, then look at the stats on the results screen. For example it's got statistics for number of grabs, throws, ground attacks, air attacks, smashes, and projectiles. Not only is it easier, it's also more accurate. If your amiibo gets interrupted during a startup animation it can be hard to tell what it was doing. Unfortunately it doesn't break it down into specific attacks, but it's good enough for my current experiment, which is grab rate.
Also, if you're doing a test of your amiibo fighting against a "normal" opponent, it would be better to pit the amiibo against a CPU, since it'll be easier to reproduce the results.


wut
I'm pretty sure using up B to recover is something every AI does, unless you meant something else.

Our limited data so far, and the amiibo tutorial text, suggests it's a mixture. Sometimes supervised learning/imitation, sometimes reinforcement, depending on the specific thing. We need testan to figure out which things are "learned" which way. I put "learned" in quotes because it's really probably just tweaking variables.

Actually, they don't. I've been playing mirror matches with Kirby against level 9s for a while now, it helps me improve a lot. I rarely see them use Up B for recovering. They always just go back on the stage by jumping and floating and do it when they have no other choice. I taught him to do it right when he's in range no matter what, even if you barely got hit off, and he does it. It definitely copies playstyle, mine plays like me, but I don't know if it actually learns tech until maybe higher levels/not at all.

 

[mugwhump]

Here's the full list of post-battle stats (minus kos and stuff). It's good to rely on these in your tests when you can, since it's easier and more objective than counting yourself, though less detailed.

Damage given
Damage taken
Damage Recovered
Peak Damage
Launch Distance
Ground time
Air time
Swim time
Hit Percentage
Ground attacks
Air attacks
Smash attacks
Grabs
Throws
Edge Grabs
Projectiles
Items Grabbed
Max Launch Speed (getting launched)
Max Launcher Speed
Longest Drought
Transformation time
Final Smashes

Anyway, I finished raising my Samus amiibo. I used nothing but short-hop zairs, to try to teach it to do the same thing. It never did. But it got very good at insta-dodging them. So, unsurprisingly, they don't seem to be able to learn sequences of actions that aren't already in their scripts.
I want to reset and retrain it before testing the dodge rate. I also wonder if my constant short-hop zairs might have affected its jump rate, or if there's some general "attacks per second" variable for aggression.

Actually, they don't. I've been playing mirror matches with Kirby against level 9s for a while now, it helps me improve a lot. I rarely see them use Up B for recovering. They always just go back on the stage by jumping and floating and do it when they have no other choice. I taught him to do it right when he's in range no matter what, even if you barely got hit off, and he does it. It definitely copies playstyle, mine plays like me, but I don't know if it actually learns tech until maybe higher levels/not at all.

Ah. I would guess the script looks for people using up-B in the air and assumes it's for recovery purposes. My samus did something similar. It seems to have some preference for recovering with zair. Still, to say for sure, we need to reset our amiibos and retrain them without using those attacks, then compare them.

 

[Greda]

After reading all these, I have a question. Could Amiibo be hacked? Can we go in and look at how they function? Could we alter Amiibo and make them a "perfect" fighter, or make it go totally berserk? Also would we be able to change things like the rewards we get after scanning an Amiibo, or let me get 9999999 Rupees in Hyrule Warriors?

 

[cot(θ)]

After reading all these, I have a question. Could Amiibo be hacked? Can we go in and look at how they function? Could we alter Amiibo and make them a "perfect" fighter, or make it go totally berserk? Also would we be able to change things like the rewards we get after scanning an Amiibo, or let me get 9999999 Rupees in Hyrule Warriors?

Good question - I don't have an amiibo yet, but I assume that you have to touch the amiibo to the NFC pad on the Wii U to save the amiibo's progress back to it. This means that the data is stored in the amiibo's NFC chip, and if we were to selectively write to portions of it, we could determine what parts correspond to equipment stats / powers, and which correspond to its learning weights. You could almost certainly max out your amiibo's stats this way, and depending on how powers are stored (the ids of the 3 powers it knows, vs. a bitfield of powers) it might be possible to equip more than 3 powers on your amiibo.

I'm certain that half a KB isn't enough space for learning arbitrary combos, but I wouldn't be too surprised if just a bit of that memory was reserved for small combo strings.

 

[mugwhump]

Good question - I don't have an amiibo yet, but I assume that you have to touch the amiibo to the NFC pad on the Wii U to save the amiibo's progress back to it. This means that the data is stored in the amiibo's NFC chip, and if we were to selectively write to portions of it, we could determine what parts correspond to equipment stats / powers, and which correspond to its learning weights. You could almost certainly max out your amiibo's stats this way, and depending on how powers are stored (the ids of the 3 powers it knows, vs. a bitfield of powers) it might be possible to equip more than 3 powers on your amiibo.

It's worth taking a poke at for sure. Amiibo data probably gets stored in replays.

Unfortunately I'm pretty sure the data on the NFC chip at least is encrypted.

I'm certain that half a KB isn't enough space for learning arbitrary combos, but I wouldn't be too surprised if just a bit of that memory was reserved for small combo strings.

Yeah, most likely the only combos they can do are ones that have already been scripted. Learning sequences of actions can be a challenge even for legit machine learning algs, and especially when elements of the sequence need to change (to adapt to DI and percentage for instance). Perhaps you could get an amiibo to favor dthrow->nair over dthrow->fair by doing nairs over and over to increase that attack's weight, so it would be chosen as the followup?

Someone with more brawl AI hacking experience should chip in as to how combos worked there.

 

[comp666]

Hmm based on the experiments and anecdotes in this thread it seems possible they simply store a frequency value for each of the character's moves in the amiibo and then run it against a state machine like the one mugwhump posted. For the data storage, if each character has about 30 moves, this can be done in less than 100 bytes. As for the state machine, they can give each state a couple of possible transitions, and simply pick the transition with the highest move frequency. Updating the move frequency vector is trivial so I won't explain that. As a concrete example, suppose my amiibo has this frequency vector:

Down throw: 20
Up throw: 10
Fair: 20
Nair: 10
etc.

And if 2 transitions of the state machine looked like this:

(opp grabbed) -> (Down throw OR Up Throw)
(opp stunned in air) -> (fair OR nair)

Then when the amiibo grabs the opp, he will most likely down throw, followed by fair. (The down throw has 2x the probability as the up throw and the fair has 2x the probability as the nair). There was a user who said he trained his fox to do this.

This would also make it possible to train the amiibo to taunt after killing, or to grab more frequently, or to recover with up-b more than with jump. This would also explain why it's not possible to train an amiibo to taunt in random situations - the state machine has no transition for (idle) -> (taunt).

This would require very little additional code. The state machine is probably already there. All they have to write is reading/writing the move probability vector.