Real-time Game Data Extraction [Requesting ASM Help]

[Fizzi]

Background

I started work a while ago on a project which I hoped would be capable of getting game information directly out of Melee such that it could be passed on to the outside world via bluetooth, WiFi, or otherwise. I'm far more knowledgeable in the hardware side of these things so I set my sights on building a prototype rig that could read messages being sent to the memory card, decipher whether they were meaningful, and if so use that data for something. In short - I succeeded. The trouble came when it came to actually passing the data I wanted to the memory card turned device.

The current hardware prototype. Inserted into the Wii is a memory card breakout board which is wired to a memory card board (left). The data is then sniffed by an FPGA development board (middle), and relevant data is passed to an Arduino Due (right).

The original plan I wanted to go with was the following:

1. During a match I would write interesting information to unused memory. This includes things like what move a person killed with, what percentage and stock counts both players were at when a stock was taken, etc.
2. After the match had ended I would write the data back to the memory location from which the game reads when saving to the memory card. This would be in a location that wouldn't corrupt any important memory card information - such as after the name tag list.
3. The device would look for the address where it knows the data will be written, read it, and pass it on to somewhere.

It turned out that there was a major flaw in this plan - during the game's bulk save procedure, it appears to encrypt and/or compress the data written the memory card. In other words, there was really no easy way for me to read out the data I had written.

​

What do I need?

So in working on the project I've learned a bit about melee hacking but I'm still very much a novice. I was able to find memory locations that contained things like the last move a player connected with and start to build a list of which IDs matched to which moves. To be completely frank though I just don't have a lot of free time and it's difficult to commit it to struggling to understand the memory card stuff.

The major thing that I need right now to get something working is a way to write data to the memory card in an un-encrypted fashion. I'll describe a bit more about what I know about how data is transferred in the information section.

I'm also open to help extracting information out of the game that would serve for interesting statistics.

Technical Information

The Gamecube communicates with the memory card via a 27 MHz EXI bus. The EXI protocol is basically an SPI protocol with some extra lines. SPI is constantly bi-directional. This means that even when the game is reading from the memory card, it is still sending "bogus" data at the same rate it is reading from the memory card. The result of this is that it might be possible to safely send data to my device by triggering a read operation to a special memory address and filling the "bogus" data with good, unencrypted data.

Here is a small sample of real data being read from the memory card:

Count|MOSI|MISO|MOSI ASCII|MISO ASCII
0|82|127|R|
1|0|128|#NUM!|?
2|32|255| |ÿ
5|222|71|Þ|G
6|186|71|º|G
7|58|71|:|G
8|58|71|:|G
9|71|71|G|G
10|71|65|G|A
11|71|76|G|L
12|71|69|G|E
13|122|48|z|0
14|26|49||1
15|117|255|u|ÿ
16|232|2|è|
17|71|83|G|S
18|65|117|A|u

MOSI = Master out, Slave in (Gamecube to memory card)
MISO = Master in, Slave out (memory card to Gameubce)

The count is used to keep track of what the current bytes mean. When count = 0, the byte refers to the command being issued to the memory card. In the above case, the command being written is 82 (0x52). If you look at the dolphin source code, you can find that this command is called "cmdReadArray", basically it's a read operation. I ran dolphin using Visual Studio to debug and add breakpoints to help me understand how data is actually transferred.

Achilles referred me to this post which describes the known memory card functions http://smashboards.com/threads/patc...racters-stages-and-hats.405623/#post-19378526 . Perhaps it might be possible to use one of these to pass data over?

Potential Uses

• Statistics - Extract data out of the game and push it to a website for archiving
• Live match overviews - if being used on non-streaming set ups the data could be browse-able by fans and they could keep track of how matches are going even if they can't see them
• Automatic match reporting - Match results can be pushed automatically to a website and selected/confirmed by a TO

The following are some farther-future potential uses:

• It may be possible to get interesting information from the internet and change the actual game state based on that. For example, each device can have an ID number linked to a station ID. It can then find out whether a tournament match has been scheduled for itself and pass a flag to the game telling it to lock the set up. It then might also be possible for the station to only become unlocked when one of the players in the match comes up to the station and scans his ID badge, or types in his username/password.
• Using special codes it might be possible to increase the amount of data that can be loaded from a memory card. Data could be loaded from an SD card for example and this might allow things like loading a bunch of custom models into a real system.

Acknowledgements

• Achilles: For answering my questions and pointing me in a helpful direction related the memory card stuff
• Dan Salvato: For answering my questions and helping me understand a bit quicker how the data is transfered

 

[Achilles1515]

I have no words at the moment to express how cool this is.

I see GALE01 in that ASCII! I'm assuming if the list kept going, it would either spell out "Super Smash Bros. Melee" or just the memory card filename highlighted below.

 

[Fizzi]

I have no words at the moment to express how cool this is.

I see GALE01 in that ASCII! I'm assuming if the list kept going, it would either spell out "Super Smash Bros. Melee" or just the memory card filename highlighted below.

View attachment 69070

Yep. I specifically picked that section of data because it turns out that stuff is the only data that is stored directly in ASCII. I've attempted to find specific tags I added to the game but couldn't find them. I can only conclude they are encrypted.

 

[Fizzi]

Here is the continuation:

Spoiler

Count|MOSI|MISO|MOSI ASCII|MISO ASCII
9|71|71|G|G
10|71|65|G|A
11|71|76|G|L
12|71|69|G|E
13|122|48|z|0
14|26|49||1
15|117|255|u|ÿ
16|232|2|è|
17|71|83|G|S
18|65|117|A|u
19|76|112|L|p
20|69|101|E|e
21|48|114|0|r
22|49|83|1|S
24|2|97||a
25|83|115|S|s
26|117|104|u|h
27|112|66|p|B
28|101|114|e|r
29|114|111|r|o
30|83|115|S|s
31|109|48|m|0
32|97|49|a|1
33|115|49|s|1
34|104|48|h|0
35|66|50|B|2
36|114|57|r|9
37|111|48|o|0
38|115|51|s|3
39|48|51|0|3
40|49|52|1|4
41|49|0|1|#NUM!
42|48|0|0|#NUM!
43|50|0|2|#NUM!
44|57|0|9|#NUM!
45|48|0|0|#NUM!
46|51|0|3|#NUM!
47|51|0|3|#NUM!
48|52|0|4|#NUM!
57|29|0||#NUM!
58|66|1|B|
59|229|0|å|#NUM!
60|106|3|j|
64|64|19|@|

 

[Achilles1515]

What does the encrypted data look like that you are reading from the Arduino? Can you post a whole bunch of it and how does it compare with what is on the .gci file from a memory card? Are you reading every single byte that is in the .gci file during the save process?

I'm especially interested in seeing if you read the data at 0x2040 in the .gci file.

 

[Fizzi]

What does the encrypted data look like that you are reading from the Arduino? Can you post a whole bunch of it and how does it compare with what is on the .gci file from a memory card? Are you reading every single byte that is in the .gci file during the save process?

I'm especially interested in seeing if you read the data at 0x2040 in the .gci file.

Here is a link containing a lot more data:
https://docs.google.com/spreadsheets/d/1emelxxOh5WMQyAox_0M5_2woIreniARa4_LCLL7qkfg/edit?usp=sharing

If you go down to row 656 you will see the place where it reads the game data date.

I grabbed the data I displayed from the tab called "Exclude MOSI 00 and FF". Basically this is all* the data read and written during a save operation after exiting the results screen minus the ones where the MOSI byte was 0x00 or 0xFF. I was basically just trying to find some interesting stuff at this point. If you see a gap in the count number, it means some data was skipped because MOSI was equal to one of those values.

* - If you scroll down low enough you'll see the count jumping all over the place. This is because 27 MHz is rather fast and the communication from the FPGA to the Arduino is far slower. In order to deal with this I have a queue module of about 8 KB which is used to write the data at the speed it comes in and read out much slower for the Arduino. When that queue fills up the data becomes gibberish. That said it's very easy for me to look for specific addresses that are being written to and pre-filter the data I'm interested in at the FPGA level - as long as I know what data I want.

The point at which data becomes very scattered is row 2778. This makes sense because each row is 3 bytes so 3 * 2778 = 8334 B. This is a little bit over 8 KB because the device has time to write a little bit of data out to the Arduino before the queue fills up.

Here is the data mentioned above (btw when count is 1 and 2 the MOSI bytes appears to tell the memory card the address from which to read):

Spoiler

Count|MOSI|MISO|MOSI ASCII|MISO ASCII
0|82|127|R|
1|1|128||?
2|48|0|0|#NUM!
5|222|83|Þ|S
6|186|83|º|S
7|58|83|:|S
8|58|83|:|S
9|83|83|S|S
10|83|117|S|u
11|83|112|S|p
12|83|101|S|e
17|83|97|S|a
18|117|115|u|s
19|112|104|p|h
20|101|32|e|
21|114|66|r|B
22|32|114| |r
23|83|111|S|o
24|109|115|m|s
25|97|46|a|.
26|115|32|s|
27|104|77|h|M
28|32|101| |e
29|66|108|B|l
30|114|101|r|e
31|111|101|o|e
32|115|32|s|
33|46|32|.|
34|32|32| |
35|77|32|M|
36|101|32|e|
37|108|32|l|
38|101|32|e|
39|101|32|e|
40|32|32| |
41|32|71| |G
42|32|97| |a
43|32|109| |m
44|32|101| |e
45|32|32| |
46|32|68| |D
47|32|97| |a
48|32|116| |t
49|71|97|G|a
50|97|32|a|
51|109|50|m|2
52|101|48|e|0
53|32|49| |1
54|68|53|D|5
55|97|47|a|/
56|116|48|t|0
57|97|55|a|7
58|32|47| |/
59|50|50|2|2
60|48|50|0|2
61|49|0|1|#NUM!
62|53|0|5|#NUM!
63|47|0|/|#NUM!
64|48|0|0|#NUM!
65|55|0|7|#NUM!
66|47|0|/|#NUM!
67|50|0|2|#NUM!
68|50|0|2|#NUM!

 

[SinsOfApathy]

Wow. You invested a lot of effort there lol. You could've just written the EXI output with Dolphin by writing a log command to EXI_MemoryCard.cpp instead of your current system.

You should look into making a patcher by abusing the memory card exploit. I mean, we could pretty much turn Melee into the PSO hack now.

 

[Fizzi]

Wow. You invested a lot of effort there lol. You could've just written the EXI output with Dolphin by writing a log command to EXI_MemoryCard.cpp instead of your current system.

You should look into making a patcher by abusing the memory card exploit. I mean, we could pretty much turn Melee into the PSO hack now.

I actually did do that, mostly. I say mostly because I still haven't found the location in dolphin's code where it does DMA reads and writes.

Regardless, I wanted to prove out that this could actually be done on a real system anyway (see potential uses).

If you look at the spreadsheet I linked earlier, there is a tab called "All Data Dolphin". If you compare the data there to the data of the real console you'll actually find that they don't match exactly. And back to what I mentioned earlier, you also won't see the large blocks of data because dolphin seems to handle that data differently.

 

[Achilles1515]

Here is a link containing a lot more data:
https://docs.google.com/spreadsheets/d/1emelxxOh5WMQyAox_0M5_2woIreniARa4_LCLL7qkfg/edit?usp=sharing

If you go down to row 656 you will see the place where it reads the game data date.

I grabbed the data I displayed from the tab called "Exclude MOSI 00 and FF". Basically this is all* the data read and written during a save operation after exiting the results screen minus the ones where the MOSI byte was 0x00 or 0xFF. I was basically just trying to find some interesting stuff at this point. If you see a gap in the count number, it means some data was skipped because MOSI was equal to one of those values.

* - If you scroll down low enough you'll see the count jumping all over the place. This is because 27 MHz is rather fast and the communication from the FPGA to the Arduino is far slower. In order to deal with this I have a queue module of about 8 KB which is used to write the data at the speed it comes in and read out much slower for the Arduino. When that queue fills up the data becomes gibberish. That said it's very easy for me to look for specific addresses that are being written to and pre-filter the data I'm interested in at the FPGA level - as long as I know what data I want.

The point at which data becomes very scattered is row 2778. This makes sense because each row is 3 bytes so 3 * 2778 = 8334 B. This is a little bit over 8 KB because the device has time to write a little bit of data out to the Arduino before the queue fills up.

Here is the data mentioned above (btw when count is 1 and 2 the MOSI bytes appears to tell the memory card the address from which to read):

Spoiler

Count|MOSI|MISO|MOSI ASCII|MISO ASCII
0|82|127|R|
1|1|128||?
2|48|0|0|#NUM!
5|222|83|Þ|S
6|186|83|º|S
7|58|83|:|S
8|58|83|:|S
9|83|83|S|S
10|83|117|S|u
11|83|112|S|p
12|83|101|S|e
17|83|97|S|a
18|117|115|u|s
19|112|104|p|h
20|101|32|e|
21|114|66|r|B
22|32|114| |r
23|83|111|S|o
24|109|115|m|s
25|97|46|a|.
26|115|32|s|
27|104|77|h|M
28|32|101| |e
29|66|108|B|l
30|114|101|r|e
31|111|101|o|e
32|115|32|s|
33|46|32|.|
34|32|32| |
35|77|32|M|
36|101|32|e|
37|108|32|l|
38|101|32|e|
39|101|32|e|
40|32|32| |
41|32|71| |G
42|32|97| |a
43|32|109| |m
44|32|101| |e
45|32|32| |
46|32|68| |D
47|32|97| |a
48|32|116| |t
49|71|97|G|a
50|97|32|a|
51|109|50|m|2
52|101|48|e|0
53|32|49| |1
54|68|53|D|5
55|97|47|a|/
56|116|48|t|0
57|97|55|a|7
58|32|47| |/
59|50|50|2|2
60|48|50|0|2
61|49|0|1|#NUM!
62|53|0|5|#NUM!
63|47|0|/|#NUM!
64|48|0|0|#NUM!
65|55|0|7|#NUM!
66|47|0|/|#NUM!
67|50|0|2|#NUM!
68|50|0|2|#NUM!

This is a hard format to try and read a lot of data from, because it is not in hex and especially since 99.9% of the time we are not going to be interested in the ASCII. I’m also looking at this on my phone, which is the most limiting part.


So do you know how the data you are reading relates to the binary of a .gci file?

 

[SinsOfApathy]

I actually did do that, mostly. I say mostly because I still haven't found the location in dolphin's code where it does DMA reads and writes.

This is the interface, which the EXI_DeviceMemoryCard actually overwrites.
https://github.com/dolphin-emu/dolp...f90e62/Source/Core/Core/HW/EXI_Device.cpp#L43

They also have a dummy EXI already written if you look below that. You could always debug with that.

This is the actual implementation, which doesn't write a single byte at a time.
https://github.com/dolphin-emu/dolp...ce/Core/Core/HW/EXI_DeviceMemoryCard.cpp#L551

Regardless, I wanted to prove out that this could actually be done on a real system anyway (see potential uses).

USB Gecko devices are just a USB <-> SI <-> EXI device, I mean, I'm glad you did it but it's been done lol.

I could write a proof of concept to do:

It may be possible to get interesting information from the internet and change the actual game state based on that. For example, each device can have an ID number linked to a station ID. It can then find out whether a tournament match has been scheduled for itself and pass a flag to the game telling it to lock the set up. It then might also be possible for the station to only become unlocked when one of the players in the match comes up to the station and scans his ID badge, or types in his username/password.

I'd just do it with a push-button switch and a microcontroller, since I'm not going out of my way to buy an RFID reader or something. In the end, it's pretty much just hijacking Gecko's existing opcode for "freezegame", which just runs a while loop in the injected region, since Gamecube and Wii are single-core processors meaning you can just pin the thread.

As for your original idea, it'd probably be easier to just read/write from an unused memory location than to try doing it with the memory card. Again, something I could hack together. I may later.

Using special codes it might be possible to increase the amount of data that can be loaded from a memory card. Data could be loaded from an SD card for example and this might allow things like loading a bunch of custom models into a real system.

That'd require knowledge of the HSD archive functions and such, which we need more documentation on to be practical as far as custom models go. I've got a few of the functions for it saved in my IDA, but kinda didn't give enough of a **** to look at them.

As it is, ProjectM's alternate stage loader just points at different file names (Edit: And a few codes I've seen lately), which we can do with other things like characters. Writing it into the memory card would be interesting though.

If you look at the spreadsheet I linked earlier, there is a tab called "All Data Dolphin". If you compare the data there to the data of the real console you'll actually find that they don't match exactly. And back to what I mentioned earlier, you also won't see the large blocks of data because dolphin seems to handle that data differently.

I'll look over it later. On my way to my Operating Systems class in a bit.

 

[Fizzi]

This is a hard format to try and read a lot of data from, because it is not in hex and especially since 99.9% of the time we are not going to be interested in the ASCII. I’m also looking at this on my phone, which is the most limiting part.


So do you know how the data you are reading relates to the binary of a .gci file?

My guess right now is that they're identical. I think the data is stored in the memory card exactly the same way it is transferred - though I haven't checked to make sure. I added new columns that convert the decimal numbers to hex (keep in mind if you try to compare the data to the .gci file that the transferred data is missing ASCII.

This is the interface, which the EXI_DeviceMemoryCard actually overwrites.
https://github.com/dolphin-emu/dolp...f90e62/Source/Core/Core/HW/EXI_Device.cpp#L43

They also have a dummy EXI already written if you look below that. You could always debug with that.

This is the actual implementation, which doesn't write a single byte at a time.
https://github.com/dolphin-emu/dolp...ce/Core/Core/HW/EXI_DeviceMemoryCard.cpp#L551



USB Gecko devices are just a USB <-> SI <-> EXI device, I mean, I'm glad you did it but it's been done lol.

I could write a proof of concept to do:



I'd just do it with a push-button switch and a microcontroller, since I'm not going out of my way to buy an RFID reader or something. In the end, it's pretty much just hijacking Gecko's existing opcode for "freezegame", which just runs a while loop in the injected region, since Gamecube and Wii are single-core processors meaning you can just pin the thread.

As for your original idea, it'd probably be easier to just read/write from an unused memory location than to try doing it with the memory card. Again, something I could hack together. I may later.



That'd require knowledge of the HSD archive functions and such, which we need more documentation on to be practical as far as custom models go. I've got a few of the functions for it saved in my IDA, but kinda didn't give enough of a **** to look at them.

As it is, ProjectM's alternate stage loader just points at different file names (Edit: And a few codes I've seen lately), which we can do with other things like characters. Writing it into the memory card would be interesting though.



I'll look over it later. On my way to my Operating Systems class in a bit.

Thanks for the info. I do think debugging with that is the right way to go lol I just haven't really had anything to debug yet .

I did look up the USB gecko device to help come up with ideas (in fact I'm using the same FPGA). I agree it's definitely similar but I think you're not quite understanding the goal of the project. What you see in the picture is just a prototype. I plan on designing a single board that plugs into the memory card slots and serves to add functionality to Melee (WiFi, RFID, etc) that is supported by a hacked version of Melee. As far as I know the USB Gecko does not double as a memory card does it?

In any case if I appreciate any help you're willing to provide.

 

[SinsOfApathy]

I did look up the USB gecko device to help come up with ideas (in fact I'm using the same FPGA). I agree it's definitely similar but I think you're not quite understanding the goal of the project. What you see in the picture is just a prototype. I plan on designing a single board that plugs into the memory card slots and serves to add functionality to Melee (WiFi, RFID, etc) that is supported by a hacked version of Melee. As far as I know the USB Gecko does not double as a memory card does it?

In any case if I appreciate any help you're willing to provide.

Well, the Geckohandler isn't part of the device. It's injected into memory at runtime (for GC, via an exploit, which we have with the Memcard exploit). It's entirely possible to write a different handler, as I did, in order to provide different functionality. What Gecko does it read the EXI buffer for 1 byte op codes, then perform functions, or it'll write data into the EXI buffer which is subsequently transferred to the computer, such as memory reads.

You'd still need a memory card to save to, but you're just as capable of writing/reading arbitrary data.

Edit: That goes to say, the novelty of it would be a microcontroller providing the functionality you're talking about.

 

[Fizzi]

Well I've been working on the part of the project that writes interesting information to free memory such that it can be transferred to the device later. This is basically my first code so please provide feedback. It's very likely I'm doing things I shouldn't be doing, not doing things I should be doing, or doing things in an inefficient fashion.

Source code:

Spoiler

#Inject at address 0x80033d48

#execute replaced instruction (this updates a player's stock count)
stb r0,0x008E(r4)

#load start address of stored data
lis r12,0x8000
ori r12,r12,0x2A00

#find the amount of stocks that have been played so far
li r9,0
lis r11,0x8045
ori r11,r11,0x30EB
lbz r10,0x0000(r11)
add r9,r9,r10
lbz r10,0x0E90(r11)
add r9,r9,r10
lbz r10,0x1D20(r11)
add r9,r9,r10
lbz r10,0x2BB0(r11)
add r9,r9,r10

#death count has already been incremented at this point so subtract 1 from stock count
subi r9,r9,1

#increment write position by the amount of stocks played
mulli r11,r9,0x34
add r12,r12,r11

#get and write game time played (counted in frames)
lis r11,0x8046
ori r11,r11,0xB6C4
lwz r4,0x0000(r11)
stw r4,0x0000(r12)

#increment write position to player 1 write position
addi r12,r12,4

#initialize player id for search, start with player id 0
li r8, 0

LOOP_PLAYERS:
#execute function to get character data pointer
addi r3,r8,0
lis   r11,0x8003
ori   r11,r11,0x4110
mtctr   r11
bctrl

#once here r3 is equal to character data pointer, if pointer is equal to 0, skip to next player
cmpwi r3,0
beq INCREMENT

#here the character data pointer is valid, read data and store in memory somewhere

#load and store stock count, this is a global address, must fetch
lis r11,0x8045
ori r11,r11,0x310E
mulli r10,r8,0xE90
add r11,r11,r10
lbz r4,0x0000(r11)
stw r4,0x0000(r12)

#load and store percentage
lwz r4,0x1890(r3)
stw r4,0x0004(r12)

#load and store last move connected
lwz r4,0x20EC(r3)
stw r4,0x0008(r12)

INCREMENT:
addi r12,r12,0xC
addi r8,r8,1
cmpwi r8,4
blt LOOP_PLAYERS

Gecko code:

Spoiler

C2033D48 00000017
9804008E 3D808000
618C2A00 39200000
3D608045 616B30EB
894B0000 7D295214
894B0E90 7D295214
894B1D20 7D295214
894B2BB0 7D295214
3929FFFF 1D690034
7D8C5A14 3D608046
616BB6C4 808B0000
908C0000 398C0004
39000000 38680000
3D608003 616B4110
7D6903A6 4E800421
2C030000 4182002C
3D608045 616B310E
1D480E90 7D6B5214
888B0000 908C0000
80831890 908C0004
808320EC 908C0008
398C000C 39080001
2C080004 4180FFB0
60000000 00000000

Basically the above code will set up memory to look like the following:

Now all that's left is a mechanism for transferring it. I should also note that this is not by any means complete. This is just a starting point - much more data will be captured in the future.

 

[thetaiter]

Is this still being worked on? This is amazing. Anyone trying to do this for smash 64? I am trying to write an AI program using NEAT that can teach itself how to play smash. The last thing I need to achieve this is extracting data from the game in real time. Currently I am just taking screenshots 60 times/sec and using that as the input to the algorithm. I'd like to use more detailed information from the game to determine the fitness of the different neural networks.