• Welcome to Smashboards, the world's largest Super Smash Brothers community! Over 250,000 Smash Bros. fans from around the world have come to discuss these great games in over 19 million posts!

    You are currently viewing our boards as a visitor. Click here to sign up right now and start on your path in the Smash community!

Overriding PJ64k's Kaillera Client to add AQZ/more features

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
Legend
- confirmed for release - stable
- unconfirmed for release - could be removed!
- development note entry

Original Post 5-12-2012
I am a software programmer, fluent in C/++/#, Java, VB6, and x86 assembly.

I used PJ64k somewhat in 2004, and was disappointed by the lack of features in Kaillera. I assumed after downloading the updated client in 2012 it would have been far upgraded by then, however only few features it seems have really been implemented since then and they are mainly in terms of stability.

I have considered writing a replacement Kaillera override client, that would allow the original stable Kaillera client to run in the background however invoke all of it's features - including playerlist, chat, games list, etc. I also think it should reflect some of the recent additions to the online playing community, including AQZ support. AQZ and Kaillera players would not be able to play together, but at least people running the AQZ plugin could use a central Kaillera server to find and play AQZ games (which, in my opinion, desync far less often).

I would like to ask this community what additional features (if possible) they would like added to this client override, and any other suggestions would be nice. Right now this project is not a reality, however I have had success with projects in the past and can make this a dedicated reality if the demand is there for it.

AQZ has been developed since around 2009, and it still has not gained the popularity it deserves primarily to the fact that there's 1) no central server and 2) it's not user-friendly.

The disadvantages of the AQZ netplay plugin are:
1) No central server / master list
2) Does not synchronize cheats or save states

The advantages of AQZ over Kaillera games are:
1) The frame delay is redundant and lag is minimal.
2) Less frequent desyncs if all players are using the same plugins. I've honestly played for almost 5 hours without a single ds.

Kaillera p2p games are good and don't desync often, however the stability of Kaillera is arguable. So please post your suggestions as far as AQZ implementation and if you haven't heard of it, here's the netplay plugin topic:
http://forum.pj64-emu.com/showthread.php?t=1973

The idea is to implement this into a kaillera-like client to where there are both AQZ and Kaillera games listed, and perhaps recode the p2p client to where the hoster can optionally broadcast their p2p game onto the main server list.

This will essentially localize all game servers, whether they are AQZ, P2P, or Kaillera, into a single network (i.e. Galaxy) and allow users to more easily connect. I fear the reason AQZ and P2P are not used as much is because of the fact the client was not coded in a user-friendly manner, however I have much experience with doing so and would like to know the demand from this excellent community.

Update 5-25-2012 - concept


It currently relies on a secondary master server to broadcast the primary kaillera server along with any additional aqz or p2p matches that are listed as public.

This is a concept gui and is not final and far from complete. For example it currently shows the host as a username for aqz/p2p matches and also does not update the Users amount for them yet.

However, double-clicking a p2p game will (optionally) close the current client and reload the client in p2p mode, which is now possible by adding executable arguments. These arguments are:
-p2p tells the client to automatically load in p2p mode.
-mode specifies whether to start as a client or host.
-ip specifies the IP/PORT to connect to
-framedelay to autoset a custom frame delay
-broadcast is a boolean (0/1) value that specifies whether the p2p match should be listed to the secondary master server. (currently not implemented)


So if a user currently double-clicked my p2p match listed in this screenshot, it would restart the client with these arguments:
Project64k.exe -p2p 1 -mode client -ip smashbros.servegame.com:27886

I still haven't developed server time-outs, and there are a few bugs that need fixing as well, but I thought I would keep you posted on the progress. Also, users that are not using my client will see a /me chat message when loading an AQZ or P2P game, and if broadcast is specified as true it will include the host IP/port. This is to maintain somewhat backwards compatibility, as updated clients will not see this chat message but will list the specified game instead.

Update 5-27-2012 - bugfixes
- fixed the kaillera chat window from constantly 'refreshing' on new updates. This means while scrolling the chat will update without auto-scrolling to the bottom, and selections are also saved so copying and pasting is a lot easier in busy areas (i.e. Galaxy).
- fixed some memory leaks (some repeated at least 8 times) and wrote functions to replace a lot of repeated code structures, which will reduce the size of the client and is more conventional and secure.
- *undecided* maintains backward compatibility with 0.13 01 Aug 2003 release. There is no "Warning: Emulator mismatch may cause desync" when playing with this emulator because the netcode, thus far, remains the same (excluding any additional broadcast netcode)

Update 5-28-2012 - GUI improvements
There's also a bug that if you click anywhere in the chat, the text cursor prevents the chat from auto-scrolling properly. This means if the chat has focus, the furthest it will scroll is the line of the text cursor. Without some buggy hacking, the use of RichEdit is not logical. Owna acknowledges this in his code.

A workaround would be to disable focus on the chat window, but that would prevent copy/pasting. The cursor is required to make selections. So maybe we should make the focus more apparent, i.e. gray out the chat textbox when it's not in focus, and disable the autoscroll while the main chat has focus. Then when the chat textbox regains focus, toggle autoscroll to true.

I've rewrote Owna's rigged re_append function which works with the richtext object directly rather than external API messages. I considered using QTextEdit, an alternative to RichEdit. It's lightweight, cross-compatible and has other features. But the library dependencies were over 1GB so I'm looking at an alternative GUI controlset, wxWidgets. The new function will append the data to this object, and I'm working on a bugless autoscroll now.

I am considering updating the source, but not sure how many servers will adopt the change. Nevertheless, an is_spoofing user variable would be useful. There is a commented portion of the client that used to append "IS_SPOOFING" to the username if they were spoofing, a client-side indication. This could still be used somehow, but perhaps with an asterisk symbol appended instead. I think server owners should have the right to know who is spoofing and who is not, and if they desire, ban spoofing on their server. Let me know your thoughts on this.

This in combination with some of the control issues we're having with RichText has convinced me to restart the GUI from scratch using the wxWidgets control library. wxWidgets even has a very nice socket layer, but I am very hesitant to change socket modules without causing any desync implications, so I will most likely just change the GUI controls to use wxWidgets and leave the socket layer untouched.

wxWidget controls have a lot of awesome features, such as animated emoticons in chat, background images in all controls - if we had a graphics designer, we could come up with a killer GUI using these updated controls. I'm ok with Photoshop, but I don't want to get too distracted with the GUI because there's simply too much code that needs updated. Perhaps we can introduce this concept later using skins.

This will not be a copy-paste job because the whole core is built around these crappy windows controls. The trivial part will be making sure the client stays in sync with 0.13 despite all the core improvements, so I'm still testing with a p2p 0.13 client after every little change I make to ensure nothing is broken, because I am far from a Kaillera expert. Speaking of which, is Project64k meant to be ran on Windows only? I hope so, because the wxWidgets library I'm using is the Windows version and I'm not sure what results would incur from running it on WINE. I also don't even know how many of you use PJ64k with WINE, or if it's ever been possible, so let me know if this is an issue.


Update 5-29-2012
- in the p2p host dialogue, there is now a list of servers that mimics the list in the main interface. The purpose of this is the optional broadcast. It will need to know a server to broadcast the game info to, but due to the way the Kaillera server is designed this will require an established UDP connection with the server. Hopefully this will not impact the benefits that p2p offers with more bandwith being used. Maybe we can add a server-side variable just for p2p broadcasters, to force the server not to send them redundant data (such as the userlist, gamelist, chat, etc). Either way this was necessary to establish a ping/pong between the p2p server and the kaillera server.

- with the new wxWidgets GUI, the bug 'You must close all Kaillera windows first!' when trying to close Project64 (despite all Kaillera windows being closed) has vanished. I myself have found this annoying on many occasions, forcing me to close PJ64 from task manager. Not anymore.

- with the adoption to wxString, a buffer overflow prevention string class that comes with wxWidgets, the majority of the buffer overflow exploits in the existing Owna client are no longer an issue. I saw a blogspot that referred to Kaillera hacking and many of the alleged hacks were buffer overflow exploits. I am pleased to inform you that these exploits are no longer a reality. Improved security ftw.

- implemented gamelist filters. Now you can optionally ignore all games you do not have installed in your ROM directory. You can also specifically filter out p2p matches, aqz matches, or kaillera matches. Chats are not affected by this filter yet.


- when redesigning the GUI for wxWidgets, I had an idea for a button specifically for connecting to The Galaxy. From my experience The Galaxy is the most popular N64 Kaillera server, but some may not agree with this. So maybe we should have an Options dialogue with a checkbox 'Autoconnect to Galaxy.' Please post any GUI or Options requests you have here. Now would be the time to implement them.

To-Do / etc
- since the author of the AQZ topic seems to be ignoring my request, I've been thinking about a method to approach the AQZ broadcast. This is my idea: to bind the AQZ plugin to an additional plugin I will write so that both libraries are loaded simultaneously. However, in order for this to work, people will need to update their AQZ plugin to my new binded one that will essentially be hacking in GUI elements. Alternatively, I could position the window slightly above/below the AQZ window and just add more options. Maybe we should make the client check for which AQZ version the user has, and if they don't have the updated one, load an unbinded secondary DLL; if they do have the binded one, just load the binded DLL. This is the best solution I can think of after a lot of brainstorming and I am still not satisfied, so hopefully before I get to this point we can work something out with the author.

- wxList is capable of listing images in it's listbox control, so maybe the gamelist can be improved with default images for select games. i.e if a user loads a Mario Kart game, there would be an image of the Mario Kart game displayed next to it's list item. The disadvantage of this would be the list items would have to be more spaced apart to give room for the image. Right now it's just an idea, and goes against my initial goal of reducing the client size, but I'm still up for any and all GUI improvements.
 

Battlecow

Play to Win
Joined
May 19, 2009
Messages
8,746
Location
Chicago
Are you familiar with Ownasaurus' kaillera client? Not sure if that covers any of what you're intending to do, but it's what most people use and it was a big improvement over what we used before it.
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
Are you familiar with Ownasaurus' kaillera client? Not sure if that covers any of what you're intending to do, but it's what most people use and it was a big improvement over what we used before it.

Yes, I am familiar with that client. However, it does not have user-friendly AQZ support. AQZ has been developed since around 2009, and it still has not gained the popularity it deserves primarily to the fact that there's 1) no central server and 2) it's not user-friendly.

The disadvantages of the AQZ netplay plugin are:
1) No central server / master list
2) Does not synchronize cheats or save states

The advantages of AQZ over Kaillera games are:
1) The frame delay is redundant and lag is minimal.
2) Less frequent desyncs if all players are using the same plugins. I've honestly played for almost 5 hours without a single ds.

Kaillera p2p games are good and don't desync often, however the stability of Kaillera is arguable. So please post your suggestions as far as AQZ implementation and if you haven't heard of it, here's the netplay plugin topic:
http://forum.pj64-emu.com/showthread.php?t=1973

The idea is to implement this into a kaillera-like client to where there are both AQZ and Kaillera games listed, and perhaps recode the p2p client to where the hoster can optionally broadcast their p2p game onto the main server list.

This will essentially localize all game servers, whether they are AQZ, P2P, or Kaillera, into a single network (i.e. Galaxy) and allow users to more easily connect. I fear the reason AQZ and P2P are not used as much is because of the fact the client was not coded in a user-friendly manner, however I have much experience with doing so and would like to know the demand from this excellent community.

Thanks again, and I look forward to your suggestions.

donkey.KONG
 

Battlecow

Play to Win
Joined
May 19, 2009
Messages
8,746
Location
Chicago
I'm not the most technically-minded fellow, but what you're proposing sounds excellent. I think everyone would be very happy with a less desyncy option, if it were implemented in a user-friendly manner. If you did this, I'd weep tears of thankfulness onto your manly coding shoulders and shake your hand firmly and gratefully.

AQZ is the thing people were using for p2p teams like a few months ago, right? I could never figure that out...
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
You might want to talk more with firo, who currently owns the Galaxy server and knows a bit about kaillera and the netplay clients, as most of us here don't know much about this/don't understand/can't help.

This does sound like a fantastic idea, and there is much glory to be had if you could implement it. P2P is king and this is a great idea.

Talk to firo to see if something like this is feasible.
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
I'm glad the demand is starting to appear. I have had much success in the past with similar projects, while they did not have anything to do with emulators, I did turn GTA:SA into a Multiplayer experience where 500 players can join a server with a centralized master server list. See here: www.sa-mp.com
At this moment, there are 14718 players online purely from a external client I created (similar to Kaillera was created for PJ64, I created SA-MP for GTASA).

So the experience is there. I ask any coders that wish to be a part of this team to join forces, as this will be a 100% open source project.

P2P is king, but AQZ is arguably better. If the cheats and savefiles get synced properly, which will be CAKE compared to converting a SP game into a MP game, AQZ has far less desyncs and if it were listed in the Galaxy game list then I can have the client automatically change the input plugin to AQZ.

I will also approach the developer of the AQZ Input plugin and perhaps he can assist me in seamlessly implementing it into Kaillera without me having to do unstable API hacks.

I can easily hide the AQZ console window, and invoke the /connect /host /lag commands, but I would rather have direct access to AQZ via a .dll that I can call at runtime. This will be determined in the near future.

Thanks for your support, and I plan to have a concept GUI posted by the end of the night. I will also try to get in touch with Firo as to his suggestions since this will primarily be based on his server, since it is the most active Smash64 server today.
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
Wow. Holy ****. I had no idea of your credentials! ;)

The more power to ya! I'm sure I speak for everyone here--hell, everyone who uses kaillera--when I say we certainly need an upgrade. If you can pull this off, this could greatly impact the online emulator scene and might be just what we need to attract more people.

pleasepleasepleaseplease do this!
 

Sangoku

Smash Master
Joined
Apr 25, 2010
Messages
3,931
Location
Geneva, Switzerland
I think the main problem we have now is delay and lag. There's probably nothing we can do about delay (or is it possible to reduce it?), but lag is a big issue. If someone else is browsing or downloading it gets unplayable. Again, I don't know if it's theoretically possible to do anything about it.
 

M!nt

Smash Champion
Joined
Jan 26, 2011
Messages
2,087
Awww yeah

Sounds great, Can't wait to see it :awesome:
 

dandan

Smash Lord
Joined
Feb 11, 2009
Messages
1,373
with sort of stable internet, you can solve lag by using dynamic delay. kaillera chooses a delay bracket at the start of the session and it does not change according to changes in the connection.

delay is not something you can solve, as it is caused by physical distances between clients, or clients to server in cases of centralized server solutions. if it takes 50 ms on average for packets to arrive from you to the server, you simply cannot make commands influence the game faster.

the main problem i see in kaillera is the lack of dealing with missed packets and problems in connections.
as kaillera uses udp, it does not confirm the arrival of all packets and as such desyncs can occur because of packets that include commands sent by users such that the server will not send that command to teh other user though the first user will see the command take place.
this can be solved by using more stateful communication protocols, but has adverse effects on the speed of transmission of data.
 

Sangoku

Smash Master
Joined
Apr 25, 2010
Messages
3,931
Location
Geneva, Switzerland
I know there is the physical limitation, but what I meant was: have we already reached this limit?

And "transforming" lag into delay won't help (when I see I'd need 15 frames delay to run it smoothly, that's pretty useless). I don't play other online games, but does it always lag (or delay) like that when others use your connection? I thought not, but maybe it does and there's nothing we can do. Then fake smash sucks lol.

:phone:
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
As it stands, I don't mind the input delay. That really can't be improved upon, and it's fine as is. If we could somehow figure out how to, like dandan said, dynamically change the delay as it was needed, that would help reduce desyncs, but I'm not sure it would be good for the game. Who wants to play a game where delay is constantly changing? Fixed is bad enough, I can't image fluctuations between 2 and 15 frames at any given instant.

I'd imagine dynamic delay is also very difficult to incorporate, especially since I don't think any online services currently do that. Hell, Nintendo didn't want to and just stuffed a 30 frame delay in there as a catch-all to prevent lag-outs.


What's really killer are the desyncs and how often they occur. I don't really know the differences between server play, AQZ, and P2P, but I imagine it has something to do with packet paths and as it stands, we're currently using the worst option for playing against random people. We need a more reliable way to ensure packet transfer success, and AQZ looks to be that method.

I would absolutely love a way to easily incorporate AQZ in the same way we connect to the server, or maybe a way to broadcast P2P requests that are easily connectable. This would eliminate the need for a server, and just require somebody to host a constantly updating list. But I dunno. I certainly can't implement any of this, so they're just ideas....
 

Sangoku

Smash Master
Joined
Apr 25, 2010
Messages
3,931
Location
Geneva, Switzerland
Looks like I'm the only one fine with DS... Also less delay means I could play with Americans and not be stuck here playing CPUs... But I understand others don't need to.
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
Like dandan said, there's no way to decrease delay. It's already at a minimum (so low it causes desyncs), and the only way to change that is to physically move closer.

But on the bright side, the delay with kaillera is lower than all other online services! Nowhere else would SSB be even remotely playable. So I guess there's that...

it's also the most unstable, but hey, you get what you pay for get
 

firo

Smash Ace
Joined
Jul 27, 2008
Messages
600
Location
Champaign, Illinois
Hey there donkey.KONG

Obviously this is a good idea. I've toyed around with this before. Adding basic p2p "support" to the server is fairly simple. Upon start all the client needs to do is send the game status packet and open a p2p window at the very least.

AQZ is also a very good plugin, although technically it can have more delay depending on the location of the players. There are a few problems with it though, most importantly being that there is no source available (to my knowledge). I'm not sure what's possible without the source.

I've tried to work on this stuff before, unfortunately my knowledge of windows programming is lacking and I'm usually busy with school. Having someone else around to work on this would be great, and I'll have time over the summer. The current kaillera client is poorly documented and was developed using an old version of visual studio and was ridden with security holes that were recently patched up by some helpful people. I've gotten to compile on 2010 though.

Coding on the client side is doable, but what would be annoying is figuring out and modifying the server to accommodate this. Maybe it won't need to be changed. But if it does, then the server will have to be decompiled (it is written in java, and there is no source). I've done it before, but it's not pretty. I'd honestly be down to make a new server (maybe without kaillera server game support) and possibly add it later.

What I'd really want to eventually do is integrate netplay into mupen64plus, which is cross-platform, open source, and is updated often. Desynchs occur less on newer/better emulators. But I haven't had the time to even start looking at it.

In any case, I would really want to contribute to this if this is actually going to happen. Most projects involving kaillera have started with a lot of potential but then fizzled out when people had other things to do. Hopefully this can get off the ground.
 

MATTS!

Smash Ace
Joined
Jul 31, 2008
Messages
575
Location
Section 476
This will take ALOT of work. Ill support you guys in anyway possible. Good luck guys.


The Great MATTS!
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
http://en.wikipedia.org/wiki/GGPO#Design

That's really freaking cool.

http://skullgirls.com/2011/09/skullgirls-ggpo-and-you/

About halfway down, it starts to describe rollbacks.


TL;DR
With "rollbacks", there is zero lag on your part, but since realistically there is lag, the game guesses what your opponent is going to do.

So you see one thing happen, but depending on what your opponent does a few frames later, the game will roll back and revise itself to correct what you see with what your opponent did.

How is this possible? Well, it leads to choppy gameplay, depending on latency and how often the game has to roll back. If, for example, you're playing at 15 frames, and you jab somebody, you may see them briefly flinch, but if your opponent shielded in that 15 frame window, the game jolts back to the point before the jab and registers a jab on shield.



I honestly have no clue how smooth this will look when actually playing (it sounds pretty bad on paper), but I imagine it won't be terrible if you're playing with really low ping (2-3 frames). Anything too large, and the game would become unplayable. I suppose the idea isn't to visually see two "alternate endings" for every action, so large frame delay would ruin the experience.

Still, really really cool.


EDIT: It also looks like you can combine roll backs and input delay for high ping situations, if purely rollbacks and zero input lag creates too choppy gameplay. So in the 15 frame scenario, you could hack down input delay to 7 frames, and use rollbacks for the other 7 frames. It would still be pretty bad, but better than purely one or the other.
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
I personally think it's a great idea. Probably the best idea ever in terms of any netplay service in existence, be it middleware or a professional operation. *****es can stop complaining about delay, and in reality, a few frames of rollback is hardly noticeable.

Here's a cool example: http://www.youtube.com/watch?v=q0TZdY9_XHo

You can barely tell where the rollback occurs, and hadn't it been for the "KO" sound, you'd never know it was there.

IS THIS NOT WHAT YOU ALL WANT???

more: http://www.youtube.com/watch?v=5BCsltPNWWE
Rollbacks are constantly happening, but you can't even tell.

I think that even implementing ONE frame of GGPO would reduce a frame of input delay, and satiate all those people *****ing about 3 vs. 4 frames.

This is seriously cool and definitely the future of online gaming, but I have no idea how hard it'd be to implement for Smash, or if the project developers would allow it.
 

firo

Smash Ace
Joined
Jul 27, 2008
Messages
600
Location
Champaign, Illinois
Someone mentions GGPO and rollbacks about once a year. Simply put, for many reasons, it's not going to happen. Much better to focus on what is currently being done.
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
Packet data prediction is feasible for 3d games with coordinate systems in which the accuracy of prediction has minimal impact on the smoothness of gameplay. For example techniques such as linear interpolation are used to minimize artifacts of lag, if a packet predicted a player was 1.6 units ahead of where they actually were than the difference is interpolated and the player experiences a smooth correction.

This is not relevant in terms of emulation and keystrokes as we are not dealing with native game libraries on a native architecture. It is my belief that if such a system were implemented in Kaillera, it would result in something like this from a network/gameplay perspective:
SA-MP Vehicle Synchronization

This is what I've defined an artifact of latency, and you can quickly see how client-side network data prediction would cause issues on a Kaillera game.

AQZ is also a very good plugin, although technically it can have more delay depending on the location of the players. There are a few problems with it though, most importantly being that there is no source available (to my knowledge). I'm not sure what's possible without the source.
For obvious reasons I would prefer to either have the source of rewrite the plugin, however the latter option would only delay the underlying cause of this project. Although, you would be legitimately surprised at what one can achieve without the source - GTA's source was never released and SA-MP was done purely through code injection and reverse engineering techniques. There are other options such as API hacking, in which the client would simply hide the AQZ window but still input it's console commands (such as /start or /name) via sending API messages to it's window handle. This could result in some additional cross-compatibility work due to the API differences between XP/7/etc, although at the moment it's probable.

I have not decompiled his plugin to check, but I'm guessing he compiled it in Release mode (without debug symbols) which would drastically delay any reverse engineering approach. If the author could simply compile the plugin in Debug mode, the function and variable labels would be available through IDA and a more legitimate approach could be made, still waiting on his response.

Firo said:
The current kaillera client is poorly documented and was developed using an old version of visual studio and was ridden with security holes that were recently patched up by some helpful people. I've gotten to compile on 2010 though.
I would prefer not to work from an insecure code base. Please send any updated source you have to cmcjacob@gmail.com.

Firo said:
Coding on the client side is doable, but what would be annoying is figuring out and modifying the server to accommodate this. Maybe it won't need to be changed. But if it does, then the server will have to be decompiled (it is written in java, and there is no source). I've done it before, but it's not pretty. I'd honestly be down to make a new server (maybe without kaillera server game support) and possibly add it later.
I was under the assumption that the server source was already released. I'm unable to determine whether rewriting it will currently be necessary based on the few feature propositions I've made. The client will be overridden and the server side may operate as an add-on process.

I will ask this: is the Kaillera netcode well documented? If so, optimizing a new server may be a good idea. The goal here is to essentially reduce desync and populate the gamelist with other netplay options, and while I'd like to say that I'm a renounced emulator developer, the truth is I'm not. My approach is far more subliminal and consists of low-level hacks to existing code structures.
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
Also, the packet reliability issue is an issue with UDP. If Kaillera uses Raknet we can consider ourselves extremely lucky because they have made some very nice updates in terms of their UDP reliability layer. A simple update to the Raknet libraries would have immediate results on reducing desync.

There's the option of switching over to TCP entirely, but there's a huge playerbase that would have to adapt to this network change so at the moment both options should be on the table.
 

dandan

Smash Lord
Joined
Feb 11, 2009
Messages
1,373
changing to TCP can and probably would cause sever spikes while trying to mend missed packets.
so yeah, you can use it to remove desyncs almost entirely, but the decrease in performance might not be worth it.

furthermore, if you had the source code of the servers, changing the sockets would be easy enough, but can you do such a change with RE without much problem? or will you have to rewrite the whole thing.

anyway i look at it, this seems like too much of an undertaking for such a low user base.
the connectivity issue is one of the main ones that caused me to almost stop playing entirely, and i would have tried to fix them myself, but i do not think this is very feasible/cost-effective.
 

blaze3927

Smash Ace
Joined
Aug 12, 2009
Messages
832
Location
Melbourne
TCP has been discarded after quite a bit of input.

low user base, mmmm, i dunno, theres probably 10,000+ users that use or have used kaillera in the past 6 months. 1000~ users that hop on weekly etc.
 

firo

Smash Ace
Joined
Jul 27, 2008
Messages
600
Location
Champaign, Illinois
No, kaillera does not use Raknet or any well-established network engine. It uses its own transfer layer and protocol, the only documentation of which can be found here
http://sourceforge.net/projects/okai/
In the files->protocol folder. The original kaillera was not made open source, so the only documentation available is stuff reverse engineered by the open kaillera group years back. There's not much, but its adequate.

For the server, the most recent source available is here:

http://sourceforge.net/projects/emu...aillera-server/emulinker-kaillera-server-1.0/

Emulinker 1.0.2 is what all the current servers are based on. From what I understand all of the netcode has remained the same. The only things that have changed in recent versions, atleast to my knowledge, has been user access stuff and a couple more commands. I only took over the server a few years ago and that was after the current source was lost.

TCP is always an option, but I'm not sure what type of delay that would require. I'm sure I speak for a lot of people saying that I'd rather have as little delay as possible, even if that means the possibility of desynch is a bit higher. The most recent kaillera client and source comes from the index: https://code.google.com/p/p2pkaillera/downloads/detail?name=Ownaclient.dll&can=2&q=
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19


It currently relies on a secondary master server to broadcast the primary kaillera server along with any additional aqz or p2p matches that are listed as public.

This is a concept gui and is not final and far from complete. For example it currently shows the host as a username for aqz/p2p matches and also does not update the Users amount for them yet.

However, double-clicking a p2p game will (optionally) close the current client and reload the client in p2p mode, which is now possible by adding executable arguments. These arguments are:
-p2p tells the client to automatically load in p2p mode.
-mode specifies whether to start as a client or host.
-ip specifies the IP/PORT to connect to
-framedelay to autoset a custom frame delay
-broadcast is a boolean (0/1) value that specifies whether the p2p match should be listed to the secondary master server. (currently not implemented)

So if a user currently double-clicked my p2p match listed in this screenshot, it would restart the client with these arguments:
Project64k.exe -p2p 1 -mode client -ip smashbros.servegame.com:27886

I still haven't developed server time-outs, and there are a few bugs that need fixing as well, but I thought I would keep you posted on the progress. Also, users that are not using my client will see a /me chat message when loading an AQZ or P2P game, and if broadcast is specified as true it will include the host IP/port. This is to maintain somewhat backwards compatibility, as updated clients will not see this chat message but will list the specified game instead.
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
Wow, great work! This is one of the first times somebody has come in here claiming to change things and actually followed through! The potential here is fantastic. Keep it up; I eagerly await the finished product!

Simply being able to broadcast P2P and AQZ matches is the best possible step to take. Cuts out the middle man (MSN/chat)!
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
I have attempted to contact the author of the AQZ Netplay Plugin in his release topic here:
http://forum.pj64-emu.com/showthread.php?t=1973&page=19

Hopefully he can assist me approach the AQZ broadcast method with native code or data structures, or perhaps even a library that I could use to interact with the plugin internally.

I also find it frustrating that the Project64 1.7 source has not been made public. Closed-source development should always go open-source when development activity ceases for the sake of the community. Even if the code has unfinished portions, it would still be useful to squash bugs and/or even manually implement kaillera into 1.7.

Unfortunately that's not a reality at the moment so I'm focusing on this build, and currently would like to avoid any subclassing or API hacks so hopefully the AQZ plugin author will reply with good news. If not, it will be simple to subclass the AQZ window and input commands into it, however that approach is not always reliable in terms of cross-compatibility. It might work on 7/Vista, but not 8 or XP or WINE on *nix, etc.
 

M!nt

Smash Champion
Joined
Jan 26, 2011
Messages
2,087
:happysheep::happysheep::happysheep::happysheep:

I don't think they will help you there.
They are very negative towards kaillera and don't support it at all.

Good work tho!!!
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
Yeah, but I guess the fact that he's branching out (accepting kailler sux) and trying new options (i.e. THEIR plugin) has gotta mean something, right?
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
They're unresponsive so far. Maybe my proposition was asking too much.

Some minor updates in the meantime:
- fixed the kaillera chat window from constantly 'refreshing' on new updates. This means while scrolling the chat will update without auto-scrolling to the bottom, and selections are also saved so copying and pasting is a lot easier in busy areas (i.e. Galaxy).
- fixed some memory leaks (some repeated at least 8 times) and wrote functions to replace a lot of repeated code structures, which will reduce the size of the client and is more conventional and secure.
- maintains backward compatibility with 0.13 01 Aug 2003 release. There is no "Warning: Emulator mismatch may cause desync" when playing with this emulator because the netcode, thus far, remains the same (excluding any additional broadcast netcode)

The more I look at this code the more I think it would be better off rewritten. It's rarely commented and the variable naming convention is, my apologies, pathetic. The variables 'xx' and 'xxx' are repeated in every other function despite their varying purpose. It's also not object-oriented at all.

I think the client would be better structured as a class with localized variables rather than all these ugly global vars that rarely get used or were there for debugging purposes. A #IF _DEBUG precompilation header before this debugging code would have sufficed, but it is evident this was rushed to a release based on the fact that "it works."

I also think the (N02.RES) resource file should be completely rewritten. There are many instances that certain internal strings are obtained via SendWindowMessage API to the client's own resource handle. I think this was a counter-measure to prevent people from using hex editors to edit things like their emulator version, but it's also not as efficient as simply encrypting your strings or packing the executable.

So far, when I've made changes to the resource file the client crashes upon connect. Not sure why yet, perhaps it has something to do with the fact that Express doesn't support MFC although I still had to link the MFC libraries. But since so much code is written around the controls in that resource file, which contains all the dialogues interfaces and listboxes etc, a restart of that resource would demand almost a rewrite of the client as a whole. A lot of the code could be copied and pasted, but again with the naming convention and memory leaks I've encountered in my brief experience so far I still think Ownasauraus's work would be better off rewritten.

I suppose the "sketchy API subclassing" I was referring to in regards to the last-resort AQZ broadcast method wouldn't be so sketchy afterall, considering such hacks are a foundation of Owna's client.
(see all the unnecessary references to SendWindowMessage) :p

cmcjacob
 

KnitePhox

Smash Lord
Joined
Oct 17, 2005
Messages
1,838
Location
Chicago, IL
- fixed the kaillera chat window from constantly 'refreshing' on new updates. This means while scrolling the chat will update without auto-scrolling to the bottom, and selections are also saved so copying and pasting is a lot easier in busy areas (i.e. Galaxy).
GOD-****ING-LIKE.

- maintains backward compatibility with 0.13 01 Aug 2003 release. There is no "Warning: Emulator mismatch may cause desync" when playing with this emulator because the netcode, thus far, remains the same (excluding any additional broadcast netcode)
what about noobs who join a game with mupen emulator while the host is using pj64k...???
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
It will still warn the user if the mismatch is anything except what I have created. For example, if a user with this client tries to join a Mupen64k game, the warning still exists. It only provides an exemption to the mismatch between this client and 0.13, because they are essentially the same (this client was continued from that one).

I wanted to maintain backwards compatibility so I've changed the name of the emulator displayed in Kaillera but prevented it from warning you if it interacts with 0.13. But now that I've thought about it, 0.13 users will still be warned if the emulator name changes, so I'm not sure how to handle it yet.

If anything, 0.13 users may still detect a 0.13 client if we keep the name the same to prevent the emulator mismatch warning; however, a localized variable could be made so that 0.14 users could still differentiate between 0.14 and 0.13.

An alternative would be to change the name and simply update the 0.13 source on it's google code repository so that it does not warn users of this specific mismatch. It seems irrelevant to do that when we're already updating 0.13 and shouldn't have to change 0.13 just because of things we've added in 0.14. -.-
 

donkey.KONG

Smash Rookie
Joined
May 5, 2012
Messages
19
There's also a bug that if you click anywhere in the chat, the text cursor prevents the chat from auto-scrolling properly. This means if the chat has focus, the furthest it will scroll is the line of the text cursor. Without some buggy hacking, the use of RichEdit is not logical. Owna acknowledges this in his code.

A workaround would be to disable focus on the chat window, but that would prevent copy/pasting. The cursor is required to make selections. So maybe we should make the focus more apparent, i.e. gray out the chat textbox when it's not in focus, and disable the autoscroll while the main chat has focus. Then when the chat textbox regains focus, toggle autoscroll to true.

EDIT:
Code:
	// Owna: The following contains my horribly rigged way of handing the scrolling,
	// which is still bugged in specific situations.  I hope someone fixes this!
Beneath this friendly acknowledgement is more of the sketchy API hacking I've been referring to. Instead of directly editing the control's properties, for some reason external API messages are sent.... internally. I really don't like the way this is done and obviously he doesn't either.

I've rewrote Owna's rigged re_append function which works with the richtext object directly rather than external API messages. I considered using QTextEdit, an alternative to RichEdit. It's lightweight, cross-compatible and has other features. But the library dependencies were over 1GB so I'm looking at an alternative GUI controlset, wxWidgets. The new function will append the data to this object, and I'm working on a bugless autoscroll now.

UPDATE:
- I am now considering updating the source, but not sure how many servers will adopt the change. Nevertheless, an is_spoofing user variable would be useful. There is a commented portion of the client that used to append "IS_SPOOFING" to the username if they were spoofing, a client-side indication. This could still be used somehow, but perhaps with an asterisk symbol appended instead. I think server owners should have the right to know who is spoofing and who is not, and if they desire, ban spoofing on their server. Let me know your thoughts on this.
 

firo

Smash Ace
Joined
Jul 27, 2008
Messages
600
Location
Champaign, Illinois
Hey donkey.KONG,

Thank you so much for your work on the client! You definitely have a lot more experience than anyone who has worked on this so far. The source is messy, and I don't have enough windows progamming experience to really understand what is going on everywhere, so I do appreciate you going into some of the technical details behind the work. I wouldn't blame Owna for the messy source, he only made some smallish changes and took time out of his busy schedule to work on it.
A few notes:

The ping spoofing issue has existed for quite a while. For a time after Owna first implemented this, I told him to add a message, "<-- This user is ping spoofing!" to the chat whenever the user joined a game. We talked back and forth a lot about it but we eventually decided to get rid of it, as it really didn't give any useful information as far as server owners go. We already have their ip, which is enough as far as information goes in my opinion. I originally had fears of people spoofing and then lagging games without the host knowing, but this ended up not happening. People were also doing funky things with their names to confuse people. I wouldn't bother with it, unless you can think of a specific reason to do it, especially since someone can just edit the source and take it out (which actually happened just before Owna removed it).

I wouldn't worry about servers not adopting the new client. Users pretty quickly moved over to owna's client, and I put a warning at the top of the server along with a change log and link with each new version. With the really nice changes you have here people will want to upgrade, and will. In a similar manner, I don't think warning the user of a mismatch is that bad of a consequence - we want people to upgrade, and adding an inconvenience will only help to speed the process by letting people know who is lagging behind.
 

Sempiternity

Smash Lord
Joined
Sep 11, 2010
Messages
1,695
Location
Connecticut
This is way cool. I still don't know where you came from or why you decided to tackle ****ty N64 online play, but I'm sure I speak for everybody when I say I'm glad as hell you did! People here would be willing to test any new client you put out and we'll do everything we can to help you.

It's about goddamn time we got somebody experienced to work on this.
 

malva00

Smash Master
Joined
Mar 1, 2002
Messages
3,864
Location
54th and 5th
can you make it so we can rearrange the order of the servers in the "local list" (move them up and down)?
 
Top Bottom