Compo4All starts with a simple yet brilliant idea. Many people like retro gaming, and enjoy playing those ancient games that we use to love at the arcades. However, the said arcades have mostly disappeared, or the said games been replaced by newer ones. Therefore it has become almost impossible to go and play with friends and compete on such old games for the higher score. Compo4All has an interesting take on that issue…
Compo4All lets players run these old/ancient/retro games via M.A.M.E. (note that the M.A.M.E. emulators are not modified at all, so that it is not impacted by M.A.M.E. licensing), and then uploads the high scores to a remote server where all players’ achievements are recorded.
The client running on the Pandora gets the data back from the servers, in real-time, so that you can see what are the latest high scores to beat, and the competition can take place again. Note that there are still numerous limitations in this early release, such as the fact that you need to be online at the time of playing to get a new high score recorded. The current version sends the data back to the remote server only once and if your connection is not working, your score is not saved remotely. In subsequent versions this will be apparently fixed by adding a retry delay for the transmission of the scores should it fail at first.
While the client on Pandora has a very simple interface to display the high scores, there is also a web version with a neat retro look to check out the latest achievements.
I asked Skeezix (the author of Compo4All) to share more info about this project in order to feature his work here. Here is how this all started:
I used to run a barbecue at my place, let people play some games (my basement is full of old arcade machines) and hang out, but that was just for hanging out… not for competition. Someone else said it, and nailed it… you can play 30 seconds of Donkey Kong and its cool and all, but you die and feel put off and just blop over to another game. The emus make it too easy to switch sometimes, so you don’t commit. You’re sometimes just not engaged, unless you give yourself some goal. In my machines, I often set them to charge for play (not on free play) and keep a small cup of quarters on top of the cabinet. I can surely just reach down and hit the coiner or pull the quarters out of the machines bank but its psychological – if you say you’ve only got 6 or 8 quarters on tap, you’re going to skill up and use them better each time. Likewise, if you’ve got a scoreboard challenge, it gives it meaning… you don’t just play 30s and move on. You will complete the game, and fire up another, and another! So you can play a bit of Donkey Kong and move on. But if you have just been topped by someone, it becomes suddenly addictive, just like it was in Galaxian 1978 again… it is funny, but just seeing the scoreboard update makes it more fun. Sometimes you just don’t want to compete, or its not your thing.
But sometimes _ITS ON!_
Tying the desire for some competition and drive together with being a coder and arcade nerd.. there you go. Idea has been bouncing around in my ‘reserve projects’ jar for a year or two.
And hey, I have got a lot of code lying around that can be leveraged for this. I have proven to myself years ago I can do small projects, and medium projects (stuff like BattleJewels).. so these years I tend to bite off really large projects. Fine and dandy, but lately I just don’t have the time to pull them off in a useful time-frame This big game server I have been building on and off for a year or more… shaving off a piece of it could turn it into the score server overnight. So it did.
Since Compo4All does not use custom Roms, the program needs to find a way to extract the score data and the initials generated by each game for the high scores records. And it is not always straightforward, as you can imagine. There is quite some work involved every time:
Likewise, due to the nature of the beast — interpreting scoring data in the virtual machines RAM — I have to add each game, game by game, decoding the RAM as I go. Its not like I’m writing emus for each game here (I’ve _done_ that, that is way more effort), but nonetheless, some games encode their scoring data in really bizarre ways.. and some are simple. I also have to grab the vanilla data, play through enough levels to register some high scores and enter initials (to figure out how they encode the character sets) .. now that I’ve gotten good at it, I can occasionally add a game in 20 minutes (Pulstar and Bubble Bobble) or an hour if unlucky (some games have decoy copies of the data that looks right but is funky and then a hidden copy..).
Even when you find where the data is, understanding how it is encoded is another piece of the jigsaw puzzle. You need to understand the data structure behind it, as well as the encoding used.
There are Single-score games (Galaxian, Ms Pacman) versus Multi-score tables (most games). Do they store things in row-blocks (score, initials, stage for row1, then repeat for row2) or do they stripe-it (score1, score2, score3, initials1, initials2, initials3), or something else? if a game stores a single score, its pretty easy to figure out… there is probably just a block of 20 bytes somewhere… a score, and some initials, and maybe a stage-reached, or how long it took to get to. For high score tables, it can be messier, but usually they’re fairly easy to spot. Especially with some quick fiddling… the MAME guys have done an awesome job of saying ‘where the high score is, roughly’, but have not decoded it. But you run a game, wait for the attract mode to cycle and show the table, and you pause and write it down. 50,000, 40,000, 30,000… okay, look for those encoded one way or another in a block of RAM. When you find it, you’re gold.
Decoding… is it BCD (binary coded decimal), or pure byte value? Big or little endian? Or display-wise (where 30,000 is 5 bytes in a row, for each character) ? Are letters using ASCII like many modern games, or using a game-specific sequence? i.e. byte value 0-9 for numbers and 10-36 for English uppercase? What about punctuation? Or some other scheme?
Once I’ve got the vanilla untouched table, and a couple modified (scored on) tables, I can generally figure it out; the server needs to have these untouched scores on its side, so it can compare to incoming scores to figure out whats changed, and to send down the untouched (or generated or live streaming) scoreboards.
Currently when you launch Compo4All, it shows the list of games supported and once you select one of them, it loads the latest high scores from the remote server. This way you know what are the top 5 scores to beat in order to have your name in there. But once you start the say Rom through the interface, the Rom starts in virgin state: none of the loaded high scores are displayed in the arcade game itself.
For upcoming versions, Skeezix is trying to encode the high score data directly in the Roms, live from the remote server:
I have to figure how to encode back. Currently, to simplify, I’m just decoding… but I have toyed with encoding and sending back down to the emu live, so I need to be able to go both ways. I am currently doing it game by game and not through a generalized parsing engine, since the rules of the game are not clearly defined yet. If it turns out there’s only say, 8 ways of encoding things in common use, that changes everything. But as is, I keep finding them doing it in different ways, so I need to keep screwing with it…
Compo4All is far from being “done” as Skeezix is constantly working on it to add new features and all, but it did not that that long to reach a first working build.
[it took only] a few weeks really. Its not ‘done’, as there’s so much potential that can be done. (And I’m rather assuming people aren’t going to try to cheat; there’s a thousand avenues for it and no real sensible way to block it. With Punkbuster and Blizzards live anti-cheating tools, etc, you can easily spend 10-times your effort on avoiding cheating than just doing what you are here to do. So its better (always in life imho) to just trust people. i.e. don’t under-clock your machine down so everything runs slow, don’t fiddle with dip-switches or hacked Roms that run slow/fast/easy, and don’t swap in alternate builds of the emulators, and don’t attack my servers and screw with the protocols, etc… I really don’t want to get into signing builds, forcing recording of your whole play, licensing Roms and encrypting them into the binary…
We’re working on a fancier new website that’s retro looking and full of game artwork; much more attractive than my dumb ASCII displays now, and I hope it will really draw people in. Not just a text high score, but click on a Galaxian marquee and boom, see a nice scrolling high score table surrounded by artwork. Sweet stuff.
Here’s a preview of that new website, by the way. Not official, yet…
I was curious to learn what tools Skeezix used to make this project a reality. Not that I expected anything in particular. I was rather interested to learn about the choices he made.
It’s mostly homegrown stuff, no real dependencies. The current client is Python+GTK so its trivial to port form Desktop to Pandora, but it does give a boring desktop-looking UI. I’m working on a pure GLES interface with some artwork sliding around and so on, so it will be more attractive and also run on Caanoo, Wiz, etc easier (author note: other Open-Source Handhelds, if you are not familiar with them). The protocols and Server are C and Python driven. Initially I was being pretty careful and did it all in C and embedded into emus in question so it was harder to cheat and did real-time magic (showing the scores updating live in the emu, working in multiple emus, etc) but then went for a more pragmatic “lets get it out the door and see if people like it before we get fancy” and started getting sloppy… the current incarnation is less fancy, but works very well, so it is a great “functional prototype.”
Because of the way the application is designed, everything is very portable. One can probably expect some multi-platforms support later on… such as the Raspberry Pi, since it’s technically very close to a Pandora.
The in-emu code would certainly port pretty easily, but the way I do it now with pre- and post- emu run sync, would also work on most platforms. The Python-GTK menu probably wouldn’t port very nicely to say, Caanoo or Wiz, but a new frontend I’m working on should make it much more portable and pretty. We should do a port to Raspberry-pi, since it’d be trivial; if I get another front-end done (GLES like I’m working on, say), then I could roll a mini-distro. I’ll check in with the Picade guys… there may be a good tie in there. If we can get Compo4All roped into the whole r-pi uptake, that could rock.
Otherwise, it runs fine on Windows, OSX, Linux as well (naturally!), and could do other platforms pretty easily. But if we want to grow it up big like that, scale it to many kinds of emus, many kinds of platforms, we just need to think about how to integrate all that scoring. For example, let people run their own servers, so that clans can have a server; also, one server per platform or group, or have one server that does them all with users being able to filter by platform (See all Pandora scores, or see All Platforms scores regardless of origin), etc etc. If people like the idea, there are many options to explore… but so far, I’ve been fairly specific to keep the design simple to see if the idea has legs. Hence, Pandora and a few games. We can add iOS and Android devices just as easily as desktops.
All pieces of the code will be made open source down the road, but it is still a little early at this stage, while Skeezix is committed to do so.
The ‘frameworks’ (client, server, protocol, decoders and encoders etc) will probably be open source down the road a bit, but in the name of obfuscation, I have not published it _yet_. I will certainly describe how it works with anyone who wants to know… its not all that clever, and security through obscurity is bad security, but keeping it mysterious also lets me do drastic changes without worry over confusing anyone or screwing up anyone trying to tie into it. At the same time, if people like this idea enough to keep it rolling, then it would be smart to open it up and make it more general so we can rope in more games (indie, ports, other emus..) and add features (achievements? chat rooms and parties? lobby.. all that watering down stuff).
For now the community of players using Compo4All may be limited, as the only available version is currently on the Open Pandora hand-held. But as more people toy with it, and more platforms become supported, leading to more users, one may wonder… will all of this scale well ?
Its still an early system, but yeah, its designed to scale somewhat. Right now if your wifi goes down it cannot submit your score, so I will make it have a retry/wait option so you can hold it up if your train goes through a tunnel. Likewise, I can set up a round-robin port easily enough so that if my server goes down you can hit another. The servers are stateless, so even if they crash and restart 1sec later, nothing is really lost. Extra servers (software) on other servers (hardware), no big deal. Its also very low load. Think of it like a webserver, except that it’s not serving out images or 100,000 hits per second. It gets queries before/after a game, not every second during gameplay (right now, though I have done that, too. Even that was not a lot of load). I am an infrastructure developer by trade, so having scalable frameworks is a second nature. I also only tend to write cross platform code .. why limit yourself?
Thanks to Skeezix for his time and Compo4All ! If you are interested to know more about this, I recommend following the related topic on the Open Pandora forums. And here’s the Repo link if you own a Pandora.