Game Development

Replica Manager 2 Done + Video

I spent an entire day on Sunday thinking about the architecture on how to write a generic system to create, destroy, serialize, and control scope over networked objects in C++. It’s a very hard problem to solve because you know nothing about the game objects themselves, or what complexities the end-users might introduce with their specific architecture.

In the end I decided on 3 classes. One class, ReplicaManager2, is the base plugin for the system. All it does is essentially serialization, deserialization, transmission, and bookkeeping. Stuff you wouldn’t normally care about. I have the user pass a class factory that generates connection instances, one per connection. This controls high-level and non-object specific functionality such as how to create object instances. Lastly, there is the base class Replica2, which game objects should derive from.

The main reason I did it this way, rather than putting everything in the ReplicaManager2 plugin (as I did in its predecessor) was so that the functionality can be easily overridden. This should solve a problem I had with the old system, where I ended up passing a ton of data to the callbacks in order to force it to get it to do what I wanted. By assuming users will override functions, this also gives me the ability to make assumptions and thus much more elaborate functions.

Improvements it has over the old system are:

1. Relay across systems actually works correctly. So you could have a server cluster, with n clients on each server, and everything would just get sent out properly. It automatically deals with cyclical connection graphs up to a depth of 1 away from the source node.

2. Defaults to polling for changes. Each tick, if an object is now constructable, or in scope, and it was not in the last tick (or vice-versa) the object is created or destroyed appropriately. The object is also serialized every n milliseconds, and if it serializes differently under the same context, the newly serialized value is transmitted. This is easier to use than event-based updates (which are also supported) so the user has to write less code.

3. Downloads construct all objects first, then deserializes them, and in the order they were originally created. This is likely what a user would do were they to hand-write the download system, and maximizes the chance that pointers and other dependencies will be valid on deserialization. This also allows you to have objects that cross-reference each other when deserializing, each both have already been created anyway.

4. It’s easier to modify events and more information is given about what triggered those events. For example, it’s now trivial to send using different reliability types based on the context of what is being sent, something that was hard to do before.

5. Changes are easier to initiate on the client, especially construction, which before would have taken custom game code.

Here’s a video that goes over the system. I start out too fast because I was tired of starting over when I made mistakes, but later on it gets better.

Game Development

Replica Manager 2

I’m working on the second iteration of the Replica Manager class. For those that don’t know it, this class is supposed to handle the details of sending and synchronizing objects across the network as new systems connect.

My first approach was data based. All the code was in the Replica Manager itself, and it would query an interface to get the data it needed to perform operations, such as serializing objects. In most cases, this worked OK. However, the problem with games is there is always complexity to work around, especially with existing systems, and if cases came up that I didn’t anticipate it was hard to go around the system. One example in particular was the initial object download. The first system would gather up all objects, serialize, and send them to you. However, this didn’t work if objects already existed globally (such as created when the map loaded). It also didn’t work if you cared about what order objects were downloaded in, which you would if they had dependencies on each other (such as cross referencing pointers).

My new approach is more code based, with data queries exposed only for very common or unlikely to change operations. So now the Replica Manager class itself mostly just encodes and decodes queries, and holds the data structures, with the actual functionality in the Replica class, the class you derive your own game objects from. This gave me a degree of flexibility I didn’t have before, because now I can implement complex operations since I know the user can override the code if necessary. One example is having a client locally create an object which is verified by the server, and deleted or synchronized on the client accordingly. Another example is automatically scoping objects with a simple callback, where before you had to do this by writing your own system.

The test case I’m solving is:

// Preexists on all systems, has objectID, still want to register it so I can call serialize automatically
class World {
int totalKills;

class Player : public Participant {
Soldier *soldier;
char name[256];
int teamNumber;
char killMessage[256];
int gameInstanceId; // Players may be spread out between multiple game instances

class Soldier {
Player *owningPlayer, *lastDamagingPlayer;
Gun gun;
float health; // Sent reliable ordered
float positionX, positionY; // Sent unreliable sequenced
bool isCloaked; // If cloaked, do not send position to non-teammates

class Gun {
Soldier *owningSoldier;
Player *owningPlayer;
int ammo;

class Bullet {
Gun *firingGun;
Player *owningPlayer;
RakNetTime timeFired;

So you have:

1. Classes that have pointers to classes that have pointers to themselves in return
2. The world object is assumed to preexist, so should not be dynamically downloaded, yet still needs to be updated
3. Cloaking, where an object either won’t be serialized at all, or only partly serialized (if a teammate)
4. Pointers which may be NULL
5. Object composition with serialized objects (so their parents should be dynamically downloaded, but not the objects themselves).
6. Serializing the object using a different send type, depending on what part is serialized.

These could all be solved with the old system too, but it was much harder and you had to really understand the code to know what to disable. If I do my job this test case should be solvable trivially.


How to tip like a Gentleman?

I was reading How to tip like a gentleman. It seems like most people support what he wrote, but personally I think tipping is way out of hand.

One incident I remember was at Supercuts about two years ago. It’s $20 for a 20 minute haircut, they spent the whole time I was sitting there trying to sell me cosmetics, and did a bad job too. So for the tip I put a slash, and got a nasty look. Why should the lady expect a tip for doing her job at all, much less doing a bad job, and spending the whole time trying to advertise to me?

Another incident in was Las Vegas about 3 months ago in a taxi on the way back to the airport. The guy was nice enough but the fee is near $100 an hour. So I gave him like $40 for a $38 fair, told him to keep the change, and got a nasty look for it. I don’t know what’s supposed to be appropriate for a taxi, but good grief at that rate (if they keep half the money) they are pulling in probably 75K a year to drive a car, with no education and minimal training. I know it’s a risky job but it’s not like they are making minimum wage. That’s how it is in Europe, leave them the change if you want to, but it’s not expected. Now it’s like the opposite, you are expected to give a tip, so not only is a small tip not appreciated, they are mad for getting it.

About 3 months ago I went to semi-fast food restaurant. I paid by credit card, and there was a tip line to be filled in before you even got your food. And they didn’t even serve the food there, you had to get up and get it yourself. And even if they did, I don’t see the sense in tipping before I even get the service, so put a slash there. I got a bad look for that too.

A few dollars for good service is one thing. But I don’t think it should be expected, and certainly not for cases where someone is doing a job that is not interpersonal (driving a taxi, taking your order at a fast food outlet).


Empire Earth III review

Empire Earth III, developed by Mad Doc Software, is an RTS where your civilization upgrades through the ages from about 1,000 BC to far in the future. This sounds like a fun design, like an RTS version of Civilization that goes beyond the modern era. The game is split into a world component (like XCOM) and an RTS component. In the world component, you assign each territory to produce military, commerce, research, or imperial points. You also move military units, and buy militia to defend provinces. There are several other AI controlled sides doing the same thing.

When conflict arises against either the native tribes defending each province, or an AI player, the game switches to the RTS map. Your starting units’ technology and numbers are based on your play in the world component. Base build-up is fairly standard, with the exception that you don’t really take specialized paths or make major decisions in base building. Most of the meat here lies in troop upgrades and deployment, with special abilities such as ranged units entrenching or using calvary vs. infantry.

The game was semi-fun for a day or so to see the different types of units you could get through research. However, I think the AI is too weak, and ultimately caused me to lose interest in the game. The main positive thing I can say about it is that your units will help each other out, so if someone is attacking you offscreen your units will move in to help out. On the negative side, units lack sufficient autonomous behavior, pathfinding is sometimes a problem, and most significantly about half the maps do not provide significant gameplay. Of those maps, half the time all opponents are neutral or cannot reach your base. So you build your base for 20 minutes until you have a good income stream, then just bribe all opponents to join your side, without a single bullet fired or sword swung. The other half of the time combat is not a significant challenge, and play progresses the same way. For example, if you build 3 towers (for nearly the same price as 3 regular units, buildable at any time) on most maps this renders you impervious. You could attack your opponents if you want, but it’s usually faster, cheaper, and easier just to save enough resources to bribe them. Towards the end I found myself dreading the RTS part of the game, trying to blow through the tedious parts as fast as possible so I could get back to the world component. On the last map I played before uninstalling, I built my base for 20 minutes, wondering where the attacking army was, and found them floating near the left edge of the world, never having disembarked their troop transports.

A few maps had pre-scripted challenges, and those were the most fun in the game. But they are far enough apart that it eventually became not not worthwhile to play through the unchallenging parts to get to them. It’s too bad too, I can see a lot of work went into the game, to have it all for naught.

I’d wait for this one to hit the bargain bin.


React Operating System

Yesterday, I was reading about the React Operating System. I was chuckling about this for the rest of the day. I really admire what they are doing and I agree with their premise and solution. It is true that Windows has the problems of being bloated and closed source. It is true that Linux has the problems of requiring non-trivial operations through the command line, and having compatibility problems. So their basic solution is correct – an open source rewrite of Windows that is binary compatible with existing programs.

Going by their roadmap they are on version .34 and have been doing it since 1996. That’s a long time, but when you consider it’s a few guys accomplishing in their spare time what a billion dollar company has done… Great job!

Game Development

Sourceforge is a great service

Quick note: I’m really glad a service like Sourceforge exists. After all the trouble recently with source control providers it is really awesome that I can update RakNet without having to think about if the server is up or down, or security for my code, or paying for it. Plus you get a mailing list. It’s hard to set up but I’m glad I did.


Don’t forget to vote

Tomorrow is the big voting day “Super Tuesday.” It will also be the first time I’ve been actively interested in a political campaign. Prior to this election I’ve always looked at the candidates and saw they were all crooks and liars, so why bother? It’s like voting for what kind of gun you want to shoot you in a firing squad.

I was telling my friend Ron Paul is the first candidate I’ve ever seen in my life that is honest. Whatever else you think about him, I think it is undeniable the other guys weasel out of questions they don’t want to answer. Every interview of Ron Paul I’ve ever seen has him saying nothing but the honest truth, and the truth does not depend on who he is talking to. Just last week I was watching an interview by MTV and a girl asked him what he would do about the high price of birth control. His answer was basically (paraphrasing) “The president is the commander in chief of the military, but the Constitution does not grant him/her powers over the economy. I would get goverment out of the industry, which would have the effect of lowering prices across the board.” I can’t imagine any other candidate giving such a candid and straight answer.

The problem with Ron Paul is most people don’t want to hear the truth. They don’t want to hear that the dollar is crashing, the state is bankrupt, and it’s our own government’s policies that blow terrorism back at us. But it’s all true and obvious to any educated and knowledgeable person. 6 years ago when I worked in Europe I was unhappy because the Euro was worth much less than the dollar. Now it’s worth much more. And it’s not like the dollar just magically becomes worth less. It’s because there are more dollars in circulation. When the federal reserve was created 5 cents back then was worth a dollar today. The debt is approaching 10 trillion dollars. I heard on the radio today it is estimated to be at 10.5 trillion by the time Bush is out of office. And that’s being creative; if you use corporate accounting methods Social Security and Medicare every one of us owes over $200,000. And who honestly believes that terrorists in the Middle East attack us because they hate freedom?

It’s not over – Ron Paul may win. Most state’s votes are not in yet. People’s awareness of Ron Paul is growing daily. I’m going to vote and do my part to help him win. But if he doesn’t I think he’s done a lot of good by campaigning. People are more aware now of the problems we are facing as a nation. If most people are not ready for Ron Paul this election, maybe they will be the next. Right now it’s only the smart and educated 15% that see this, but as conditions deteriorate and our freedoms are taken more people will sign on. This is truly a revolution, and like all revolutions it’s not about a single leader, but the people.

Game Development

Blizzard hiring

I heard from 3 unrelated sources today that Blizzard is on a mad hiring spree. I’ve never worked there but from what I’ve heard

A. They pay hourly, 9 hours a day max, with 1 hour of that overtime
B. They pay up to 40% of your yearly pay as a bonus (not sure if this is true, or how true it is, or who it is true for)
C. The work is easy (at least on WoW) because you are tweaking a game that’s already done.

They are only a couple of miles from where I live too. The problem is I don’t see anything on their job board I am well qualified for. I have too much experience for their network programmer position. Tools I could do, but I don’t know C# (though I could learn quickly enough). Also, I’m sort of ambivalent about working on tools, since that is most of what I’ve done in my career.

Looking over their listings makes me think of what kind of job I would like to do. In my opinion here’s how things rank:

Most to least enjoyable

Graphics – You get all the attention from the big wigs
API – You get to design systems
AI – See above
Gameplay – Fun to do a good job and make a good game, but can be high pressure
Audio – Easy to do, low pressure
Physics – Moderately hard, low credit and visibility
Networking – Very hard, doing a good job means people don’t complain.
Maintaining systems – Who likes guessing at bugs caused by other people?

Best pay to worst pay:

Graphics – Highly specialized, high demand
Networking – Highly specialized, critical in games that do MP
AI – Everyone thinks they can do it, but if you have experience you can get a higher price
Physics – Many middleware libraries drives down price
Tools – Slightly specialized
Gameplay – Not specialized
Maintaining systems – Grunt work


Networking – Takes less knowledge than graphics, but much more thought and bugs are harder to fix. Everything is asynchronous.
Graphics – Requires lots of up-to-date knowledge, great math
API – Doing a good job is hard, which is why so many do it badly
Maintaining systems – It’s all about experience at reading other’s code, which is hard
Gameplay – Average
AI – It’s not hard, it’s just a matter of designing what you want before you write it
Physics – Lots of APIs out there, just integration
Audio – Lots of APIs out there, just integration

Probably I would like to do graphics, but my skill isn’t great enough to get a job doing that on its own at the pay I usually get. It would be good to do this in a training capacity again. After that gameplay or AI. There’s too little demand for API authoring, and usually lead programmers take that upon themselves. Speaking of which, I’ve gotten 3 interview offers for lead programmer over the last few months. Galactic Melee really helps on this point. Maybe one of these days soon I’ll be moving up on the industry.

Game Development

PS3 giving me a hard time / API usability engineer?

I am having a very difficult time developing on the PS3.

If the PS3 is not hooked up from the day before, I spend on average between 15 minutes and 2 hours getting it running again. Today I spent an entire day, and it still does not work correctly. I probably spent 20% of my time dealing with hardware or setup issues preventing me from doing my work.

There’s 3 things I think they need to work on

1. The causes of failures are often seemingly unrelated to the results of failures. Either A has to be done before B, or if you do A, B will not work, and one would not normally suspect that A and B are related. I’ve been thinking a lot about the analogy of how a chain is only as strong as its weakest link, and in this analogy the PS3 has a very long chain, with many potential failure points. It is actually similar to the difference between gaming on consoles and the PC. On the consoles you plug in the game and it just works. On the PC there are many reasons why a game would not work. Your video card is too old. You did not reboot after uploading drivers. You have some particular combination of drivers the game does not like. If any of these things go wrong, usually you just get a crash with some obscure message that does not necessarily indicate why things went wrong. The PS3 is like that, with some factors variable or semi-random.

2. Errors due to your failure to take some precondition often do not show up until much later, and the manner in which they show up often do not reflect the precondition. It’s like bad pointers when running in release mode. Or put more simply, you try to start it, and it says “Fubar because of 0x8327283” and it turns out it was because you didn’t hold down the reset button long enough when you turned off the machine the day before (that’s a made up example, but if it happened I would not be surprised).

3. On the API, I think a lot more attention should have been given to documentation and generally making things clear. Especially given the name and resources behind it. Or maybe I’m spoiled given the relative high quality of the documentation for the Windows API and DirectX.

I’ve never heard of such a job, but someone ought to make up the job “API Usability Engineer.” If they did I would apply. I would even help Sony if they asked me. I think I have relatively high standards of documentation and what construes acceptable quality for public APIs. A few things come to mind immediately:

* If B relies on A, and there is no reason not to do A, then B should do A implicitly. I can think of some cases immediately where this ought to be but is not true on the PS3
* If 95% of the users would do A the same way, and if that assumption is made B can be written much more simply, provide both A and B.
* If B will fail because A is not done, B should indicate this immediately, and not crash or cause problems later in apparently unrelated code.
* Documentation should cross reference itself, so that if A is related to B…n then B…n is linked to on the same page.
* Failures should not be silent
* All non-trivial functionality should be documented.
* Documentation should not just repeat the name of a function, except in the most trivial cases.
* Samples should be as simple and modular as possible, with an eye towards clarity over repetition or cleverness. By modular I don’t mean just focusing on that one feature, but writing the feature in such a way that the user could copy/paste it and use it with minimal problems
* Error messages should accurately reflect why something went wrong at the API level, not why something went wrong because of some internal reason. Error messages should always indicate how to fix the problem, not just state what the problem is.
* It’s worth spending a day of your time to safe a few minutes of an end-user’s time, because you only have to spend that day once but each end-user will have to spend that time over and over again.

I could think of more but I’m tired and frustrated right now. RakNet is not perfect, so if one wanted to bash me back they could. But I think it reflects these standards more than your typical library and my users sometimes write to tell me how much they appreciate this (And I appreciate the compliments!)