Category: Game Development

I’m adding a generalized clan system to the lobby in RakNet. Because it’s generalized, anybody can use it for any game.

It’s commercially viable in quality:

Create, list, update, rename, and destroy clans.
Clan names have to be unique, and can’t be in the disallowed words table.
Users can be in more than one clan at a time.
Each clan has a bulletin board
You can invite members to the clan, and members can ask to be invited. You can also have open clans where anyone can join.
DB maintains integrity via constraints and cascading deletes.

This is actually the second time I’ve done something like it. The first time was with Galactic Melee, but it was too specific to the game.

As with the ranking, email, and cd key systems, the systems are reasonably independent so you can use one without having to use all the features.

Probably 3 days total for the implementation. You wouldn’t think that just creating a clan, which is just a list of players, would take a long time. But it’s much harder because you have to enforce conditions in the database and process the requests asynchronously. Once the database is working you have to do it all over again, now serializing the request to the server, having the server enforce code-level restrictions (such as only 1 user per clan), validating input, having the server serialize the reply back, and serializing and calling the proper callbacks on the client. Plus the database schema has to be reasonably efficient. Plus the system is generalized, and it’s always harder to write general than specific.

It’ll be in the next version.

I let Galactic Melee die for a while. I lost heart to put any time into it and I didn’t want to pay the server fees anymore. However, last week I met with someone who expressed interest in possibly buying it, and who may have the resources to do so. I’m putting it back online on my second computer, and now that I have a static IP I can actually host it myself. This has been bothering me for a long time. In fact some nights I wake up in the middle of the night filled with anxiety over all the time and money I lost. So I’m happy it’s going to be up again.

The main problem is I don’t know how the provisioning system works with Aria, and there was some weird thing being done to connect to the database. OBDC or something like that. So I’m going to have to figure that out over again. I wish I had used PostgreSQL originally for the database stuff. The server will probably be back up tomorrow in any event. The webpage is up as soon as the DNS change propagates through the internet.

There’s really two ways to go with the game. One is to hang onto it as I have been doing in case a buyer or investor comes around. There’s a very small chance I could make back my money that way. However, as I’ve heard finding an investor is a full-time job, and not something I’m interested in actively doing. The other is to open-source it and import it as a game demo into RakNet. I’ve wanted a good game demo in RakNet for a long time and nothing could possibly beat this. Doing that doesn’t make me money directly, but it does increase interest in RakNet. It’s also satisfying because it means I invested in a year and almost $100K into RakNet rather than investing a year into nothing.

Probably the best thing about RentACoder is that you don’t have to pay if the work doesn’t get done. I hired an Indian programmer to fix my TCP based EmailSender class to work with Gmail. I allowed him to use OpenSSL if necessary. Almost two weeks later he tells me OpenSSL can’t send email, and if he can use a 3rd party library to send email. That makes about as much sense as a graphics programmer being asked to fix shader code, and two weeks later asking why C++ doesn’t work in shader code, and if he can use a different graphics engine.

I get this feeling of annoyed despair that I’m wasting time and money, the same feeling I often felt with bad programmers while doing Galactic Melee. I’m really happy that I don’t have to pay if the work isn’t done. This was a major source of lost money on Galactic Melee. Since the project deadline is only 3 days away anyway I’ll just let it elapse and get my money back through arbitration.

My previous experience was good though. I think it’s pretty easy to tell within the first two days if a contract is going to make progress or not. In the future I’ll just go to arbitration right away, rather than giving the contractor the benefit of the doubt and waiting out the whole contract for nothing.

I have a feeling I’m going to end up posting here again how I did the fix myself in a couple of hours.

Over this week I was talking to Linley Henzell about porting Dungeon Crawl to the Wii. It’s one of those 10 year old undiscovered gems like Subspace. Amazing gameplay, great design, actually more fun than some multi-million dollar RPGs to come out recently. It’s undiscovered because the graphics were originally ASCII based, and are now 2D tile based. If you have work to do, don’t download this game, because you won’t do anything else for the next 3 weeks.

However, I was talking to my prospective partner and we agreed it probably wouldn’t sell well as a 2D turn based game. To be appealing on the Wii, it would have to be 3D real-time with a close-up interface like Fate. The problem is that would require a redesign of major parts of the game, so rather than being a 3 month port, it’s now probably a one year project. It’s really too bad too, because I can see the game being insanely addictive and fun.

On the bright side, I came up with an even better idea for a Wii game. From a business perspective it’s great: trivial development costs, wide demographic appeal, holds its value, and it expands upon an already successful genre. I pitched it to my prospective partner tonight, and if he goes for it I think we can do very well in a very short timeframe.

I worked with a guy in the UK on rent-a-coder “fuchsia” who did a really great job on my AutoRPC system. It now supports 32 and 64 bit OSes, and also the Power PC (though untested). He also came up with a way to automatically support pointers. So this call, with a pointer as parameter f, is now valid:

struct ThreeBytes
{
char bytes[3];
};
struct TwentyBytes
{
char bytes[20];
};

void __cdecl cFunc7(char a, short b, int c, long d, unsigned long long e, ThreeBytes* f, TwentyBytes g, AutoRPC* networkCaller)
{
if (networkCaller==0)
autoRpc.Call(“cFunc7”,a,b,c,d,e,f,g);
printf(“cFunc7: %i, %i, %i, %i, %i, %i, %i, sender=%s\n”, a, b, c, d, (unsigned long) e, f->bytes[0], g.bytes[0], networkCaller==0 ? “N/A” : networkCaller->GetLastSenderAddress().ToString());
}

Imagine a game engine where you can make games without a single line of code. Instead, you design and create through a GUI by dragging, dropping, setting properties for, and linking modules. Each module represents some user-level component of the game world. By user-level I mean something a game designer would refer to, rather than what a programmer would refer to. So you would have a skybox module, a level module, an enemy module, and so on. Each module is tweaked through a property list, such as what the skybox looks like, what model the enemy should use, and so on. One way to think of it is as a game design engine, that happens to also create the game that you are designing.

As the value of the editor comes from the editing tools, and the interpretation of modules, it is not necessary to write libraries such as a graphics engine or a sound engine. Instead, one could take free or low-cost best-in-class libraries and put them together to power your editor. This will massively reduce the programming cost and time, while delivering the same value.

What makes this possible is that most of the programming in a game solves the same fundamental problems. Characters walk around, guns shoot in the direction they are pointed, enemies die and disappear, triggers react on proximity or speech interactions, and so on. So the key to successfully programming this is to define how low level to solve the problem. Too low level and people will complain they are just programming in a GUI (See Mental Mill). Too high level and people will complain you can only make one type of game. The approach I would take is the same as I do with RakNet: Be high level and just do a good enough job of self-managing that people don’t care about the low level. Focus on one type of game (shooter, RTS), do a good job with that genre, then add modules for other genres as time goes on.

I think this could be programmed by a team of two people over one year to reach beta. Another year for release where you could actually make interesting games. And another year for the product to really be polished, where people won’t complain too much in the forums.

It could be monetized by running the game in a web-browser only, only allowing registered users to make games, and showing ads for unregistered users.

If I ever hit it big with RakNet this would probably be the next project I do.

I removed a couple of conditions, inlined and unrolled a function, and replaced the generic memory pool with a stack of pointers.

RakString vs. std::string
Creation: RakString roughly fixed cost. std::string improves over time. Depending on when the test occurs, RakString ranges from 3X faster to roughly equal.
Remove at head (shifting a large list): RakString is consistently 4.25X faster
Deletion: Both are fast enough to not register one millisecond.

5000 iterations
Reference is new / delete and strcpy with a straight char*

Insertion 1 Ref=5 Rak=13, Std=48
RemoveHead Ref=18 Rak=150, Std=674
Insertion 2 Ref=5 Rak=6, Std=13
RemoveTail Ref=0 Rak=0, Std=0
Insertion 1 Ref=6 Rak=7, Std=5
RemoveHead Ref=30 Rak=153, Std=673
Insertion 2 Ref=8 Rak=6, Std=5
RemoveTail Ref=0 Rak=0, Std=0
Insertion 1 Ref=6 Rak=7, Std=8
RemoveHead Ref=62 Rak=151, Std=666
Insertion 2 Ref=7 Rak=7, Std=8
RemoveTail Ref=0 Rak=0, Std=0
Insertion 1 Ref=6 Rak=7, Std=13
RemoveHead Ref=61 Rak=156, Std=669
Insertion 2 Ref=11 Rak=7, Std=12
RemoveTail Ref=0 Rak=0, Std=0

10000 iterations

Insertion 1 Ref=13 Rak=26, Std=141
RemoveHead Ref=58 Rak=1167, Std=3425
Insertion 2 Ref=12 Rak=13, Std=7
RemoveTail Ref=0 Rak=0, Std=0
Insertion 1 Ref=13 Rak=13, Std=30
RemoveHead Ref=108 Rak=1122, Std=3334
Insertion 2 Ref=12 Rak=14, Std=29
RemoveTail Ref=0 Rak=0, Std=0
Insertion 1 Ref=10 Rak=14, Std=22
RemoveHead Ref=878 Rak=1116, Std=3363
Insertion 2 Ref=12 Rak=13, Std=23
RemoveTail Ref=1 Rak=0, Std=0
Insertion 1 Ref=13 Rak=13, Std=14
RemoveHead Ref=886 Rak=1128, Std=3349
Insertion 2 Ref=14 Rak=14, Std=14
RemoveTail Ref=0 Rak=0, Std=0

Summary

Reference is much faster in shifting large lists because it is just copying a single variable internally. Neither std::string nor RakString know that contextually they don’t need to do intermediate copies, so there is more overhead and thus they are slower. However, RakString uses a reference counted pointer, so the shifting is limited to a few variables vs. std::string which actually does a full copy. RakString is about 20% slower than reference, but std::string is over 350% slower.

Reference is much faster the first iteration, because RakString has to create the memory pools. Presumably std::string does this as well. After the first iteration, RakString varies from equally fast to 50% slower. std::string varies from equally fast to 250% slower. However, oddly enough in one test std::string was faster than both – this was probably a fluke due to thread context switching.

While it doesn’t show in these numbers, in some later tests reference got slower the more tests I ran, until it was actually far slower than both RakString and std::string. This was probably due to memory fragmentation.

If you are going to do a lot of shifting around in lists, just use char* that you allocated yourself. Otherwise, use RakString. RakString is faster than std::string, and doesn’t cause linker errors and export warnings when linking as a DLL, a big plus. The constructor and Set function is varidic. It complies with camelCase conventions. And as I use it more I can add inherent functionality that std::string is missing, such as splitpath.

Here’s the code. It’s stand-alone except for DS_List, and SimpleMutex. For the export and assertions you can just replace those with your own versions. SimpleMutex could be removed if this was only used in a single thread at a single time.

Header
Source

I wrote my own string class, that is 5X faster than std::string for creation and 2X faster for copying. However, it is slower if I create a string that was previously deleted. I’m not sure what they are doing, but in that case creation is 2X faster than mine.

It would be really cool if I could get it faster in all cases.

Here’s the code:
RakString.cpp
RakString.h

Times are in milliseconds
Ref is just inserting a number, to show the speed of the list itself.
Rak is my string
Std is the std::string

Insertion 1 Ref=0 Rak=25, Std=102
RemoveHead Ref=106 Rak=4964, Std=8577
Insertion 2 Ref=1 Rak=22, Std=12
RemoveTail Ref=0 Rak=0, Std=0

Insertion 1 Ref=1 Rak=24, Std=102
RemoveHead Ref=107 Rak=4868, Std=8585
Insertion 2 Ref=0 Rak=23, Std=13
RemoveTail Ref=0 Rak=0, Std=0

Insertion 1 Ref=1 Rak=25, Std=104
RemoveHead Ref=107 Rak=4879, Std=8553
Insertion 2 Ref=0 Rak=22, Std=13
RemoveTail Ref=0 Rak=0, Std=0

Test code

static const int repeatCount=15000;
DataStructures::List rakStringList;
DataStructures::List stdStringList;
DataStructures::List referenceStringList;
unsigned i;
RakNetTime beforeReferenceList, beforeRakString, beforeStdString, afterStdString;

beforeReferenceList=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	referenceStringList.Insert(0);
beforeRakString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	rakStringList.Insert("Aalsdkj alsdjf laksdjf ;lasdfj ;lasjfd");
beforeStdString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	stdStringList.Insert("Aalsdkj alsdjf laksdjf ;lasdfj ;lasjfd");
afterStdString=RakNet::GetTime();
printf("Insertion 1 Ref=%i Rak=%i, Std=%i\n", 
beforeRakString-beforeReferenceList, 
beforeStdString-beforeRakString,
 afterStdString-beforeStdString);

beforeReferenceList=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	referenceStringList.RemoveAtIndex(0);
beforeRakString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	rakStringList.RemoveAtIndex(0);
beforeStdString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	stdStringList.RemoveAtIndex(0);
afterStdString=RakNet::GetTime();
printf("RemoveHead Ref=%i Rak=%i, Std=%i\n",
 beforeRakString-beforeReferenceList,
 beforeStdString-beforeRakString,
 afterStdString-beforeStdString);

beforeReferenceList=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	referenceStringList.Insert(0);
beforeRakString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	rakStringList.Insert("Aalsdkj alsdjf laksdjf ;lasdfj ;lasjfd");
beforeStdString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	stdStringList.Insert("Aalsdkj alsdjf laksdjf ;lasdfj ;lasjfd");
afterStdString=RakNet::GetTime();
printf("Insertion 2 Ref=%i Rak=%i, Std=%i\n",
 beforeRakString-beforeReferenceList,
 beforeStdString-beforeRakString,
 afterStdString-beforeStdString);

beforeReferenceList=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	referenceStringList.RemoveAtIndex(referenceStringList.Size()-1);
beforeRakString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	rakStringList.RemoveAtIndex(rakStringList.Size()-1);
beforeStdString=RakNet::GetTime();
for (i=0; i < repeatCount; i++)
	stdStringList.RemoveAtIndex(stdStringList.Size()-1);
afterStdString=RakNet::GetTime();
printf("RemoveTail Ref=%i Rak=%i, Std=%i\n",
 beforeRakString-beforeReferenceList, 
beforeStdString-beforeRakString, 
afterStdString-beforeStdString);

I need to add a moddable GUI system to RakNet so that users using the lobby don’t have to implement the 40 or so client features themselves. The problem is that this is such a huge task when you think about it. I would need a cross-platform graphics system that can work at the same time as the user’s game. Then there needs to be a GUI system on top of that, and not just windows but all the little effects too. Effects usually include sound, so that’s another issue (at least I can use FMOD there). Of course, to interact with the GUI system it also needs an input system, and the input system has to be either stand-alone or work with the end-users own input system.

There’s some free GUI systems out there (wxWidgets among others) but none of them support consoles as far as I know.

There’s Scaleform but I don’t want to learn flash and it’s not free. Still, that’s probably the best compromise I’m going to find. It’s certainly easier than trying to roll my own complete solution.

Traditional RPC requires several steps to setup and receive the call. First, you have to serialize the input parameters. Second, you send the serialized data, along with an identifer for what function to call. Third, you deserialize the data, and put it in some known format. Lastly, you call the function, with that format as the parameter.

For example, to call this function on a remote system:

void PrintNum(int i)
{
printf(“%i\n”, i);
}

I would need 3 functions:

Old way: Serialize data

void SerializeNum(int i, BitStream *bs)
{
bs->Write(i);
}

Old way: Call with serialized data

void SendRPC(BitStream *serializedData)
{
CALL(“DeserializeAndCallPrintNum”, serializedData, …);
}

Old way: Deserialize and call real function

void DeserializeAndCallPrintNum(BitStream *bs)
{
int i;
bs->Read(i);
PrintNum(i);
}

I came up with a way to get rid of Serialize and DeserializeAndCallPrintNum(). To the sender, the RPC is essentially varidic, in that the receiver can have an unknown and arbitrary number of parameters.

New way: Remote function call, passing parameter list directly

CALL(“PrintNum”, 5);

It’s less efficient, but much easier to use. The key gain from this is that no hooks are necessary. When a function is called, it can trigger itself on remote systems. Furthermore, I can inject extra parameters into the parameter list so that I know if the function was called from the network or not.

New way: Self-triggering networked function

PrintNum(10, false);

void PrintNum(int i, bool calledFromNetwork)
{
if (calledFromNetwork==false)
CALL(“PrintNum”, i); // Over the network, last parameter is injected by network code with true
printf(“%i\n”, i);
}

The process to do this is:

1. Use templates to serialize the input parameters, such that I can tell the number and size of each parameter.
2. Send the parameters across the network, along with the function name
3. Lookup the previously stored function pointer. Deserialize the parameters, expanding parameters under the size of a register to the size of a register (4 bytes).
4. Push parameters on the stack
5. Call the function

Here’s proof-of-concept code:
Autoserialized, hook free, RPC proof of concept

Drawbacks:

1. Only works with parameters that are actually copied correctly into the stack (shallow copy, no arrays)
2. No pointers (old system didn’t have this either)
3. Lose flexibility to do algorithmic encoding, such as if (hasPosition) Write(position)

I might be able to fix the first point. If I can somehow detect what type of template parameter was passed maybe I can query if the object fulfills some kind of interface. The biggest drawback to the first point is that you can’t pass strings or arrays of non-fixed length. So you either have to be very wasteful, or fallback to the old system.