Category: Game Development

My current Lobby system for RakNet separated out the database calls from the C++ calls as follows:

1. Lobby operation is via a generalized operation, such as changing the status variable of a row. There is a unit test in a separate project for each lobby operation.
2. C++ operation is higher level, and uses the lobby operation in a more specific manner, such as setting your status to be a clan member.
3. Client code serializes the operation, does some local checks, and sends it to the server
4. Server code deserializes the operation, performs as many C++ checks as possible (returning error codes on failure) and performs the lobby operation in a thread
5. When the lobby operation completes, server serializes the result back to the calling client, updates internal state variables (such as creating a room), and sends notifications to other clients if needed.
6. Lastly, the client gets the serialized result, updates its own state variables (such as now in a room), and calls a callback that the user registered.

It’s a lot of work to even type all that, and every operation takes a HUGE time investment. I mean like half an hour of solid nonstop typing just for one operation, and there’s over 50 operations. That doesn’t even account for time spent debugging and documenting.

The problems I’ve been having is that there is too much code, tons of copy/paste, the system is somewhat buggy due to so much code, the same data can be duplicated as many as 3 times (database, server, and client), and it’s very hard and painful to change or write anything new. The system IS quite efficient though – many operations can be performed in C++ without ever accessing the database.

I am thinking of changing this as to use a stored procedure for each specific operation. ALL commands (login, create room, etc) are encapsulated into structures. The same structures are used for both input and output data. The structure is a functor, meaning it can be operated on in a thread. Therefore, all the client has to do is allocate a command, fill in the input details, and send it to the client interface. The client interface does nothing but calls the serialize function, which sends it to the server, which automatically adds it to a database processing thread. The thread will query and get results from a stored procedure on the database, serialize the results, and send it back to the client in the same structure. Lastly, a callback is called of the same name, and generated automatically through macros.

I think this would cut down on the total amount of code by 1/2. The lobby client and lobby server now do little more than call stored procedures. I would no longer have problems with complex database operations getting out of synch. And the system becomes scalable for free, because database operations are scalable to begin with. All I’d really need to handle on the server is network connectivity.

One of the most annoying things with database calls is creating a thread to process the call.

Here’s some psuedocode to illustrate the issue:

void QueryDB() {
CreateThread(SelectResults, callback);
}
int SelectResults(Callback *callback) {
result = Process(“select * from db”);
callback(result);
}
void Callback(Result *result) {
printf(“result is %i”, result);
}

Doing the same thing inline is much easier. It’s one function rather than 3, and you also have context about the call.

void QueryDB() {
result = Process(“select * from db”);
printf(“result is %i”, result);
}

However, it blocks your entire program.

I came up with an “InlineFunctorProcessor” system that has the benefits of both. It allows you to do blocking calls all within the context of one function, but runs the blocking call in a thread while your program continues running.

printf(“Enter file to read: “);
ReadFileFunctor *rff = new ReadFileFunctor;
gets(rff->fn);
// Nonblocking, but processes as easily as if it were
YieldOnFunctor(rff);
if (rff->data)
printf(“Got file data\n%s\n”, rff->data);
else
printf(“Open failed\n”);
delete rff;

The way the system works is through recursion, thus saving prior calls on the stack. The yield function calls the main application update loop – or any other function you want to call while the function is processing. That function is responsible for calling InlineFunctorProcessor::UpdateIFP(), which if it returns true, the function should also return.

Here’s the code for the base class:
Header
Source

Here’s a solution that demonstrates the functionality
Download

Joel on Software has a great article about the proper use of Hungarian notation. Basically, most people these days that use Hungarian notation use it to indicate fundamental type to some degree – i for integer, str for string, p for pointer. Usually these types reflect the machine, such as indicating if a variable is a pointer.

However, the original design had Hungarian notation indicate intent, not type. For example, if you had a calendar display with a month, date, and year, you would have 3 integers prefixed with an m for month, d for date, and y for year. Not i, even though they are all integers.

Joel says it better than I can, but to put it succinctly the first is a waste of time, because you are repeating what the computer already tells you and then having to maintain it as well. The second is quite useful – just as all comments should indicate intent, so should Hungarian notation which is just an abbreviated comment.

In RakNet I will use numBits vs. numBytes. An even better approach is to comment the purpose and scope of variables where declared.

Every game company I’ve worked at except nFusion has used Hungarian notation. Most of the middleware libraries do as well. A notable exception is Irrlicht, which also happens to be one of the friendliest and best designed, though not the most feature-rich.

I’m curious what other game companies and libraries do not use Hungarian notation, vs. those that do.

I don’t know if it’s just me, or if programmers outside of the gaming industry are just this bad in general, but I really ran into a terrible programmer this time.

I hire a guy to port RakNet’s PostgreSQL implementation of the autopatcher to MySQL for $700. I expect this to take about 3 days of part-time work, and the bid I get is pretty in-line with this:

This is quite straightforward, I’d expect to turn this around with two or three days – although for the purposes of the expert guarantee I’d like to allocate a week in case of problems.

About me: I’ve been doing C++ for longer than I care to remember 😉 on Unix and Windows.

11 days later I get the submission from the coder. Problems include:

1. Code doesn’t compile
2. Literally every SQL query had to be rewritten, about half wouldn’t even run, and the other half didn’t work right.
3. Over half a dozen memory leaks
4. No Windows project included
5. Zero attention to detail, for example the #define at the top of the header file was the same as for the PostgreSQL version he copied from
6. Disregard for project requirements, such as not using stl
7. Limited file sizes to 8.4MB via MEDIUMINT (stupidity or malice?)
8. Upload files escaped rather than binary, allocating memory once for the escaped file, and again for the query.

It was not much more than a copy/paste job of my existing PostgreSQL implementation, and a find/replace to replace the PostgreSQL calls with MySQL calls, and an additional hour or so to screw things over.

Over the next two days I report bugs, and he fixes some of them, but I eventually get fed up and take the project to arbitration. The arbitrator has already said he will rule in my favor unless the coder can explain by Monday why this should not happen. That was on Friday and the coder didn’t yet respond.

One of his excuses:

But my understanding is that deadlines are for delivering code, not fixing all the bugs

Arbitrator:

The deadline is for the coder to have the full project uploaded in working order. This should also be bug free as the coder is supposed to test the project before uploading it.

Good grief. I just redid all his work in an hour, then did the port myself over the next day days (part time).

In other news…

Today, I took half a day to port my EmailSender class to support GMail SMTP over SSL. I wrote about this previously, how the coder took two weeks to do nothing and lied about it besides. I can’t believe how I can do in half a day what some Indian programming team can’t do in two weeks. It wasn’t like the existing code base helped me either, since this is all new stuff. In hindsight, I’m offering way too much money for these projects.

What is the secret to hiring competent programmers?

I hired another guy on rent-a-coder, this time to do a php based webpage to store uploaded game servers and send them back to C++.

Here’s the page

The whole job was a 2 day project, 3 at most. I gave the coder two weeks. He did a bit here and there, but I could tell by his communications that 75% of the work was started the day before, and it ended up being two days late.

I had to spend half a day refactoring his code, testing, and reporting bugs. There was definitely a lack of investment, by which I mean doing the absolute minimum unless I complain. The webpage is an example of this, where he used a big block for each server, unsuitable for more than 10 or rows. The use of obviously conflicting reserved column names such as “ip” and “name”, embedded into the code, is another example. Were things strictly wrong? No. Could they have been done better by someone who cared? Absolutely.

This is one of the reasons I liked working with Rui before. He would spend the time to do a good job, at the level of quality I would have done myself, if not more. There’s a lot of things you can’t tell in an interview, and you can’t get no matter how much you pay for, and really caring about doing a good job is one of them.

I’ve been on the other end of things so I still rated the coder very good at a 7.

My one week Gamedev.net ad has run its course.

1. New user registrations was slightly more, maybe 20%
2. During that one week period, I got one sale that paid for the ad, and one business communication of indeterminate value
3. Got 306 visits due to the ad in total, 2/3rds of which were on the first day of the ad.

Overall I’m pretty happy with how it turned out. To VERY roughly estimate, I think the ad paid for itself and probably generated its own cost in increased sales as well.

I also contacted Gamasutra but the sales person I emailed about advertising did not reply to me.

I’m pretty happy as last week Codemasters purchased a site license to RakNet 3.x. I feel it’s a vindication of my hard work and effort over these 6 years because it means even against a big competitor, no real advertising, a programmer-centric website, and few connections, I still won in the end based on technical and quality merits. With most of my past sales I had some kind of edge – I knew someone at the company, or the company was too small to afford anything more expensive. But this time it was a straight-up victory with all the major hurdles in place. Codemasters is a great company that I like too, and the tech director there is a very nice guy.

I think these kinds of sales are more rare than they should be because a lot of people don’t realize how far advanced RakNet is at this point. Of course there are always parts that can be better, or one of my competitors does better to some degree. But on overall technical merits I think it’s right up there with my most expensive competitor, even better in some ways. I think with the increased funds my new prices are earning I can afford some marketing and finally get some serious market share.

I’m pursuing various official and unofficial channels in getting official console support too. Talk about beating your head against a wall… Fortunately, while I can’t say that I support console X, I can say that my customers do, and that my customers use RakNet. Maybe 10% as good, but better than nothing. If I can get that out of the way I’d probably double my sales. My best bet so far is to partner up with an established console developer and have them sell RakNet, thus bypassing the need to get licensed to begin with.

I keep reading about how Google always asks the interview question on how to pick the median of a large list in a distributed network. Here’s my answer, which I think is pretty optimal:

1. Randomly divide the list equally among all computers.
2. Pick one element (the key) of the list at random.
3. For each computer, for each element in that computer’s sublist, compare that element against the key. If it is less, move the element to the lesser list. If it is more, move it to the greater list.
4. Add up the sizes of the greater and less than lists among all computers.
5. If the sizes are equal, the key is the median. Done.
6. Else set the list to the greater than list. Go to 2

I paid for a ‘featured spotlight’ ad on Gamedev.net. So far I haven’t gotten any apparent boost in traffic. Two people that I know of in the industry have ad-blockers so didn’t even see the ad. I wouldn’t say that adblockers are stealing, but it’s true that pretty much nobody wants to see ads. Taken to its logical conclusion pure advertisement paid content would not be a viable business model. Those sites would have to massively downgrade content, require subscription fees, rely on free user submissions, put ads in a more obtrusive context, and/or a combination of these measures. None of these are good solutions, and I don’t know what a good solution would otherwise be.

Considering that in the best case scenario I stand to make $50,000 from a site-license to a publisher, $500 is a pittance. But that’s only worthwhile if the link that gets me that sale is the ad itself, as opposed to other measures such as a new post.

Spent like 8 hours straight today working with a customer on getting NAT punch through working with symmetric routers. The technique I was using with RakNet was essentially STUN, with an early datagram with TTL set to 2. This avoids the problem of one message arriving early, despite attempts to send simultaneously, and ending up IP banned on one router.

Very few sites on the internet go farther than STUN. I found this article at newport-networks on other techniques, however none of them were practical. Boiling down the 6 pages to two sentences, the first technique was to route the messages. The second was to have a filter past the router that modified the destination addresses.

I later found a much better paper at cornell.edu. It’s long but gives a good explanation of the problem. To boil it down to one sentence, it says it is impossible to connect peers both behind symmetric NATs in a reliable way, but you can guess the remote port based on typical router implementations.

I use the external port … external port + 4 and the internal port … internal port + 4. It worked for me connecting to the guy I was working with, but not the other way around. However, I suspect he was doing something wrong since it was 3 AM his time and his output didn’t make sense.

Here’s the code

I’m feel exhausted after doing this all day although all I’ve done is sit here in front of the computer.

*** EDIT ***

Came up with the idea to also have the recipient send datagrams to the sender’s possible open ports. If one of those datagrams arrives, the sender will use the external port therein. This ought to halve the failure rate by handling the case where the receiver could have connected to the sender but not vice-versa. Because I can send these datagrams all at once (rather than connecting one attempt at a time), it should help the system complete faster too.

I’m pretty happy about this because I have known for a while my NAT punchthrough wasn’t tested to the degree it should have been. I think it’s pretty robust now. The only thing I’m not doing is uPnP, and maybe I’ll add that further down.