Categories
Game Development

Indy Game programming: India outsourcing postmortem

Ever since I started in the games industry, I, and most games professionals I know, have a pet game design, and a dream to start our own companies. Most of us never have a chance to do that, but with a promise to my wife to ‘only’ spend 3 months and 20K (so about 50K […]

Ever since I started in the games industry, I, and most games professionals I know, have a pet game design, and a dream to start our own companies. Most of us never have a chance to do that, but with a promise to my wife to ‘only’ spend 3 months and 20K (so about 50K counting my own lost salary), I started down the path of potential riches. That was 5 months ago.

The initial schedule called for 4 months of game programming work for myself. In order to cut that down to the required 2 months, I decided to hire 3 juniors, plus one senior graphics programmer so my game would have all the cool new graphics effects. Of course I can’t afford that for 20K, and I heard that India was the best outsourcing location and that the programmers there were cheap and hardworking. So I decided to give outsourcing a try.

To get my programmers, I posted on a general job site, naukri.com. I thought I would get 20-30 resumes, interview maybe half of them, and pick the best of the bunch. What actually happened was I got got flooded with about 800 resumes, about 200 of which came in the first 24 hours. Because of the way their site is designed, anyone can apply to any job, regardless of qualification, and without reading the job description, and with a single click. You can imagine the resulting average quality applicant I got. I also had no idea what Indian programmers were worth at the time. I got prices ranging from $250 a month to $5000 a month, all of whom claimed a college education, and most of whom could not even program “Hello World” in a real-life application.

I ultimately hired 4 guys at, on average, $1500 a month. One disappeared on the day he was supposed to start work. One was fired after wasting 3 days just trying to compile the game and had to get help even opening and copying files. Another supposedly worked for two days, then disappeared, and told me he took another job else a week later. The last was the best qualified of the bunch: my former graphics programmer – a diamond in the rough comparatively speaking. He claimed to be a former lead programmer at nVidia, claimed 6 years of experience, could speak well, and could answer most of my general interview questions.

Yet at the same time, his graphics knowledge wasn’t what I was expecting. I’m a networking guy, not a graphics guy, so was prepared to interview a graphics programmer who knew so much more than I did it was like Chuck Norris beating up a schoolyard bully. What actually happened was closer to an untrained sumo wrestling match in a mud ring. I couldn’t really get a grip on what he knew, I knew some things he didn’t, and overall we were pretty evenly matched as graphics programmers. But I attributed this lack due to in part the language barrier, in part because he was awesome compared to the 849 buffoons before him, and lastly because I was so desperate to save money. My own inexperience in hiring didn’t help either.

I thought hiring an ‘expert’ with 6 claimed years of experience would enable me to be very hands-off about the graphics, and focus just on the gameplay and networking. Don’t meddle in affairs you know nothing about, right? Unfortunately, this approach messed things up even more. He put a lot of hours in – easily 7 days a week, over 10 hours a day. He got work done, from the standpoint of writing code and checking it in. Yet the game never made much graphical progress. The problem he solved were the wrong problems, solved the wrong way, and even that inefficiently and poorly written. Many tasks ended up taking 4X longer than they should have – once to explain every minor nuance, once for him to program it, once for him to redo it after I point out obvious errors by glancing at the code for 30 seconds, and the final time for me to give up in exasperation and do it myself.

As time progressed, and this kept happening, I had to learn more and more about graphics programming in Ogre 3D just to keep the game from degrading. Eventually it reached the critical threshold where it was easier and faster to do everything myself. I forced him to resign, and only regret not doing so sooner.

What conclusions can one draw from this? I have a good sample set of applicants and interviewees – the average applicant on a general job site is worthless. To be fair, this is probably also true on general job sites in the US as well. There is a huge variation among asking salaries – a twenty-fold difference between the cheapest and most expensive. I’ve found that you get what you pay for – ignoring the outrageous endpoints, the more expensive guys are clearly better.

I hired 4 guys – not a statistically meaningful sample set, but for what it’s worth, all 4 of the guys that did work worked hard and put a lot of time in. The comments about Indian programmers being cheap and hardworking is actually true. It’s only their competence that is lacking.

If I have to pick one common problem among all the applicants and workers I’ve had from India it’s literalism. If you say “Write a ship explosion effect” they will – but it will look stupid, be inefficient, and ignore the game design you just told them about two days before. And they honestly won’t know what’s wrong. So if you be more specific, and say “Write a ship explosion effect and that looks like this image, and do so efficiently” they will bombard you with a dozen stupid questions. It’s like programming a computer – you have to give ALL the information. Whatever information you don’t give will produce arbitrary and meaningless results, such as only supporting 1 lights in shaders (I didn’t say we needed more than one, I just said we needed light support), or doing 3D effects in 2D (Our game is in 3D, but I didn’t say the effect had to be in 3D), or making a cloaking effect where the enemy player’s 3D name still shows up (I didn’t say the name had to be invisible, only the ship).

Not all outsourcing is bad – I have some artists and a programmer in Europe that are doing an excellent job. But outsourcing to India has been a definite bust for me.

Fortunately, the project is not at risk, since graphics is just marketing, 90% of graphic quality is your artists anyway, and I have fantastic artists. But you definitely get what you pay for. Next time I’ll just hire someone in the US, at full salary.

20 replies on “Indy Game programming: India outsourcing postmortem”

hi, i noticed this entry in digg.com and was intrigued.
What you say is unfortunately true, albeit partly.
Not at all programmers are dumb as you are saying, there are really skilled people here but the problem is this: The various companies that are setting up their businesses here hire people with good academic record and not their skills. This resulted in a situation where colleges have started concentrating more on achieving marks and less on real-world skills.
Also, many of the companies hire people from different engineering streams like Electrical, Mechanical and even Civil !!! citing lack of good resources in the Computers stream. Therefore, these people from irrelevant streams have to undergo a rigorous training schedule in programming and the objective is always, unfortunately on finishing with the schedule rather than knowledge impartation.
A very famous company, Infosys does the same. It hires people from different streams imparts rigorous training and pushes them into the projects right away. Now, let me remind you that almost all the *software* companies in India are services-based and not product-based. So, emphasis is more on finishing the project in time rather than quality.
Also, you don’t need an engineering degree to do these kind of work anyone with a sound knowledge of programming will be able to do that.There are no new innovations to be made; no new algorithms to be designed; just write code according to the requirements,guidelines or whatever and thats it.
99% of the work done in the software companies here is a *no-brainer*.
Adding to the woes is population. For every job here in India, there are at least a thousand applicants,as you may have already experienced it. Most of these are people with excellent academics but no real-world skills.
This trend is going to continue until a saturation point is reached and then all hell breaks loose! Until we (India) start showing interest towards R&D and other sectors, we will have to face a very uncertain future.
Its widely known why many companies prefer India, Cheap Labor, hard-workers etc; Thats all and for no other reason.
What exactly is required is, a good finishing school or let the colleges themselves impart the necessary skills and give the students a wide exposure.Let them be creative, don’t curb their creativity.

Excuse me for keeping this post so long…i couldn’t help myself!

A good analogy for your experience:
I wanted to see the pyramids of Egypt and soak in the Egyptian culture. I could do this in 2 ways.

1. I ask my friends who have been to Egypt about the positives and negatives, weigh the options and then decide on the hotels, travel plans etc and then go there and enjoy the place as I am already aware of the pitfalls and I am prepared for them.

2. I land in Egypt without asking anyone. After entering Cairo, I go ask the travel agency in Cairo to make my travel and stay plans. Depending on whether the agency that I have chosen, I could either have a good experience or a bad experience. The options are egyptian govt run travel agencies or one of the big american travel companies..

If I choose the second option with egyptian govt run travel agency, what do you think would be my impression of Egypt as a travel destination?

In a way, your blog post helps people understand that there are more issues to outsourcing than just saving money.

Also, I haven’t known even one good programmer who gets his job/consulting assignments through naukri.com directly in India. (I stay and work in India)

As a suggestion, when talking to Indian programmers, ask for something that they have created and see if they can explain it properly. If they can, you have pretty much got yourself a good programmer working for you. This is something that every Indian recruiter would know but might be something alien for the non Indian recruiter.

P.S : I was not able to comment on your blogpost before and hence commented on reddit. I am replicating my comment here.

Ask an American or European if he understands some made up crap and he’ll usually say something like “WTF is that? I’ve never heard of x!”

Ask an Asian or Indian about some made up crap and he’ll almost always pretend to be an expert in it.

Without honest feedback, it’s impossible to do any non-trivial work efficiently.

dude, your story sounds correct , but let me correct you . Do you think that the best programmers in india would be out there surfing n waiting every day on naukri .com . If they would be as good as you want them to be , they would be making their own game , rather then writing game for some one who is just paying 1500$ .

Its like askin the next John Carmack , hey would you work for me for 1500$ , n yeah you are trying to find John Carmack on a general job site.

Try some game programming forums , that might help you ..
p.s : software field is an ocean

If you think this is bad, you should try dealing with the Chinese.

Outsourcing in India will lead to failure at least nine times out of ten. BEWARE.

You are better off with college kids in the usa or euro and hiring them as interns in house.

Hi,

I am an indian researcher and I am in perfect agreement with you. The problem is that India is producing too many so called engineers but there is a serious dearth of quality engineers. Barring a few reputed institutes such as the IITs, the rest of the institutes do not train engineers to solve problems.

The other angle to this problem is that many of these engineers are pushed into engineering as career owing to peer pressure although sadly most of them lack the aptitude to do. Unlike in US/Europe where college kids thrive in a “Do what you want” kind of culture, in India parental and peer pressure is very high and so many students move into engineering just to eke out a decent living and not because it is there passion.

Finally all said and done, there are good engineers too. I worked in US for a few years and came back to India for personal reasons. I did enjoy the freedom of working in US and I did learn a lot while I was there, but I have continued to learn over the years even while I am in India. It is because I am passionate about my work.

So I do feel it is kind of premature of you to jump to a conclusion about Indian engineers although I agree with you that there is a dearth of quality.

-Hari

That’s cheap exeprience for someone thinking cheaply to get some cheap game program done at cheap rate.
get THIS. Indian salaries have grown up so much. If you are only ready to pay 1500$ you should may be get some one making Burgers at Mcdonalds in US rather than going elsewhere.

I commiserate with your experience and am sorry for your giving up on looking talent so easily. Yes S/w programmers are dime a dozen in India. Big names in S/w in India are labour shops. 99% of the programmers in India are intellectual Coolies. (A coolie is some-one who lugs your luggage in the railway station and gets paid a pittance). Secondly game programming is a black hole – you should get an idea from the Gaming market here. it has just started. And finally 90% of the engineers make the worst programmer – I mean hard core programming, not just working for a big named s/w company. Ask them something and they will quote a book answer right to the page number and paragraph section.

You talk about good things, you say there are good people. But your intention seems to project your bad experience which can happen anywhere.

Why did you not figure out their capabilities when you interviewed them. It only means your interviewing was not effective. You pick people who are not good and you complain thaat they are not good. ????

if outsourcing to India were not good, so much of work will not be outsourced. I am sure companies are not dumb to get the job done in India despite no quality.

I think your blog has lot of prejudice. If you want to look something bad, it will definitely look bad.

Well even though i feel sorry for ur bitter experience in the Outsourcing experience that u had i must say ur at fault for all that u experienced as well.
The main reason we do have an interview is to judge and decide whether a candidate is skilled and equipped enough to fit into a particular job. So it is the interviewers job to ask the right questions for example if it is a programming job then test his programming skills or if it is a job which needs creativity accordingly give an excercise which will let u decide if he is worth the job which u’re giving him. If at all u’re not qualified enough to interview for a position then hire someone to do that.
There is no point in handpicking guyz whom u can pay the least and then complaining and passing a generic observation that offshoring is bad.

This is 1 example which u gave for proving that offshoring to India is a waste of time but i could give u 10 examples to prove otherwise.

U cannot take a one off case as a generic case.

[…] #3 Implement a growth plan that you can be comfortable with. Pace your growth to keep control of your expenses as well as to preserve your equity. There’ll be that point in the business when more hands may be needed on deck; which means hiring people — most likely, consultants — to assist with the skills gap. It is cost beneficial to hire free lancers when you’re not at the stage to employ full time workers, especially when there’s not enough work for anyone else to be able to productively tackle. We learned that outsourcing to another country can be a risk for a small company at this stage, so we’re staying out of that path for now. […]

Lemme tell a bit about me before I comment.

I am from India, an engineering drop-out, self taught game developer. I have been developing mostly engines \ renderers for PC and consoles and in the process of my own startup. Over the years, I built a network of talented game developers scattered in India ( mostly self-taught like myself), guided newbies, placed them within Indian game studios, and so on.

Almost all comments had at least some of what I wanted to reply to your post. Your experience is an obvious one, most of the ‘engineers’ are not ready/capable of your requirements.

First things first. Naukri.com or most Indian job portals are the last place to find proper game developers in India.

India is a massively populated country, with a major chunk of youngsters choosing/forced to choose Engineering/Programming streams because of better job opportunities. In almost anything in this world, the good ones out of big numbers are always minority.

– Majority of coders in Outsourcing companies are basically code fillers, donkey-work machines. And Naukri.com is the website if you want these kinds. Thats what the site is for. Thats what you got. As someone said, you cannot generalize and blame an entire nation based on one experience.

– You cannot compare India with US at the moment, where Game industry is Big and lot of universities, schools exist to provide 3D / Game specialization courses.

– There are many kick ass programmers in India, second to none. Few of them happen to be into Game development. Its a recent thing here. And those few developers had the balls to learn this shit even without proper books available. Like me, all they had is Gamedev.net and Gamasutra.com.

– I am really surprised you haven’t started with the obvious place. IGDA Forums.

– Did you check up when nVidia started its dev-center in Banglore ? How can a guy have six years experience in graphics ? even if its not totally at nVidia.

Finally, I am sure you did not ask the guy supposedly from nVidia, the right questions. Or may be I am too good at filtering out the fakers. 🙂

Anti-Outsourcing US Cribbers. Stop cribbing here please. Lets talk about Corporate Oil wars first.

Peace,
SkeletalMesh

“That’s cheap exeprience for someone thinking cheaply to get some cheap game program done at cheap rate.
get THIS. Indian salaries have grown up so much. If you are only ready to pay 1500$ you should may be get some one making Burgers at Mcdonalds in US rather than going elsewhere. “

Well said Aussie.

Oh Btw, nVidia’s Bangalore center is not into ‘Out sourcing’ or anything close.

nVidia’s flagship chipset nForce 600i, was developed Completely In-House, from scratch by a 70 member team at their Bangalore center. Two Indians started and run that center, hired the right guys and delivered it in months’ time.

The problem is finding game dev. expertise in a country that historically doesn’t have any. Most entertainment jobs can’t be learnt through books or the Internet – it’s this weird black art that’s passed through experience.

Special effects + animation have experienced the same things as you – poor quality work, but not because the labour is lazy or stupid. It’s because they are completely inexperienced. Taiwanese animation studios used to be like this, but after 20 years now rival western studios in skill and costs.

Sorry about what you faced. Indeed frustrating and time wasting stuff. If I were you, I would have hired someone who is passionate about gaming and love what he does.

The best way to do that is find through community forums/mailing-lists. Generally, most of the good guys are working for a few good *product* development companies but I bet I can still find someone.

I am correcting someone (the first comment) on product companies scenario, fyi! India has Adobe, Microsoft, Google, Yahoo and many more product companies, well yea Yahoo might be Internet service for end users for engineers each property (section) in Yahoo! is a product.

Things like game development are not very popular in India but there are some guys who are passionate enough to do anything for it. One such guy is a friend of mine, who is deep into Algorithms, Data-Structure, Maths, Physics, Graphics, 3d and programming languages (C, C++, ActionScript etc) are no brainer for him, but still thinks he needs to learn more and more before he would join some good game development company..Though I feel he has enough knowledge to go ahead and follow his dreams.

I would use Google-Search to find folks, I guess that’s the way recruitment would happen in future. Your contribution to community, open-source projects etc speak in itself.

Thanks for sharing your thoughts, it indeed helps.

-abdul

If you like I can join you and help you from my place on net for free(2hrs a day except exam times). I am really intersted in 3d game prgrammin and have done atleast 4 projects related to 3d opengl graphics. I love both palyin and programmin games .I hav also worked on mobile games…….

I am not that comfortable (but read a lot and wrote some basic prog ) with directx but its becouse I didnt worked on that but with opengl (may be more simple ) ,I can do anythin.

I am from IIT Kanpur (4th yr mechanical engg)…and have all basic knowledge of maths, physics, DS etc….

I tried to make a 3d bike racing game from myself but becouse of bad animation skills its sucked. Presently I am working on some3d visualization software in germany using opengl and will be back to IIT Kanpur in 1 week.

So if you think I can join your team than I would be very happy if you contact me.

Before writing all this, do give them a thought. We are the persons on whom US and most of the countries rely for getting their job done.

YOU YOURSELF ARE NOT ALWAYS CAPABLE TO COMPLETE THE JOB.

AND NEVER EVER IN FUTURE, TRY TO TARNISH THE IMAGE OF INDIANS.

I can’t do surgery, but that isn’t a valid argument to say that it’s therefore reasonable to hire someone else incompetent at surgery. I don’t have to try to tarnish the reputation of Indians, as they do very well at it themselves. I went into the outsourcing market with no preconceptions other than

A. I heard Indians do hard work
B. I heard Indians are cheaper
C. I heard many other companies outsource to India.

9 months later, after losing countless hours and tens of thousands of dollars I had enough evidence to form my present conclusions. I’ve worked with half a dozen Indians I’d call total idiots and rejected hiring dozens more.

Leave a Reply to jude Cancel reply

Your email address will not be published. Required fields are marked *