OpenEngine
A recent discussion that I've been having on a private industry forum regarding openness and secrecy has gotten me thinking about the issue of secrecy and openness in game development, and I've come to the conclusion that the industry could really do with opening up a lot. I'm especially talking about codebases, game engines and tools.
You've heard it before, both from here and other places, that a huge portion of the effort that happens across many developers is this issue that they continually reinvent the wheel. Most development teams, whether independent companies or in-house publisher teams, spend a great deal of time redesigning game engines, making tools on top of tools, and essentially managing these huge code beasts that they then intend to use to to make multiple games.
In many cases, the actual usage value of these engines is limited, especially in re-use, and so while you will see multiple games built upon the same base, the base itself changes over time to the point that it eventually has to be redesigned. This is a classic symptom of a closed-shop mentality, because it relies on the judgement of a few programmers and designers (who may move on, new ones get recruited etc) and it basically means that all the work done is special-case.
The biggest solution to this problem that has been mooted is, of course, middleware, ranging from the simple end like a plug-in rendering engine or a bit of physics code, to the full solution like Unreal 3 or all the bells and whistles of Renderware. However, there are two pretty large problems that middleware encounters which really detract from its status as a real solution.
The first is that it tends to be expensive. Middleware isn't just an application package that you license for use in your company in general. It tends to be a per-title license, for a huge fee (north of a million bucks) or a tasty cut of sales, or some combination of the two. That means that the middleware solution usually costs as much as the development of own-generated software anyway.
The second issue is whether it's really any good. Many programmers that I know are unhappy with working with middleware because they end up having to figure out what it does for a long time and then realise which bits of it don't work. Support becomes a real issue for middleware as a result, as does openness over how it works, how to evaluate its usefulness in regard to your game's needs, and so on.
At its root, I think that the problem that developers have with the cost of their technology is a solvable one, but it's one that they've generated for themselves. The reasons that many developers hold on to their technology with such secrecy and closed-shop mentalities (and keep re-inventing that wheel) basically derives from the belief that having their own engine technology is a valuable asset for the company. A codebase makes up a part of their intellectual property, in their eyes, and is therefore something to be treasured and protected. Many developers essentially look to id, Epic and Valve and see that those companies are making a good fist of licensing their technology, and in the truest tradition of the industry they seek to emulate that success.
IP
Let's start with why developers are so concerned over IP, and what IP IS exactly.
According to wikipedia, "Intellectual property (IP) refers to a legal entitlement which sometimes attaches to the expressed form of an idea, or to some other intangible subject matter. This legal entitlement generally enables its holder to exercise exclusive rights of use in relation to the subject matter of the IP. The term intellectual property reflects the idea that this subject matter is the product of the mind or the intellect, and that IP rights may be protected at law in the same way as any other form of property."
IP is a handle often used in the games industry (especially in the last few years) and has basically become synonymous with the idea of 'value'. Companies that have IP have value. That means that they have an appreciable price tag that is born out in the marketplace, where development companies that manage to generate their own IP are highly valued, whereas those that are pure license houses are not.
Let's be clear on this, though. 'Generate their own IP' means conceive of an original idea, get it funded, get it made, retain the rights to it, sell it in stores and make enough of a splash with it to warrant a sequel. And that's what gives the company its value and usually makes a publisher come calling with a chequebook.
Developers and publishers also treat the software that underpins their games as part of their intellectual property. The idea goes that since the software is part and parcel of the package, it therefore follows that it too is IP. This rather romantic notion does not really bear up to scrutiny, however.
Firstly, the value of an IP is not in the code that runs it. It's in the brand, and this is demonstrable with the longest-running franchises in the world. Zelda and Mario, for instance, are brands. There are many games that they have featured in, and the code of all those games has differed radically over the years. There is not a stitch of code from Super Mario Brothers left in Mario Sunshine. It's the characters that are the property. The same is true with most franchises good and bad.
Secondly, the price of developers that have created their own engine tech but no good games from it is vastly lower than those that have used existing engines to create amazing games (such as Rockstar). A lot of developers operate under the idea that the codebases etc have inherent value, but the market tends to not bear this out. While middleware vendors who have specifically created their engines and tools with the intent of having others use them do generate value, an engine developed for a particular game usually requires so much work to be made generally useful that in fact it has very little real value.
So what you have is a load of developers who have a lot of codebases that do broadly similar things in different ways, who regard all this code as an intellectual property. That it may be in the most literal sense, but it's a worthless property for most. It never gets sold to anyone and so it has no value.
Worse, the continual maintenance and rewriting and redesigning of said 'intellectual property' is actually a net drain, so the intellectual property becomes like a decrepit old building that's always in need of repair, built on land of very little worth.
The Development Silo
So it doesn't make a lot of sense, I think we can agree, to pour millions of dollars in hundreds of companies down a collective drain so that they all do roughly the same job as each other. Indeed it does not, but it's exactly the sort of result that emerges when companies don't talk, and that describes development companies to a tee.
There is a lot of back-channel talk in game development, such as secret industry forums etc, but in the official light of day, developers do not talk to each other. Most development houses, whether independent or inside a publisher, are protective, insular affairs where real information is at a premium, and they tend to develop their own fairly neurotic cultures. It doesn't help that these developers tend to be located in remote areas (like Rare's famous barns in the middle of the countryside, or EA Chertsey), but mostly the neurosis comes from habitual secrecy.
Development companies are like ICBM silos. Rumours filter in, information comes guardedly from the top down, and suspicion of what the other silos are doing is rife. Everybody's got their big missile (or IPs if you like) as against the eventual day when it's put to use, but in fact it never does get used. In most silos you have the warily suspicious who are looking to get promoted to a better silo, you have the unquestionably loyal who stick close to the letter of the place and hear no outside argument, and you have the powerless grunts who know that sooner or later the place is going to get bombed or decommissioned.
The problem with the silo mentality is simply that it encourages narrow perspectives. Drips of news that come via gamasutra or gi.biz are treated with over-compensated importance, and the silo fuels the imaginary arms race. We must have bigger and better shaders, we must have more and more horsepower, we must have more realism and kinetics and whatever other nonsense phrases because the other guys do. This causes numerous problems:
1. It means that evey developer feels that they must do everything themselves if at all possible.
2. It means that developers must retain artificially large staff levels all the time.
3. It means that that developers spend an inordinate amount on recruiting new talent and training them up.
4. It means that the culture eventually encourages the brightest sparks in the company to leave.
5. It means that making games becomes massively expensive.
6. It means that there's a lack of trust both internally and externally
7. It means that the developer is weakened in the eyes of the rest of the industry, and can have terms dictated to it so much more easily.
But mostly, silos encourage developers to regard each other as enemies rather than fellows because they come to believe that they are in competition with each other. Silos are good only for the companies that exist at the top of the food chain, the manufacturers, but in practise, silo-ification is bad for everyone else. But when you have an in-grained narrow perspective that dates back to the 80s, it's hard to convince some developers of that. They witter on about the good nature of competition etc while watching their prospects sink like stones.
Bunker busters
Many developers still have a very 80s sense of capitalism and competition which the silo mentality creates. In the rest of the world, collaboration has become a part of regular business, particularly collaboration on underpinning technologies. Groups of companies regularly come together to try and solve common problems that enable their whole industry to flourish because they understand 'Game Theory'.
Not as in the theory of game design. The theory of how multiple collaborators can work together toward common goals and common benefit as pithily explained in the movie 'A Beautiful Mind'. Game theory holds that there are many situations in business where competition is harmful to all sides.
The classic example is 5 guys and 5 girls hanging out in a park. The 5 guys each fancy one of the 5 girls. They have two choices. They can compete against each other to try and get one of the girls, but that is as likely to produce a result for none of them despite a lot of effort as the girls get repulsed by such blatant posturing. On the other hand, they can work as a team, in the hope that their co-ordinated and much more effectively targeted practises will net a result for all of them.
Game theory is the idea that underpins the DVD Forum, for instance. A whole lot of companies involved in movie-making could have gone all their own ways. They could have produced a dozen disc formats, all with proprietary content, on proprietary technology, but the upshot of that would be that nobody wins. By collaborating toward a common goal, the movie studios and the hardware owners created a market in which everybody wins, everybody is able to make more money individually than they would have if they had not con-ordinated. It's capitalism, because Warner Brothers and Fox are each still competing to sell their movies, but it's a positive kind of capitalism.
Similarly, HTML, various technical standards, the Human Genome project, OpenOffice, Firefox, many other collaborated technical efforts produce positive results, and they do so far more nimbly than silo solutiuons. A healthy dose of game theory is, ironically, exactly what the games industry needs. Collaboration between developers is vital, openness is vital, honest information flow is vital.
OpenEngine
And the form that that collaboration should take is that of a common-ground development engine and tools. As I was talking about a few dozen paragraphs above, the process of building and maintaining and redesigning every developer's own personal engine system is a massive drain on each of their resources. Middleware is an expenisve solution that all too often solves nothing. But a collaborated engine, possibly even an open source engine, is a concrete way to get away from that. Developers may be in competition with each other to get publishing contracts, but they could save themselves an absolute fortune if they decided to collaborate on the technical basis of their industry at the very least. If Warner and Fix can do it, why can't we?
Some of the benefits of this effort are:
1. Better engine
What many of the open source community have found is that exposure of code to a relatively non-political environment has remarkable effects in terms of honesty of opinions and innovative solutions. Put more simply, the more exposure the software has, the easier it becomes to fix what's wrong.
2. Focussed staff
Not less staff, but better-used staff. One the one hand it means that staff can mnore directly be applied to the problems of production rather than endless proto-stage development. Production becomes a more standardised and efficient affair so the developer can get more done.
3. Ease of recruitment
It is far more straightforward to hire someone into a position if they have skills in the standard systems that an industry uses, and most technology industries recognise that. Unfortunately, the games industry invariably has to retrain everyone that it hires for an excessively long period of time. Every studio has its own engine, its own set of awkard tools. Imagine an industry where the tools are standarised to the point of easy. That's what openness can provide.
4. More open design
If the common-ground engine is well understood, then basing designs off its known technical constraints are also well understood, and so design becomes a more straightforward sort of affair.
5. Co-operative spirit
In the film industry it is common for producers from different companies to help each other out, do each other favours and so on. This is because film is a fairly standardised medium, established through co-operation, and so business conditions are favorable to the point that taking a punt is not always a matter of risking everything. There is no similar spirit of community in the games industry. Hollywood may be cut-throat, but the games industry's silos are in many ways much worse because they offer no flexibility. Wouldn't it be nice to see EA take a punt on a small indie game because it basically costs them very little to do so because the whole thing is developed in standard open tech?
6. Empowerment
Collaboration means that the different software houses will come to realise that they are actually all alike, that they have the same problems, and it will empower them to find solutions to them. It will also help them realise that some of the external forces, such as hardware manufacturers, are being unreasonable. Sony's PS3 devkits are, on the grape vine, a complete nightmare to work with, but nobody can say in open court what it is that's wrong with them, or indeed fix them. It's all back channels. A collaborative development industry could make Sony try harder for them rather than keeping them on its leash (which is what happens with silos)
7. Getting back to the games and not the tech
This is the games industry. The GAMES industry. Yet the amount of effort and marketing devoted to talking about the technology is ridiculous. Collaboratiom enables developers (And journalists eventually) to stop so much with the shiny-lights talk and focus on competing where it matters. Making games.
8. More focussed businesses
Similarly, it's better for the companies involved because they can focus on expanding their markets with clear objectives rather than fuzzy ones.
9. Creating an easier path into the industry
All those students studying for game degrees could be obtaining valuable skills that will eventually yield dividends for the developers.
10. Forcing commercial providers to get their act together
Standards make for a more open evaluation environment for already-established tools. For instance, hugely expenisve 3D applications. Does the industry really need all the bells and whistles of Max and Maya presented with such tediously provided interfaces that they increase the workload by a large factor? Not if it's a collaborative industry it doesn't. A collaborative industry can decide to make a *better* 3D modeling package based on what it needs rather than a general solution that's based more on what the film industry needs, but does for games sorta kinda. The same is true for all sorts of providers.
But collaboration is the key. Come on developers, make friends with each other and help yourselves out of this hole. Nobody else is going to do it for you.
