The Future of Game Engines...

In the year 2013...

As technology advances beyond our wildest imagination, we begin to see the affect it has on the gaming industry, in specific, the effect it has on game engine coming out nowadays. Games like Crysis 3, are the perfect example of the limits that are being pushed to release high-quality content that looks fantastic and runs smoothly. So this wekk we talk about the future of game engines as they move toward real-time photo-realistic rendering and natural character animation.

This blog covers what's in store for games coming out on next-gen consoles. I will go over what the rendering technology of today has to offer in games and later we will discuss what methods and techniques are being employed in the game of the future. Information from the blog is taken from Xiaomao Wu's GDC China talk on 'The Future of Game Engines,' in 2010. This article can be found on the Crytek website here. 

The Current State of Affairs

There are several techniques and tools at the disposal of game developer nowadays that allow them to create the astounding AAA titles that we see clutter the video game market. Once someone figures out how it's done, the method is picked up quite quick in the industry and developers go out and do better, continuously trying to raise the standard with their next game and set the bar just a little higher. This results in healthy competition in the market and fantastic games to appeal to every player's unique play style. So let's take a look at some the technology currently being employed by game engines (primarily CryENGINE 3). 

In terms of Global Illumination, light propagation volume allows for dynamic scenes with dynamic lights. Unreal 3 makes use of Lightmass for static lighting and effects. Middleware solutions such as Geomerics, Beast (employed by Unity) and Lightsprint are also available. SSAO (Screen Space Ambient Occlusion) is widely used in games since its premier proposal by Vladimir Kajalin at Crytek. It was used for the first time in Crysis and made popular by the game's sequels in addition to Gears of War 2, Uncharted 2, Halo, etc. The advantage of using this technique is high-quality with no pre-processing or loading time allocations. It is also memory efficient and independent on scene. The only con of using SSAO is that only visible occluders on the screen are considered which may produce artifacts.

Deferred rendering was first proposed by Michael Deering and his colleagues at SIGGRAPH in '88. It essentially is the process of decoupling geometric scene complexity from the shader in order to reduce computation and memory bandwidth to visible portions. It tends to be more difficult to handle transparency with depth peeling for order-independent transparency (additional batches and g-buffer size). However, DirectX 10 and 11 can perform batches fast which makes the process no less effective than forward shading. Shadows are advancing as well with dynamic occlusion maps, shadow maps with scree-space randomized look-ups, those with light-space randomized look-ups and shadow mask textures. There are cascaded shadow maps (for directional light sources) and deferred shadow mask generation as well as unwrapped shadow maps for point lights and variance shadow maps for terrain. 

Anti-aliasing is also big nowadays and exists in the form of edge anti-aliasing as demonstrated in the CryENGINE as well as morphological anti-aliasing in God of War III. Volumetric environments are popping up and can be seen in fog, sun shafts, terrain and clouds. Post-processing effects that are popular in most games include depth of field (with bokeh), motion blur, lens flare, bloom and glow, and color grading. Color grading entails changing the color style of a game in real-time. Generally a reference chart is provided by the user as the engine is able to change the game scene using this color chart. Color grading is popular because it helps to change the game style and create expressive tones. 

Motion control in modern game engines (specifically CryENGINE) consists of time warping, parameterization, real-time motion controlling of the player or AI, and weight-shifting (uphill or downhill). Animation trees have been made popular by the Unreal Engine. CryENGINE features real-time inverse kinematics. This consists of analytic IK, example-based IK, CCD (Cyclic Coordinate Decent) IK, and Jacobian IK. Human IK includes types such as foot IK, look IK, aim Ik, and full-body IK. This is all integrated with Physics in the form of rag-dolls, hit-reaction and smooth transitions between the physics and animation data. Additive animations are being used for breathing, looking around, flinching, and posture changes. Behavior trees (table which defines a character's behavior), script-based AI logic and animation graphs are all being used to simulate animation behavior. 

Skinning is still done with linear blending which reveals artifacts like collapsing and candy wrapping. More advanced skinning techniques include dual quaternions which are GPU -friendly and preserve rigidity by reducing skin collapsing. Of course there still exists a "flipping" artifact with discontinuity beyond 180 degrees of rotation. Multiple CPUs and GPUs are also gaining popularity. 

Around the corner

DirectX 11 and OpenGL 4.0 have several advanced features such as DirectCompute, better tessellation, improved multi-threading and shader model 5.0. All this might mean real-time high-quality subdivision surfaces, real-time micro-polygon rendering and convenient dynamic shader binding. However, this could also mean a pipeline change on the code and art side of things. Point-based rendering is a new approach which naturally fits an automatic Locally One-Dimension (LOD) scheme and features freeform modeling as well as anti-aliasing. On the other hand, however, point-based rendering only makes sense on super high-detailed scenes as editing tools still rely greatly on triangles and GPUs focus on rasterization. Real-time ray-tracing brings a lot to the table as it is easily parallelizable and naturally leads to high quality shadows and lighting. It also simplifies the rendering pipeline and supports anti-aliasing. The downside would be that it's still too slow for a real-time application and once again GPUs focus on rasterization. 

Server-side rendering is also an interesting concept that seems to be making waves. 4G networks have a good ground for this type of technology with low ping (a strong requirement for real-time games). It is predicted that this will be deployed in the next 5-7 years. Advancements in video compression mean temporary decomposition of the videos details in addition to the use of perception-based importance (salience maps and user-side eye-tracking). Motion control continually grows to be more and more accurate with geostatistics. This optimizes both kernel parameters and weights to make them a better fit. There is more accurate interpolation than Radial Basis Function (RBF), inverse weighting, and k-nearest neighbors. It is about half the speed of RBF thanks to optimization tricks. 

There is current buzz about interactive and dynamic ragdoll physics which will make things a lot more realistic. CryPhysics and Nvidia Physx both support this though there are ever-present challenges associated with real-time performance and stability. Natural and real-time IK is also something worth looking into. This combines with biomechanics information and extends 12-DOF (Degrees of Freedom) analytic IK. An interesting concept circling around is that of a so-called "UniversalK" which means HumanIK, MonsterIK, AlienIK, AnimalIK are all supported. The system will essentially learn from artist-defined sparse poses and use data from biometrics studies. A more automated rigging system could improve a lot of things. Current theories behind hot to do this include Region-of-Influence, Heat-transfer and skinning by example. Pure automatic skinning won't be available in 2-3 years, but a highly-automated method that can decrease manual weight adjustment to a large extent is currently possible. 

Robust and real-time motion retargeting could be possible with different skeleton topology and allowing manual interaction within a game editor. The process would essentially follow decomposition and re-combining, analyzing the behaviors of different creatures, using an intermediate skeleton, and relying on a fast, robust full-body IK solution to maintain constraints. Smarter AI is definitely something a lot of gamers are looking forward to. Improvement in perception (speech, facial expressions, environment) and social intelligence are current areas of focus. This includes predicting the actions of others by their motives and emotional states through game theory, decision theory and emotion modeling. Another interesting technique would be learning game players' behavior and applying this to the AI characters both online and offline. This would happen by constantly updating AI characters' skills and knowledge at run-time basically making them human-like. A neural network and reinforcement learning are also theories yet to be implemented. 

In Conclusion

There is a lot to look forward to in the world of game engines. Technology is changing and the gaming industry is rushing to keep up with it all. There are some marvelous things happening behind closed door and several theories worth delving into for practical application and research purposes. CryENGINE is a great example of a game engine that keeps pushing the boundaries and setting the standards for game industries everywhere. There's a lot going on and massive sandbox in which it's taking place.