Around the turn of the century, evaluating COTS game engines was not necessary; you only had several to choose from.  Today, there are many, but picking the correct engine can be an overwhelming task, especially for someone who doesn’t even understand exactly what a game engine is.

            First, let’s define a game engine.  A game engine is a collection of classes and code that utilizes graphics APIs, networking, input, and sound.  A game engine executes the main game loop, the mechanism we use to render a single frame of a 3D application.  By running and updating the main loop every 16.6 milliseconds, a game engine displays an animated interactive scene 60 times a second.  Game Engines range from 50,000 lines of code to 500,000, and although the amount of code is not a proper delineating factor, the size of a game engine does make the task of comparison more difficult.

            Choosing a game engine for commercial games is difficult; choosing a game engine for MS&T applications is even more difficult.  Why?  Because games generally fall into 6 different genres, and games engines are usually focused towards one of those genres.  Simulations often require engines that span the technical boundaries of any single genre.

            Since we are all busy people and deadlines are always approaching, let’s change the format of the article into a more palatable one you can use as a checklist when evaluating your next engine.  Each section contains questions you should ask your vendor.  If a vendor is unable to answer these questions, you should consider it a red flag.  The more unanswered or poorly answered questions, the worse then engine (or salesperson) probably is for MS&T.

           

 

Game Engine Evaluation

 

Efficiency

           

Content Creation Turnaround – A common tactic by game publishing companies is to evaluate the time it takes from asset creation to asset display in the engine.  If this process relies on a programmer, it could take days.  If the artists cannot place the model in the 3D world themselves using a level editor, it will limit their abilities.  If only one artist is able to work in the level at one time, they will be less efficient.

 

Ask the vendor the following questions:

• Does your engine require a programmer to add the models to a 3D world?
• Can more than one person work on the level at one time or is the mission/level not mergable?
• Does your engine have an editor for GUIs?

 

           

Rendering Performance – When measuring performance, determining the amount of possible polygons is a good measure of the vendor’s ability to optimize their engine.  The number of polygons that an engine can support is measurable by the number of triangles per second they can render and maintain 60 frames per second.

 

Ask the vendor the following questions:

• How many frames per second does your engine render?
  
• Does your engine use SIMD math on complex algorithms?
  
• Does your engine use Vertex Buffers?
  

 

Robustness

 

An engines ability to render many different scenes efficiently is a sign of good design.  Many engines support only one genre or forced to be genre independent as a post- design consideration.  An engine that has a design driven on the demos or games will not be the best for MS&T.  I once hear a story about a firefighting simulation rendered with an engine used for a war game.  Due to a lack of flexibility, the simulation rendered the fireman holding a hose with the same pose of someone holding an M16.

 

            Ask the vendor the following questions:

• Does your engine support interiors?  If so, what method of culling are you using?
• What is the size limit of the terrain?
• What genres does your engine support?
• (Most will say all genres; ask to see demos of each.  How would they know the support them if they don’t have demos? )
• How many of your team members are on the design team? 
• How often do they meet? 
• Explain your design process.

 

Reliability

           

Testing – an engine’s stability is dependent on its testing; this is especially true on engines with few developers (meaning less than 1000).  The more users that an engines has, the more stable it probably is (otherwise they would use something else).  Some methods test the engine in an automated way.  This is great, but only if they are writing good tests.  If a vendor is using an automated building system, they are more likely to have well tested code.  Programmers are biased tester.  Prefer engines where their testing team is different than their programming team.

 

Ask the vendor the following questions:

• Explain your QA process.
  
• How many testers do you have?
  
• Are you using automated testing?
  
• Do you use an automated building system?
  
• If so, how often do you do a build?
  (The shorter the time the better, some teams build releases several times a day)
  
• May I see your testing lab?  How many machines/configurations do you test your application code?
  

 

Flexibility

           

Cross-platform

If an engine supports more than one platform, it is a good sign.  In order to support multiple platforms, the vendor must design their engine well.  If an engine supports multiple rendering APIs, that’s even better.

 

Ask the vendor the following questions:

• What platforms does you engine support? (The more the better)
• Does your engine support both OpenGL and DirectX?
• What platform does your engine support the best?

 

Configurability

 

Engines that are easily configurable have a better, more flexible design.  Highly configurable engines are often data driven.  Data driven applications can alter the flow of the application by configuring data outside of the application.  Engines that are easily configurable require less code changes and allow for a more testable environment.

 

Ask the vendor the following questions:

• What components of your engine are data driven?
• Does your engine support a scripting language like Lua or Python?

           

Extendibility

 

No engine will support every feature that you will need; therefore, you will need an engine that allows you to extend its basic features.  An engine that supports extendibility via sub-classes will require you to have full source code.  Do not expect to be able to extend an engine given to you as a library without the source code.  Relying on documentation to do this is nearly impossible.  Extending an engine using a scripting language is the second best option.  Having the complete source code is the most flexible way to extend an engine..

 

Ask the vendor the following questions:

• How do we extend the base functionality of the engine?
• Do you offer full source code support?
• Does your engine support a scripting language like Lua or Python?

 

Maintainability

 

In order for an engine to continue to be a world class engine it must evolve.  Creating new versions of the engine is useful unless upgrading to the new engine requires too much work for your organization.  The engine must support versioning and preferably, an element of backwards compatibility.

 

Ask the vendor the following questions:

• How does your engine support versioning?
• How often do you release updated versions of your engine? ( the shorter the better since your integration will be easier.)

 

The author of this publication is a course director teaching graphics optimization at Full Sail University. He has 6 years experience in the game/simulation industry. His services are available for contract. For more information, go to http://www.commandtoclient.com