OK, let’s see, what’s new in FE…

• The type system used for throwing around basic values (ints, floats, pointers, etc.) in the reflection code is much improved. To summarize, StaticType(T) (WordPress is eating my angle brackets, bear with me) implements traits and traits functions for certain basic types, and you can create a DynamicType ‘binding’ of the static type information to pass around at runtime. (StaticType(T)::Alignment becomes dynTyp->getAlignment() where dynTyp is a DynamicType*.) Information about whether types need constructors/destructors/copy operations is included, so you can do things like create properly aligned struct layouts in memory based on purely dynamic information.
• Implemented a function call system that works off of this, so it’s fairly easy to pass parameters to a function dynamically but also generally without ANY heap allocations — no boxing, pretty darn fast.
• PyBind (the new binding system works entirely off dynamic reflection info rather than SWIG) is getting pretty functional. Unfortunately most of the classes aren’t currently marked up or designed to work with the reflection system, so that’s an issue
• Finally shoveled all the C++ reflection-related stuff into a subnamespace (Common::Reflect), which is a lot nicer cognitively I think.
• Working on a system that should supercede making *actual* scripts for many cases where it’s just value fill-in, and also allow a common base for serializing non-binary data. I could have gone with XML for the format but I ended up writing a parser for what I’m currently calling “def files”, which look very similar to OGRE scripts. It’s much easier on the wrists to write than XML, and simpler to read, but of course the parser isn’t as well-tested as TinyXML yet The reason this is neat is because 99% of the entity definitions Tim wrote in Python were actually just filling in properties, so I’m going to try to direct things towards more of a “property-fill-in” approach than a script subclass approach. (In fact, it would be really simplifying to just throw out scripts entirely — not sure if that’s doable though.) Anyway, we can do several things with a ‘centralized’ definition file system:
• For named “resource objects”, i.e. things that represent individual, named, unchanging object instances (physics materials, map information), it will use a cache so you can basically go “get me object “, and it will try to locate and deserialize it if necessary, or just return the already-loaded object if it’s in the cache.
• For “templates”, i.e. object definitions that end up being used to initialize multiple instances throughout the process’s lifetime (ragdolls, UI, entities), you can go “make me an object “, and it should deserialize and fill it in for you.
• For miscellaneous configuration information (keybinds, highscore, etc.) it should be possible to represent these as normal reflection-enabled Objects and serialize/deserialize to specific filenames directly without having to write any load/save code, just mark up what properties exist.

I’ve also been doing some preliminary work on SHilScript++, and have a small prototype (VERY small — don’t get excited) started in C#. So far the current plan is:

• ANTLR-generated frontend (grammar almost finalized)
• Stack-based safe bytecode (like JVM or .NET — compact, easy to generate)
• Bytecode packages represent an entire group of classes, more like a .NET assembly than a Java .class file
• Eventually an LLVM-based backend. I might mess around with generating CIL from C#, which might end up being fairly easy because of the similarity between CIL and SS bytecode. (You might ask what the purpose of SS bytecode existing separate from CIL would be in this situation, and the reason is that eventually I’m gonna do this in C++ and won’t have the luxury of .NET just sitting there for me to use. So it has prototyping value, basically.)
• It’s also pretty easy to implement a “reference” VM for stack-based bytecode, even though it’d probably be hilariously slow.

I’ve been looking at the mcs (Mono C# compiler) implementation some to get a feel for how they organize things, because C#/.NET and SS are “similar” as far as syntax and runtime model. I’ve also been brushing up on my optimization/machine code/compiler building/grammar building skills, which is fun because you get to feel much more smug than normal if you have a reason to talk about pipeline stalls or register allocation or branch prediction.

I’m still in the process of absorbing a minor epiphany I had earlier this month about data-drivenness. You’d think I’d have all of these types of epiphanies out of my system by now, but I suppose I missed this one because it tends to go against certain OO/minimalism ideas. Anyway, the thing I noticed was, it’s way easier to to add a few more properties to a base class and complicate that class’s code a bit rather than than deal with lots of base classes or strategy objects. Another component of it is that it’s kind of an un-mentioned pattern in gamecode to have LOTS of these kinds of properties. A year ago, if I saw that a certain game object in an engine had 50 or 100 properties, I’d have thought someone was just not trying hard enough to get it down to a set of classes with each around 10 or so. However, in the long run, it’s much easier to have one moderately complex class and be able to make 95% of the behavior variations you need by just changing values — no actual “scripting” or subclassing or anything involved — than to go and treat everything that COULD be an object hierarchy as an actual object hierarchy. This also has benefits because if you work on a team with dedicated, non-programmer designers, it’s way easier to comprehend setting values to change behavior than defining logic. In summary: when you see “data driven”, think property/value settings in files before actual logic scripts in files. And when you think of class customization, think small, shallow hierarchy that can allow for most of the detailed variations just by changing properties, not by doing virtual function overrides. I’m not sure if that makes sense to anyone, but it’s an idiom I seem to have zoned out on and now I’m happy I found it, because it makes a whole lot of things I see somewhat more rational.

At the moment, FE is something like 30,000 lines of C++ with some extra MaxScript, Python and SWIG interface scripts. I think we’re using about a dozen file formats for resources, and if we’re not now we will be soon easily. (Let me count for a moment: .mesh, .material, .program, .compositor, .particle, .cg, .png, .tga, .fnt, .py(c), .femap, .femap.xml, .physics, .cfg (OGRE’s), .cfg (ours), .con, logfiles… okay, so that’s about 16.) That’s easily gonna be in the low to mid 20′s by the time things are done. (We’re going to have a package manifest, separate map metadata/rendering/collision/entity layers, cinematics, savegames…) The point is, there’s a bunch of shit going on and you can get design vertigo if you try to think about it all at once instead of adding things one-at-a-time until it’s complete.

I also noticed that we’re ending up with several separate file hierarchies that I’m gonna have to document… 1) the source tree layout; 2) the installation layout; 3) the user profile layout; 4) the package “pattern” layout (standard subdirectories etc. for resources in packages.) If you take into account that the installation layout might end up with a Linux and an OS X variant, and that the source tree would have to have build scripts for both of those cases, that alone is gonna be a few days of work for each case to set up build & install behaviors.
I guess I’m just feeling scattered from not being on a real overall plan at the moment. There’s a goal to get the engine ready to do some kind of a demo by December but that’s very vague and leaves me with a lot of flexibility as far as what needs to be in place by then. I think the most prudent plan would probably involve getting the entity system improvements done (adding nice light & camera support, cinematic animation, trigger/raycast-only collision volumes, and rudimentary load/save and possibly network ability) whilst actually using them to implement some basic behaviors in test maps so they can be proved to actually work.

I do still need to make several decisions regarding the entity system, such as how to actually set up cinematics (the current plan is to allow assignment of controllers to property slots, and use this to hook entities up to sampled movement curves from 3DS), how to organize the volume system (some volumes, like the hit-volumes for limb parts on entities, will actually change from raycast-only volumes to physics bodies, which ight mean the collision volume can be in a “physics” or “raycast-only” state, which is very weird), how to set it up so you can set up complicated, animating entity groups that only save the minimum amount of data when serializing to file/network (I have a few ideas on this, but I’ve not gotten to implementing them yet), and how to actually interface this with Python scripting. I also need to add more UI element types and improve UI scripting ability so scripts can actually create proper HUDs.

On the plus side, in the past week I’ve done some overhauls that were long overdue, such as adding a proper ingame console, adding proper keybind support (which also means player movement is driven by changes to cvars), adding a less primitive (and not Python-bound) configuration system, and moving all of the user profile files to a user directory instead of splattering them in various places in the installation tree. (This also means that theoretically Tim and I can make personal configuration changes without them ending up in SVN and resulting in a ping-pong config tweaking effect. This happens quite a lot with the default map setting and things of that nature.) The OGRE configuration dialog, which I intend to remove entirely in favor of our own in-game UI, can now be disabled if you don’t want to change config on startup by passing +ogreconfig 0 on the commandline (the commandline more or less works like Quake 3, and is basically just an interface to the console.)

I haven’t done much extra work on the LLVM-backed scripting engine concept. I’ve got more pressing stuff to work on with FE. I also haven’t submitted a patch for LLVM to compile properly on VS7.1 either Nor have I done anything on EL with Pouya for a good month, I think.

I think one of the things that’s bugging me about FE is the lack of visual changes recently. I’ve been making big, important changes to the engine side, but it’s still just the same handful of maps with enemies that are dumb that don’t have collision so you can’t shoot them. I spend a lot of my time wishing I was making something more visually impressive (or making tools to make more visually impressive things, which is more classic SHilbert.) I think FE definitely has the capability to be impressive, but of course it’s all C++ and any effects I’d want to do have to be possible within OGRE’s framework, which isn’t always the case. I still need to add a proper lighting system and set up a set of shaders for characters and map geometry that look nice and allow for all the different effects I’m likely to need.

October is starting soon, so the whole Halloween atmosphere is starting to creep into grocery stores and newspaper ads around here. If I didn’t have experience showing that it’s incredibly unlikely for such a thing to work *cough*, I’d make a small game for Halloween… I’m still toying with the idea, but I’ll probably be too lazy to actually do it. Plus I’ve got way too many other games I’m supposed to be working on anyway. Maybe I’ll just make some kind of pumpkin-head mod for FE or something.