void SHilbert::Update(float delta) {

Okay, so the semester started again and now I don’t have as much free time anymore. Which is not great, because when I had free time over winter break I didn’t get much of anything done. Current project statuses follow:

PLATFORMER.

I didn’t work much on this over the break. My laptop is running Linux now because the poor thing is dying and somehow it seems to work alright with Ubuntu (as in, it only freezes intermittently, as opposed to Windows which can’t get through the installer without bluescreening.) So, being that I was away for the majority of my break, and thus using my laptop, which runs Linux, I didn’t get much done on a game that uses XNA. I have done a little more design work but it’s mostly on the silly kinds of details I worry about that noone else is interested in. I’m hoping that eventually I’ll get to a point where I can release a demo (or at least a video) every month or two of the progress, it’ll depend on how classes go.

PORTAL TURRET.

Not a whole lot since the last update. I have some more details worked out for motor control but it’s still all basically resting on fabricating the body at this point. I am going to talk to someone at NCSU about using the school of design’s CNC router to do this. I am also still unsure about how the thing should be controlled – complete autonomy is sort of boring, and adding a proper remote control unit would be a giant pain because I’m not that great at electronics. Besides that, the rest of it is just details, hopefully.

SHILSCRIPT++

I got some work done on this over the break. I’m working on static checking for expressions right now. Basically I need to clean up the classes used for types and implement some kind of overload resolution function. I haven’t implemented overloads yet, as expression static checking isn’t even done yet, but the C# spec defines the available builtin operators in terms of function overloads, and that seems like a clean way to do it. Hopefully I’ll get to codegen sometime this century. Other things that are outstanding are storing location information on AST nodes for error messages with actual locations, and adding memory tracking so I can make sure there aren’t any leaks.

So, that’s the project status. I am feeling kind of crappy recently because I haven’t gotten much of anything done and I feel like I’m atrophying just sitting here not coding anything. Usually when that happens I open up SS++ a little or think about tinkering with FE or low-level rendering, but never get around to it.

HOLIDAY STUFF

I got to see Tim while I was in California. He’s pretty much always up to something and this time was no exception. Here is a brief list of things we did:

  • Spent forever trying to get Defcon netplay to work on a hotel wireless network. Once we finally did get it to work we found out my laptop hard freezes a few minutes into the Defcon stage, so that was kind of lame. Next time I am going to just bring a wired switch so I don’t have to start considering scavenging a hotel ethernet cable to make a crossover cable. Oh, and get a new laptop that isn’t subtly broken.
  • Watched Tim do some parkour crap in public places. I am pretty sure somewhere there is a little kid jumping off his roof because he saw a little Asian man do it at the park.
  • Walked around in downtown Pasadena while everyone was sitting on the roadside claiming spots for the Rose Parade. We were looking for some bizarre kind of shoes Tim wanted but the closest he could find were freeking expensive.
  • Played Megaman X ‘together’ by exchanging the controller. I totally beat Sigma on the first try. I wasn’t so hot at Mega Man X2, though, I had to actually use the boss’s weaknesses against them. :(

The whole thing made me really want to work on FE more, despite already having too many projects and not getting any work done on any of them.

Also, apparently captchas (or the captcha I added) aren’t good enough for comment spam protection, let alone trackback spa protection. I am really getting tired of this.

Fallen Elements & SS++ update

OK, let’s see, what’s new in FE…

  • The type system used for throwing around basic values (ints, floats, pointers, etc.) in the reflection code is much improved. To summarize, StaticType(T) (WordPress is eating my angle brackets, bear with me) implements traits and traits functions for certain basic types, and you can create a DynamicType ‘binding’ of the static type information to pass around at runtime. (StaticType(T)::Alignment becomes dynTyp->getAlignment() where dynTyp is a DynamicType*.) Information about whether types need constructors/destructors/copy operations is included, so you can do things like create properly aligned struct layouts in memory based on purely dynamic information.
  • Implemented a function call system that works off of this, so it’s fairly easy to pass parameters to a function dynamically but also generally without ANY heap allocations — no boxing, pretty darn fast.
  • PyBind (the new binding system works entirely off dynamic reflection info rather than SWIG) is getting pretty functional. Unfortunately most of the classes aren’t currently marked up or designed to work with the reflection system, so that’s an issue ;)
  • Finally shoveled all the C++ reflection-related stuff into a subnamespace (Common::Reflect), which is a lot nicer cognitively I think.
  • Working on a system that should supercede making *actual* scripts for many cases where it’s just value fill-in, and also allow a common base for serializing non-binary data. I could have gone with XML for the format but I ended up writing a parser for what I’m currently calling “def files”, which look very similar to OGRE scripts. It’s much easier on the wrists to write than XML, and simpler to read, but of course the parser isn’t as well-tested as TinyXML yet ;) The reason this is neat is because 99% of the entity definitions Tim wrote in Python were actually just filling in properties, so I’m going to try to direct things towards more of a “property-fill-in” approach than a script subclass approach. (In fact, it would be really simplifying to just throw out scripts entirely — not sure if that’s doable though.) Anyway, we can do several things with a ‘centralized’ definition file system:
    • For named “resource objects”, i.e. things that represent individual, named, unchanging object instances (physics materials, map information), it will use a cache so you can basically go “get me object “, and it will try to locate and deserialize it if necessary, or just return the already-loaded object if it’s in the cache.
    • For “templates”, i.e. object definitions that end up being used to initialize multiple instances throughout the process’s lifetime (ragdolls, UI, entities), you can go “make me an object “, and it should deserialize and fill it in for you.
    • For miscellaneous configuration information (keybinds, highscore, etc.) it should be possible to represent these as normal reflection-enabled Objects and serialize/deserialize to specific filenames directly without having to write any load/save code, just mark up what properties exist.

I’ve also been doing some preliminary work on SHilScript++, and have a small prototype (VERY small — don’t get excited) started in C#. So far the current plan is:

  • ANTLR-generated frontend (grammar almost finalized)
  • Stack-based safe bytecode (like JVM or .NET — compact, easy to generate)
    • Bytecode packages represent an entire group of classes, more like a .NET assembly than a Java .class file
  • Eventually an LLVM-based backend. I might mess around with generating CIL from C#, which might end up being fairly easy because of the similarity between CIL and SS bytecode. (You might ask what the purpose of SS bytecode existing separate from CIL would be in this situation, and the reason is that eventually I’m gonna do this in C++ and won’t have the luxury of .NET just sitting there for me to use. So it has prototyping value, basically.)
  • It’s also pretty easy to implement a “reference” VM for stack-based bytecode, even though it’d probably be hilariously slow.

I’ve been looking at the mcs (Mono C# compiler) implementation some to get a feel for how they organize things, because C#/.NET and SS are “similar” as far as syntax and runtime model. I’ve also been brushing up on my optimization/machine code/compiler building/grammar building skills, which is fun because you get to feel much more smug than normal if you have a reason to talk about pipeline stalls or register allocation or branch prediction.

I’m still in the process of absorbing a minor epiphany I had earlier this month about data-drivenness. You’d think I’d have all of these types of epiphanies out of my system by now, but I suppose I missed this one because it tends to go against certain OO/minimalism ideas. Anyway, the thing I noticed was, it’s way easier to to add a few more properties to a base class and complicate that class’s code a bit rather than than deal with lots of base classes or strategy objects. Another component of it is that it’s kind of an un-mentioned pattern in gamecode to have LOTS of these kinds of properties. A year ago, if I saw that a certain game object in an engine had 50 or 100 properties, I’d have thought someone was just not trying hard enough to get it down to a set of classes with each around 10 or so. However, in the long run, it’s much easier to have one moderately complex class and be able to make 95% of the behavior variations you need by just changing values — no actual “scripting” or subclassing or anything involved — than to go and treat everything that COULD be an object hierarchy as an actual object hierarchy. This also has benefits because if you work on a team with dedicated, non-programmer designers, it’s way easier to comprehend setting values to change behavior than defining logic. In summary: when you see “data driven”, think property/value settings in files before actual logic scripts in files. And when you think of class customization, think small, shallow hierarchy that can allow for most of the detailed variations just by changing properties, not by doing virtual function overrides. I’m not sure if that makes sense to anyone, but it’s an idiom I seem to have zoned out on and now I’m happy I found it, because it makes a whole lot of things I see somewhat more rational.

More FE updates

I haven’t written an update about FE features since July 2006, so here we go :)

  • Fixed all namespaces & include guards to be more uniform and correspond to project/directory names.
  • Lots of work done on the interface for game entities.
  • Quite a lot of work done on the Common reflection system, allowing for all sorts of neat magic at runtime you normally don’t get with C++.
  • Cleaned up MaxScript tools.
  • Removed a whole bunch of legacy code, making initialization of the game much simpler.
  • Added a proper ingame console system. Quite a few useful commands available.
  • Player control now works through console keybinds.
  • Updated configuration system to be simpler and not rely on Python. (Not everything is moved off the old system, though.)
  • Build information (date/platform/compiler) now embedded into the game.
  • Work done on improving shaders ingame.
  • Work on Clockwork (object creation/destruction/variable replication basically working over a loopback connection)
  • Began working on a system to base the Python bindings off the Common reflection system, eliminating the need for SWIG (mostly) and enforcing more uniformity and explicitness in the API.
  • Currently implementing a type marshalling system for doing dynamic function calls (to work with reflection & new python binding system.)
  • Profile information now goes in a user-specific directory, so you can theoretically run off of read-only media and have multiple users on one machine with different configurations.

Things you can now do in the console (executive summary):

  • Define console command aliases.
  • Manage key binds.
  • Edit configuration settings.
  • View reflected class tree.
  • Run Python scripts.
  • View build identification information.
  • Toggle profiling/debug readouts.
  • Edit mouse sensitivity/Y invert.
  • Control time speed and view time statistics.
  • Perform ingame movement commands.

Complexity is getting to me

At the moment, FE is something like 30,000 lines of C++ with some extra MaxScript, Python and SWIG interface scripts. I think we’re using about a dozen file formats for resources, and if we’re not now we will be soon easily. (Let me count for a moment: .mesh, .material, .program, .compositor, .particle, .cg, .png, .tga, .fnt, .py(c), .femap, .femap.xml, .physics, .cfg (OGRE’s), .cfg (ours), .con, logfiles… okay, so that’s about 16.) That’s easily gonna be in the low to mid 20′s by the time things are done. (We’re going to have a package manifest, separate map metadata/rendering/collision/entity layers, cinematics, savegames…) The point is, there’s a bunch of shit going on and you can get design vertigo if you try to think about it all at once instead of adding things one-at-a-time until it’s complete.

I also noticed that we’re ending up with several separate file hierarchies that I’m gonna have to document… 1) the source tree layout; 2) the installation layout; 3) the user profile layout; 4) the package “pattern” layout (standard subdirectories etc. for resources in packages.) If you take into account that the installation layout might end up with a Linux and an OS X variant, and that the source tree would have to have build scripts for both of those cases, that alone is gonna be a few days of work for each case to set up build & install behaviors.
I guess I’m just feeling scattered from not being on a real overall plan at the moment. There’s a goal to get the engine ready to do some kind of a demo by December but that’s very vague and leaves me with a lot of flexibility as far as what needs to be in place by then. I think the most prudent plan would probably involve getting the entity system improvements done (adding nice light & camera support, cinematic animation, trigger/raycast-only collision volumes, and rudimentary load/save and possibly network ability) whilst actually using them to implement some basic behaviors in test maps so they can be proved to actually work.

I do still need to make several decisions regarding the entity system, such as how to actually set up cinematics (the current plan is to allow assignment of controllers to property slots, and use this to hook entities up to sampled movement curves from 3DS), how to organize the volume system (some volumes, like the hit-volumes for limb parts on entities, will actually change from raycast-only volumes to physics bodies, which ight mean the collision volume can be in a “physics” or “raycast-only” state, which is very weird), how to set it up so you can set up complicated, animating entity groups that only save the minimum amount of data when serializing to file/network (I have a few ideas on this, but I’ve not gotten to implementing them yet), and how to actually interface this with Python scripting. I also need to add more UI element types and improve UI scripting ability so scripts can actually create proper HUDs.

On the plus side, in the past week I’ve done some overhauls that were long overdue, such as adding a proper ingame console, adding proper keybind support (which also means player movement is driven by changes to cvars), adding a less primitive (and not Python-bound) configuration system, and moving all of the user profile files to a user directory instead of splattering them in various places in the installation tree. (This also means that theoretically Tim and I can make personal configuration changes without them ending up in SVN and resulting in a ping-pong config tweaking effect. This happens quite a lot with the default map setting and things of that nature.) The OGRE configuration dialog, which I intend to remove entirely in favor of our own in-game UI, can now be disabled if you don’t want to change config on startup by passing +ogreconfig 0 on the commandline (the commandline more or less works like Quake 3, and is basically just an interface to the console.)

I haven’t done much extra work on the LLVM-backed scripting engine concept. I’ve got more pressing stuff to work on with FE. I also haven’t submitted a patch for LLVM to compile properly on VS7.1 either :P Nor have I done anything on EL with Pouya for a good month, I think.

I think one of the things that’s bugging me about FE is the lack of visual changes recently. I’ve been making big, important changes to the engine side, but it’s still just the same handful of maps with enemies that are dumb that don’t have collision so you can’t shoot them. I spend a lot of my time wishing I was making something more visually impressive (or making tools to make more visually impressive things, which is more classic SHilbert.) I think FE definitely has the capability to be impressive, but of course it’s all C++ and any effects I’d want to do have to be possible within OGRE’s framework, which isn’t always the case. I still need to add a proper lighting system and set up a set of shaders for characters and map geometry that look nice and allow for all the different effects I’m likely to need.

October is starting soon, so the whole Halloween atmosphere is starting to creep into grocery stores and newspaper ads around here. If I didn’t have experience showing that it’s incredibly unlikely for such a thing to work *cough*, I’d make a small game for Halloween… I’m still toying with the idea, but I’ll probably be too lazy to actually do it. Plus I’ve got way too many other games I’m supposed to be working on anyway. Maybe I’ll just make some kind of pumpkin-head mod for FE or something.

What has SHilbert been up to?

I feel like I’ve been spinning my wheels on Fallen Elements 2 lately. So, I’m going to try to write a list of what I’ve actually gotten done in the past few months, to see if this is actually the case. Let’s see:

  • Original Object Return (a la wxPython) for scripting. This allows the same Python object to stay attached to any given C++ object over its lifetime, and for C++ objects that do not have a Python shadow class instance yet, a shadow class instance of the correct type is created. This is actually some fairly neat code, I might post it online here in a while.
  • New event system. This makes subscribing to events on a certain object a lot more flexible (no Java-style GroupOfEventListener classes.) It also works from Python in a nice, clean way too. There’s also the concept of EventChannels that makes listening to several objects at once very very simple. There are some issues with it that still bother me, though:
    • Defining event ID’s is a little annoying
    • Binding/unbinding has ugly syntax (might need a macro)
    • It always holds a strong reference to the listeners on the emitter class
    • The old event system (which was much worse) is still in use in certain places.
  • UI system begun. It happens to have a fairly clean design and although there aren’t many widgets set up yet, it’s fairly easy to use from Python. There are some things I still need to fix with the way keyboard input gets fed to it, but it’s basically in pretty good shape.
  • A lot of the physics code was cleaned up and refined. It’s all in a nice pretty phys:: namespace right now, and the Visualizer class that displays meshes of physics objects works using only the event system; it’s very very clean.
  • The ‘fe2′ Python module generated by SWIG is now actually a package; SWIG actually generates a module called fe2.fe2core, which fe2/__init__.py imports symbols from. This means Python-side utilities can show up under the fe2 package without having to hack them into SWIG definition files.
  • The SWIG definition files were reorganized to use proper %includes for the classes they need to build on, instead of having to rely on being ordered correctly by me (which is a pain to keep up with.)
  • The ConfigSystem that was originally implemented in Python (don’t ask) is in C++ now, as it should be
  • Removed binary font formats, we just read AngelCode .fnt files directly now
  • Renamed all the instances of ’4E4′ to ‘fe2′ that I can find, including in repository paths.
  • Have a basic console system coded up; however, it’s not integrated into the game yet
  • More efficient profiling system. It was originally intended to use RDTSC, but it turns out that’s not usable for profiling on my Athlon 64 X2 processor because it’s not synchronized on both cores, so it uses QueryPerformanceCounter.
  • Some of the code is verified to work in gcc again; I haven’t tried to do a compile under Linux since January or so. Now that VMWare Server is free I think it’ll be much easier to do test compiles.
  • Added lots of unit tests to the support code, using CxxTest
  • Clockwork persistence system overhauled to use a much cleverer “flat” persistence scheme, which should make saving/reloading and eventually networking and demorecording much easier on my brain
  • Started maintaining an SCons build script for some of the modules; this isn’t properly set up to compile all the way yet, because I don’t have all the dependencies on my laptop, but it’s not difficult to finish
  • Finally got a nice high-level architectural view of the engine in my head in such a way that I don’t expect it to shift all over the place by next month
  • Preliminary designs for how to efficiently render San Diego while allowing decently fluid motion across a several mile wide area
  • Preliminary designs for a package system that should make working with assets a lot simpler overall
  • Lots and lots of source code cleanup

Okay, well, that makes me feel like at least I’ve gotten something done since my last post. A lot of that stuff is inane obsessive-compulsive type fixes, and it’s largely undirected work even though I theoretically have a milestone I’m supposedly working towards. I haven’t been working on it too intensely in the past month or so, but I intend to get back on it soon.

Oh snap

Man, this is what happens every time I try to use one of these things, I just get lazy and don’t update for weeks.

Uhh… let’s see, FE has ragdolls (xvid). And new computer machine parts are coming via FedEx, which has me giddy inbetween the normal bouts of self-loathing.

That is pretty much it. Not feeling very verbose right now.