virtual world rant# 29

Some thoughts about virtual worlds, avatars, prims, mesh, and maybe a little politics.

Lately I've been spending some time working on openviewer (www.openviewer.org), a project underway to build a BSD licensed virtual world client. Initially, the target server platform is SL/opensim. My goal for the project was just to have a sandbox to learn about the client side of things, without 'tainting' myself with the GPL code from Linden (their viewer is licensed under GPL, which is incompatible with opensim's less restrictive BSD license).

Anyway, what this has allowed me to do is get deeper into the technical underpinnings of physics, animation, and client/server relationship in this architecture. That in turn has led me to refine and focus my ideas about 'existents' and such, and the general issue of making virtual worlds safe for AI-controlled v-bots.

The upshot is that modern game engine front-ends (the graphics library part; we're using Ogre, but I suspect this applies generally) -- have a set of features that has evolved over the last 20+ years to include all the tools necessary to visually represent a world that looks something like the 'real' world, or at least looks pretty damn cool, and appeals to human perceptual sensibilities. Starting with the basic atomic 3D building block, the triangle, these engines have built up a vast set of capabilities, such as texture mapping, shading, deformable meshes, 'skeleton' animation, and other gizmos, which collectively form the palette that designers can access.

The kinds of things you can simulate with this bunch of tools runs the gamut of things we experience in real life. Assuming we're outdoors, we start with terrain, sky, and sea. Then we add fixed forms, both natural (plants, vegetation), and artificial (buildings, roads, etc). On top of that, we have inanimate objects that move about -- vehicles mostly, but they could be anything, like a ball, a door, or a (mechanical-looking) robot.

Finally, we move on to living things -- animals and humans (not to quibble; obviously humans are animals, but they get pretty special treatment because of their status within most applications). Humans (and human-like creatures, ie trolls, monsters, aliens etc) have been the recepients of a massive amount of design and implementation refinement. 'Skin' (deformable mesh) is stretched over 'bones' (linked sets of rigid sticks), with complex mathematical relationships so that animation (generally derived from motion capture) will look natural. Then you have clothes (algorithmic mesh), hair (various approaches), and finally additional inanimate accessories (attachments, in SL parlance) such as jetpacks, earpieces, glasses, etc and so on.

The point of going over all this is to focus on the fact that these graphics engines are not really general-purpose reality simulators. They have some pretty specific features that are for the most part tweaked to perform functions that relate to specific elements that we might want to visualize in the context of virtual reality. Actually, the game engines themselves are usually quite flexible; you don't have to have a humanoid skeleton -- you can put the bones together in any strange way you choose. You can utilize the various tools in ways not envisioned by the designers of the engine, as long as you have the resources to code and test it.

On top of these grahics engines, we have (in the case I'm analyzing) one more layer -- the client/server foundational architecture, starting with Linden's Second Life protocol, and presently being extended by folks like RealXtend. Obviously there are other platforms (multiverse, wonderland, Qwaq...) but this is the one I'm familiar with, and frankly it looks like the one free and open source platform that is getting enough traction to have a good shot at becoming some sort of defacto standard. Anyway that's a different discussion; for now, it's the case I'm familiar with, and I'm willing to bet the others have a similar set of features and limitations.

What Linden did with Second Life is to make some pretty deep assumptions about what kinds of entities can exist in their virtual worldscape. At the top of the heirarchy is the 'avatar' -- a nominally human form, customized to varying degrees by users. These entities have a special status, and (shades of the old nobility) sometimes their status includes restrictions on what they can do (kind of a karmic quid-pro-quo maybe).

On the other side of the tracks, we have what Second Life calls 'prims' -- primitive shapes with varying adornments; potentially linked together in complex ways, and possibly scripted with computer code.

It might be helpful to spell out in list form what the various capabilities and limitations of these two entity types are:

Avatars:
controlled by: human users or 'bots' (pseudo-users running a program that acts like a client)
Motion and behavior: animations (.bvh), simple physics (capsule-based)
basic form: skeleton + deformable mesh, human-based
visual enhancements: clothes, hair,
special status: name, groups, friends, ownership (inventory)

Prims:
controlled by: server-side scripting (could be plug-in for opensim)
motion and behavior: scripting and more realistic physics (all parts can be physical)
basic form: sets of geometric shapes with algorithmic modifiers (cuts, shear, rotation, taper etc)
visual enhancements: custom textures, particle systems, special lighting (?)
special status: owned by regions &/or avatars

In general terms, when you see something that looks vaguely human, and it talks to you, and buys stuff, it's almost definitely an avatar. If it looks like a vehicle, or a machine, it's almost definitely prim-based.

The problem I have with this state of affairs is this. My interest in virtual worlds revolves around the idea that, eventually, we might be able to make major advances in AI, by leveraging the co-existence of human and computer-controlled entities in the same world. However, platforms like Second Life have been developed with the animate/artificial distinction so deeply baked in, that this goal becomes somewhat difficult, if not impossible.

Getting concrete, what I want in a virtual world system is as follows:

* Whether you are controlled by a human or a computer should not have a major impact on your capabilities (modulo issues of privacy and transparency -- social issues are real, but the technology should not dictate them).

* Computer-controlled entities should be able to look like humans, animals, aliens, robots, or anything else the graphic engine could conceivably generate

* Human-controlled entities should be able to look non-human, be fully physical, and use low-level control within the parameters of a realistic simulation (as opposed to relying solely on pre-cooked animation sequences)

* There shouldn't be a-priori distinctions between clients and servers. Issues of trust and access should be generalized, and dealt with in a rigorous and flexible manner. Any process that can possibly be distributed should be able to be distributed among an arbitrary number of machines with arbitrary network topologies

* Animation and physical simulation need to be seamlessly integrated. There are techniques for doing this at the bleeding edge of game design and movie-making. The hooks for these capabilities need to be in the protocol.

* All of this has to be done in such a way that today's model of entertainment-based games and worlds is not negatively impacted (otherwise no one will care, and my ideas will go nowhere)

Sometimes I feel as if I'm pissing into the wind here. Who really cares about this stuff? Very few people, from what I can tell. There are a few more who vaguely understand what I'm getting at, but have their own work and agenda, so can't really be bothered to help beyond just saying "yeah, that sounds nice". I have so far failed to really make the case that this stuff matters, and so it's falling on deaf ears. Perhaps a tree needs to fall down and get itself back up multiple times, before anyone in the forest happens to notice.

-dan