Blendo 2

Motivation

Introduction

In late 1998, the Sony Distributed Systems Lab initiated a research project to investigate the concept of Steerable Media: giving users control of a rich, declaratively authored, multimedia experience. The engine created to execute this content was named Blendo, for its ability to seamlessly blend many media types, such as 2D and 3D graphics, still and moving imagery, video and audio streams into a cohesive experience. Targeted at a range of consumer appliances, from PCs to internet terminals to set top boxes, the design goals of the engine make it appropriate for a wide spectrum of multimedia applications. Furthermore, its VRML heritage makes it an appropriate candidate architecture for the next generation of that technology.



Blendo is a declarative system for describing interactive objects using a variety of media types. In spite of its declarative nature, Blendo includes a powerful built-in scripting capability as well as accessibility to external procedural engines such as Java. Blendo also provides a mechanism for integration with page description languages such as HTML or XML. This makes it appropriate for a wide variety of multimedia applications, from consumer appliances to large simulations and multi-user games.

The Design of Blendo

The Blendo architecture was designed to satisfy a number of requirements. Some of these have been derived from the design goals of VRML 97. Others are a result of a VRML community survey about perceived requirements for the next generation of that technology. Still others have been derived from one of the foundation concepts of the Steerable Media Project itself.



"Our development process will consist of three interleaved phases. We will develop content scenarios that we feel are compelling in this space. Then we will design a platform on which to run these content scenarios. We will implement this design and the content scenario. We will use what we learn from this experience to modify the scenario and then the design." - Steerable Media project summary



The design goals presented here are broken into categories which represent the major capabilities needed for a powerful multimedia authoring solution.

Compatibility

Compatibility VRML97 is of primary importance. In addition to leveraging existing content, it also leverages knowledge gained by authors. But due to the fact that VRML97 contains a small number of under specified features, full compatibility is not possible. These requirements attempt to balance the two:



Common syntax - Where possible, the VRML97 textual format shall be maintained.

Migration - Where VRML 97 syntax leads to poorly defined constructs, syntax changes are allowed. To allow easy migration, tools shall be provided to convert as large a portion of existing content as possible.

Transparency of conversion - In some profiles, migration tools will be transparent, providing VRML 97 compatibility for as much content as possible.

Support for JSAI and EAI - At some profiles and levels it shall be possible to run EAI and JSAI based VRML applications unmodified.



Core Runtime

Some applications of VRML require a browser with a relatively small footprint. Others require a large amount of functionality. It is important to meet both of these seemingly contradictory goals with the core runtime engine. The core provides the underlying functionality for the entire system. At the lowest profile levels, the core is very small and grows as profiles are added and levels increase.



Small Core - The core runtime engine must be small at the simplest profiles and lowest levels.

Extensible Core - It shall be possible to add functionality to the core to increase its capability without breaking the existing runtime features.

Profiles and levels - The author must have a mechanism to indicate the capabilities of an incoming file or stream, to allow browsers to adaptively load capabilities or to reject a file containing unsupported functionality. This must allow the indication of a set of features (profiles) as well as the functional level (levels) of each feature set.

Basic types - A set of basic types shall be available at all levels and profiles. These include a base object from which all nodes and fields may be extended, integral and floating point numbers including a double precision floating point number, strings and boolean values.



Scalability

VRML 97 strives to be a scalable technology. But this is a goal that has never been fully realized. Blendo attempts to solve the technical problems involved and give authors the tools necessary to create very complex and/or very media rich environments:



Load control - There shall be mechanisms available to the author to control when a given asset is loaded. There shall also be a mechanism to allow the explicit unloading of a given asset.

Load notification - There shall be a mechanism for informing the author of the load status of a given asset, including an indication of when loading is complete or has failed.

Preloading - Mechanisms shall be provided to allow an author to indicate that the use of some asset is imminent. This gives assets with a finite startup latency time to become ready when needed.



Authorability

One of the most important successes of VRML has been its ease of authoring. A VRML text file is easy to author using a simple text editor and the composition capabilities allows a world to be made up of many separate files. The ability to write programmatic behaviors in an easy to use scripting language has also been a great benefit. Blendo will extend this capability to make authoring even easier and giving authors even more powerful tools.



Composability - A mechanism shall be made available to authors to compose a final scene from elements located in many separate files and streams.

Consistent representation - A file shall be usable as a root scene or included as an element of a larger scene. The mechanism for inclusion shall be the same whether the element is included directly or as a file.

Element interfacing - The mechanism for composing elements into a larger scene shall have the ability to provide a field interface to each element.

Textual editing - At some profiles and levels an easy to author, VRML 97 like text format shall be made available to authors.

Built-in programming - A built-in scripting language, tightly integrated with the file syntax, shall be provided at some levels and profiles.

Consistency - There shall be a high level of consistency exhibited in content run across multiple platforms and multiple runtimes. This includes high fidelity of the rendering appearance as well as consistent behavior execution.

Metadata - A mechanism for adding author supplied data to a given object type shall be provided. It shall furthermore be possible to perform a variety of search operations on this data.



Object Model

VRML 97 suffered from an incomplete object model. This made extending the functionality of existing nodes difficult and prevented the implementation of a strong typing system. For instance, the SFNode class did not have encoded into it any notion of allowable node types. Blendo will correct this shortcoming to provide a more authorable declarative environment.



True object model - It shall be possible to create a new object with characteristics derived from a given object or objects. This new object shall present the appropriate interface of the derived objects to its clients. The implementation of these interfaces can be obtained from the derived objects or directly from the new object.

Common objects - The system shall allow the creation of new field types in addition to new node types.

Typed node fields - A node field shall require the specification of the object types allowed in that field. Allowable objects are the specified object as well as any object derived from it.

Common type querying - Whether an object has inherited the implementation of a given object or just its interface, there shall be a common mechanism for querying a supported interface.

Object hierarchy - A profile and level comprising a given set of objects shall fully specify the hierarchy of those objects. New objects must be able to inherit from any object in this hierarchy.

Namespaces - A notion of namespaces shall be used to allow access to named objects in various elements of the scene. A mechanism for disambiguating namespaces in both the file formats and at runtime shall be provided.



Event Model

The VRML event model is problematic for authors because the order of event propagation is undefined and several steps of the event sequence are deferred until later in the event cycle. This makes it difficult to create predictable behavior in complex event networks. Blendo makes the event propagation mechanism simple and predictable with both syntactic and semantic modifications.



Determinism - For a given set of inputs, the path through the event graph and the order of event execution shall be deterministic across all implementations.

Simplicity - The event model shall be simple. It shall be possible to handle events with minimum runtime overhead. The event model must also adhere to the rule of least surprise (an action resulting from an event should be easy for an author to understand).

Author control - The author shall be able to exert control over the event model to change its behavior. The author shall be able to intercept the processing of an event in order to defer or reorder it.



Extensibility

In addition to the notion of profiles and levels, used to tune the size of a runtime engine for a given application, the author must have the ability to add custom programmatic functionality to a specific content piece. This allows the author to provide features not present in the runtime or to provide enhanced performance for a specific function. Throughout this document this feature is referred to as native extensibility, regardless of the language used:



Common extensibility mechanism - Regardless of the language used, a given extension shall appear the same inside the runtime. A natively implemented object shall present the same interface to the system as an object created directly in Blendo.

Common subclassing - It shall be possible to create a new object by deriving existing built-in or authored objects, regardless of the language used to implement either. For instance, if the system supports both ECMAScript and Java, an authored object written in Java can be derived by an object written in ECMAScript.

Language optimizations - A given implementation may provide optimized mechanisms for creating an extension using a given language. For instance, an implementation might provide a Java hierarchy of classes to make implementing extensions in that language especially efficient.



Programming API

VRML has a very limited set of programmable API's. Authors need a more powerful set to allow the implementation of custom functionality and to exert more control over the system. Blendo attempts to solve this with a richer set of procedural interfaces and a more efficient and more tightly integrated built-in scripting mechanism.



Built-in language - The core, at some profile level, shall provide a built-in and fully integrated language. This language shall be lightweight but must provide full access to the underlying API.

Common API - Whether the built-in language or a supported native language is used, the same API functionality shall be provided.

Implementation dependence - Some supported native languages (such as Java) shall have a well specified API as part of this specification. Others (such as C++) may have an implementation dependent API.

Introspection - The API shall allow access to components of a given node. This includes access to the type, name and value of fields, the DEF name and bounding box of a node (if appropriate). For certain object types access to the dictionary of exported DEF names and prototype names shall also be made available.

Completeness - For a given profile and level, it must be possible to implement every object contained therein with the corresponding API. For instance, a profile containing an IndexedFaceSet node would contain an API to provide programmatic access to an underlying renderer for this shape. This would allow the author to, for instance, prototype new nodes with similar functionality to that of the IndexedFaceSet.

Object specification - The implementation of each object of a given profile and level, using the built-in language and the API of that profile and level, shall be provided as part of its specification. Given a runtime engine supporting the API of a given profile and level but without any of its objects, it shall be possible to execute any appropriate content using the object implementations contained in the specification. As a compliance verification, it shall be possible to configure the runtime to perform this test.



Media Integration

VRML 97 had a very simple media model which had difficulty dealing with the streaming aspects of some media types. Blendo strives to solve these problems by providing a rich and common base for audio, video, animation timing and still images. Additionally Blendo will provide complete integration of all media types, including 2D and 3D graphics, audio, video, still images and application generated elements:



Synchronization - It shall be possible to specify a common time base for a set of media to allow for very high quality synchronization of audio, video and animation.

Streaming formats - Each profile and level shall have a well defined set of supported formats. The format of the resource descriptor for a given stream type shall also be well defined.

Media tracking - Each timed media element shall provide a mechanism for tracking its current temporal position.

Surface rendering - It shall be possible to render a subset of scene elements to a separate 2D surface for further composition as a texture or other 2D element.

Application rendering - At some levels and profiles it shall be possible for the main rendering loop to use surfaces rendered by other applications, possibly in other threads or process spaces. There shall be well defined mechanisms for synchronization between these renderings.



Page Integration

A large body of work has been done since the creation of VRML in the area of integration of various media types in a document. Blendo uses these concepts to allow both the direct integration of Blendo elements in a XML document as well as interaction between the page and the Blendo scene using DOM. Blendo also attempts to improve the integration of 2D geometric elements into the 3D scene.



Common support - There shall be some level of page integration at every level of the runtime intended for use in a HTML or XML environment. This shall include passing events and values to and from the page and may include integrated rendering of HTML or XML content inside a Blendo scene.

Runtime integration - The page integration feature shall handle value changes at runtime intended for the scene. Also, events sent from the scene intended for the page at runtime shall properly affect the rendered page. For instance, if an interface is provided to change the color of a body of text, an event sent from the scene at any time shall signal the page to appropriately affect the text.

DOM integration - The W3C Document Object Model shall be used to affect the interface between the page and the scene.

Encapsulation - At some profiles and levels it shall be possible to represent rendered page elements inside a Blendo scene. DOM interfacing shall be used for communication between the two.



Sensory Apparatus

VRML 97 provides a rich set of sensors. Blendo will improve on this set and provide more capability to existing sensors:



API access - To give the author the ability to more finely control the operation of a given sensor, its underlying API shall be exposed. For instance, a PlaneSensor shall provide a scene picking API, as well as transformations to convert between a 2D and 3D coordinate system.

Collision - As mentioned above, a profile providing some declarative collision sensing capability shall also provide an API to this functionality.

Full set of sensors - Sensors and/or API's shall be provided at a given profile and level to provide full access to the set of user input devices provided on a given platform. This means access to keyboard events as well as alternate mouse button presses and other user valuator devices (e.g., joysticks).



Advanced Rendering

In recent years many advanced rendering techniques have become available to cost effective hardware. Blendo will allow these capabilities to be used when available:



Rendering support - At some profiles and levels, advanced rendering capabilities will be made available to authors, in both declarative and programmatic forms.

Graceful degradation - There shall be a well defined behavior on every platform for a rendering feature not available on that platform.