eZ component: Webdav, Design, 1.0 - Extensibility
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:Author: Kore Nordmann, Tobias Schlitt
:Revision: $Rev$
:Date: $Date$
:Status: Draft

.. contents::

=====
Scope
=====

The scope of this document is to define the extensibility features of the
Webdav component. It does not affect the current design provided in
design.txt, but only extends this one with design details.

This chapter gives an overview about the state of the Webdav component while
this document is created and a high level view about the problems faced to
make the presented solution necessar and a high level view about the problems
faced to make the presented solution necessary.

-------------
Current state
-------------

The current state of the Webdav component is, that a fully running server can
be built as a "proof of concept". We successfully have run 2 different clients
(cadaver and Nautilus) on such a server and recorded them as test suites. We
also pass the largest amount of tests in litmus, a test suit for WebDAV
servers and have a test suite generated out of examples from the RFC.

=========
The issue
=========

This document takes care about different aspects of extensibility of the
Webdav component. To design extensibility features, first an analysis of the
already existing features and an examination of the requirements are
necessary.

Current features
================

Currently the component allows to replace the backend and transport objects of
a server. The inheritence tree of the transport classes is meant to reflect
client specific adjustments, which is already an extensibility feature.

The backend is completly custom implementable and must follow a certain set of
interfaces to be fully compliant with the base RFC. Backends can be extended to
add additional features, since inheritence is not taken as a matter of
extensibility here.

The server layer does not provide any extensibility features. First off,
because it does not exist, yet. Second and more important they are not
defined, yet.

Requirements
============

The requirements are defined by 3 different issue groups in this feature
set:

- RFCs
- Custom extensions
- Tieins

All 3 groups will be analyzed shortly here:

RFCs
----

There are a lot of different RFCs that extend the Webdav layer with additional
functionality like versioning support and authorization features. We will not
be able to implement any of these in the first stable release of the
component, but must define the API these may use to extend the current feature
set. A good `overview of the RFCs_` related to Webdav can be found at
GreenBytes.

.. _`overview of RFCs`: http://greenbytes.de/tech/webdav/common-index.html

The range of different points where a possible extension affects the basic
Webdav component classes is huge. This includes even the introduction of new
request methods, which need additional parsing in the transport, handling the
backend or any other module and possibly new response classes, which need
serialization code in the transport.

The requirements set by additional RFCs affects all layers of the Webdav
component and needs to access almost any functionality. At least, hooks in
ezcWebdavTransport and ezcWebdavServer are needed.

Custom extensions
-----------------

The Webdav RFC recommends custom extensions of the protocol so we will
definitly have to face the issue, that we need those on our own or someone else
needs it.

A custom extension usually adds custom properties to existing
requests/responses, which does not require any interaction at all, with the
current design. Dead properties and unknown life properties are simply parsed
and passed to the  backend for storage. The problem here comes into turn, when
these additional properties should be associated with functionality.

There are different places where we might allow access to properties, for
example while parsing and serializing them during request/response to perform
conversions as we do for life properties. This migh even lead to the invention
of new property classes.

Functionality would be more likely be placed on the server level, if no storage
functionality is required or this functionality can be performed using the
standard communication API. Or on the backend level, if heavy storage access is
required. The latter decision most likely limits an extension to a certain set
or even a single backend.

Anyway, functionality for custom extensions must be, as with RFCs, provided on
every layer of the component. Hooks in ezcWebdavTransport and ezcWebdavServer
are mandatory, hooks for ezcWebdavBackend are desirable. The latter one can
also be handled by inheritance.

Tieins
------

eZ Components already provides a mechanism of adding functionality to a
package. This mechanism is useful for both cases described before, since we
might offer an Authentication tiein or a Template tiein, which will realize
different RFCs. Beside that, 3rd parties can use the Tiein mechanism to provide
their extensions.

Summary
=======

There are 3 issue fields that require the Webdav component to have a very
flexible and extensible API. This affects all 3 layers of the component, while
in the transport layer the possibility to use inheritence is already taken by
the need to react on missbehaving clients.

============
The solution
============

The solution to the described requirements is to define a plugin API similar to
the terms of `Aspect Oriented Programming`_. This new way of extending the
object oriented programming paradigma allows you to hook into different layers
of a program with common functionality.

.. _`Aspect Oriented Programming`: http://en.wikipedia.org/wiki/Aspect_oriented_programming

We call the whole system designed here a "plugin system", therefore the packages
that provide extended functionality are called "plugins".

Basics
======

This section should clearify basic terms and the basic idea of the plugin
system.

--------
The idea
--------

The basic idea is, to invent a global plugin system, which offers hooks for any
imagineable functionality to offer extensibility support for the Webdav component.

A single object, which takes care about the dispatching of plugin hooks. This
very specific class is not extendable and an instance might only exist once per
request (ezcWebdavServer instance). For now we call this object the plugin
registry.

This registry takes care about the management of extensions all over the
component. It allows the user of the component to add extension packages to the
registry, which then interact with this to influence the Webdav component in
any layer.

The registry knows about the hooks available throughout the component and
dispatches them centrally, if a hook is signalized by a layer of the component.
Each time a hook is announced by any of the layers, the registry dispatches
this information to every attached callback. Hooks can be specified to send a
number of arbitrary parameters to the attached plugin methods and can expect a
return value for further processing.

A hook may have any number of callbacks from several extensions assigned, which
are processed in order of registration, when a hook is announced. Therefore,
the plugin developer is encouraged to perform only such manipulations on the
objects received by parameters, that are harmless for other extension and
especially the basic RFC implementation.

-------------------
Definition of terms
-------------------

Before a detailed description of the proposed design can follow, some terms
need to be clearified in this area. Terms explained in this list are are
written as capitals.

Plugin-System
  The Plugin-System is the part of the Webdav component that should be designed
  in this document. The Plugin-System will take care about Plugins, that can
  be installed via tie-ins or be customly developed. The Plugin-System ensures
  that the component stays flexible without allowing its basic API to change
  and keeping other extensibility levels for different purposes.

Plugin
  A Plugin is a package of classes, that provides optional new functionality to
  be used with the Webdav component. A Plugin can be configured by the user
  of the component through a central instance of the main server: The
  Plugin-Registry. All communication is handled through this class. A Plugin
  may consist of any number of classes and must contain 1 certain
  Configuration-Class, that follows a specific interface and implements the
  Registration as well as the Initialization of the Plugin.

Plugin-Registry
  A single instance of the Plugin-Registry care of managing plugins and
  dispatching hooks to them. A Plugin informs the Plugin-Registry about
  the Hooks affected by it during Registration. The Plugin-Registry stores this
  information and calls all registered specific callbacks if a certain Hook is
  announced. The Annnouncement is provided by the classes of a certain layer to
  the Plugin-Registry, including arbitrary Parameters. The Plugin-Registry
  forwards these parameters to the callbacks registered for the specific hook.

Registration
  When a user installs a Plugin, it does not automatically register itself to
  be used with any ezcWebdavServer instance, since any plugin will need a set
  of configuration values and simply installing a plugin does not mean to use
  it in every server. Therefore the Plugin must implement a process of
  Registration as soon as a user adds its configuration to the Plugin-Registry.
  During Registration a Plugin must inform the Plugin-Registry about the Hooks
  it wants to subscribe to.  Registration is performed by the Configuration
  class.

Initialization
  When a Plugin has performed successful configuration and Registration, it
  will be notified by the Plugin-Registry about any Hook it is subscribed to.
  Before any of these Hooks are populated, the Plugin-Registry will call a
  method for Initialization on each Plugins Configuration-Class. This allows
  the plugin to instanciate required objects and initialize values.

Hook
  A Hook describes the reaching of a certain point in th code. A Plugin may inform
  the Plugin-Registry that it needs to be informed about a certain Hook. When
  the point of code is reached, the Webdav component will pause processing and
  send the Announcement for the Hook to the Plugin-Registry. The
  Plugin-Registry will then dispatch the hook to all attached Callbacks and
  return controll.

Configuration-Class
  Each Plugin must provide exactly 1 main configuration class. An instance of
  this class might be created by the user and submitted to the Plugin-Registry,
  to enable the plugin. During this, a method on the configuration object
  (instance of the Configuration-Class) will be called that makes the Plugin
  register all necessary Hooks. Before any Hook is disptched, another method
  will be called for Initialization.

Callback
  A Plugin may attach any number of Callbacks to any number of Hooks through
  Registration at the Plugin-Registry. Each Hook defines a concrete Interface
  of Parameters, that the callback assigned to it must fulfill. Beside these
  restrictions, a callback is defined in the usual terms of the abstract PHP
  datatype. A callback will be called when the Plugin-Registry receives a Hook
  Announcement.

Interface
  A Hook behaves like a method, but the other way around. It defines the
  signature of the Callback assigned to it and therefore its Interface. The
  Interface consists of the Parameters submitted, when the Hook occurs. Hook
  Interfaces are not defined using PHP code and phpDocumentor syntax, but an
  RST document.

Parameters
  A Hook may specify any number of parameters that will be submitted to the
  attached Callbacks when the Hook occurs. The assigned Callback methods must
  accept these parameters. A Parameter may be defined to be read/write or
  read-only. The Plugins assigned to a Hook must accept these rules and may not
  violate it. A malicious Plugin can easily destroy or even silently exploit
  the server, so only trusted sources should be used.

Announcement
  The Announcement of a Hook is performed by any of the layers of the Webdav
  component and send to the central Plugin Registry. This instance dispatches
  the Announcement, including Parameters, to the Callbacks assigned to the
  Hook.
  
API
===

As the basic section already introduced, several elements will design the API
of the Plugin-System. There are 2 major goals for this design: 1. The
extensibility of the component. 2. The consistence of its internal API. The
first part will be ensured by the Hooks defined in the second part of this
section. The second goal will follow now.

-----------
Inheritence
-----------

The current communication between the layers is kept as small as possible and
each layer uses independent objects. To ensure this, the base interface on each
level will define a certain set of public final methods, that handle the base
communication and dispatching.

Inside these methods, which will perform the main dispatching tasks of each
layer, the hooks of the Plugin-System will be established. Only there the
Plugin-Registry will be informed about Hooks.

This will ensure the stability of 2 points in our API: 1. The communication
between the 3 layers and their affected classes. 2. The Plugin-API. And this
will not be affected by inheritence.

Inheritence can still be used on any level of the component to perform other
specialization tasks, except for the Transport layer, where inheritence is
already taken by the fact of client specific adjustments.

-----
Hooks
-----

The Hooks of each layer are defined in the Plugin-Registry and are not
influenceable by any external mechanism at run time. The final/private methods
of each layer dispatch these Hooks to the Plugin-Registry, the necessary
parameters attached. The Plugin-Registry will then perform all necessary
callbacks in the order of their registration.

As decided after discussion, hooks will only be offered by the layers Transport
and Server. The Backend layer is to specific to offer any hooks, except for
addition of new processing instructions for new request types. Those can also
be dispatched by the Server layer, since all request and response objects pass
this one before/after being processed by the Backend.

Callbacks issued by hooks may issue any public API call that is defined by the
Webdav component. This way it is possible for plugins to perform any task.
Plugins need to rely on APIs defined in interfaces and base classes, to
interact with e.g. the backend or the transport layer, to ensure highest
compatibility with the component and other plugins.

Transport
---------

The Transport layer (represented by the ezcWebdavTransport class and its
children + utility classes) will issue 3 different types of hooks:

- Request hooks (parse*Request and parseUnknownRequest)
- Response hooks (handle*Response and handleUnknownResponse)
- Property hooks (beforeExtract*Property and afterExtract*Property)

Request hooks
^^^^^^^^^^^^^

For the request section, a hook will be offered for each HTTP request method
defined in `RFC 2518`_. This hook will be invoked right before the request is
parsed and will receive the parameters of the accordingly named protected
method on ezcWebdavTransport, which is an additional orientation for the
developer using the plugin system. For example, the hook "parsePropFindRequest"
will be announced before a PROPFIND request is parsed, given the raw URI and
body as parameters.

.. _`RFC 2518`: http://tools.ietf.org/html/rfc2518

This hook will commonly be used to extract custom XML from the body of a
request or to check custom headers. The manipulation of the XML and headers has
possibly effects on the base component or other plugins and must be taken
highly serious by plugin developers. But it might even be imangineable that a
plugin fixes certain XML errors, removes or adds XML elements.

An additional hook will be provided to hook into the parsing of unknown
requests. This hook will provide the HTTP method name in addition to the data
of the hooks presented above. The first callback that returns a valid
ezcWebdavRequest implementation will be the last one executed and the request
will be dispatched to the server.

.. note: This behaviour is not recommended, if the request method might be used
   by other clients or even possibly in the future, too. In this case, a plugin
   developer should consider to store the information he needs internally and
   let others handle this request properly, too. The server layer hooks,
   introduced later, will give possibilities to catch up with desired and
   undesired behaviour and will even allow the plugin to issue requests on its
   own and handle them.

Response hooks
^^^^^^^^^^^^^^

For response hooks, a very similar behaviour is realized by the transport
layer. A hook is realized right before the final display information is send
out, for every handled response object. It is possible at this stage for the
plugin manipulate the headers and body of the response to be send to the
client, right before this happens.

At this stage, the plugin developer could (theoretically) even replace the
display information struct with a different implementation (e.g. an empty
response with a xml response). Therefore, plugin developers must take
manipulations here extremly serious.

As with requests, an additional hook will be provided for all responses that
are unknown by the transport layer. The first callback returning a valid
display information struct will be used to satisfy the response. A plugin
should define its own response classes for this purpose, to not collide with
other plugins or the base component.

It is possible to dispatch additional responses to the currently active
transport and retrieve back the display information generated trough the public
API of the component. This way, a plugin can rely somewhat on the specialities
of each client in addition.

Property hooks
^^^^^^^^^^^^^^

The property section affects both previously mentioned ones, since it affects
the parsing and handling of properties. The hooks there are devided into
before and after hooks, as well as hooks for live and dead properties. Live
properties are those that need generation and validation by the server while
dead properties are simply stored by the backend.

The before hook is announced directly before a property is parse, just like the
request hooks, the after hook is announced directly after the mehod for parsing
a property is called. An example: The hook "beforeExtractLiveProperty" is
called as soon as any request or response requests to parse a property in the
DAV: namespace and receives the affected DOMElement as a parameter. The
corresponding hook "afterExtractLiveProperty" receives the result of the method
call, which is either a live property, recognized by the sever, or a dead
property. In the latter case, a plugin is allowed to replace the dead property
with a live property by returning the new object. Processing of following
plugins attached to this slot will be given the new property as the parameter.

The same API will be realized for the before/afterExtractDeadProperty. The
second range of hooks provided for properties: The serialization hooks. As with
extraction of properties, before and after hooks are to be implemented here.
The "beforeSerializeDeadProperty" hook will be called right before a dead
property is going to be serialized. The dead property will be received as the
parameter.  The "afterSerializeDeadProperty" hook will be announced right after
the property has been serialized, given the resulting DOMElement to the plugin
callbacks.

As for requests and responses, there are special hooks for live properties,
each for extraction and serialization: The hook "extractUnknownLiveProperty" is
announced as soon as a live property was attempt to be parsed and this attempt
failed. The first plugin returning a live property here will make the race. But
beware, the live property must pass the after parsing hook possibly, after it
was parsed here. In case no plugin reacts, the base component will parse a dead
property in the DAV: namespace, so this could still be replaced in the after
hook.

.. note: Using custom live properties is not the recommended way of storing
   data. Dead properties suite this need as well as live ones and are generally
   handled.

Similar rules apply to the corresponding "serializeUnkownLiveProperty" is
announced as soon as the transport layer receives a live property it can not
serialize to XML. The first plugin callback that reacts on this hook will make
the job. If no hook attempts to react on this hook, an exception is thrown.

.. note: When a Plugin attaches to a hook of the Transport layer, it might
   not expect anything about the client it is talking to, but needs to inspect
   the User-Agent header itself, if it attaches to new request methods or
   reponses.

Server
------

The server will provide a very limited number of hooks:

- Request received (receivedRequest)
- Response generated (responseGenerated)

The receivedRequest hook is reached each time after a request object is
returned from the Transport layer. A Plugin may attach here, to perform any
number of operations. For it's internal usage, it may issue public methods on
the Server and Backend layer (and even on the Transport layer, although this
might be of raw use).

Each hook attached to this slot may manipulate the request object or issue new
requests. All those generated requests SHOULD again be passed to the hook, to
be optionally processed by other plugins following the current plugin in the
call order. If no more plugins are left, which could handle or modify the
request, it is passed to the backend and the response goes its way back up
through the plugins. This enables plugins to issue multiple requests to the
backend and generate subsequent requests based on the responses of the server.

In some cases a plugin MAY pass the request directly to the backend, if it
certain, that no other plugin will ever has anything to do with this request
type, this mechansim should be used with care, but may be useful to request
custom dead properties from the backend, for example.

If some request is not affected by a plugin, which registered for the hook, it
may just return the request, so it will be finally handled by the default
handling mechanism in the server. Each plugin MUST return a last request object
to be handled by the default mechanism in the server.

The seconde hook (responseGenerated) will allow a plugin to be informed,
whenever a response was generated by the server. This hook is the preferred
form for a plugin to interact with responses of commonly known requests (e.g.
GET, PUT, PROPFIND).

.. Note:
  The previously explained receivedRequest hook can be used to gather request
  information and a before hook from the Transport layer might be used to parse
  custom XML information and attach it to the request object with the after
  hook.

---------------------------
Base infrastructure changes
---------------------------

There is 1 point where the base infrastructure must be changed. For now we
asumed that it is enough to attach unknown XML tags to properties and
requests. This is a) not realizeable as it turned out and b) not valueable.
Therefore, we will remove this base class (ezcWebdavXmlBase) and replace it
with a new one.

This new class will provide a common property for every request, response and
property class. The property will be generated on the fly, when it is requested
once and contain a collector object, that allows the storage of any arbitrary
data in the plugins namespace.

The mentioned namespace must be provided by each plugin to identify data of
that plugin uniquely in the base storage and to avoid conflicts with other
plugins. Data contained in this storage will not be affected by any other part
of the system except for the plugin itself.

================
Proof of concept
================

To proofe that the above described concept works for extending the Webdav
component should be to implement the locking facilities described in `RFC
2518`_. This functionality was orginally meant to be part of the base package
and to belong into the Server layer. Since it is connected to parsing
properties, parsing different new requests and old ones and because it needs
to issue multiple internal requests to the backend to perform its work.

The locking facilities described in `RFC 2518`_ require the usage of almost
every part of the plugin system. The LOCK and UNLOCK requests (currently parsed
by the Transport layer) are added as new requests and need to be handled as
such in the Server layer hooks. The handling requires to issue new requests to
the backend for setting/getting properties.  Beside that, locking affects
multiple requests known by the base class, like parsing live properties and
reading headers. The exact requirements for locking can be extracted from the
RFC overview document.


..
   Local Variables:
   mode: rst
   fill-column: 79
   End: 
   vim: et syn=rst tw=79
