[silva-dev] customizability

[Martijn Faassen]

===================================
Silva, Customizability, Flexibility
===================================

In this document I'd like to describe some of my thinking about
customizability of Silva. First I'll go to a list of existing
customizability/pluggability features in Silva. Then I'll describe
some directions we may be going into in the future. Hopefully this
will inspire some discussion and ideas, as well as a vocabulary
of ideas to help us talk to each other.

Customizability and pluggability take on a lot of dimensions, and each
dimension tends to need its own solution. Sometimes a desire for new
customizations appears. We can sometimes extend an existing solution
(extend pluggability API) to make the customization possible in a new
release of Silva. If not, we may be dealing with a new dimension and
we need to consider ways to open it up.

Customizability and pluggability are tied inherently to maintenance
issues. If a pluggability API is maintained in the Silva core and
changes only infrequently and backward-compatibly, plugins can survive
upgrades. Sometimes however the need occurs to change things rather
drastically in the core to unleash a major new functionality; this
does impact the plugins. A good example of this is the changeover to
unicode in 0.9.2. Silva extensions now can break if they try to mix
8-bit strings with unicode strings, where they worked before. It is
all for the good cause of supporting lots of languages in a
maintainable way, however.

Examples of customizability in Silva today
==========================================

Silva extensions
----------------

Extensions are a powerful mechanism to extend Silva's
functionality. They take the form of Zope products and are distributed
that way, but need to be installed into Silva in the
service_extensions to be enabled. Silva extensions allow developers to
create new Silva objects that appear in the add list. Existing Silva
core objects can be subclassed to gain basic functionality. Existing
view facilities can be used to create new views for these objects in
the Silva layout. This mechanism builds on the way Zope is extensible
with new Products.

Authentication and Membership system
------------------------------------

A specific kind of Silva extension can be used to plug new membership
facilities into Silva. This takes care of interfacing with external
soures of user data (such as LDAP) and exposing this information to
Silva. Examples of this information are email address and full name.

By default Silva uses a membership service that stores this
information in the member object in the ZODB. We have also written
extensions to Silva that retrieve this information from other sources,
such as specific LDAP servers.

Silva can use the existing authentication pluggability of Zope to make
it possible to authenticate with something else than the ZODB-based
user information, such as LDAP.

Silva also has a groups component. Other groups services can be
ipmlemented that retrieve their group membership information from
external sources such as LDAP.

Public layout
-------------

While Silva manages the way content is presented, site managers can
change how this content is embedded in the site totally. All the power
of page templates and python scripts can be employed. Silva offers an
API which the writers of public layout can use to access information
about Silva content.

Code element
------------

The code element makes it possible to extend the Silva documents with
new functionality. A code element generates HTML which can then be
included in a document by authors.

File storage
------------

Silva can be configured to store file object information in the ZODB,
but another option is to make it store this information directly on
the local filesystem. This can be configured per branch of the Silva
site.

Metadata
--------

In Silva 0.9.2 it is possible to add new metadata sets to existing
objects. This can also be done from an extension object. Metadata can
also be automatically indexed in the cataloged. This allows a number
of powerful possibilities.

There are some kinks in the metadata system that we are in the process
of working out. We hope to have some improvements from the Python
programmer's perspective landed in 0.9.3.

Import system
-------------

The XML importer system is also pluggable. Importer functions that
know how to import certain XML can be registered centrally. I'm in the
process of revising the XML export and import functionality of Silva.
 
Upcoming customizability
========================

We are working on some other features that exhibit customizability. Some
are new dimensions, some extend different dimensions.

Container index policy (new dimension)
--------------------------------------

Right now Silva objects automatically create an 'index' document
whenever a container such as a folder or publication is created. The
index object is used whenever it is attempted to display the content
of a folder publically.  

In Silva 0.9.3 it will be possible to set a different policy per
publication, so that different index objects can be created. This
makes it possible to create custom index objects that for instance
display the contents of the first published item in the container
instead. The container index policy can also be used to change the
behavior of the 'container title' (also managed by the index document)
to be versioned or non-versioned.

For endless discussion about this topic see issue:

http://issues.infrae.com/silva/issue488

View overlays (new dimension)
-----------------------------

A system very similar to the CMF's ability to customize existing Silva
views. Because of the way the MultiViewRegistry infrastructure
currently works this is unfortunately a non-trivial change. The hope is
that a hack in the right place will make things work, though. :)

This is part of the 'vineyard project' that investigates more general
customization features for Silva.

Internationalization (new dimension)
------------------------------------

The Silva UI is being adjusted so that it is fully
internationalizable. This means that localized versions of the Silva
user interface can be created by translators.

Layout packaging and installation (existing dimension)
------------------------------------------------------

We are currently working on extending functionality so that different
layouts can be packaged and distributed more easily. There will also
be a user interface from within Silva where a specific layout can be
selected. This is the first stage of the 'garden' project.  See issue:

http://issues.infrae.com/silva/issue502

At a following stage we are planning to extend the toolbox for layout
builders significantly. Standard layout components will be delivered
which a layout designer can then use in pages.

Parameters and cachability for code element (existing dimension)
----------------------------------------------------------------

The code element will be extended in the future to allow for
parameters. We will also be able to indicate cachability. See issue:

http://issues.infrae.com/silva/issue270

File storage on external servers (existing dimension)
-----------------------------------------------------

We are considering further extensions to that (large multimedia) files
can also be stored on external servers. Zope is not optimal for
uploading and downloading huge binary files, storing them in the ZODB
is not very useful (filesystem is better), and it is not capable of
streaming.  Other servers are capable of doing so much better, so we
plan to make Silva interoperate with these.

Customizability Futures
=======================

We also have some customizability that is further in the future, and
mainly just speculation at present.

Zope 3 Component Architecture
-----------------------------

Zope 3 has facilities for adapters and views. Adapters make it
possible to attach new functionality to existing objects, much like
the metadata does right now but in a more generalized sense. You could
do this from an extension without having to change the core code.

Views in Zope 3 can be attached to interfaces and also have the
advantage that a new view can be attached to an existing object
through an extension package.

It is even possible to override existing views or adapters from
another package. In case of conflicts the system administrator can
make a decision which goes through. The framework to do this is rather
complicated, however, as I noticed at the last sprint. You might've
heard my grumbling far away. :)

We don't have to switch to Zope 3 for some of this to become usable
from Zope 2, though it is hard to say what form it will take exactly
and when this will happen. I'll continue to follow this, though.

Conclusion
==========

There is a lot, there will be a lot more, and I'm sure there is a lot I
at present forget. :)

Regards,

Martijn