An Approach for Evolving XML Vocabularies
Using XML
Schema
Noah Mendelsohn
IBM Corporation
June
15, 2004
ROUGH DRAFT: DAVID HAS ASKED ME TO GET THIS OUT ASAP. I’LL PROOFREAD TOMORROW AND REPUBLISH IF
NECESSARY. COMMENTS ARE
WELCOME. THANK YOU.
Introduction
At the May, 2004 face to face meeting of the XML Schema WG I
discussed some ideas for managing evolving XML vocabularies using XML
schema. This note is an attempt to
summarize that proposal. These
ideas remain at best incomplete, and they do not necessarily represent the
preferred approach of my employer, IBM.
I do hope they are helpful in facilitating workgroup discussions of XML
versioning.
Assumptions & Rationale
In this exposition, the term “vocabulary” is used broadly and
somewhat informally to refer to a user-defined XML language that is used for
some purpose or other. It is
assumed that a given version of such a vocabulary will be describable by some
XML schema that can validate element information items conforming to the
vocabulary. Though informal, this
definition should suffice to introduce the design. Indeed, some of the characteristics of a
vocabulary (I.e. what can be validated, what can be versioned, etc.) become
clearer from the design itself.
Note that a vocabulary is not necessarily a single namespace, and a
single namespace may embody all or part of more than one vocabulary. Changes to a vocabulary may or may not
be made using the same namespace(s) as the original.
There seems to be wide divergence of opinion as to how
vocabularies will evolve in various scenarios, who will be involved in making
changes, how often the same vocabulary will be revised over time, what sorts of
incompatibilities may be introduced, which users will have which versions of a
schema, how documents that use multiple evolving namespaces are to be managed,
and so on. Some of the specific
proposals that have been discussed make particular assumptions about the answers
to these questions. A thesis of
this proposal is that XML Schema can relatively easily be adapted to support a
quite broad range of answers to the above questions.
Specifically, this proposal is based on the following
assumptions and design goals:
·
The same vocabulary may be versioned or fixed
repeatedly. Accordingly, the design
should be convenient to use even after 20 or 30 such revisions.
·
The versioning mechanisms should not presume particular
instance constructions such as <extension> elements.
·
In some but not in all cases, forward and/or backward
compatibility is be required: I.e.
it should be possible but not essential to write early schemas that will somehow
accept content that is not fully defined until later, and schemas for later
versions will often but not always validate earlier forms of the vocabulary.
·
Conversely, breaking changes should not be
forbidden. For example, it may be
that an early construct is deprecated at some later time, and perhaps completely
disallowed eventually. Likewise,
later versions may introduce constructs that are rejected outright by earlier
ones.
·
It should be possible to check for or force various
sorts of forward or backward compatibility when desired.
·
Schemas for versions of a vocabulary may but need not
form a sequence or tree, in which later versions somehow directly reference
particular schema documents for earlier versions. This choice allows for redefinition of
the same vocabulary by multiple organizations or in multiple schema files, as
XML does today. In this design, you
can rewrite the schema for my vocabulary to adapt it for your own use (this may
or may not be good common practice, but the versioning mechanisms do not
prohibit it.)
·
A consequence of the point above is that the schema for
version x is not necessarily expressed as a delta to the schema for version x-1,
if in fact the versions form a sequence at all. Such incremental definition schemes are
convenient, but do not necessarily scale to the case where the same vocabulary
is revised 20 or 30 times. In such
a case one would need up to 30 schema documents to assemble the effective
schema. Accordingly, this design
allows for but does not require such incremental definition.
·
As discussed above, no unnecessary assumptions should be
made regarding the relationships between vocabularies and XML Namespaces. Often, a vocabulary will be
expressed primarily as a single XML namespace. Often, to maintain forward and backward
compatibility, that same namespace will be used in subsequent versions as
well. Nothing in this design
prohibits the use or coordinated evolution of multiple namespaces, the addition
of new namespaces in subsequent versions of a vocabulary, etc.
Separation of Concerns and Goals of the Core Mechanism
Obviously, this design is targeted at a very flexible set of
assumptions as to how XML is used.
This is achieved through a more careful attention to separation of
concerns than seems to be the case in some other proposals. Indeed, the key feature of this
approach is that it’s core mechanisms are not fundamentally addressed at
defining “versions”, but rather with these two goals:
1) Making it convenient to write a schema that, when
validating, distinguishes in the PSVI content that is truly expected from
content that is tolerated but not fully understood (I.e. to tolerate later
changes to the vocabulary) and to distinguish both from content that is to be
completely rejected.
2) Given two such schemas, making it convenient to check
whether there is any content that one will allow that the other will not
tolerate. This is to facilitate the
situations in which you do want such compatibility.
Note that the above refers to schemas, not schema documents,
so all of the above applies to mixed namespace scenarios, and independent of how
many schema documents may have been imported or included to make the
schema. The means by which this is
achieved are discussed below under “Core Design”. We then allow for but do not require the
invention and optional use of mechanisms that:
·
Facilitate the writing of schema documents that embody
incremental changes to earlier versions (e.g. mechanisms like
<xsd:redefine>. Indeed,
the design specifically allows for one to make say 5 or 6 successive changes
using such an incremental mechanism, and then for cleanliness rewrite the next
version of the schema document from scratch.
·
Declare an intensional tree or other graph of schema
documents or schemas to facilitate management of evolving definitions. These would include mechanisms such as
version=”x.y” attributes, attributes asserting that one schema is an evolution
of some other designated schema etc.
The next section discusses the means by which the core
mechanism achieves the two goals set out above.
The Core Design
The two main goals of the core design are listed above. These are achieved as follows:
Distinguishing expected from tolerated or disallowed content
using wildcards
XML schema provides a so-called “wildcard”, which is
expressed in schema documents as <xsd:any>. The fundamental thesis of this design is
that content truly expected by an application can usually be validated by a
non-wildcard particle; wildcards
can be used to designate places in the content model where additional content is
tolerated to facilitate interoperation with other versions of the
vocabulary.
The Unique Particle Attribution Constraint of XML schema
ensures that applications can tell from the PSVI which content has been
validated by a wildcard and which by an element declaration. Consider the following two schemas:
Version A:
<xsd:sequence>
<xsd:element name=”x”
type=”xsd:integer”/>
<xsd:any minOccurs=”0” maxOccurs=”unbounded”
processContents=”skip”/>
</xsd:sequence>
Given the instance:
<x>123</x>
<y>abc</y>
this schema will create a PSVI associating <x> with the
element declaration and y with the wildcard. Now let’s assume that the reason this
instance showed up was that it was created by an application that knew about
schema version B:
Version B:
<xsd:sequence>
<xsd:element name=”x”
type=”xsd:integer”/>
<xsd:element name=”y” type=”xsd:string”/>
<xsd:any minOccurs=”0”
maxOccurs=”unbounded”
processContents=”skip”/>
</xsd:sequence>
An application validating with version B accepts the same
instance but associates y with the second element declaration in the PSVI. This application presumably has
first-class knowledge of both x and y.
Weak Wildcards avoid UPA conflicts
Use of wildcards in roughly this manner has been considered
on and off for years, but has until know been inhibited by UPA, which prohibits
schemas such as the following:
<xsd:sequence>
<xsd:element name=”x”
type=”xsd:integer”
minOccurs=”0”/>
<xsd:any minOccurs=”0” maxOccurs=”unbounded”
processContents=”skip”/>
</xsd:sequence>
The above causes a UPA violation, because a single element
<x> matches either the first or the second particle.
The schema workgroup has recently given serious
consideration, for other reasons, to changing wildcards to behave in a so-called
weak manner. This would not change
the behavior of existing schemas, but would allow schemas such as the one shown
above. In such cases, the explicit
element declaration would always take precedence, thus removing the ambiguity.
This proposal presumes the existence of weak
wildcards in the schema design.
Indeed, this is the only incompatible change that is absolutely required
in comparison to XML Schema 1.0.
Given that we have weak wildcards for other reasons, it
becomes possible to use wildcards freely at any point in a content model. Thus, this proposal allows at user
discretion for extension of vocabularies not just at the end of each model, but
anywhere that a weak wildcard can be used.
Possible further changes to facilitate wildcard use
The above analysis shows how weak wildcards can be used to
facilitate construction of extensible content models. In fact there are at least two sorts of
further changes to XML schema that we would want to consider in conjunction with
this proposal:
·
Our current wildcards allow content from any namespace,
other namespaces, or a list of designated namespaces. These may not necessarily be the most
useful options for our versioning scenarios. One proposal is to introduce a wildcard
that would validate any element not explicitly declared elsewhere in the schema
(regardless of namespace, or perhaps intersected with the existing namespace
controls.) This supports an idiom
in which: if I know about an
element and I don’t explicitly call for it, that means I don’t want it. I personally think we would want to use
something like this in the schema for schemas.
·
Purely as a convenience, we could introduce wildcard
defaulting mechanisms into the transfer syntax. So, we might have something like:
<xsd:schema
… defaultExtensionModel=”{openAtEnd,
openEverywhere}>
where
“openAtEnd” causes by default a weak wildcard to appear at the end of every
content model, and openEverywhere puts one between each particle. This is an admittedly vague proposal,
but it’s just a convenience and we can decide later what if anything we want
along these lines.
Comparing two schemas to determine subsumption
The sections above outline an approach that uses weak
wildcards to facilitate the construction of schemas that distinguish tolerated,
from expected, from disallowed content.
The 2nd goal is to facilitate checking of whether one schema
will fail to tolerate any of the content allowed by another.
The schema workgroup has recently devoted significant effort
to proving that the subsumption relation between any two content models can be
tractably checked. That work was
done to support a simplification of the rules for complexType refinement.
This design proposes to use the same subsumption algorithm to
achieve our second goal.
Specifically, given an element declaration from schema 1 and a
declaration for a similarly named element from schema 2, the subsumption
algorithm allows one to determine whether any content accepted by one of the
schemas will be rejected by the other.
This design does not require that such checking be
done. The assumption is that in
many scenarios application developers will wish to enforce such discipline
between evolving versions, and we show how development tools can do the
necessary checking as schemas are developed and before they are deployed. Conversely, if a user wishes for
whatever reason to introduce an intentional incompatibility, then a suitably
written checker can be used to ensure that only the intended incompatibilities
have been created.
Note also that nothing in the design as presented so far
states that versions must evolve in linear or tree-like form. Indeed, completely independent
organizations can create schemas that purport to validate similar or identical
vocabularies, can check the degree to which the goal is achieved, and can deploy
as appropriate. Debug versions can
be checked against production versions, and so on.
Optional Features
As implied above, optional layers can be defined to meet
additional goals such as the following:
·
Facilitating the creation of one schema document based
on another, particularly if the changes are small. We should see whether or not redefine
meets the whole need.
·
Facilitating the automatic insertion of weak wildcards
to create content models that are by default open.
·
Supporting some sort of standard labeling for versions
(e.g. version=”x.y”). Note that the
mechanisms above are oblivious to such labeling, but various schema document
management and deployment systems may find them useful.
While not a separate layer, we should also consider the
proposal above to:
·
Provide new options on wildcards to accept only content
likely to be used in evolution of a given vocabulary (e.g. from the current
namespace but not explicitly declared in the current schema)
Conclusion
Careful readers will note that the only truly required
incompatibility with XML Schema 1.0 is the introduction of weak wildcards, a
change that is already contemplated.
Other changes to facilitate version management or incremental development
of schemas may also prove desirable, but are not strictly required. Accordingly, an interesting feature of
this design is that it seems to achieve a broader range of goals than many
others, using a single design change that has already been contemplated for
other reasons. Indeed, that one
change does not invalidate any existing schemas, but merely allows the use of
wildcards in situations where they were previously prohibited.
Among the next steps I suggest are:
·
Check this proposal against a broad range of use cases
and with our user community.
·
Check with the schema implementation community to guage
their willingness to deploy weak wildcards and to deal with the attendant
incompatibility with XML Schema 1.0.
·
Consider the optional features above, whether they are
worth implementing, and if so whether implementers of the schema language will
support them.
