Re: Charters for review

On Wednesday, November 22, 2006, 12:35:31 AM, L. wrote:

LDB> On Tuesday 2006-11-21 21:05 +0000, Ian Hickson wrote:
>> TIMETABLE
>> 
>> The milestones in the charter are somewhat unrealistic. I would suggest 
>> the following timetable would be far more likely, based on past experience 
>> with HTML4 (which is still not fully implemented by any two UAs), DOM, and 
>> CSS2.1 (which is the only large W3C specification to have attempted a 
>> serious disambiguation period):
>> 
>>     First Working Draft in October 2007.
>>     Last Call Working Draft in October 2009.
>>     Call for contributions for the test suite in 2011.
>>     Candidate Recommendation in 2012.
>>     First draft of test suite in 2012.
>>     Second draft of test suite in 2015.
>>     Final version of test suite in 2019.
>>     Reissued Last Call Working Draft in 2020.
>>     Proposed Recommendation in 2022.

LDB> This seems a bit slow to me.

Positively glacial.

Daniel Glazman gave some helpful feedback earlier today on what a
realistic schedule might be.

LDB>   Ambitious schedules encourage faster
LDB> progress.  And I think given good organization, adequate tools,
LDB> and adequate amounts of people's time, things could move a good bit
LDB> faster than they have in the CSS WG.

Yes, I would expect that to be the case.

LDB> I think a good bit of the tension over the schedule reflects tension
LDB> over how many improvements are needed to make it worth pushing the
LDB> whole set of improvements through the process as a new version.  I
LDB> usually prefer small and frequent increments to large and infrequent
LDB> ones.  But I think the W3C process has a tendency towards large and
LDB> infrequent increments due to administrative overhead (rechartering
LDB> per increment, publication overhead) and the tendency of commenters
LDB> to repeat the same comments at every increment.

Yes, good point.

LDB> To put it another way, schedules like this don't reflect the fact
LDB> that some parts of the specification will likely be stable long
LDB> before others are.

Yes. its a question of granularity.

LDB> CSS has worked around this problem by splitting CSS3 into modules
LDB> that are independently versioned and advanced.  However, this
LDB> workaround also has significant problems:
LDB>  * It is harder to write, harder to publish (extra administrative
LDB>    overhead) and harder to read (can't find all the pieces, and some
LDB>    parts are unnecessarily abstract so they are modular).
LDB>  * The separation between modules started off as logical separation,
LDB>    but then gets tweaked so that features of the same stability
LDB>    level are published at the same time.  This makes the
LDB>    specifications even more confusing.
LDB>  * Interactions between features in different modules are unlikely
LDB>    to be properly tested.

LDB> The problem with the approach where a single large document advances
LDB> according to its slowest part is that many of the checks provided by
LDB> the process (such as test suites to verify that the specification is
LDB> being implemented interoperably) happen much later than they should.
LDB> Sections that are stable now should have test suites written now so
LDB> that implementations can continually improve.  For example, the
LDB> WHATWG <canvas> specification has been implemented by three browsers
LDB> (so clearly that *part* of the specification has reached
LDB> call-for-implementations), but there's no test suite, and there are
LDB> interoperability problems that would have been caught by a test
LDB> suite.

LDB> Ideally, different parts of a large document could be marked as
LDB> being at different stages in the process.

Thats an interesting idea.

LDB>   Perhaps that's a lot to
LDB> ask (both of W3C management and of document reviewers), but I think
LDB> it would help align specifications with reality.

LDB> Perhaps this could be approximated by maintaining a master document,
LDB> using a tool to remove the parts that aren't at a given maturity
LDB> level, and publishing the incremental updates to each maturity level
LDB> as HTML 5.00, 5.01, etc.?

Interesting. Yes, that sort of compositional approach might work
well.


LDB> I'd rather define a comprehensive test suite as one that:

LDB>  * has tests adequate to verify that each normative sentence in the
LDB>    specification applying to browser conformance (as opposed to
LDB>    document or authoring tool conformance) is correctly implemented
LDB>    in at least the basic cases, and

LDB>  * has tests adequate to verify correct interaction of features
LDB>    whose interaction is interesting or potentially problematic.

Good points about testing based on class of conforming product 9eg
browser, generator etc)

LDB> The rule about sentences should cause the complexity of the test
LDB> suite to scale with the complexity of the spec, and also provides
LDB> further incentive to keep the specification simple.

:)



-- 
 Chris Lilley                    mailto:chris@xxxxxx
 Interaction Domain Leader
 Co-Chair, W3C SVG Working Group
 W3C Graphics Activity Lead
 Co-Chair, W3C Hypertext CG