[GHC] #669: negative indentation in Text.PrettyPrint.HughesPJ

#669: negative indentation in  Text.PrettyPrint.HughesPJ
---------------------------------------------+------------------------------
    Reporter:  waldmann@xxxxxxxxxxxxxxxxxxx  |        Owner:         
        Type:  bug                           |       Status:  new    
    Priority:  normal                        |    Milestone:         
   Component:  hslibs/text                   |      Version:  6.4.1  
    Severity:  minor                         |     Keywords:         
          Os:  Unknown                       |   Difficulty:  Unknown
Architecture:  Unknown                       |  
---------------------------------------------+------------------------------
The pretty printing library has a long-standing bug:
 it sometimes creates negative indentation.
 Currently, the symptom is cured, but the cause is unknown.

 I tried to dig a bit into the code, and here's what I found.

 short summary: the law
 {{{
 (n6)  x <> nest k y = x <> y, if x non-empty
 }}}
 seems broken by the implementation.


 test case:

 this seems OK:

 {{{
 *Main> text "foo" <> nest 20 (  nest 20 ( text "foo" ) $$ text "bar" )
 Beside (TextBeside (Str "foo") 3 Empty) False
        (Nest 40
              (TextBeside (Str "foo") 3
                          (NilAbove (Nest -23 (TextBeside (Str "bar") 3
 Empty)))))

 }}}

 but thes seems wrong (the Nest 40 is gone but the Nest -23 is kept,
 this will lead to negative indentation later on)

 {{{
 *Main> reduceDoc $ text "foo" <> nest 20 (  nest 20 ( text "foo" ) $$ text
 "bar" )
 TextBeside (Str "foo") 3
            (TextBeside (Str "foo") 3
                        (NilAbove (Nest -23 (TextBeside (Str "bar") 3
 Empty))))
 }}}

 Now the formatter thinks that he has to indent by 3 + 3 - 23 = -17 spaces.

 In the source, I found these problematic lines:

 {{{
 618:nilBeside g (Nest _ p) = nilBeside g p
 657:sepNB g (Nest _ p)  k ys  = sepNB g p k ys
 700:fillNB g (Nest _ p)  k ys  = fillNB g p k ys
 862:        lay2 k (Nest _ p)          = lay2 k p
 }}}

 I think that no function (that produces a document)
 is allowed to ignore the nesting level
 because this adds to the total (relative) indentation
 that must be kept in a consistent state.

 At least that's my guess that these numbers represent relative indentation
 -
 the source is not exactly verbose on the meaning of all these numbers
 (which are not in the original paper by John Hughes).

-- 
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/669>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler_______________________________________________
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@xxxxxxxxxxx
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs