Re: generating correct sfnt ttf files

Hi,

On Tue, 29 Aug 2006 13:27:19 -0400
Qianqian Fang <fangq@xxxxxxxxxxxxxxxxxxx> wrote:
>I posted a question on fontconfig mailing list asking
>about the recognition of a SFNT TTF font. *mpsuzuki
>pointed out that this is due to the font file is not 
>well posed. The discussion can be found at:
>
>*http://lists.freedesktop.org/archives/fontconfig/2006-August/002364.html
>
>I haven't got chance to verify this by myself, but
>from from *mpsuzuki's detailed reply, it seems* the
>FT_FACE_FLAG_SCALABLE is somehow left on.

For first, I (re-)summarize the original reason why
fontconfig cannot handle wqy bitmap font. Sorry, this
is too off-topic in fontforge people.

-------------------- off-topic start ------------------

fontconfig is a font management system designed for
vector font, using persistent database of font files.
Its database does not provide sufficient informations
to manage bitmap glyphs in TrueType font file.

Why? TrueType font can include multiple bitmap glyphs,
e.g. 12/14/16pt.. In following, I call a collection
of bitmap glyphs for specified pixelsize by the name
"strike". The availability of glyph data is separately
defined per strike. For example, it is possible to
make a bitmap-only-TrueType whose 8pt strike includes
ASCII only, 10pt strike includes UCS2-BMP. On the other
hand, fontconfig can store only single database of glyph
availability (PCF & BDF are easy, because they have only
single strike). So, how we can compute the database of
glyph availability for multi-strike-bitmap-only-TrueType?
The sum of each strike coverage? It is bad idea, because
it cause a trouble like "when I magnify a document, some
characters are lost". The intersection of each strike
coverage? It is not good idea, especially for the font
for large character set, because fc-cache must crawl all
strikes. I guess, fontconfig developer couldn't determine
generic solution, and decided to ignore bitmap data simply.

fc-cache, a program to generate/update fontconfig
database works as:

1. when the file is identified as scalable by FreeType2
   (the flag FT_FACE_FLAG_SCALABLE: by exist of glyf
    or CFF. Yet I've not checked sfnt-housed PS font),
   fc-cache ignores included bitmap glyphs forcibly.

   for first, fc-cache creates glyphID list by expanding
   cmap table. in next, fc-cache checks the availability
   of glyph in the list by FcFreeTypeCheckGlyph(), to
   remove the glyph that is declared in cmap but displayed
   by fallback spacing.

fontconfig-2.2.3/src/fcfreetype.c
 1536: static FcBool
 1537: FcFreeTypeCheckGlyph (FT_Face face, FcChar32 ucs4, 
 1538:                       FT_UInt glyph, FcBlanks *blanks,
 1539:                       FT_Pos *advance)
 1540: {
 1541:     FT_Int          load_flags = FT_LOAD_NO_SCALE | FT_LOAD_NO_HINTING;
 1542:     FT_GlyphSlot    slot;
 1543:     
 1544:     /*
 1545:      * When using scalable fonts, only report those glyphs
 1546:      * which can be scaled; otherwise those fonts will
 1547:      * only be available at some sizes, and never when
 1548:      * transformed.  Avoid this by simply reporting bitmap-only
 1549:      * glyphs as missing
 1550:      */
 1551:     if (face->face_flags & FT_FACE_FLAG_SCALABLE)
 1552:         load_flags |= FT_LOAD_NO_BITMAP;

2. when the file is identified as not-scalable,
   fc-cache checks the glyph existence without specification
   of strike.

   (before this part, as in above, fc-cache don't know whether
    this face is scalable, how many strike, etc etc not at all,
    so, fc-cache cannot choose strike)

fontconfig-2.2.3/src/fcfreetype.c
 1554:     if (FT_Load_Glyph (face, glyph, load_flags))
 1555:         return FcFalse;

3. If given face comes from bitmap-only-TrueType, FreeType2
   calls TT_Load_Glyph() to load a glyph data.

freetype-2.2.1/src/truetype/ttgload.c
 1803:   TT_Load_Glyph( TT_Size       size,
 1804:                  TT_GlyphSlot  glyph,
 1805:                  FT_UInt       glyph_index,
 1806:                  FT_Int32      load_flags )
 1807:   {
...
 1824:     if ( size->strike_index != 0xFFFFFFFFUL      &&
 1825:          ( load_flags & FT_LOAD_NO_BITMAP ) == 0 )
 1826:     {
 1827:       error = load_sbit_image( size, glyph, glyph_index, load_flags );
 1828:       if ( !error )
 1829:         return TT_Err_Ok;
 1830:     }

Here, strike_index was set to 0xFFFFFFFF when strike is not
chosen. Even if fc-cache does not set FT_LOAD_NO_BITMAP
(we can force that by hiding glyf/CFF tables), FreeType2
does not execute bitmap glyph data loading without strike
specification.

 1838:     if ( load_flags & FT_LOAD_SBITS_ONLY )
 1839:       return TT_Err_Invalid_Argument;

As a result (bitmap only, but strike is not specified),
TT_Load_Glyph() returns error.

-------------------- off-topic end ------------------

In summary, removing glyf/CFF is not enough to make fc-cache
to handle bitmap-only-TrueType.

>I created the wqy sfnt font file from fontforge 20050310
>(later version will crash when saving to sfnt ttf/otf).
>I looked at the "General" tab of Font Info dialog, there is
>a "Scale Outlines" checkbox, looks like the place to set
>this bit, however, it seems does not working: if I uncheck
>it and save the font to ttf, and reopen the generated
>sfnt file, this flag is still on.

>Under "OS/2-misc" tag, I saw there is a "Only embedded bitmap"
>checkbox, I am wondering if this corresponds to the
>FT_FACE_FLAG_SFNT flag for freetype2 ? 
><http://www.freetype.org/freetype2/docs/reference/ft2-base_interface.html#FT_FACE_FLAG_XXX>
>does fontforge set this flag (and other necessary) automatically
>when generating sfnt ttf/otf or I need to do it manually?

As I've analized in above, changes of fontforge, FreeType2,
and a few flags in font file is not enough to use current
WQY font in fontconfig.

One of the smooth solution might be insertion of phony outline
data for each glyphID. It will cheat fc-cache as if WQY font
were outline font. Although I'm not sure if it may misguide
users to use WQY font in graphical application (e.g. GIMP etc),
I think such cheat will work all existing systems.
George, is it difficult for fontforge to add phony outlines
(I think empty square is enough) for all glyphID used by bitmap
data?

More honest solution might be modification of fontconfig, to
scan glyph availability. I've already finished a trial patch
to scan single strike (the strike is specified by environment
variable and available pixelsizes). I think there are more
bitmap-only-TrueType whose glyf does not cover all glyphID
(yes, there had been many, on old MacOS). To use them without
violation of commercial font license, modification of fontconfig
will be easier.

But, fontconfig database registers the bitmap only font as
scalable, and cannot store the information of available pixel
sizes (so, fontconfig client cannot know WQY provides 12,13,
15,16 only). It can cause misguiding problem again. I think
further extension of fonts.cache syntax is required.

Qianqian Fang, How do you think of?

Regards,
mpsuzuki

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642