Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!wuarchive!gem.mps.ohio-state.edu!apple!well!shf
From: shf@well.UUCP (Stuart H. Ferguson)
Newsgroups: comp.sys.amiga.tech
Subject: Response to DR2D format specification
Message-ID: <14491@well.UUCP>
Date: 10 Nov 89 06:44:37 GMT
Reply-To: shf@well.UUCP (Stuart H. Ferguson)
Organization: The Blue Planet
Lines: 305

+----  Ross Cunniff (cunniff%hpfcrt@hplabs.HP.COM) writes:
|
| The following is the format of the DR2D IFF FORM file produced
| by ProVector, a 2-dimensional structured graphics program.

I see failings with this format in two main areas:

1.  Transportability across applications.  Some aspects of the format
were clearly designed to work well with this one drawing program but 
will hinder other developers from using it for different applications.
Since the main reason for IFF is this tranportability of data, this
is a serious consideration in any format that wants to become a 
"standard."

2.  Conformance with IFF philosophy.  Although it's never been explicity
documented what constitutes a "good" IFF FORM, it is possible to glean
some of the basic design notions from studying the IFF documentation, 
the standard FORM's and the IFF format itself.  Being a something of a 
student of IFF, I think I can illuminate some areas where this proposed
format clashes with general IFF principles.


Let me first address the transportability and generality issues; those
issues that will affect how well this format can adapt to other 
applications than the one for which it was specifically designed.

| Coordinates are specified in IEEE format (see note 1 for a
| description of the IEEE format).

Storing coordinates in floating point is not an ideal choice.  For some
things floating point numbers are essential, but not for drawings.  The
advantage that floating point numbers have over fixed point is that they
are valid over a much greater range, and are therefore useful for 
calculations involving multiplications.  Fixed point numbers, however,
are more accurate and operations on them are faster, but fixed within a 
narrow range.  This makes them more appropriate for calculations 
involving additions and subtractions.  Since coordinates of greatly 
differing magnitude cannot be mixed in the same drawing, and the edges
of a drawing provide a natural boundary for the scale of the numbers, and
the fact that vectors are often added and rarely multiplied, fixed point 
coordinates (integers) are a more natural choice for drawings.

This does not mean that any given program couldn't use floating point
numbers internally.  It just means that the Interchange format for drawings
would use the more natural fixed point notation for exchanging data
between varied applications, some which use floating point interanlly
and some which do not.  But none are required to parse it when they don't
have to.  For programs which want floating point, the drawing header chunk,

| 	struct DRHDstruct {
| 	    LONG	ID;
| 	    LONG	Size;		/* Always 18 */
| 	    IEEE	p_xmin, p_ymin,	/* Minimum and maximum */
| 			p_xmax, p_ymax;	/*   coordinates of the drawing area */
| 	};

could be expanded to include the bounding box of the fixed point 
coordinates as well as their equivalents in floating point.  Thus a
floating point program such as ProVector could transform the integer
values in the DR2D file into their equivalent floating point values,
with no loss of precision, with one floating multiply.  Since a drawing
requires at least this much calculation each time it's displayed, I
can't see how this extra computation would be objectionable.  It
saves a lot of effort for those who only want to deal with integers.

| 	typedef struct {
| 	    UBYTE	FillType;	/* One of FT_*, above	*/
| 	    UBYTE	FillValue;	/* See note 3		*/
| 	    UBYTE	EdgeType;	/* One of ET_*, above	*/
| 	    UBYTE	EdgeValue;	/* Edge color index	*/
| 	    USHORT	WhichLayer;	/* Which layer it's in	*/
| 	    IEEE	EdgeThick;	/* Line width		*/
| 	    IEEE	XMin, YMin,	/* Bounding box of obj. */
| 			XMax, YMax;	/* including line width	*/
| 	} ObjHdr;	/* Size == 26 */

This "Object Header" record comes along with almost all primitive
objects in the file.  Although most of these parameters are reasonable,
I have problems with two of them:  WhichLayer and the X/Y-Min/Max
bounding box.

WhichLayer is somewhat ambigious.  Can you have, for example, a group
with components in more than one layer?  If so, what does it mean?
The "use a symbol" chunk has its own WhichLayer field, as do each of
the components of the symbol.  Which layer wins?  I think that the
layer information does not belong in the data for each object, but 
rather as a property of a whole collection of objects.  I will spell
this out later on.

The bounding box information simply doesn't belong here.  What useful
purpose does it serve in the file?  None.  It does make it much more 
difficult for simple programs to write one of these files, because it 
makes them compute the bounding box for each object, even if they don't 
*use* that information themselves.  Because the box includes the line 
width, mitering makes this a non-trivial calculation.  In fact, because 
there's no specified limit on the length of a miter, the bounding box is
_undefined_ in the general case.

I want to emphasize how utterly absurd it is to store this information in
an Interchange file.  1) It makes simple readers and writers more complex.
2) Even a program that could use the information can't, because 3) the
information might not be correct.  Since this is a published format, you
can't tell where it might be coming from.  A program that wants to use the 
bounding box to speed things up or whatever would still have to *verify* it
against its idea of what the object is.  And the most ridiculously 
redundant aspect of this whole thing is that 4) a drawing program already
has to have the logic in it to compute the bounds given the parameters
of an object.  So why not just compute them as they get read!?  Saving 
this information might save *some* programs a little time when reading
and writing their own files, but it will cost *everybody* pain and grief 
in the long run.  Be smart -- throw it out now.

| /* 8x8 bitmap fill patterns */ /* OBSOLETE */

If it's obsolete, why publish it?

| XTRN (0x4554524E)	/* Externally controlled group */

It would help transportability if there was some clue what this chunk
was about.

| /* Color map */
| 	struct CMAPstruct {
| 	    LONG	ID;
| 	    LONG	Size;		/* Usually 256*3, or 768 */
| #define			NUMCOLOR(Size)	(Size / 3)
| 	    UBYTE	ColorMap[NUMCOLOR(Size)][3];
| 	};

I assume (since it's not specified) that the colors are in R G B order
with 0=min intensity and 255=max intensity.  If so, then this chunk is
the same as the ILBM.CMAP chunk, which is probably a good thing.

| RAST (0x52415354)	/* Raster image */
| 
| 	struct RASTstruct {
| 	    LONG	ID;
| 	    LONG	Size;
| 	    IEEE	XPos, YPos,		/* Virtual coords */
| 			XSize, YSize;		/* Virtual size */
| 	    SHORT	NumX, NumY, Depth;	/* Actual size */
| 	    UBYTE	Colors[3*(1<<Depth)];	/* Color map */
| 	    UBYTE	Closest[(1<<Depth)];	/* Closest equivs in CMAP */
| #define RSIZE(x,y,d)	((x+15)/16)*y*d
| 	    USHORT	Rast[RSIZE(NumX,NumY,Depth)];	/* Lookups into Colors*/
| 	};

With the exception of virtual coordinates and size, this is just like a
simplified ILBM.  Using nested ILBM's here would facilitate 
transportability and would serve the IFF charter a lot better.  Again,
the "Closest" array is information that is best computed by the target
application and should not be stored in the file.  Same reasons apply
as for the bounding box.


Now we get into the area of IFF philosophy.  These are in some sense
far more important issues, because they can have such far-reaching
effects.  Each format that is made a standard evolves the standard,
and once a format is accepted, either by Commodore or by the 
marketplace, we are forever stuck with whatever slant it takes on
the IFF concept.

| Note that most chunks have
| both a chunk Size and an object Count.  I know this is redundant;
| however, this allows the data structures to be extended in a
| compatible manner if it should become necessary in the far future.

The general way IFF provides for this future extensibility is by 
obsoleting old chunks and creating updated versions.  The SMUS FORM
contains an example of this with the INST and INS1 chunks.  INS1
is the new version of the now obsolete INST chunk (new versions of
chunk ID's are created by appending a number, INS1, INS2, etc.).
Old files still work because the new readers can interpret the old
chunks, and old readers don't crash because they skip chunks they
don't understand.  Once a chunk is publicly defined, it should not
be changing (even to get bigger).

| PREF (0x50524546)	/* Application preferences */
  [ ... ]
| 	   Other applications may safely ignore this chunk,
| 	   although it is requested that applications that
| 	   manipulate DR2D files do not delete preferences
| 	   settings that they do not understand.  Some possible
| 	   settings include:
| 
| 		DATE=10/28/88
| 		COPYRIGHT=Copyright 1989 by Taliesin, Inc.
| 		PROGRAM=ProVector
| 		ARTIST=Ross Cunniff
| 		LMARGIN=1 inch
| 		ORIENT=portrait
| 		ZOOM=1.0 1.0 2.0 2.0

First of all, it looks like some of this information would be of general
interest and would be better to be stored in documented chunks. The
Copyright "(c) " chunk already exists in SMUS, for example.

Although there is nothing wrong with a specific application sticking a 
private chunk in a public FORM, the rule with these is for programs
that do not understand them to throw them away.  The reason is that there
is no way for these other programs to verify if the values in the chunk
are still valid.  If the program has edited the data at all, then the 
ZOOM might be off in never-never land, for example.  If ProVector is 
willing to deal with this possibility, then it can certainly *request*
that other programs leave this chunk intact.  Don't expect competitors to
leave it in though.  :-)

| /* Font table */
| 	struct FONTstruct {
| 	    LONG	ID;
| 	    LONG	Size;
| 	    SHORT	NumFonts;
| 	    CHAR	FontNames[Size-2];
| 	};
| 	/* FontNames is an array containing the names of all the fonts
| 	   used in this drawing.  The names are stored as null-separated
| 	   strings. */

Dumping all the font names in one big lump is not a very flexible way
to indicate fonts, from an IFF standpoint.  For example, there is no
way to selectively override a single font in a nested FORM.  For a better,
more IFF-like way to specify fonts, look at the FTXT.FONS chunk.  Better
yet, use it!

| NOTE 4: The  data in a FILL,  DSYM  or  GRUP chunk is a series of nested
| 	object chunks.  All object chunks except DSYM, CMAP, PATT, FILL,
| 	and FONT are allowed in a DSYM or GRUP.

This one is a serious, serious problem.  This type of nesting is not 
supported by IFF, and is inconsistent with the provided mechanism.  While
there is no technical reason why the content of the chunks couldn't be
defined this way, doing so flys in the face of a consistent IFF 
philosophy.  These nested chunks cannot, for example, be read by the
generic iffparse.library, since the library was designed to deal with
standard IFF nesting constructs.  Reading this would require every
programmer to code their own custom reader, and would futher set back
any progress toward providing general IFF parsing tools.

The only place properly nested chunks can occur is within FORM's.  In
this case, it should be a FORM DR2D, since that's what kind of chunks 
they are.  In other words, a group would look like a nested drawing:

	FORM {
		DR2D
		...		-- root level objects
		FORM {
			DR2D
			...	-- grouped objects
		}
		...
	}

The different types of nested drawings could be flagged in numerous
ways, a simple one being different drawing header chunks.  Symbols
could have a SYMB chunk in them giving the name of this symbol.
Fill patterns could have a header that specifies the local coordinate
system and other stuff.  Another way to flag them would be with
chunks in the containing FORM just before the included FORM.  There 
are probably better ways waiting to be discovered.

The basic point is that the nesting *must* look something like this 
in order to be IFF.  The mechanism specified in note 4 is vaguely
like IFF, but it isn't, and never will be.  If this format is released 
in its current form, I will certainly do everything I can to prevent it 
from ever being accepted as a Commodore standard IFF FORM.  The damage
to IFF integrity would be too great.


On a more positive note .. I mentioned earlier that there is a more
IFF-ish way to encode layers.  Rather than having the layer number as
a parameter in each object in the file, the layer ID could be a
property chunk with the file organized as a list of drawings, one in 
each layer.  This would be a "MultiLayer" LIST format:

	LIST {
		MULY
		PROP {
			DR2D
			...	-- shared properties (such as header).
		}
		FORM {
			DR2D
			LAYR { 1 }
			...	-- data for layer 1.
		}
		FORM {
			DR2D
			LAYR { 2 }
			...	-- data for layer 2.
		}
		...		-- more layers.
	}

This takes full advantage of the built-in IFF semantics for lexically
scoped properties within LIST's.  It is, of course, somewhat harder
to read, but not that much.


So let's hear from you Ross.  I'm sure that there's a way to make this
format more in the spirit of IFF without holding up your delivery 
schedule too much.  I'd be glad to work with you on this one.  A few 
conversations should be all it takes.
-- 
		Stuart Ferguson		(shf@well.UUCP)
		Action by HAVOC		(ferguson@metaphor.com)