Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!clyde.concordia.ca!uunet!cs.utexas.edu!usc!apple!well!shf From: shf@well.UUCP (Stuart H. Ferguson) Newsgroups: comp.sys.amiga.tech Subject: Re: Response to DR2D format specification Message-ID: <14892@well.UUCP> Date: 10 Dec 89 03:38:58 GMT References: <14491@well.UUCP> <9310004@hpfcso.HP.COM> Reply-To: shf@well.UUCP (Stuart H. Ferguson) Organization: The Blue Planet Lines: 324 +---- cunniff@hpfcso.HP.COM (Ross Cunniff) | shf@well.UUCP (Stuart H. Ferguson) writes: | >Storing coordinates in floating point is not an ideal choice. [ ... ] | > Fixed point numbers, however, | >are more accurate and operations on them are faster, but fixed within a | >narrow range. This makes them more appropriate for calculations | >involving additions and subtractions. Since coordinates of greatly | >differing magnitude cannot be mixed in the same drawing, [ ... ] fixed point | >coordinates (integers) are a more natural choice for drawings. | | I disagree. Floating point numbers are infinitely more flexible | than integers. Perhaps. They are undoubtedly more complex, which is my objection. Since a drawing can be represented just as well with integers (better, in fact), and integers are simplier than floating point numbers, why not use them? | Why can't coordinates of greatly differing | magnitude be mixed in the same drawing? [ ... ] What if I wanted | objects of great detail spread across a large distance (a map of the | solar system, perhaps)? You can't do it -- it's an inherent limitation of floating point. FP numbers are specified by a mantissa (M) between 1 and N, and an exponent (E) which gives a magitude multiplier for the mantissa. The complete number is given by the expression M * (N^E) where N is the base of the number system. For IEEE floating point numbers, N is 2, E is an integer between -127 and 127, and M is a 24-bit fixed point integer greater than or equal to 1 and less than 2. (There's also a sign bit, of course.) To give a simple illustration of the limitations of the format, imagine that we have a floating point number in base 10, with three digits of mantissa. This means that we can represent numbers like 999, 87600 and .00765. We cannot represent 1.234 accurately because we only have three digits of mantissa -- the best we can do is approximate it with 1.23. So, let's pretend that we have a ruler 2000 units long and we want to put a mark along the ruler every 1 unit. So we start putting marks at 1, 2, 3, 4, etc., until we get to 999. Now we put a mark at 1000. Fine. But what happens when we try to put a mark at 1001? We can't -- we've run out of digits of mantissa. We can mark positions at 1000 and 1010, but we can't mark the nine intermediate positions, because we were forced to increment the exponent of our number to express a greater magnitude and gave up some precision at the low end in the process. The exact same thing happens with binary floating point formats. Now, IEEE has 24 bits of mantissa, and double precision has 56 bits, but this only makes the effect harder to spot. But the example of the solar system will do it. Suppose we want to mark out the distance from the Sun to the Earth with one mile increments. We will do fine until we get about 16 million miles out when we run out of mantissa and are forced to start placing our marks TWO miles apart. At 32 million miles we can only make marks FOUR miles apart and so on. Each time the distance doubles it halves our effective precision. By the time we reach the Earth, at about 100 million miles from the Sun, our marks are 64 miles apart. This means that 64 miles is the smallest distance you can measure on the Earth in this scale model of the Solar system. Now consider, if we took that 100 million miles and represented it with a 32 bit integer, the smallest distance we could measure would be about 1200 FEET -- more than 250 times smaller than what we can measure with an equal sized FP number. | What if I want to specify coordinates in millimeters? Easy -- just call each integer unit a micron. Then 1000 units is a millimeter. Maximum size of the drawing: 4.3 km. My proposal from last time allowed drawings to do this by specifying the bounding box for the drawing in both integer and floating point. The floating point bounding box allows you to specify the size of the drawing in whatever units you want (which floating point numbers are good for); the integer bounding box allows you to impose a very fine uniform grid on that bounded area (which integers are good for). Best of both worlds. | [ ... ] Integers are no more natural than floating point. [ ... ] One advantage of the proposal above is that programs that are not interested in the real-world scale of the drawing can use the integer information without being forced to deal with floating point. Background art for windows, for example, could be drawn without having to know the actual size of the drawing. It would be a shame to force Intuition to load the IEEE floating point library just to draw drawings, especially since it can be avoided so easily. (If anything, it will make assembly language programmers happy :-). | >The bounding box information simply doesn't belong here. [ ... ] | >Because the box includes the line | >width, mitering makes this a non-trivial calculation. In fact, because | >there's no specified limit on the length of a miter, the bounding box is | >_undefined_ in the general case. | I should have specified that the miter limit is assumed to be the PostScript | default - 10x line width, if I remember correctly (I left my info at home). It's still a difficult calculation. I would bet that 8 programs out of 10 would just fudge it and add the miter limit plus 15% to all sides of the bounding box. | As for useful purposes, a display program could trivially decide whether | an object is visible in its current viewport and not display objects | which are obscured. This is true, but this is a run-time consideration which argues that the bounding box is useful information to have in a program that is continously updating the screen. What purpose does it serve *in the file*? -- that's my question. In my experience, an IFF format should be the distilled-down essense of the data to be exchanged. The file should represent only the substance of the data objects and their relationships in the most abstract possible way. But if you take away the bouding box information, the picture is unchanged! Also, given the objects in the drawing, you can *derive* their bounding boxes. This strongly suggests that the bounding box is not an essential part of an object and should not be in the file format. | I would be willing to extend the meaning of these numbers as follows: [ allows for bounding box to be marked invalid ] It would be even better if you would make the whole structure optional, or leave it out altogether. Those 16 bytes per object could be put to much better use. | >| /* 8x8 bitmap fill patterns */ /* OBSOLETE */ | We haven't decided how | useful this is in an object-oriented program. All Mac programs seem | to have them (a powerful clue...) Yeah, but a powerful clue which way? :-) Actually, can't this be done in DR2D with a PATT containing a RAST? | >| XTRN (0x4554524E) /* Externally controlled group */ | On the Amiga, an externally controlled group is one where for each | manipulation performed on that group (i.e., resizing, rotating, moving, | etc.) the ARexx macro named 'ApplName' is called with appropriate flags, This doesn't seem very portable. I would recomend that you make this an optional chunk, such that reader that can't deal with it can safely ignore it. It's an interesting idea, and I think I know what you're trying to accomplish, but I wonder if this works for an IFF format. I don't feel I have enough grasp of the implications to say you should remove it, so I'd say give it a try and see what happens. We'll find out in a few years if it was the right decision or not... :-) | >| /* Color map */ | > [ ... ] this chunk is the same as the ILBM.CMAP chunk, which is | >probably a good thing. | Yep. Exactly the same (except that all 8 bits of each of R G and B are | potentially significant...) Nitpick time: All eight bits in the ILBM.CMAP chunk are potentially significant too, it's just Amiga programs that truncate them to four. I imagine that a drawing program on the Amiga would do the same thing given a DR2D.CMAP. | | RAST (0x52415354) /* Raster image */ [ ... ] | >the "Closest" array is information that is best computed by the target | >application and should not be stored in the file. | Have you any idea how long it can take to calculate the closest colors in a | large bitmap? Well, yes, I have some idea. The naive algorithm will be O(N*M), where N and M are the sizes of the source and destination color tables. With N & M up to 32 or so, this straight-ahead search will be plenty fast. If the color tables get into the 256 color range, you might need some hashing on the color space to speed up the search, but I would think that would still be pretty easy and fast enough to do on the fly. Note that the time to compute this mapping has nothing to do with how large the bitmaps are, but only depends on the size of the color tables. | I might consider, however, a chunk that looks like: | FORM { | RAST (put virtual coordinates and 'closest' stuff here) | FORM { | ILBM | ... (etc) Except now you're creating a new FORM, which is not a great idea. See below. | >Dumping all the font names in one big lump is not a very flexible way | >to indicate fonts, from an IFF standpoint. For example, there is no | >way to selectively override a single font in a nested FORM. For a better, | >more IFF-like way to specify fonts, look at the FTXT.FONS chunk. Better | >yet, use it! | Well, for one thing, these fonts are *not* the standard Amiga fonts. Neither are the fonts in FTXT.FONS. Remember, nothing in IFF is Amiga specific (at least none of the required chunks) -- it is intended as an interchange format. The FONS chunk also provides some other general information about the font to help make a good decision in case a different font needs to be substituted. | [ ... ] it simplifies text objects, since all they have to include is an | index into the font table to tell which font they use. That's how FTXT.FONS chunks work too. Each chunk has a index which is then used to select the font later. | Also, it's very nice to have a list of the fonts all in one place. Why? I gave you an example of something you can do when they're separate; you give me an example of an advantage of having them together. | OK. Now for the most serious objections: | >| NOTE 4: The data in a FILL, DSYM or GRUP chunk is a series of nested | >| object chunks. All object chunks except DSYM, CMAP, PATT, FILL, | >| and FONT are allowed in a DSYM or GRUP. | >This one is a serious, serious problem. This type of nesting is not | >supported by IFF, and is inconsistent with the provided mechanism. | > [ ... ] These nested chunks cannot, for example, be read by the | >generic iffparse.library, since the library was designed to deal with | >standard IFF nesting constructs. | How about: | FORM { | DR2D | ... (a bunch of objects) | FORM { | GRUP (has an object header) ^^^^ ^^^^^^^^^^^^^^^^^^^^^^ | ... (a bunch of objects) | } [ ...etc. ] I can't tell what you're suggesting here. This "GRUP" ID appears right at the start of the FORM, which makes it look like the FORM-type for the FORM; however, it also looks like it has content, which makes it look like a chunk. This doesn't make sense. If you mean the latter, i.e. that the GRUP is a chunk, then the FORM-type should be DR2D. In this case, the file would look like this: "FORM" { "DR2D" "DRHD" { drawing header } ... -- object chunks "FORM" { "DR2D" "GRUP" { group header } ... -- more object chunks } ... -- even more object chunks } If this is what you mean, then I agree that this is a good way to go. If, on the other hand, you were proposing the former possibility, i.e. that "GRUP" is a FORM-type in itself, then I don't think this is a good way to go. Creating a new FORM-types is conceptually creating a whole new entity with its own syntax and content. Since a "GRUP" FORM is essentially exactly the same as a "DR2D" FORM, it really should not be its own FORM-type. Same with the other types: FILL, PATT, etc. | I'm not sure we'll be able to change it though. | We have on the order of 3-5 different companies working on importing | our file format at this point (I'll not name names). We'll have to | talk with them and determine how far along they are toward implementing | the readers. Well, I can certainly appreciate the need to get product out the door, as well as the kinds of compromises needed to do that in a timely manner. It is sometimes more expedient to accept a kludgy solution now and fix it later; however, this is one case where this is not true. This format is a public interface, not just an internal issue. There will be no opportunity to fix it later! Unless you make this a private format -- which you cannot unless the other companies you mention are willing to update their programs at the same time -- the decisions you make now will stick. And all of us in the Amiga community will have to live with the consequences. | Is iffparse.library not flexible enough to do something like this: [ code fragment deleted ... ] | Actually, I don't know how iffparse.library works at all, as you can | probably tell :-) Let me know how off-base I am... The code fragment that was deleted showed an example of recursive-descent IFF parsing code being used to delve into a GRUP chunk as if it were a nesting construct. No, you can't do this with iffparse because iffparse is not a recursive-descent parser. Hard-coded in the heart of the library is an explicit state-machine that controls the traversal of IFF file structures. To treat GRUP chunks like FORM's and LIST's would require re-wiring this central part of the parsing logic. | >On a more positive note .. I mentioned earlier that there is a more | >IFF-ish way to encode layers. Rather than having the layer number as | >a parameter in each object in the file, the layer ID could be a | >property chunk with the file organized as a list of drawings, one in | >each layer. | I don't particularly like this. It makes moving objects around from | layer to layer rather difficult. Unless you're editing your pictures with NewZAP, this is a run-time issue. You can structure your internal data objects to have a layer ID in each one if you prefer. While writing, your program would stick all the objects with like ID's into the same DR2D FORM, and arrange those in a LIST. | >This takes full advantage of the built-in IFF semantics for lexically | >scoped properties within LIST's. It is, of course, somewhat harder | >to read, but not that much. | I think it is quite a bit harder to read, myself. This was one of the | hot points of discussion when the format was originally posted. The | consensus was that 16-bit layer id's on each object was a better | way to go. What was the basis for this consensus? My position here is the same "distilled essense" argument about the content of IFF formats. A simple drawing has no layers. A layered drawing is actually several drawings, each one in a different layer, constructed using IFF syntax. | Sorry it took so long to respond - business trips and all. I'm looking | forward to the next set of comments... At least you'll understand why this response took so long. :-) -- Stuart Ferguson (shf@well.UUCP) Action by HAVOC (ferguson@metaphor.com)