Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!sdd.hp.com!hplabs!otter.hpl.hp.com!hpltoad!hpopd!hpcpbla!kev From: kev@hpcpbla.HP.COM (Kevin Jones) Newsgroups: comp.periphs.scsi Subject: Re: Random Access DAT Message-ID: <9850028@hpcpbla.HP.COM> Date: 17 Jun 91 08:02:40 GMT References: <1991Jun14.211246.4340@newshost.anu.edu.au> Organization: HP Computer Peripherals Bristol, UK Lines: 223 > 1. Now that DAT is available with fairly high speed random access, is > there any software available to actually USE this? (Or for Exabyte > which is only slightly slower, or indeed any older software to use > seeks on a raw character device?) > > 2. Do the drivers normally provided on Unix systems (e.g. ISC 386/ix 2.2) > allow for random seeks or would new drivers as well as application > software be needed? (If so, are any such drivers available?). > The DDS format has counts of: #blocks, #filemarks, #setmarks written in each physical "frame" of data placed on tape. This enables a drive in "fast search mode" (typically 200x read/write speed) to skip directly to any specified block, filemark or setmark. (PS. Setmarks are "super filemarks" and tend to be unique to DDS. You don't have to use them if you don't want to. Most people don't know what they are, and are probably no worse off as a result :-) A drive will automatically use "fast search" mode when told to skip blocks/filemarks/setmarks (SCSI "SPACE" command), or to position to an absolute location on tape (SCSI "LOCATE" command). Most drivers out there should be able to handle block/mark skipping. The problem with fast searching is knowing where to go, thus either you have a smart backup application which keeps track of the order in which it puts files onto tape, or you have a dumb backup application and keep track of the order of files yourself. You could then use something like the 'mt' command to position the tape in front of a file you want to read. > 3. Main application I have in mind is for offline archiving of large > volumes of files to be available for mail requests. Tape could be > written once, or with supplements added in batches after filemarks. > Requests get batched or handled individually but either way > spin the tape at high speed between locations where actual file > data has to be extracted rather than scanning the data at ordinary > speed (and tying up the CPU with scanning). > > 4. Is it practical to do this with filemarks instead of fully > random seeking? (i.e. THOUSANDS of filemarks?) Seems undesirable > to me as there would be a stop-start after each logical file was With DDS, a filemark uses 4 bytes of space, and a drive can "write" one in approx 5 milli-seconds. (this assumes you write the filemark in "immediate report" mode to prevent a time/space wasting flush of the drive's data buffe to tape). Thus there is not neccesarily a stop-start after a filemark is written. (NB. This is not true for 8mm, where filemarks waste a lot of space and time). > written, and also I imagine that scanning for a filemark is slower > than scanning to a specified block count? Is that right? Makes no difference in DDS. > 5. A nice way to do it might be something that makes a directory > showing the byte offset to each file as files are being streamed > out through cpio or tar. Then to read you just sort the requests. > Normal recovery through cpio or tar would be available, > and faster direct access as well. You mean: have a number of cpio/tar archives on tape interspersed with filemarks, and skip to the archive you're interested in ??? I guess this might work, provided cpio/tar doesn't cause a rewind when it "opens" the device for read/write. (I don't know much about this sort of stuff, so I skip to the next question ...........) > 6. I understand that dataDAT allows random writing as well, but only > DDS is popular which does not allow random writing. Is there any way with DDS > to leave huge gaps and waste tape in order to allow a primitive form > of random writing between file marks or pseudo end of media marks? > Seems to me that should be PHYSICALLY possible regardless of the > format, but I can't figure out how to do it. Alternatively, or perhaps > it is the same question, is there some way to use numerous "partitions" > with DDS instead of just two? No and No :-( The way that most helical scan tape formats work means that you cannot equate a specific size of data to a specific length of tape, eg. "10 Meggabytes of data = 1 meter of tape" doesn't work. Mechanisms that undermine this sort of algebra: 1. Auto-rewrites of bad frames. To ensure data integrity, data is read by a "trailing read head" soon after it is written. If the read shows problems then the data is repeated (Rewritten). Rewrites occur until the data is known to be written OK. Rewrites can thus result in "more tape for the same amount of data". 2. Flushes of partial groups. A "group" in DDS is a 126Kbytes of data which can contain multiple blocks/filemarks packed and written in a "single indivisible operation". Groups are built in a drive's buffer and written once they are "full" of data/marks. For data integrity purposes it is possible to force the writing of a group before it is full to "ensure the stuff is on tape". It still uses 126Kbytes-worth of tape however. Reserving "spaces" into which you can fit "known amounts of data" therefore becomes very difficult. Leaving "huge gaps" can pose great difficulty, especially to the fast-search algorithms. The muck up the tape format badly. This is, in effect, what the DATA/DAT format did (along with more exotic things like spares tables on tape etc....). The DDS format thus restricts you to 2 partitions and you will find drives will not allow you to read "beyond the most recent end of data" (ie. you will not be able to get past the point at which you last stopped writing in order to read "old" data). > 7. If such software isn't yet available how about some DAT vendors > (or even SCSI adaptor vendors) developing some? Seems to me it would > widen the DAT (and/or Exabyte) market beyond backup streaming > (and incidentally provide for fast recovery of files from backups). > Once a crude prototype was freely available from a DAT vendor, others > would develop new and better applications. > Would also widen appeal of SCSI (less substantially). > Its a matter of convincing DAT vendors (people who make iron) that developing a bit of software for a change is in their own best interests. Why hasn't this happened ??? I don't know. > 8. If this isn't the best newsgroup to ask in, which should I try? > > P.S. While I am at it, an entirely separate issue, which also > happens to relate to Random Access DAT. > > I have read of VERY expensive "juke boxes" for tapes as well as for > WORMs and CD ROM. > > 9. Is there any reason a crude version of such a juke box could not > be constructed quite cheaply? (e.g. a rather slow one, using toy > components?) It doesn't sound all that complex to me for a toy robot > to pick tapes from a rack and drop them into a slot and vice versa. > Anybody working on this? > > 10. Are audio juke boxes for DAT available or planned? If so is > anybody going to convert them for computer use? (Just take the > the electronics from a computer DAT and use the mechanical parts > from an audio juke box). I havn't seen any coming from Japan. There are some "stacker" devices for DAT on the market. Typical capacity approx 12 cassettes. > > Hmm, that reminds me of another issue. > > 11. The Wangtek DAT I am using has a JVC drive and an entirely > separate Wangtek SCSI controller which is merely screwed together > to make it look like a single full height drive. Would the JVC > drive be an ordinary audio DAT drive and would there be some > simple way one could feed output from an ADC to it or input to > a DAC from it (at least standalone audio DACs are readily > available for use with high end CD players)? A lot of DAT drives use mechanisms developed for audio. The similarity ends at this point. Most of the electronics (doens to the read/write amplifiers in some cases) will have undergone radical change to support computer data. Bolting on ADC's/DAC's becomes a non-trivial operation. > > 12. Likewise, could one just add a SCSI controller board to > a consumer audio DAT drive to get a computer DAT (as I > believe Wangtek may have done)? I guess consumer audio DAT "SCSI Controllers" for DDS-DAT drives tend to be fairly complicated. In order to generate DDS format, handle the SCSI interface, maintain data integrity etc.... they usually end up with a component list that goes something like: 1 16 bit processor, A couple of 8 bit microcontrollers, 0.5 - 1 Meg of DRAM, 128K-256Kbytes of PROM/ROM/FLASH memory "containing" 30,000 lines-worth of source code (that count doesn't include comments). 1 SCSI protocol controller chip, 1 or more DAT formatting chips ........ Assuming that you could get a consumer DAT drive, a controller board "off the shelf" that implements SCSI/DDS, plug them together, perform a LOT of data integrity and reliability testing and modify the design as you go so that you end up with a quality product......... In other words, "designing and adding a SCSI controller" may be all that WangDAT or any other DAT vendor has done, but that's where the difficulty is, and where a lot of the development and product costs are incurred. That's what you're paying any perceived "extra" for. > will soon be dramatically cheaper than before and it > might be worth looking into. (Or are computer DAT drives > expected to quickly follow consumer audio prices down?) > Seems to me that with CD ROMS there may be some technical > differences in seek-time requirements etc to explain a > continuing large price differential, but for DAT the > ONLY technical differences should be in controller > electronics and price differentials may be purely a > marketing decision that could be challenged by some > small PCB producer buying audio drives and adding their > PCBs. Price "erosion" (I think that might be a marketing term) will occur. I wouldn't expect DAT to become as cheap/cheaper than CD rom. There's an awfull lot more mechanism and electronics in a DAT drive than in a CD ROM. > > > -- > Opinions disclaimed (Authoritative answer from opinion server) > Header reply address wrong. Use cmf851@csc2.anu.edu.au ----------------------------------------------------------------- Kevin Jones. | Hewlett Packard Ltd, | Computer Peripherals Bristol, kev%hpcpbla@hplb.hpl.hp.com | Filton Road, | Stoke Gifford, Tel: 011 44 272 799910 (ext 22351) | Bristol. BS12 6QZ. | ENGLAND. ----------------------------------------------------------------- This response does not represent the official position of, or statement by, the Hewlett-Packard Company. The above data is provided for informational purposes only. It is supplied without warranty of any kind.