Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!nrl-cmf!ukma!rutgers!att!alberta!ubc-cs!ubc-bdcvax!holm From: holm@ubc-bdcvax.UUCP (Terrence W. Holm) Newsgroups: comp.os.minix Subject: de(1) minix disk editor (part 1 of 5) **REPOSTING** Message-ID: <416@ubc-bdcvax.UUCP> Date: 2 Feb 89 19:54:55 GMT Lines: 492 EFTH MINIX report #63 - January 1989 - de(1) Over Christmas I was cleaning up our test Minix system, and I accidently rm'ed a file (uptime.c) that had not yet made it to the production machine. I was slightly upset, but using fgrep(1), od(1) and dd(1) I recovered the three blocks used by the file. I thought, "boy, wouldn't it be nice if I had a program that would display a block from the file system as 1024 characters. The program would allow me to PAGE UP and PAGE DOWN, and search for an ASCII string anywhere on the disk, and then write out any block to a new file." So, I wrote such a program. It allows movement through- out a file system device, displays information in a couple of formats, will write blocks from the device onto another file, and allows rewriting words on the disk (I assume no responsibility for your use of that command!). The command is called de(1) "disk editor". -------------------- I also made a few changes to the Minix file system to aid recovering files. I-node numbers are retained in directory entries now (they get moved to the end). And all the i-node information is not zeroed-out when a file is unlinked. So, after a file is accidently rm'ed, you can find the old i-node, and then manually go to each of the freed blocks and write them to a new file. The movement and write commands are set up for doing this. And I was happy. I told Ed about it (well, actually I eventually tested it on his file system (his better than mine right!)). He wanted to know if the recovery could be automatic.....I thought it could be.....so I added the 'X' command and an "-r" option. So, believe it or not, you can accidently "rm file", and then immediately "de -r file" and everything comes back! -------------------- You can use de(1) without the file system changes, this gives you just the observation and manual recovery commands. The automatic recovery commands can be used if you decide to do the file system changes. This EFTH report includes a "README", the de(1) sources, a "make" file, a "man" page and 3 "cdiff"s to the Minix file system. Terrence W. Holm holm@bdc.ubc.ca ---------------------------------------------------------- echo x - Makefile sed '/^X/s///' > Makefile << '/' XCFLAGS = -Di8088 -T. -F XDEOBJ = de.s de_stdin.s de_stdout.s de_diskio.s de_recover.s X Xde: $(DEOBJ) X cc -i -T. $(DEOBJ) -o de X chmem =20000 de X chmod 4755 de X chown root de X chgrp bin de X X$(DEOBJ): de.h / echo x - README sed '/^X/s///' > README << '/' X de - A Minix Disk Editor X X Terrence W. Holm, Jan. 1989 X X XINTRODUCTION X X The de(1) disk editor allows a system administrator to X look at and modify a Minix file system device. Commands X allow movement throughout a file system device, displaying X information in a couple of formats, writing blocks from X the device onto another file, and rewriting words on the X disk. X X A few changes to the Minix file system aid recovering files. X I-node numbers are retained in directory entries now (they X get moved to the end). And all the i-node information is not X zeroed-out when a file is unlinked. So, after a file is X accidently rm(1)'ed, you can find the old i-node, and then X manually (or automatically) go to each of the freed blocks X and write them to a new file. X X XUSES FOR THE DISK EDITOR X X 1) EDUCATION. Students can look at a file system in X a painless manner. For example you don't have to X use od(1) to look at the zone numbers in i-nodes. X X A simple assignment is to change the size of an un-mounted X floppy disk file system from 360 to 300 blocks. (A more X difficult assignment is to explain why this works, even X though fsck(1) and df(1) do not report the correct number X of free blocks. :-) X X 2) ADMINISTRATION. You can visually check inconsistencies X reported by fsck(1) before letting fsck(1) fix them. X You can change any word on the disk, this greatly simplifies X editing file system information. For example, changing the X size of a block special device is actually fun, no more X "blind" writing to your partitions. X X Bit maps can be displayed with 2048 "bits" per screen, X (on the IBM/PC console), see how your zones are allocated! X X 3) RECOVERING LOST FILES. You can search a disk for an ASCII X string, once found, the block can be written out to a file. X X A one line change to fs/path.c allows users to get the i-node X number for a file after it has been removed from a directory. X X Another couple lines changed in the file system keep the X i-node information available until the i-node is reused X (normally this information is zeroed out when an i-node is X released.) This allows a de(1) user to go to a released X i-node, get all the block numbers, go to these blocks and X write them back to a new file. X X The whole recovery process is automated by running "de -r file". X So, IF a file is unlink(2)'ed (eg. "rm file"), AND IF no one X allocates a new i-node or block in the mean-time, THEN you X can recover the file. X X XRECOVERY SECURITY X X Normally Minix hard disk partitions are r/w only by the super-user, X and floppy disks are r/w by anyone. This means that only "root" X can look at hard disk partitions, but others can use de(1) to play X with their floppy disks. X X When recovering files ("de -r file"), a user requires access to X the major file system partitions. This can be done by: X X (a) Give everyone access to the hard disks. DON'T DO THIS, it X defeats all the file system protection we already have. X X (b) Make de(1) set-uid "root". This is the way to go, IF you X are running a Minix system that has NO ACCESS from the X outside. This allows anyone to execute "de -r file", but only X root to use "de /dev/hd3". De(1) does some checking when X retrieving lost blocks, eg. making sure they really are X free blocks and making sure the user owned the i-node. X BUT, file system information has been lost when the file X was unlink(2)'ed, so de(1) can not be 100% sure that a X recovered block really belonged to the user. THIS IS A X SECURITY HOLE. [Since the only access to my machine is from X observable terminals and their associated humans, I run X de(1) as set-uid root.] X X (c) Keep the disks rw-------, and don't set-uid de(1). This X means that only the super-user can recover lost files. X So, if you accidently "rm", you must tell the system X administrator to "su" and recover your file, (be sure to X inform the other users to stop whatever they are doing X until the file is restored). X X XINSTALLATION X X - Install de.1 in /usr/man/cat1. X X - Install the files: Makefile, README, de.h, de.c, de_stdin.c, X de_stdout.c, de_diskio.c and de_recover.c in commands/de. X Add -F and -T. to the Makefile, if necessary. X X - "make" de(1). If a header file is not found, don't worry: X You probably have it somewhere, just link it to what de(1) X is looking for. This program also requires the subroutine X tolower(3), see EFTH MINIX report #50, if you don't have it. X X - Do you really want set-uid root on de? X X - Patch the files fs/path.c, fs/link.c and fs/open.c. If X you don't patch the file system then the recover option X "-r" and associated commands ('x' and 'X') will not work, X but de(1) is still functional and useful. X X - "make" a new fs, using -DRECOVER. Rebuild a boot diskette. X X XUSING DE(1) FOR THE FIRST TIME X X De(1) starts up in "word" mode at block 0 of the specified X device. Hit the PGDN (or space bar) a few times, observing X all the information on the screen. Each PGUP/PGDN moves to X the next 1024 byte block, (de(1) only knows about 1 block per X zone file systems). Note that "word" mode only displays 32 X bytes at a time, so you are only observing the first 32 bytes X in the first few blocks when you skip using PGDN. X X Now go back to block 3, (zone bit map), using "g 3 ENTER". X Change to "map" mode "v m", and then use the down arrow key X to check each 2 Megs in the zone bit map. X X Now change to "block" mode using "v b". And go to some data X block, eg. "g 1000 ENTER". Use PGUP/PGDN to see what data X is in each nearby block. X X Remember 'h' gives you a help page. X X Try some more commands, for example: 'END', 'I', '/'. X (Note: searching through a whole disk under Minix takes a X long time: 30-60 seconds per megabyte, depending on your X machine, drive and controller, [Minix is embarrassingly slow].) X X Don't worry about looking at a mounted device, you must specify X the "-w" option before the 's' command is operational, and X this command is the only one which will try to modify the X contents of the device. X X XMINIX-ST X X Please contact me if you are interesting in attempting a port X to MINIX-ST. / echo x - de.1 sed '/^X/s///' > de.1 << '/' XNAME X de(1) - minix disk editor X XSYNOPSIS X de [-w] /dev/device X X de -r lost_file_name X XDESCRIPTION X De(1) allows a system administrator to examine and modify X a Minix file system device. Interactive observation of a X disk partition is initiated by a command line, for example: X X de /dev/hd2 X X Commands are available to move to any address on the disk X and display the disk block contents. This information may X be presented in one of three visual modes: as two-byte words, X as ASCII characters or as a bit map. The disk may be searched X for a string of characters. If the "-w" option is given, X de(1) will open the device for writing and words may be X modified. X X Lost blocks and files can be recovered using a variety of X commands. The "-r" option supports automated recovery of X files removed by unlink(2). X X X POSITIONING X X Disks are divided into blocks (also called "zones") of 1024 X bytes. De(1) keeps a current address on the disk as a X block number and a byte offset within the block. In some X visual modes the offset is rounded off, for example, in X "word" mode the offset must be even. X X There are different types of blocks on a file system device, X including a super block, bit maps, i-nodes and data blocks. X De(1) knows the type of the current block, but will allow X most positioning commands and visual modes to function X anywhere on the disk. X X The 'f' command (or PGDN on the keypad) moves forward to the X next block, similarly 'b' (PGUP) moves backwards one block. X 'F' (END) moves to the last block and 'B' (HOME) moves to the X first block. X X The arrow keys (or 'u', 'd', 'l' and 'r') change the current X address by small increments. The size of the increment X depends on the current display mode, as shown below. The X various sizes suit each display and pointers move on the X screen to follow each press of an arrow key. X X mode up down left right X X word -2 +2 -32 +32 X block -64 +64 -1 +1 X map -256 +256 -4 +4 X X X The 'g' command allows movement to any specified block. X Like all commands that take arguments, a prompt and X subsequent input are written to the bottom line of the X screen. Numerical entry may be decimal, octal or X hexadecimal, for example 234, -1, 070, 0xf3, -X3C. X X While checking an i-node one may want to move to a block X listed as a zone of the file. The 'G' command takes the X contents at the current address in the device as a block X number and indirectly jumps to that block. X X The address may be set to the start of any i-node using X the 'i' command and supplying an i-node number. The 'I' X command maps a given file name into an i-node address. X The file must exist on the current device and this X device must be mounted so that Minix can stat(2) it. X X X THE DISPLAY X X The first line of the display contains the device name, X the name of the current output file (if one is open) and X the current search string. If de(1) is being run with X the "-w" option then the device name is flagged with "(w)". X If a string is too long to fit on the line it is marked X with "...". X X The second line contains the current block number, the X total number of blocks, and the type of the current block. X The types are: boot, super, i-node bit map, zone bit map, X i-nodes and data block. See section 5.6.2 of the text for X an explanation and a diagram. If the current address is X within a data block then the string "in use" is displayed X if the block corresponds to a set bit in the zone bit map. X X The third line shows the offset in the current block. If X the current address is within either the i-node or zone bit X maps then the i-node or block number corresponding to the X current bit is shown. If the current address is within an X i-node then the i-node number and "in use" status is displayed. X If the address is within a bit map or i-node block, but past X the last usable entry, then the string "padding" is shown. X X The rest of the screen is used to display data from the X current block. There are three visual display modes: X "word", "block" and "map". The 'v' command followed by X 'w', 'b' or 'm' sets the current display mode. X X In "word" mode 16 words, of two bytes each, are shown in X either base 2, 8, 10 or 16. The current base is displayed X to the far right of the screen. It can be changed using the X 'o' command followed by either an 'h' (hexadecimal), 'd' X (decimal), 'o' (octal) or 'b' (binary). X X De(1) knows where i-nodes are, and will display the X contents in a readable format, including the "rwx" bits, X the user name and the time field. If the current page X is at the beginning of the super block, or an executable X file or an ar(1) archive, then de(1) will also inform X the user. In all other cases the contents of the 16 X words are shown to the right as equivalent ASCII X characters. X X In "block" mode a whole block of 1024 bytes is displayed X as ASCII characters, 64 columns by 16 lines. Control codes X are shown as highlighted characters. If the high order bit X is set in any of the 1024 bytes then an "MSB" flag is shown X on the far right of the screen, but these bytes are not X individually marked. X X In "map" mode 2048 bits (256 bytes) are displayed from the X top to the bottom (32 bits) and from the left to the right X of the screen. Bit zero of a byte is towards the top of the X screen. This visual mode is generally used to observe X the bit map blocks. The number of set bits displayed is X written on the far right of the screen. X X X SEARCHING X X A search for an ASCII string is initiated by the '/' command. X Control characters not used for other purposes may be X entered in the search string, for example ^J is an end-of- X line character. The search is from the current position to X the end of the current device. X X Once a search string has been defined by a use of '/', the X next search may be initiated with the 'n' command, (a '/' X followed immediately by an ENTER is equivalent to an 'n'). X X Whenever a search is in progress de(1) will append one X '.' to the prompt line for every 500 blocks searched. If the X string is found between the end of the file system and the X actual end of the device, then the current address is set to X the end of the file system. X X Some of the positioning commands push the current address X and visual mode in a stack before going to a new address. X These commands are B, F, g, G, i, I, n, x and /. The 'p' X (previous) command pops the last address and visual mode X from the stack. This stack is eight entries deep. X X X MODIFYING THE FILE SYSTEM X X The 's' command will prompt for a data word and store it at X the current address on the disk. This is used to change X information that can not be easily changed by any other X means. X X The data word is 16 bits wide, it may be entered in decimal, X octal or hexadecimal. Remember that the "-w" option must X be specified for the 's' command to operate. Be careful X when modifying a mounted file system. X X X RECOVERING FILES X X Any block on the disk may be written to an output file. X This is used to recover blocks marked as free on the X disk. A write command will request a file name the first X time it is used, on subsequent writes the data is appended X to the current output file. X X The name of the current output file is changed using the X 'c' command. This file should be on a different file system, X to avoid overwriting an i-node or block before it is X recovered. X X An ASCII block is usually recovered using the 'w' command. X All bytes will have their most significant bit cleared before X being written to the output file. Bytes containing '\0' or X '\177' are not copied. The 'W' command writes the current X block (1024 bytes) exactly to the output file. X X When a file is deleted using unlink(2) the i-node number X in the directory is zeroed, but before its removal, it is X copied into the end of the file name field. This allows X the i-node of a deleted file to be found by searching X through a directory. The 'x' command asks for the path X name of a lost file, extracts the old i-node number and X changes the current disk address to the start of the X i-node. X X Once an i-node is found, all of the freed blocks may be X recovered by checking the i-node zone fields, using 'G' X to go to a block, writing it back out using 'w', going X back to the i-node with 'p' and advancing to the next X block. This file extraction process is automated by using X the 'X' command, which goes through the i-node, indirect X and double indirect blocks finding all the block pointers X and recovering all the blocks of the file. X X The 'X' command closes the current output file and asks X for the name of a new output file. All of the disk blocks X must be marked as free, if they are not the command stops X and the file must be recovered manually. X X When extracting lost blocks de(1) will maintain "holes" in X the file. Thus, a recovered sparse file does not allocate X unused blocks and will keep its efficient storage scheme. X This property of the 'X' command may be used to move a sparse X file from one device to another. X X Automatic recovery may be initiated by the "-r" option on X the command line. Also specified is the path name of a X file just removed by unlink(2). De(1) determines which X mounted file system device held the file and opens it for X reading. The lost i-node is found and the file extracted by X automatically performing an 'x' and an 'X' command. X X The recovered file will be written to /tmp. De(1) will X refuse to automatically recover a file on the same file X system as /tmp. The lost file must have belonged to the X user. If automatic recovery will not complete, then manual X recovery may be performed. X X X EXITING THE DISK EDITOR X X The user can terminate a session with de(1) by typing X 'q', ^C, ^D, or the key associated with sigquit. X X The 'm' command invokes the Minix "sh" shell as a sub- X process. X X For help while using de(1) use 'h'. X X X COMMAND SUMMARY X X PGUP b Back one block X PGDN f Forward one block X HOME B Goto first block X END F Goto last block X X UP u Move back 2/64/256 bytes X DOWN d Move forward 2/64/256 bytes X LEFT l Move back 32/1/4 bytes ---------------------------------------------------------------