dmsdos.doc

This is the main documentation for the DMSDOS CVF module. 19NOV1998


This is HTML document has been derived from dmsdos.doc in dmsdos version 0.9.2.0.

Contents:

1. DMSDOS capabilities
2. Restrictions
3. Mount options
4. Kernel messages [moved to file messages.doc]
5. Installation
6. Troubleshooting [moved to file troubleshooting.doc]
7. Defragment procedures
8. How to contribute to dmsdos development
9. The external dmsdos utility 'dutil'
10. The dmsdos daemon 'dmsdosd'
11. The dmsdos library 'libdmsdos.a'
12. The dmsdos filesystem checker 'dmsdosfsck'
13. The dmsdos Midnight Commander external filesystem utility 'mcdmsdos'
14. Authors and email addresses

links other documentation:
 Description of the CVF-FAT interface
 Description of CVF Filesystem Layout
 DMSDOS Frequently Asked Questions


1. DMSDOS capabilities

[**] No, don't really think of burning a CVF onto a cdrom. Dos cannot read it unless you copy it to your hard disk. REALLY BAD. Linux+dmsdos can.

In detail:

Dos compressed partitions are always stored inside an uncompressed host partition as large files (CVFs, Compressed Volume Files). In order to access them from Linux, you need * the CVF-FAT interface, which is present in kernels 2.1.80 and newer and must be patched into the 2.0.xx kernel series - the patches are included in this dmsdos release * and the loopback block device. The latter lets you mount a file as a filesystem. You must compile the loopback block device into your kernel or as a loadable module in order to use this feature.

This version works with compressed hard disks and with compressed floppies. It has been tested successfully with MS-DOS 6.2 Doublespace and MS-DOS 6.22 Drivespace compressed formats. win95 doublespace is supported, also the drivespace 3 format from the Micro$ Plus package (rumors tell that this package has been integrated into Win95OSR2). Stacker version 3 and 4 are supported too. Dmsdos can even mount CVFs from cdrom (Dos can't).

Francesco Potorti(pot@foda-devel.cnuce.cnr.it) has reported that dmsdos also mounts Superstore compressed filesystems. I can't test this as I don't have Superstore. They are recognized as old Dos 6.0 Doublespace format. I'm interested in further reports about this. I still can't believe that M$ simply copied (bought?) Superstore and renamed it to Doublespace without further modifications...

Since version 0.9.0, dmsdos runs *below* the msdos/fat filesystem. This makes it compatible with all fat based filesystems including FAT32, NLS and other things :)

Dmsdos does a lot of internal caching in order to reach a reasonable speed with a compressed (and really badly organised, but simple) filesystem. As a side effect, this cache may eat a lot of system memory (you can change the size limit).

Dmsdos was initially inspired by the THS filesystem (once found under ftp://sunsite.unc.edu/pub/Linux/system/Filesystems/dosfs/thsfs.tgz) written 1994 by Thomas Scheuermann (email address out of date), which was the first filesystem available under Linux that could read some CVFs.

When mounting a compressed filesystem, the dmsdos driver does some quick and simple consistency checking. If it finds a serious filesystem error that might cause further damage on write access, it sets the compressed filesystem to read-only mode. The filesystem check can be switched off if you don't like it.

The dmsdos daemon included in this release can speed up dmsdos write access on systems that usually run at high processor load using delayed compression in the background when the system is idle. I recommend to use it if you are going to write much to compressed partitions from Linux and need the processor time mostly for other tasks. The daemon also has a significant effect on dmsdos memory management which might be important for systems with little memory (well yes, dmsdos *is* a memory monster that can eat up to 2MB of system memory for caching in the default configuration). See chapter 'The dmsdos daemon' for details.

The dmsdos library is a stand-alone tool for raw CVF programming. 'Stand-alone' means that it works independently from the dmsdos kernel module, on *raw* CVFs (i.e. those large files lying around in an uncompressed host partition). It's the same source code, though, just compiled in a slightly different way. Meanwhile, it compiles even under Windows :)

Back to Table of Contents



2. Restrictions

(See also file BUGS for a list of known bugs.)

Performance degradiation (well, not really bad in most cases...)

Access to compressed filesystems slows down your system. This is a fact that results from additional time needed to decompress or compress the data and to maintain additional filesystem structures. It's an untrue rumor (sometimes spread by compressed filesystem software sellers) that compressed filesystems speeded up slow ISA bus systems because of the smaller amount of data to be transferred through the slow ISA bus - there's too much overhead otherwhere.

Though most time-consuming actions like compression and finding free sectors have been greatly improved since dmsdos 0.8.0, there are still sudden system freezes reported. Reasons for this include (in rising order):

Compression and free sector finding may be fine-tuned with some of the dmsdos options. This is not really explained here, see mount options cf:xx and speedup:xx.

Dmsdos never freezes the system totally, it periodically unfreezes even on extremely hard disk access. You *can* login into another console and you *can* enter commands there. No keystroke gets lost though you may not see a reaction on the console immediately, but after some seconds you should see something. Just be warned and prepared :)

Compressed formats

The scheme I named the compression methods is simply this: Take the first four bytes of a compressed cluster. For the DS-0-2 method, for example, they are 'D','S',0x00,0x02.
 
Format:  Used by: Readable:  Writable:
DS-0-0 win95 doublespace yes  yes
DS-0-1 never reported to exist works in theory  :)
DS-0-2 msdos 6.2  doublespace yes yes
JM-0-0 msdos 6.22 drivespace and 
win95 drivespace 3
yes yes
JM-0-1 win95 drivespace 3 'High' yes yes
SQ-0-0 win95 drivespace 3 'Ultra' yes yes
uncompressed all versions yes yes
SD3 (no header) Stacker 3 yes yes
SD4 (0x81-0) Stacker 4 yes yes

Yes, meanwhile all currently known methods are supported (hmm, why is this still listed under restrictions...)

Fragmented clusters in drivespace 3 volumes

They can be read since dmsdos 0.8.1pl5. They can be written since 0.9.0pl10, but writing fragmented clusters is supposed extremely experimental. If dmsdos needs to rewrite data that are stored in a fragmented cluster it first deletes the old data and saves the new data in the usual linear order. Fragmentation writes are only done if the data do not fit on the disk in an unfragmented way. Currently, dmsdos becomes VERY noisy when it writes fragmented clusters... This is considered experimental and needs further testing. You can currently switch off this feature by setting speedup bit#8 (this is a mount option, see below).

Fragmented and suballocated clusters in Stacker volumes

Of course, they can be read. But currently, they cannot be written. If dmsdos needs to rewrite data that are stored in a fragmented or suballocated cluster it currently deletes the old data and saves the new data in the usual linear order. Fragmented write may come soon, suballocated write is more complex and currently considered not very important for dmsdos. We'll see...

Swapping to a file on the compressed partition

Just No. It doesn't work. Don't bother me with ideas of swapping to a compressed ram disk which would be a memory compressor... bah! That's all gibberish, crap, and dogsh*t. I won't write code to support this. You know very well that swapping has to be extremely fast, so use a partition on your fastest disk for this purpose. Not a swapfile, and never in a fat based filesystem (just imagine what a filesystem driver has to do in order to access the *end* of a large file... yeah, tons of fat access...)

attention, DOSEmu users

You may have to unmount all fat based partitions before running DOSEmu depending on your configuration. If DOSEmu is configured to use wholedisk or partition access (this is often the case to let DOSEmu access compressed partitions) there's a risk of destroying your compressed partitions or crashing your system because of confused drivers.

Let me explain why. Dmsdos does a lot of internal caching, but cache contents won't be up to date if DOSEmu directly modifies the disk. This confuses dmsdos and may mess up your data. On the other hand, dos dblspace/drvspace.bin also has a kind of caching mechanism that gets confused by the Linux dmsdos driver's write access. Confusion even occurs if one party mounts read-only and the other mounts read-write.

Note that it is always safe to redirect the compressed partitions with lredir or emufs.sys. Refer to this table for configurations that are safe (marked with +) and which are dangerous (marked with -):
 
 
DOSEmu:  Linux fat based filesystems
not mounted

mounted ro

mounted rw
wholedisk ro + + -
wholedisk rw + - -
partition ro + + -
partition rw + - -
lredir/emufs.sys ro/rw (impossible) + +

Memory mapping

*Should* work. :^)

Underlying dos filesystem limitations

None. Really, they're gone since the CVF-FAT interface is used.

Umsdos users warning

If you want to use umsdos-style long filenames in compressed filesystems, use kernel 2.0.xx or get at least kernel 2.1.94.

Win98 compatibility

HEY! Windows People ! Everyone uses it in the Windows World(TM) :)) Let me know whether it works, please. (I refuse to buy Win98 just for such a test.) Or have you all switched to WinNT yet ?

WinNT compatibility

No, WinNT does not support compressed FAT partitions. But... :)) There's meanwhile a small chance to port dmsdos to WinNT. This is somewhat interesting because neither Doublespace nor Stacker runs under WinNT. But I can't do it myself because of lack of knowledge :( For more information, see file PORT_TO_WIN32. The trick is hacking in the dmsdos library so it compiles under that operating system... I just managed to compile one dmsdos utility, mcdmsdos, under WinNT...

Last but least

Dmsdos has been written for dos/win95 compatibility. It's just a relict from the time when I programmed some software that had to run both under dos and Linux and I had to compress my drive because hard disk space was rather expensive. Times and prices have changed a lot since then (at least in the country where I live), but dmsdos has still been improved for fun.

If you don't need dos/win95 compatibility, i.e. if you want to compress Linux-only data, let me say: DON'T DO IT. Clean up your disk, or buy a larger one, but don't compress it. Really. Compress single files (e.g. manpages) with gzip. Throw away the crap you don't need. Shrink your dos partition or remove it completely :)

If you still want to compress Linux data, dmsdos may not be the software you want. The msdos filesystem is extremely unsuited for fast disk access, umsdos suffers from it too. The CVF structure msdos uses is even more unsuited, and the time required by msdos online compression may kill the rest of Linux' performance on these filesystems.

Just to let you know about some other online compression software for Linux: There's a package 'double' available on sunsite for Linux online compression. There are also compression patches for the ext2 filesystem available under the name 'hufs' and 'e2compr'. I've also heard about a 'zlibc' project which lets you access gzip compressed files as if they were uncompressed - it works by a patched libc. Look at common Linux ftp sites.

Like dmsdos, all those packages may have their specific advantages and disadvantages. Some are said to be not 100% stable, but who can claim that his software is really 100% stable :-) Please don't flame at me if the packages don't do what you want - I haven't tested them and I don't know much about their qualilty.

Back to Table of Contents



3. Mount options

CVF related mount options are usually surrounded by the CVF-FAT interface options. These are "cvf_format=xxx" and "cvf_options=yyy". The string "xxx" can be "dblspace" for a doublespace or drivespace CVF and "stacker" for a stacker CVF. For more information about the "cvf_format=xxx" option, especially in cooperation with kerneld and automatic module loading, see the dmsdos installation instructions (file INSTALL.TXT) and the generic CVF-FAT documentation.

When we speak about dmsdos mount options, we usually mean the string "yyy" in "cvf_options=yyy". That's what this chapter describes.

Summary:

The dmsdos driver currently recognizes the following options:

comp:xxx, cf:xxx, bitfaterrs:xxx, loglevel:xxx, speedup:xxx

(for backwards compatibility also comp=xxx, cf=xxx, bitfaterrs=xxx and loglevel=xxx, speedup=xxx are accepted)

The xxx has to be replaced by an appropriate parameter, of course. All options have to be specified separated by plus signs (not commas) in a list in the mount command: 'mount -t msdos -o cvf_options=option1+option2+option3 ...'. Consider the string 'cvf_options=option1+option2+option3' as *one* FAT driver option! For backwards compatibility, also a dot (.) is recognized as option separator symbol.
 

The string 'option1+option2+option3...' in the FAT mount option 'cvf_options=option1+option2+option3...' has a hard coded limit of 100 characters (this is a limitation of the CVF-FAT interface). The string is silently cut down if it is too long. Don't worry, you won't manage to break the limit without specifying senseless parameters :)

For those who want to know everything exactly, the CVF-FAT interface has a further FAT driver option (*not* dmsdos option): cvf_format=xxx. The parameters that the dmsdos module allows for 'xxx' are 'dblspace' (for all doublespace and drivespace versions) and 'stacker' (for all stacker versions). Usually you don't care about them since the formats are detected automatically. They may, however, be useful in order to trigger kerneld to load the dmsdos module on demand. For details please look at the dmsdos installation instructions (file INSTALL.TXT) and the generic CVF-FAT interface documentation (file linux/Documentation/filesystems/fat_cvf.txt).

And now a warning: If you want to mount a CVF *be sure* to have the dmsdos module loaded. The plain FAT driver also mounts some CVFs without complaining (but begins to scream loudly when you, for example, do a 'ls' on the mountpoint). As some CVFs look very similar to normal, uncompressed filesystems, the FAT driver has no chance to detect a CVF at mount time (and e.g. print an error) when the dmsdos module is not loaded.

*** If you always specify a "cvf_format=something" option you never run into that problem.

Some simple examples:

1. Your msdos partition that contains CVFs is on /dev/hda1. You want to mount it under /DOS and the CVF under /mnt.
mount -t msdos /dev/hda1 /DOS mount -t msdos -o loop /DOS/dblspace.001 /mnt

2. Like above, but you need umsdos support.
mount -t umsdos /dev/hda1 /DOS mount -t msdos -o loop /DOS/dblspace.001 /mnt

3. Like above, but you also have umsdos style long filenames in your CVF
mount -t umsdos /dev/hda1 /DOS mount -t umsdos -o loop /DOS/dblspace.001 /mnt

4. Like 1., but you need Win95 long filename support *only inside* the compressed partition.
mount -t msdos /dev/hda1 /DOS mount -t vfat -o loop /DOS/dblspace.001 /mnt

5. Like 1., but you want to feel safe and refuse write access to the compressed partitions only (but the uncompressed host should be writable).
mount -t msdos /dev/hda1 /DOS mount -t msdos -o ro,loop /DOS/dblspace.001 /mnt

6. Like 4., but you need Win95 long filenames also in the uncompressed host partition
mount -t vfat /dev/hda1 /DOS mount -t vfat -o loop /DOS/dblspace.001 /mnt

7. You have managed to burn a compressed filesystem onto a CD (This is quite easy - put the large file 'dblspace.001' into an iso9660 filesystem and burn it. Be warned, dos can't mount it from CD. Uh...)
mount -t iso9660 /dev/cdrom /cdrom mount -t msdos -o loop /cdrom/dblspace.001 /mnt

8. Like 1., but the CVF has been umssynced and contains umsdos long filenames.
mount -t msdos /dev/hda1 /DOS mount -t umsdos -o loop /DOS/dblspace.001 /mnt

Some complex examples

The option strings might look strange but they are correct:

9. Like 1, but you want the driver to repair bitfat errors automatically.
mount -t msdos /dev/hda1 /DOS mount -t msdos -o loop,cvf_options=bitfaterrs:repair /DOS/dblspace.001 /mnt

10. Like 1, but you need more speedup due to high-traffic write access (well, please note that this is not recommended, but it works):
mount -t msdos /dev/hda1 /DOS mount -t msdos -o loop,cvf_options=speedup:255 /DOS/dblspace.001 /mnt

11. Like 9 and 10 (repair bitfat errors and more speedup):
mount -t msdos /dev/hda1 /DOS mount -t msdos -o loop,cvf_options=bitfaterrs:repair+speedup:255 /DOS/dblspace.001 /mnt

12. Like 1, but disable write access, ignore bitfat error and do debug logging (be careful, this causes tons of debug messages):
mount -t msdos /dev/hda1 /DOS mount -t msdos -o loop,ro,cvf_options=bitfaterrs:ignore+loglevel:0xffffffff /DOS/dblspace.001 /mnt

Mounting via /etc/fstab:

This may be a bit tricky. A typical fstab entry for a CVF looks like this:

/DOS/drvspace.001 /DOSF msdos loop 1 0
/dev/hda1 /DOS msdos defaults 1 0

Be sure to keep the sixth field (the 'pass number') zero to prevent fsck from checking the CVF (it does not know about CVFs and very likely fails). Also the filesystems in /etc/fstab seem to be mounted in reverse order, but that may depend on your mount or libc version. Try to reorder the lines it it doesn't work.

Back to Table of Contents



4. Kernel messages

This section has moved to file messages.doc.

Back to Table of Contents



5. Installation

Please refer to the file INSTALL.TXT in the main directory of any dmsdos package. It may contain dmsdos version dependent things. Be sure to check this file in the dmsdos package you download.

Installation usually means this:

As installation may be quite complex, dmsdos comes with a set of scripts that do most of the work for a standard kernel. But the automatic installation cannot know whether you have already applied bugfix or other patches. If you do not want to care about that,  fire up your modem and download the latest non-development version of the kernel :)

Back to Table of Contents



6. Troubleshooting

This section has moved to file troubleshooting.doc.

Back to Table of Contents



7. Defragment procedures

There are three possibilities for fragmentation of a compressed partition:

How to defragment a CVF at internal MDFAT level see your dos/win95 or stacker documentation. Dmsdos does not support CVF maintainance.

Back to Table of Contents



8. How to contribute to dmsdos development

First of all, thanks a lot to all alpha testers and all who have sent problem reports, found bugs in the sources or documentation, suggested solutions to some problems and asked questions due to unclear documentation. Without your help dmsdos would not have become what it is.

Also many thanks to all who simply mailed they liked dmsdos.

At this place, I must also say many thanks to Gordon (VFAT fs) and Matija (UMSDOS fs) for the help to make dmsdos work with the standard 2.1.xx kernels.

If you have a problem that is not covered in the documentation please email me. But, *please*, be sure to read the file 'troubleshooting.doc' before. If it's not mentioned there, you may have uncovered a bug that should be fixed. Please watch your kernel log (/var/log/messages) for strange dmsdos messages too. Please include the following information in your mail message:

Back to Table of Contents

9. The external dmsdos utility 'dutil'

(For a summary, take a look at the dutil manual page (file src/dutil.1).)

Run dutil on a dmsdos mount point to obtain additional information about the compressed partition. Example: 'dutil /mnt'. This utility has some additional debugging options. These are not described here since they are for debugging only. Run dutil with debug options only when you are asked by the dmsdos author/maintainer in a response to any problem report.

Note: Debugging options require running dutil as root. Other users can only see the general filesystem information dutil prints on the screen. See file dblspace_ioctl.c for details on what can be done only by root.

You can use the external utility dutil to setup or change dmsdos options later. Use 'dutil /mnt setcomp xxx' (comp:xxx option) and 'dutil /mnt setcf xx' (cf:xx option). Note to replace the directory name '/mnt' with yours. This also works: 'dutil /mnt setspeedup xxx' (speedup:xxx option) and 'dutil /mnt setloglevel xxx' (loglevel:xxx option). The dmsdos options can only be changed by root.

To perform a simple filesystem check, use the command 'dutil /mnt checkfs'. This fs check doesn't do any repairs, but immediately reports some fs inconsistencies. There shouldn't be any unless the driver complained already at mount time (see kernel log). If there are errors and the dmsdos driver didn't find them at mount time, this is a serious dmsdos bug (it implies that data have been destroyed by the dmsdos driver) and should be reported. If the command complains about errors but the dos fs checker says everything is okay, you should also send a bug report.

To sync the write-back cache you can use 'dutil /mnt synccache'. By default, the command waits until the data have been passed to the kernel's filesystem buffering interface. (If you want to ensure that they are really physically written issue a 'sync' afterwards which causes the kernel to write all dirty buffers to disk.) You can specify an additional argument !=0 in order to pass the cache contents to the daemon instead of writing directly to disk (not useful if you want to ensure that the data are written *now*).

Note that dutil displays two different compression ratios - an internal and an external one. The internal one is always lower and is used to estimate how much space is left on the device. It depends on which information can actually be obtained from the MDFAT without scanning the whole disk (would take too much time). If you want to compare different versions of compressed filesystems to each other (e.g. doublespace vs. stacker) concerning the compression effectivity, always use the external (second) one because it is always calculated in the same manner. The external compression ratio is calculated according to this formula:

Space that would be used on an equivalent uncompressed msdos filesystem
------------------------------------------------------------------------- : 1
Space that is used on the disk

'Space' means what file and directory clusters occupy, it does *not* mean: FAT, root directory, futher filesystem maintainance information.

Back to Table of Contents



10. The dmsdos daemon 'dmsdosd'

The dmsdos daemon is a program that runs in the background and does delayed compression when the system is idle. This prevents the dmsdos driver from slowing down your system when compressing large amounts of data. In newer dmsdos versions, the daemon has also some influence on dmsdos memory usage (it shrinks the amount of memory dmsdos eats for caching if the dmsdos driver has not much to do). This might be an advantage for systems that are short on memory.

Actually, the daemon exists in two variants. The first one is a user-level process (dmsdosd) that must be started with a command line like 'dmsdosd /mnt' or 'dmsdosd /mnt xx', where /mnt is the mount point and 'xx' is the compression level like in the cf:xx mount option. If the xx is omitted it defaults to 12 (i.e. maximum compression). Please note that this variant of the dmsdos daemon does not care about the dmsdos cf:xx option or the dutil setcf command, but uses its own from its command line. This variant of the dmsdos daemon is called 'external daemon'. For a summary concerning this variant of the dmsdos daemon you can also take a look at the dmsdosd manual page (file src/dmsdosd.1).

The second variant is built inside the kernel and starts up automatically when it is needed and exits when it is no longer needed. This variant is called 'internal daemon' and cannot be killed from user space, not even by root. It also obeys the cf:xx option and the dutil setcf command.

You can only use one daemon at a time. You must decide before compiling dmsdos which variant to use. During dmsdos configuration ('make config') you are asked which one you want. Both daemon variants do the same work, but the internal one is more user-friendly (well, you don't have to start it manually :-) ) while the external one is more flexible and seems to eat up less system time (but has other backdraws - you must kill it before you can unmount the filesystem that has been given as a command line argument). If you are using the internal daemon, the external one won't start - it always exits with a "no permission" error.

If you are going to use the external daemon, please note this: DON'T start one daemon for each filesystem. The one single daemon does all the work for the filesystem given in its command line and for all other dmsdos filesystems you have currently mounted and for all other dmsdos filesystems you mount later while the daemon is running. To prevent security problems only root can start the external daemon (and even more, the driver refuses to talk to a non-root dmsdos daemon via the ioctl interface). The external daemon may be killed (by root) at any time, even when it is under hard work. Send a SIGINT to it to let it finish its actual work and then exit. Send a SIGTERM to it to make it exit immediately. In any case, the daemon catches the signals and exits cleanly.

The dmsdos daemon works like this: When it starts up, it tells the dmsdos driver not to compress the data when doing write access to the compressed filesystem. Instead, the driver writes the *uncompressed* data to the disk and maintains a list of clusters that have to be compressed. When the daemon has time to do something, it asks the driver for a cluster to compress. Then the driver looks up a cluster in its 'clusters-that-must-be-compressed' list, reads it from disk and passes it to the daemon. The daemon itself now compresses the data. When it has finished compression, it passes the compressed cluster back to the driver, which in turn writes it to the disk.

There are two situations the list gets lost and the data that have not yet been compressed by the daemon remain uncompressed on the disk:

  1. Rebooting the system clears the whole list.
  2. Unmounting the filesystem clears all the list entries that contain clusters of the filesystem to be unmounted.
Please note that this kind of "cache" has a limited size. If you exceed that size by writing *much* data at once the dmsdos driver no longer lets the daemon do the job, but starts to compress the data itself. You will notice a moderate system slowdown in that case. In extreme situations you might not even see an immediate reaction on a keystroke, but those cases have become rare. (If you suddenly see a *great* system slowdown this is usually caused by too much fragmentation - not by compression.)

The size of this "cache" can be selected during 'make config'. The number specified there means the maximum (i.e. total, not per CVF) number of clusters that can be in the list. If you want to know the amount in bytes, multiply with the cluster size (which is 32KB for drivespace 3 and 8KB for the other compressed filesystems). The dmsdos driver may use up to this amount of _uncompressed_ disk space inside each compressed partition as "cache". You can use dutil to find out how much uncompressed free space is on the disk. If it is less than the "cache" size you shouldn't write more than the amount of uncompressed free space to the disk at once. Otherwise, the disk is getting full with the cached uncompressed data and you receive a 'no space left on device' error.

If there's nothing to compress any more the daemon goes to sleep mode, but when the dmsdos driver needs the daemon it tries to awake it immediately. Furthermore, the daemon awakes periodically to check for data and inform the driver when it is time to release memory.

Please note that using the daemon causes *duplicate* write access to the filesystem. This results in a lot of more MDFAT level fragmentation. Thus, the daemon cannot really make your system faster. In fact, it only makes your system *appear* faster by doing a kind of load balancing. This load balancing avoids unnecessary locks (kind of wait loops) in the driver. It means the time some processes would have to wait can be used by other processes.

As I'm sometimes asked what to recommend: For a system with read-only or very little write access and enough memory you don't need the daemon. For a system with moderate write access and an appropriate fast processor the internal daemon is best choice. For a system with moderate to high-traffic write access to compressed partitions I recommend the external daemon. If you have a slow processor and you need CPU time for some other intensive computing, it's best to use the external daemon with a nice value (see 'man nice') - this prevents the daemon from slowing down the other processes. If the daemon does not compress fast enough try to lower the compression level in its command line. Well, and if you want to write a large chunk of data (some 100MB) to a compressed partition at once, disable the daemon before - it cannot help in this situation. It only causes too much fragmentation which in turn slows down the system horribly.

Just another hint: If you want to use the daemon only for memory management but not for delayed compression, you can clear speedup bit #7.

You see, there's much room for experimentation here, especially if you use the external daemon :) There are some pre-configured dmsdos config files which just implement some of the above recommendations. This is explaind in INSTALL.TXT so see there for details, please.

Back to Table of Contents



11. The dmsdos library 'libdmsdos.a'

This feature is currently experimental, but begins to become more and more interesting.

The dmsdos library is intended to provide some low-level dmsdos functions to user processes. The library is not new source code - it's just the dmsdos module source compiled in a different way (well, with some tricks). The dmsdos library aims towards working with raw CVFs, not filesystems. It is usable only for *not* mounted CVFs. Well, for mounted ones there's the dmsdos module :)

Documentation (including a detailed description of the interface functions) can be found in file libdmsdos.doc.

Back to Table of Contents



12. The dmsdos filesystem checker 'dmsdosfsck'

The goal of dmsdosfsck is not having a complete and reliable filesystem checker for CVFs. If you want that, use the filesystem maintainance tools that came with your CVF software package under Dos (or blame their manufacturers if they are not working well...). dmsdosfsck is just meant as a kind of workaround for problems that may show up if the dmsdos or FAT driver suddenly detects an unexpected inconsistency and might get confused. With dmsdosfsck those problems that may confuse the drivers can be detected and corrected before mounting the CVF.

The command line of dmsdosfsck looks like this:

dmsdosfsck [-a] [-r] [-v] [-l] device|file

where -a means repair automatically (uh... be VERY careful), -r means repair interactively, -v means be verbose, and -l means list the files while processing them. These options are somewhat similar to the well-known dosfsck.

Currently not all errors can be corrected. It just corrects fatal FAT errors like loops, crosslinks, illegal values, and unexpected ends of chains - things the kernel FAT driver does not like at all and tends to crash or hang if it runs over them. (Well, this is a question of philosophy. Should a kernel driver be idiot proof or may it assume the filesystem is valid in most cases at the most vital places? I don't want to give an answer, but the FAT driver of Linux 2.0.34 seems not 100% idiot proof.) So there is really a need for a dmsdosfsck utility here.

Furthermore, it can detect but not correct MDFAT errors, crosslinks, damaged MDFAT entries - you must repair them under Dos using the CVF maintainance tools that came with your CVF software package. The kernel FAT driver does not see these structures, and the dmsdos driver simply returns read errors - those errors just cannot confuse drivers or even make the system unusable.

It can detect and correct BITFAT allocation errors - this is necessary for write access. (But the dmsdos module already does it if it is told to do so at mount time.)

It detects damaged directory entries and can correct them. This is also a situation the kernel FAT driver doesn't like at all. Be warned, repairing damaged directories is a very critical job which cannot be automated completely. Some damaged directories cannot be repaired and will be converted into files instead. This gives you a chance to take a low-level disk editor and repair this by hand under Dos later (if you know how to do it). Repaired directories are saved without compression so you should be able to analyse them under Dos without further help.

It also checks whether one filename is used more than once in a directory. The file can optionally be renamed. This is usually done by appending ~%d to the filename where %d is the lowest possible number not causing a conflict.

It also detects lost cluster chains but does not correct them. They just use space and do nothing evil. For that, use the Dos tools that came....:)

*** Currently, dmsdosfsck must be considered extremely alpha test ***

It's far away from complete, but some useful things work. You *can* use dmsdosfsck with the generic fsck frontend. You can even run your usual fsck when mounting from /etc/fstab at boot time, even on CVF partitions. But that's not trivial. If you really want to do such things, take a look at the example script 'msdos.fsck-wrapper' which is suitable for the generic fsck frontend. Be sure to understand what the script does. Change it to your requirements. Then replace the link in /sbin/fsck.msdos (or /etc/fs/fsck.msdos) by a link to the script :)

Back to Table of Contents



13. The dmsdos Midnight Commander external filesystem utility 'mcdmsdos'

Dmsdos comes with an interface utility that accepts standard Midnight Commander commands for reading archive files. The utility is named 'mcdmsdos' and is compiled during usual dmsdos compile process. The utility is currently read-only.

Please refer to the documentation of Midnight Commander for further information on how to configure an external filesystem. You may want to write a small shell script as a wrapper around mcdmsdos in order to suppress output to stderr distorting the screen, e.g.

#!/bin/sh
exec /usr/local/bin/mcdmsdos $@ 2> /dev/null
Furthermore, mcdmsdos can be used as a utility to quickly look at what is inside the CVF and to extract single files from a CVF. Today mcdmsdos even works in a Win32 environment, so it may be worth to learn its command line parameters :)) If documentation of newer versions of Midnight Commander wants some more commands consider the respective operations to fail. I expect something like remove, rmdir, mkdir etc. which are missing in the current standard (I wrote this utility according to Midnight Commander 3.0 documentation). But that doesn't matter as long as mcdmsdos is read-only :)

Back to Table of Contents



14. Authors and email addresses

CVF-FAT/dmsdos is the official successor of the former dmsdos filesystem. The dmsdos filesystem was initially written from scratch by Frank Gockel, after taking a close look at the THS filesystem (a read-only doublespace fs with only DS-0-2 decompression written by Thomas Scheuermann). Stacker support and drivespace 3 'ultra' compression/decompression were added by Pavel Pisa as well as improved DS/JM compression and decompression routines. Meanwhile, it contains several parts of code that was directly provided or code that is based on the ideas from a lot of people over the net in order to fix bugs, to improve performance, and to add features.

The dmsdos code is distributed under the GNU General Public Licence (see file COPYING).

The dmsdos driver is currently maintained by Pavel Pisa (stacker access, SQ compression, meanwhile all other compression too) and me, Frank Gockel (rest of the code).

Pavel's email address is pisa@cvlinux.felk.cvut.cz, my email address is gockel@sent13.uni-duisburg.de.

If you want to contact me via email, please write in English or take a close look at the country codes in the email addresses :)

Back to Table of Contents