Re: [SLUG] On compression and safety

[Scott Howard <scott@xxxxxxxxxx>]

On Thu, Aug 16, 2001 at 12:01:59PM +1000, Jamie Honan wrote:
> 
> 
> >In fact, compression of data on backup tape is generally a risky thing
> >- if the media degenerates, and gathers a few bad sectors, if there is
> >no compression, you can still get the data on either side of the bad

This depends on the type of compression you do. If you compress the
stream as it goes to tape (eg, tar cf - . | gzip -c > /dev/tape), and
you're using a compression which doesn't do some form of fixed-block
compression (ie, you're using gzip or most any other compression
program), then thats right.

If you're compressing individual files and then writing those files
to tape, then you'll most likely be able to recover all but the actual
damaged files. There is (or at least, was) a variant of cpio (minds
gone blank on the name) which did this.  It's also what many 
commercial backup programs do.

If you're compressing at the hardware (tape drive) level then it's
probably a block-level compression, where corruption will just result
in one block of data being corrupt, not the entire stream.

> Are there standard filters that do 'spreading error codes through
> data' e.g reed-solomon to help recovery.

Don't know about software, but DLT drives do this by default - they use
some form of ECC coding which can handle up to a certain bit error
rate from the tape.

  Scott.