RAID 1 with offline elements

I created this webpage and submitted it for comments at comp.os.linux.misc.

Multipart containers

Recently, I wrote a response to a newsgroup question about spanning data blocks over multiple media**. That post refers to an earlier post with a detailed example in the "appendix" section. The advantage of using a container made from predefined data blocks is that cut/paste operations are not required. This is because the container is defined using a device mapper table. There are a couple of disadvantages. First, the size of the container must be determined in advance*, and there could be some wasted space when the last block is not full, etc. The other main disadvantage is the extra complexity that the technique introduces. Certainly, it is much simpler to work without introducing sub-blocks and work with one big file instead. Sometimes that is not possible- necessity demands a more complex solution. This technique still has a useful place for me, especially when data exceeds the size of one optical disc.

* While writing this, I realized that these containers are actually dynamically expandable if certain conditions are met. The first condition is that the filesystem used for the container must be expandable. I use XFS and it is dynamically expandable. The second condition is that data stored on the container must be compatible with expansion. If these conditions are met, then expanding the container could be as simple as allocating another block and adding another line to the device mapper table configuration.


Update: 2010-12-28
** Once again, Google decided not to archive my response on that thread. Here is the content:

>On Fri, 10 Dec 2010 21:12:03 +0000, Rui Maciel wrote:
>
> Is it possible to create a multipart gzip archive without resorting to
> the split/cat pair of commands?
>
>
> Thanks in advance,
> Rui Maciel
>
Almost anything is possible; whether it is practical is another matter.
If it is a "one of a kind" or rare event, then it may not be practical
to investigate other possibilities. That said, it was practical for me
to figure this out because I need to write in blocks which "span" across
optical media quite often.

I wrote about the technique here:
http://groups.google.com/group/comp.os.linux.misc/msg/6e34937952916a9a

Once you define a "container" you can put whatever you like into it
(simple data, compressed data, loopback filesystems, encrypted data,
etc.)

Making Windows Backups from GNU/Linux

misc. comp.os.linux.misc post,
and another (similar).

misc aols post

misc aols post

Update: 2010-10-24

Sometimes Google Groups doesn't capture posts, or in this case at least one of their servers did (How else did I get the ref number?). Anyway here it is copied from a long term newsgroup archive:

On Sun, 20 Dec 2009 02:24:34 +0000, no.top.post wrote:
 
> On my old system, I've only just figured out how to backup a tree by:--
> 
> mkisofs -R /mnt/p6/usr/lib  | cdrecord -v fs=6m speed=2 dev=0,0 -
> 
> Obviously I'd rather not have small [10% of CD capacity] dir-trees on
> each CD, so I'd want to serially write several dir-trees to the same
> CD.
> 
> Apparently the above instruction does:
>   AND_THEN 
> So then the 2nd step would start writing from the CD beginning ?
> 
> But this is too simplistic since it's seen from the
> messages-while-running that a fifo is repeatedly written and read.
> 
> So how can I read multiple dir-trees and write them to the CD ?
> 
> == TIA

IME, data written to a CD needs to be "staged" in some way. The staging
directory provides the top-level directory structure for the final CD.
You might be able to rig something different, but I am not aware of how
to do it. Multisession optical may do something _close_ to what you
want. Feel free to experiment, but be prepared to invest some time and
media in making sure it is doing what you want.
 
Here is some additional unsolicited information- FWIW. One size does not
fit all when selecting an appropriate backup strategy. YMMV.
 
I use optical discs as my primary long-term backup medium. Their
durability and  cost vs. capacity remains somewhat competitive when
compared to magnetic devices. However, making an optical disc requires
more time and effort. To minimize the time required, I use a strategy
which stages the backup data from disk-to-disk-to-optical. In my case,
daily snapshots are created automatically, and are allowed to accumulate
in a staging directory. I can simply write them out to optical when
convenient. BTW, because the data is staged, the snapshots can be "post
processed," for example, I use both compression and encryption. When the
final disc image is ready, it is written to optical as a single session,
and then verified. Multiple copies can be made for redundancy and as
necessary.