Splitting Directories across physical drives?

chard0119 · May 2011

I am looking to use DrivePool as a replacement to DE. However, one of my biggest gripes about DE was that it split directories across physical drives. This made is VERY hard to deal with a failing/failed drive (when the "remove drive" function wouldn't work) that wasn't duplicated. I ran into an issue where I had close to 6TB worth of ripped DVDs that I did not duplicate (due to the size of the share). One of the drives failed and I lost the movies on that drive (which I expected). However, the problem that I didn't realize is that even on the good drive the directories for each of the movies listed there were not complete (some of the files were stored on the failed drive). So I ended up losing close to half of my movies (and not knowing which were complete and which weren't until literally trying to load the movies in WMC7).

So... long story short -- I am hoping that DrivePool has come up with a way to maintain the "pooling" benefits of DE without the need to split directories across physical drives. This would make this new product EXTREMELY valuable to me (and I'm sure many others as well). I know that there are other ways to mitigate this issue (such as folder synch across drives, or backing up to external NAS units). But I would love it if DrivePool had this kind of functionality.

I appreciate the time to read and respond.

Shane · May 2011

If you take a look at http://forum.covecube.com/discussion/6/drivepool-file-distribution-algorithm this has come up before. Back then Alex mentioned putting plugins (so you could, say, have a "keep directories together" plugin) on the todo list, I don't know if that is still on the cards.

I agree this would be a useful "harm minimization" feature, and also of some convenience if the server itself keeled over; e.g. not having to go searching across multiple disks to collate a single directory. Though I suspect I'd write a little script to handle that (as it wouldn't be just one directory but potentially thousands).

Note that since DP allows normal access to the underlying physical drives, even if "keep directories together" is not available as a feature/plugin, creating a "balancing" utility to do it on a regular schedule should be quite feasible.

P.S. If you have a lot of disks with relatively static data but don't want to commit half of them to backups or duplication, have you considered a parity utility? Protection against single disk failure is still better than nothing.

chard0119 · May 2011

Thanks for the comment Shane. I'll look at that other discussion thread. I'm planning on using something like Synctoy to maintain synch between each main data drive on my server with another linked drive. This way, there will always be two drives that have exactly the same content (even before backing up to an external NAS).

However, I would really love to do this along with a drive pooling tech.

Alex · May 2011

How about a "pin a shared folder to a pooled drive or a set of drives" feature?

If the drives run out of space, DrivePool will have no choice but to place files on other drives in the pool. It should probably issue a notification letting you know that this is happening.

I'm going to file this under great idea, but we don't have time to do this right now.

Balancing plugins are still in the plans.

Howdy, Stranger!

Categories

Poll

Splitting Directories across physical drives?

Comments