Does DP support filling disk-at-a-time (minimize folder splits)

mvd · March 2013

Hi there, evaluating DrivePool, I'm a little confused on how to set DrivePool to fill a disk at a time instead of scattering my files across multiple disks.

This is critical because all my disks spin down when not in use, and if lets say a Bluray or DVD folder based rip, or a sequential series of photos has been scattered multiple pool disks, then the pool will appear to freeze when a movie needs the next VOB/M2TS file or my photo viewer is trying to load the next photo.

Any advice? Thats pretty much my last stumbling block before purchase. Thanks.

mvd · March 2013

After playing with the File Placement Limiter plugin some more it seems like I can manually set one disk at a time to fill up, but this is incredibly inefficient if I want to let an overnight copy job fill up say 10 disks, because it would require a manual intervention to select a new disk in File Placement Limiter every time a disk fills up.

I really hope DP adds the ability to fill a disk at a time automatically to minimize folder splits.

lee1978 · March 2013

Hi,

Drivepool is not really designed for what your trying to do your trying to switch of all the functions of drivepool and do it all manually.

While I am sure it is possible to do what you want I doubt it will be as straight forward the software is designed to combine multiple drive's and spread the data across the drives using simple file placement rather than creating complicated raid arrays.

In the end all your drives will end full with data anyway so why not let drivepool do it the easy way instead of making it complicated.

mvd · March 2013

I understand what DP was designed for, but it doesnt seem like that difficult of a programming tweak to give us the OPTION of minimizing folder splits by filling up a disk at a time. FlexRaid has this ability, however because DP introduced the ability to add the pool drive to be indexed by windows search, thats a killer feature and why I'm evaluating it.

Minimizing folder splits is important for files that are sequential in nature -- photos, videos. If I dump thousands of photos from my camera onto the pool and they end up on all different disks, then when I go to review them later the pool will appear to freeze as I'm moving through them because of having to spin up additional disks.

Or take the example of a DVD or Bluray folder based rip. If I begin playing the movie and the next VOB file is on another disk, the movie will freeze while the next disk spins up. That's a dealbreaker. And converting them to MKV or ISO is not the answer, there are reasons and advantages for keeping them folder based. If I wanted multiple disks spinning up to access pool data I'd just stay hardware raid.

Drashna · April 2013

Conversely... if the drive is already spun up, spreading the IO load between disks would grant you a performance boon.

Also, there is an API to allow you to more greatly configure the balancing, but that requires .NET programming experience.

Shane · April 2013

Hi mvd, there's been calls for this before; I'd suggest a feature request via http://stablebit.com/contact

Hopefully this would indeed be something simple to implement, and not one of those easier-said-than-coded problems. Mention what you've mentioned here.

mvd · April 2013

Hi Shane,

Alex already got back to me and is considering the option. The fact its been requested before means other people would benefit and thats a good thing.

I think the developers of DB had some great foresight building a modular plugin architecture into the software so that functionality could be extended quickly without major rewrites or reengineering.

JazJon · April 2013

Alex released two plug-in's for drivepool 2.x

http://www.stablebit.com/DrivePool/Plugins

I don't see the logic for why they are not part of the normal build though. (maybe to "test")

Drashna · April 2013

I'd say that it's more of a development thing. Include the critical balancers, and then add any additional ones as an optional install. A good way to keep things cleaner.

JazJon · April 2013

Another update

https://twitter.com/covecube/status/322132817079320576

"A new balancing plug-in for StableBit DrivePool: http://www.stablebit.com/DrivePool/Plugins … (Ordered File Placement - check it out)"

http://www.stablebit.com/DrivePool/Plugins

mvd · April 2013

*WOW* Thank you Alex. He said via PM "hopefully within a few weeks" and then he's already finished a few days later.

This is going to be a boon for people that have their disks set to spin down after a period of inactivity, since minimizing folder splits means minimizing the need for multiple disks to spin up when browsing through files that are sequential in nature.

mika91 · April 2013

Very nice!!!

I'm waiting for a "folder based" plugin, then I'll buy my drivepool licence

Drashna · April 2013

Yeah, Alex is pretty awesome.

mvd · April 2013

Very nice!!!
I'm waiting for a "folder based" plugin, then I'll buy my drivepool licence

Do you mean like a "Folder Limiter" plugin, aka "explicit merge"? As in defining specific folders and limiting them to specific disks..

Was going to bug Alex about that next, but didnt want to risk sounding ungrateful since he banged out the "Ordered File Placement" plugin so fast. Will probably wait a few months. Maybe I'll start a thread on it. There are many reasons I could rattle off that the ability to limit specific folders/files to specific disks would provide useful, which I'll spare for another time - and I'm sure there are other power users that can think of a few (keeping related files together, or keeping certain files on faster disks or SSD's).

AWerginz · April 2013

i think he means when you use blu-ray folder structure it doesnt spread them over your pool,

so the blu-ray folder will be on one disk,

but your collection will still be pooled over several drives

mika91 · April 2013

i think he means when you use blu-ray folder structure it doesnt spread them over your pool,
so the blu-ray folder will be on one disk,
but your collection will still be pooled over several drives

Yes, it's the idea.
I'd like to organise my pool, the same way unraid do: specify spreading level for folders.

The idea is to keep subtitles on the same disk as the movie, bluray/DVD folder on one disk, ...

I tried to write my own balancing plugin, but didn't understood how it works.

Drashna · April 2013

It requires using .NET for programming the plugin. So if you're not familar with... C# I think it was, and the .NET stuff, then yeah, you could get lost *very* quickly.

Howdy, Stranger!

Categories

Poll

Does DP support filling disk-at-a-time (minimize folder splits)

Comments