Adding data offline

Trunkton

Just converted from unRaid to WHS 2011 + DrivePool. The data is reiserfs and YAReG tool is way too slow. I'd like to boot into Ubuntu and move the files over with ntfs-3g. Is the process simply move them into an existing pool HDD member or is there a special location it would go?

Currently have 3x empty (added to DrivePool) 2TB HDD's and one full 2TB HDD of reiserfs data.

Shane

Partitions added to the pool have a hidden PoolPart.* folder created in their root directory. Partitions should be basic (not dynamic) NTFS volumes. If you need to move data into the pool manually (e.g. if you need to boot into Ubuntu to access the source data), you can move the data into any folder within the hidden PoolPart.* folders; when you boot WHS the folder will be visible in Explorer in your Pool drive.

E.g. you have three drives (E,F,G) in the pool (H), and you create a folder called "myfolder" in your pool drive; the contents of H:\myfolder are the virtually aggregated contents of the same folder on E,F,G as follows:

H:\myfolder | E:\PoolPart.4d4f34c9-3f30-45c0-9452-5c75b82983ab\myfolder

            | F:\PoolPart.4d4f34c9-3f30-45c0-9452-5c75b82983ab\myfolder

            | G:\PoolPart.4d4f34c9-3f30-45c0-9452-5c75b82983ab\myfolder

Where the same file exists in the same location on more than one of the pool's physical disks, DrivePool treats this as a duplicated file (I believe using NTFS's alternate streams to manage them). I would recommend that you (a) create the "myfolder" using WHS, (b) have duplication turned OFF for that folder, (c) after moving the files reset their permissions so that they inherit their permissions from that folder, (d) force a consistency check from the Pool tab. Then you can turn duplication back on if you wish.

E.g. you move the first thousand files into E's poolpart, the next thousand into F's, the next thousand into G's, or whatever order suits. Just don't move the same file into more than one of the poolpart folders.

Shane

(E, F and G don't have to have the same poolpart string, that's just me using copy-and-paste)

Trunkton

The challenge there is I definitely have duplicates. If it wasn't for that this would be so perfect!

Thank you for the help, someone who doesn't have duplicates will appreciate this guide. I may be better off merging it in from another PC running Ubuntu/Linux Mint over the network so that it gives me options when a duplicate is detected. I wish transfers went faster than 40 MB/sec using 1gbit network cards/router, moving all this data will take awhile.

EDIT: This is taking too long, I think I will use a second NTFS HDD, from my second PC move data off the reiserfs system into that, then attach it to WHS directly and move the data to the network share //whs/myfolder...

Quick question, will it be okay to move into E: (pool drive) instead of //whs/myfolder? I am thinking this will be faster since the NTFS drive will be attached locally.

Shane

Drivepool doesn't necessarily care if the source involves (partially or fully) duplicated trees.

Where H: is the pool and E: and F: are drives in the pool, Drivepool considers this to be a duplicate (even if the files have different content):

H:\myfolder | E:\poolpart.guid\myfolder\acme\brand\fudge.txt

            | F:\poolpart.guid\myfolder\acme\brand\fudge.txt

Drivepool does not consider this to be a duplicate (even if the files have identical content):

H:\myfolder | E:\poolpart.guid\myfolder\acme\fudge.txt

            | F:\poolpart.guid\myfolder\zorg\fudge.txt

Note that if, in the first example, the two files have identical content and metadata, DrivePool will be happy. If they do not have identical content and metadata, Drivepool will consider this a problem to be rectified and warn you (when it discovers the problem, which can be forced by telling it to run a consistency check).

Yes, there's no problem using the pool drive rather than the pool shares.

EDITS: typos, clarifications, etc.