Duplication and Balancing Never Finish

Otagolad · December 2012

Hi,

Hopefully someone might be able to help with an issue I am having with StableBit DrivePool. I have recently built a new WHS 2011 Server (spec's below) and transferred all my data to it from my HPEX495 WHSv1 system.

The new WHS 2011 system was running fast and smooth until I added StableBit DrivePool (latest stable release NOT the current Beta) whereupon everything has slowed to a crawl and at times has simply seized up. The issue appears to be around finalising the duplication and on-going balancing. Duplication has hit 88% but never progresses and the machine continually seems to be balancing.

I have tried turning balancing off and to just do it at 1am in the morning and although that helps for a bit the duplication (which is why I got DrivePool) just won't finish. This results in the Dashboard freezing up and crashing.

I have heard that people have installed WHS 2011 and DrivePool on some reasonably low spec systems so I was expecting that mine would be more than capable - any ideas, suggestions on speeding things up as I've gone from having a reasonably responsive WHSv1 system to having a Server that I can't stream anything from.

Spec's are as follows (let me know if you need further info):

1. Used an old HP d5000t desktop as the base - used this as the ATX motherboard was highly expandable.

2. The Server's spec's are:

(a) Processor - Core 2 Quad Q9550 (K) (2.83GHz /1333 MHz) 12 MB L2 cache.

(b) Memory - 8 GB (4 x 2 GB) DDR2 PC2-6400 (4 DIMMS)

(c) OS is loaded on a Samsung 830 256GB SSD.

(d) I have installed two of "Sonnet Tempo's SATA E2P
2 Port 3Gb/s eSATA PCI Express Host Adapter with Port Multiplication" to hang my two 5 bay enclosures off.

(e) I have 15 Drives connected to the Server as follows:
3 x 2TB Western Digital Black Caviar Drives in the d5000t case
2 x 3TB Western Digital external USB drives
2 x Sans Digital TowerRAID TR5M-B enclosures with a total of 19TB of storage between them.

JoeMatt · December 2012

Are your USB drives in your pool? With such an old motherboard they are probably running with USB 1.0 that could slow things down to a crawl

.

Otagolad · December 2012

Hi JoeMatt,

Not sure what you mean - the motherboard has "6 USB 2.0 headers supporting 8 USB ports or devices". The link to the motherboard is here:

http://h10025.www1.hp.com/ewfrf/wc/document?docname=c01357064&tmp_task=prodinfoCategory&cc=us&dlc=en&lc=en&product=3750363#N366<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

Otagolad · December 2012

Also, some further information if it is helpful:
1. The Pooled data shows as 9.64TB, however the Duplicated data shows as 8.19TB and this hasn't changed in over 24 hours.
2. The "Pool Condition" says "Duplicating... (~66.9%)" and this hasn't changed in over 24 hours - also the yellow striped bar beside the words "Pool condition" are static and are not moving.
3. Even though the Pool shows no change, when I look at Resource Manager there is a large amount of CPU and Disk activity.
4. As soon as I stop the DrivePool Service in Windows Task Manager the full speed of the Server returns.

Shane · December 2012

Hmm. Do the Task Manager and Resource Monitor (from Task Manager or Start bar) give you any indication of which particular processes are dominating CPU and disk activity?

Does disabling the Windows Search service help at all?

Have you tried forcing a duplication consistency check (Dashboard, Server Folders, Pool tab, select Pool drive, select check task)?

Have you tried sequentially removing your external drives and drive arrays from the pool, then sequentially physically removing them from the server, to isolate potential conflicts?

Otagolad · December 2012

Hi Shane,

Launching Dashboard basically locks up the system although it has been better since I disabled all balancing and duplication in DrivePool.

I've decided to take the extreme route and rebuild the server. I've gone with an Asus P8Z77-V PRO Motherboard with an Intel Core i5-3570 processor and 16GB of Ram - should be fast enough to do what I want.

Big question I have is, once I install the new parts and do a reinstall of WHS 2011 on my SSD (assuming Microsoft will let me - only had the server running a couple of weeks) what do I have to do to ensure the Pool and all the Data on the disks is available - do I simply need to plug the drives back in after reinstalling DrivePool - is there anything I would need to do prior to rebuilding the server? I have made (i) a "bare metal" backup of the Server (C drive and system - not all the data as I don't have enough spare disks) and (ii) a Server restore DVD - anything else needed?

Any help would be most appreciated.

Shane · December 2012

That sounds right - backup OS drive (since WHS formats the entire drive during install), then install WHS with only the OS drive connected, then reconnect your data disks, make sure OS and drivers are up to date, then install DrivePool. It should pick up your existing pool. The WSS Troubleshooting Tool can be downloaded from http://wiki.covecube.com/StableBit_DrivePool_Utilities to restore any missing shares and fix any permissions issues.

Note that DrivePool (especially when you first start "filling it up") can place a load on various subsystems during balancing, duplication and indexing, so what appears to be DrivePool "causing" problems can actually be it revealing drivers/hardware that aren't 100%. That's not to say it can't be DrivePool, though, and if you still run into problems with the Dashboard freezing etc, please send a bug report via http://stablebit.com/contact

(e.g. I'd be checking if the Sonnet Tempo adapters have an available firmware or driver update that may apply)

Otagolad · December 2012

Hi Shane,

That's really helpful. One last query - after I do the clean re-install of WHS 2011 do I then do a recovery from the backup I've done or is there no need for this unless I encounter problems.

Thanks also for the tip on the Sonnet Tempo adapters - would it be worth trying to get some 6GB/s adapters? I've had an initial look around and am struggling to find any with port multiplication. 3GB/s should be fine for most things surely?

Shane · December 2012

Restoring from the backup is only if things go wrong during the reinstall (note: you'd be restoring the version that had the old motherboard drivers). I try to not keep irreplaceable apps/data on the OS drive for this reason.

Re adapters, I find 3Gb/s plenty. My drives do about 60MB/s sustained each anyway; if you had five such drives all going flat out simultaneously it'd be 300MB/s or 3Gb/s (enough to saturate your link), but I'd suspect most home/office users aren't realistically going to need to do that very often. Your mileage may vary, of course.

Otagolad · December 2012

Thanks Shane - much appreciated

Otagolad · December 2012

Hi Shane,

So I have now built my new server, updated all drivers, added in DrivePool and then used the Troubleshooting Tool to show my pool again. So far so good. I then started the duplication process and after 24 hours it has duplicated only 10.5GB of the 9.71TB. The Dashboard is not freesing up and the new i5 CPU is not under any real stress, rather my concern is simply whether it is going to take almost 100 days to duplicate - seems rediculous but at the rate it is going that's what will happen.

I've tried to run a "duplication consistency check" as you suggested earlier and the message I get is "A duplication check is already running".

I've attached three screenshots showing the "Pool" tab in "Server Folders and Hard Drives" and also the "Folders" and "Disks" screens within the StableBit DrivePool add-in.

Any thoughts on how to speed this up would be most appreciated.

Otagolad · December 2012

The following pictures are for the support staff as I've lodged a support query and can't attach these to theat message for some reason.

Shane · December 2012

Any change in the speed of the duplicating? I know that disk latency can really slow down duplication if it's having to deal with a very large number of small files, but 24 hours for 10.5GB is... well, beyond terrible and deep into ludicrous. Something's just Not Right.

Hmm. Just flinging darts blindfolded at the board here, but what happens if you untick "Network I/O Boost" (StableBit DrivePool tab, Disks section), so that DrivePool isn't prioritizing network requests over its own local activity?

Otagolad · January 2013

Hi Shane,

Pleased to announce everything is now working (mainly) fine. The issue was down to a Seagate drive that once removed sped everything up immensely.

One small issue that has cropped up over the last day is that during duplication StableBit DrivePool gets to 49.9% and then says that it can't complete duplication because some files are in use. I closed all documents and programs (e.g. iTunes, Word etc.) on my laptop that access files on the Server and left it to do its thing overnight but still had the same issue the next morning. I haven't tried powering off and restarting the Server yet so will try that next and see if it resolves the issue. If you've any ideas on this or seen this before then any hints would be helpful.

Shane · January 2013

Sometimes a program can close without releasing the files it was using, or Windows decides to inspect/whatever a file and forgets to let go. A server restart should fix both of these.

If not, another possibility is a mangled permission issue, for which the troubleshooting tool is handy. http://wiki.covecube.com/StableBit_DrivePool_Utilities

Howdy, Stranger!

Categories

Poll

Duplication and Balancing Never Finish

Comments