Server Re-index time under 1.2.7145

Wonko · November 2012

Just updated to 1.2.7145. I can see all of my 12 pool folders (2.10 TB total pooled) under the StableBit DrivePool tab. Client Computer Backups showed up almost immediately at 1.11 TB. The other 11 folders are still sitting at 0 B after over an hour of re-indexing despite their containing less than 1 TB total. I can view the individual folders' size and have them computed in under a minute under the Server Folders and Hard Drives tab.

Why is the re-indexing computation under the StableBit DrivePool tab taking so much longer than viewing the individual folders and not appearing to accomplish anything?

Thanks.

Wonko · November 2012

Followup posting: Initial re-index finally stopped after almost 5 hours. Unfortunately, several folders still show 0 B, when they have in excess of 100 GB each. Hit the re-index link again, and it appears to be starting over from scratch. Any suggestions to both improve performance and accuracy would be most appreciated.

ChipMonk · November 2012

I have exactly the same experience (WHS 2011 Intel Atom D510MO). I think it's related to the new reliance on Windows Indexing. I usually try to have minimal Windows indexing so as not to drain the cpu and RAM on a limited machine used mainly for audio streaming and backup. I expanded indexing to see if this solved the issue. It did, after a fashion, but DrivePool folder info was still not very reliable. I have now completely disabled windows indexing because the drain on my system was disproportional to the limited benefits. This is the first glitch I've noticed in an otherwise excellent package.

I also tried the new Archive Optimizer Plugin and was surprised to see it only offered drives already within the pool for initial transfer. I had wanted to experiment with using the non-pooled

\ drive (remainder of the HDD holding the OS) for this purpose.

Hope these comments useful.

Wonko · November 2012

Followup posting: Reindexed overnight. Still have four folders with 100+ GB each showing 0 B. Windows Search does not appear to be a good source for this information. The last release version did a much better job of capturing and reporting folder size data.

Shane · November 2012

@ChipMonk: if you added

to the pool but set it as the landing zone for the Archive Optimizer, the result should be what I presume you intended - DP will not use

for keeping files, only as a landing zone with the files moved into the "real" pool at the scheduled balancing time (unless the pool had no room left, of course).

ChipMonk · November 2012

@Shane: ah - that makes sense. Thanks for the landing zone tip.

Did DrivePool have it's own indexing system before this latest version? I'm still trying to understand why the folder information suddenly deteriorated with this latest version.

Thanks

Shane · November 2012

It had its own routine to track folder sizes. I'd speculate (ie wild guess) that Alex figured that since Windows Search can do this job, why reinvent the wheel? Of course, it's not always a given that something in Windows actually does what its documentation says it does, or when or how it does it either....

Shane · November 2012

Oh, and note that the landing zone still obeys the rules for duplication, so if you have real-time duplication enabled you will need two drives for the zone (otherwise it will have to use one of the non-zone drives).

ChipMonk · November 2012

Thanks again Shane - no, duplication is not an issue for me, but worth knowing. Regarding Windows Indexing - I have found it to be a pretty inefficient resource hog from the earliest days of windows and not seen much sign of improvement. DrivePool is a much system than the equivalent MS package in WHS v1 ever was and I think that the same probably applies to Covecube's in-house indexing. Can we have it back please??

ChipMonk · November 2012

Oops - missing word in last post "DrivePool is a much BETTER system than the equivalent MS package in WHS v1 ever was"

TRaSH · December 2012

i would like to see the old way also back,

since the new option uses windows indexing it only gives problems for most people when reading here on the forum.

some quotes from several posts here

Windows Indexing - I have found it to be a pretty inefficient resource hog (think on low powered systems)
Reindexed overnight. Still have four folders with 100+ GB each showing 0 B.(what about people with TB's on storage will it take weeks should we turn off all other services so the re-index doesn't get interupted ?)
I usually try to have minimal Windows indexing so as not to drain the cpu and RAM on a limited machine ............expanded indexing to see if this solved the issue. It did, after a fashion, but DrivePool folder info was still not very reliable. I have now completely disabled windows indexing because the drain on my system was disproportional to the limited benefits.
Initial re-index finally stopped after almost 5 hours. Unfortunately, several folders still show 0 B, when they have in excess of 100 GB each. Hit the re-index link again, and it appears to be starting over from scratch.

seems to me the old system worked much better

Ahlitah · December 2012

I wonder if the windows indexer is what was causing my ntfs.sys blu screen in the latest version.

As when it happens, DP was indexing the drives as it was just installed and rebooted.

Then after a min or 2, then poof blu screen nfts.sys!

TRaSH · December 2012

test it.
turn it off and see if you still get this error.

Doug · December 2012

I guess it all depends on your hardware. I have 22TB of data/10 share folders and when I upgraded to 1.2.7145 all folder reported 0b cause neither the drives or the pool were being indexed by windows. Turned on Indexing for the pool drive and 2 hours later, folders were reporting properly.

How are you setting up indexing?

ChipMonk · December 2012

Sure it's all about hardware - the Windows indexing resource hog doesn't matter much if there's loads of cpu firepower. But, as I mentioned above, on a lightweight server designed for energy efficient media streaming and home backups only, Windows Indexing is slow, tedious and resource consuming and, in conjunction with Drivepool, giving less than satisfactory results for the cycles consumed. I've chosen to switch it off and miss out on the "nice to have" folder breakdown in the current implementation.

TRaSH · December 2012

Thinking of doing the same i got a nice low powered HP Microserver 40L,

with this new "feature" it's a step backward what did work without any issues with the old version.

and without a resource hog running in the background

Alex · December 2012

Just to clarify,

The indexing behavior did not change in DrivePool 1.2.1+.

DrivePool doesn't index your folders, the Windows Home Server 2011 does. It always did, this is how Microsoft set it up.

In DrivePool 1.2.1+, instead of relying on DrivePool's own folder size tracking system, it simply queries the already existing index in order to show you folder size statistics.

One thing you may notice on low powered systems is that DrivePool's Dashboard tab takes a while to populate the folder usage chart. I've noticed that Windows Search is slow to respond, especially on Atom based systems.

I think that because the Dashboard is used purely for administrative purposes, that this delay is acceptable if it means that we have to keep track of less things in real-time.

Anyone disagree? Thoughts?

Wonko · December 2012

Alex,

I have two concerns: the first is that WHS 2011 does not seem to accurately index the folders, so the data is not accurate compared to DrivePool's own folder size tracking system. Why that is, I do not know--indexing seems to be set up correctly on my server and I have re-built multiple times while still getting 0B in multiple 100 GB+ folders. The second is it takes a long time to get the folder sizes even with an i7 2600K and WD 4 RE drives even for relatively small folders. The data DrivePool provided before was reasonably accurate, always instantly available, and reflected the cost of duplication, so I was much happier with it. Appreciate your efforts to develop an extremely useful product.

Wonko · December 2012

kenergy007 · December 2012

I'm in the same boat, using 1.2.4.7226 I turned off windows search service thinking it would speed things up, but then I realized drivepool uses the windows index, so I turned back on and its taking re-indexing hours. I have 29tb of data over 15 drives.

I'm gonna let it run over night see how far it gets.

slyboy · February 2013

Same as the above comments. My Folder sizes under DrivePool 1.2.4.7226 were working earlier, but I think I mistakenly clicked on the Re-Index link on the folders tab and now I've found myself waiting hours with 9 folders still showing 0 B reported and the other four folders of mine showing incomplete totals for now. My totals populated much quicker before. I hope this is a one-time wait and that subsequent visits to this page will populate immediately because like this the information is useless.

TomTiddler · February 2013

More a question than a comment. I recently made several changes to my server all at the same time (I know - fail). Added another drive bay with 2 new drives, changed my USB 3 controller card, and installed Stablebit Scanner. (I run latest beta of both scanner and drivepool)

Net result is everythings great except for a) re-indexing times, and b) restart times.

QUESTION on indexing :- Should both teh drive pool "drive", AND the individual hard drives be set for indexing?

COMMENT on restart -- I get many alerts on a restart (most annoying is the fact that the Client Update Service fails to start every time, even on a wake from sleep), and it takes almost 10 minutes before everything is settled.

Any help/observations welcome

Shane · February 2013

Re indexing, select only the pool drive, not the physical drives that form it (unless you have non-pool data stored on those drives).

Re restarting, don't know sorry. Anyone?

TomTiddler · February 2013

Thanx. The restart delays may also be related to the re-indexing issue. We'll see.

Howdy, Stranger!

Categories

Poll

Server Re-index time under 1.2.7145

Comments