Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with Google

Our Sites

covecube.com 
community.covecube.com 
blog.covecube.com 
wiki.covecube.com 
bitflock.com 
stablebit.com 

Poll

No poll attached to this discussion.

In order to better support our growing community we've set up a new more powerful forum.


The new forum is at: http://community.covecube.com


The new forum is running IP.Board and will be our primary forum from now on.


This forum is being retired, but will remain online indefinitely in order to preserve its contents. This forum is now read only.


Thank you,

Server Re-index time under 1.2.7145

edited November 2012 in DrivePool

Just updated to 1.2.7145. I can see all of my 12 pool folders (2.10 TB total pooled) under the StableBit DrivePool tab. Client Computer Backups showed up almost immediately at 1.11 TB. The other 11 folders are still sitting at 0 B after over an hour of re-indexing despite their containing less than 1 TB total. I can view the individual folders' size and have them computed in under a minute under the Server Folders and Hard Drives tab.

Why is the re-indexing computation under the StableBit DrivePool tab taking so much longer than viewing the individual folders and not appearing to accomplish anything?

Thanks.

Comments

  • Followup posting: Initial re-index finally stopped after almost 5 hours. Unfortunately, several folders still show 0 B, when they have in excess of 100 GB each. Hit the re-index link again, and it appears to be starting over from scratch. Any suggestions to both improve performance and accuracy would be most appreciated.
  • I have exactly the same experience (WHS 2011 Intel Atom D510MO). I think it's related to the new reliance on Windows Indexing. I usually try to have minimal Windows indexing so as not to drain the cpu and RAM on a limited machine used mainly for audio streaming and backup. I expanded indexing to see if this solved the issue. It did, after a fashion, but DrivePool folder info was still not very reliable. I have now completely disabled windows indexing because the drain on my system was disproportional to the limited benefits. This is the first glitch I've noticed in an otherwise excellent package.

    I also tried the new Archive Optimizer Plugin and was surprised to see it only offered drives already within the pool for initial transfer. I had wanted to experiment with using the non-pooled D:\ drive (remainder of the HDD holding the OS) for this purpose.

    Hope these comments useful.
  • Followup posting: Reindexed overnight. Still have four folders with 100+ GB each showing 0 B. Windows Search does not appear to be a good source for this information. The last release version did a much better job of capturing and reporting folder size data.
  • Resident Guru
    @ChipMonk: if you added D: to the pool but set it as the landing zone for the Archive Optimizer, the result should be what I presume you intended - DP will not use D: for keeping files, only as a landing zone with the files moved into the "real" pool at the scheduled balancing time (unless the pool had no room left, of course).
  • @Shane: ah - that makes sense. Thanks for the landing zone tip.

    Did DrivePool have it's own indexing system before this latest version? I'm still trying to understand why the folder information suddenly deteriorated with this latest version.

    Thanks
  • edited November 2012 Resident Guru
    It had its own routine to track folder sizes. I'd speculate (ie wild guess) that Alex figured that since Windows Search can do this job, why reinvent the wheel? Of course, it's not always a given that something in Windows actually does what its documentation says it does, or when or how it does it either.... :p
  • edited November 2012 Resident Guru
    Oh, and note that the landing zone still obeys the rules for duplication, so if you have real-time duplication enabled you will need two drives for the zone (otherwise it will have to use one of the non-zone drives).
  • Thanks again Shane - no, duplication is not an issue for me, but worth knowing. Regarding Windows Indexing - I have found it to be a pretty inefficient resource hog from the earliest days of windows and not seen much sign of improvement. DrivePool is a much system than the equivalent MS package in WHS v1 ever was and I think that the same probably applies to Covecube's in-house indexing. Can we have it back please??
  • Oops - missing word in last post "DrivePool is a much BETTER system than the equivalent MS package in WHS v1 ever was"
  • edited December 2012 Member
    i would like to see the old way also back,
    since the new option uses windows indexing it only gives problems for most people when reading here on the forum.
    some quotes from several posts here
    1. Windows Indexing - I have found it to be a pretty inefficient resource hog (think on low powered systems)
    2. Reindexed overnight. Still have four folders with 100+ GB each showing 0 B.(what about people with TB's on storage will it take weeks should we turn off all other services so the re-index doesn't get interupted ?)
    3. I usually try to have minimal Windows indexing so as not to drain the cpu and RAM on a limited machine ............expanded indexing to see if this solved the issue. It did, after a fashion, but DrivePool folder info was still not very reliable. I have now completely disabled windows indexing because the drain on my system was disproportional to the limited benefits.
    4. Initial re-index finally stopped after almost 5 hours. Unfortunately, several folders still show 0 B, when they have in excess of 100 GB each. Hit the re-index link again, and it appears to be starting over from scratch. 

    seems to me the old system worked much better
  • I wonder if the windows indexer is what was causing my ntfs.sys blu screen in the latest version.

    As when it happens, DP was indexing the drives as it was just installed and rebooted.

    Then after a min or 2, then poof blu screen nfts.sys!


  • test it.
    turn it off and see if you still get this error.
  • I guess it all depends on your hardware.  I have 22TB of data/10 share folders and when I upgraded to 1.2.7145 all folder reported 0b cause neither the drives or the pool were being indexed by windows.  Turned on Indexing for the pool drive and 2 hours later, folders were reporting properly.

    How are you setting up indexing?

  • Sure it's all about hardware - the Windows indexing resource hog doesn't matter much if there's loads of cpu firepower. But, as I mentioned above, on a lightweight server designed for energy efficient media streaming and home backups only, Windows Indexing is slow, tedious and resource consuming and, in conjunction with Drivepool, giving less than satisfactory results for the cycles consumed. I've chosen to switch it off and miss out on the "nice to have" folder breakdown in the current implementation.
  • Thinking of doing the same i got a nice low powered HP Microserver 40L,
    with this new "feature" it's a step backward what did work without any issues with the old version.
    and without a resource hog running in the background
  • Covecube
    Just to clarify,

    The indexing behavior did not change in DrivePool 1.2.1+.

    DrivePool doesn't index your folders, the Windows Home Server 2011 does. It always did, this is how Microsoft set it up.

    In DrivePool 1.2.1+, instead of relying on DrivePool's own folder size tracking system, it simply queries the already existing index in order to show you folder size statistics.

    One thing you may notice on low powered systems is that DrivePool's Dashboard tab takes a while to populate the folder usage chart. I've noticed that Windows Search is slow to respond, especially on Atom based systems.

    I think that because the Dashboard is used purely for administrative purposes, that this delay is acceptable if it means that we have to keep track of less things in real-time.

    Anyone disagree? Thoughts?
  • edited December 2012 Member

    Alex,

    I have two concerns: the first is that WHS 2011 does not seem to accurately index the folders, so the data is not accurate compared to DrivePool's own folder size tracking system. Why that is, I do not know--indexing seems to be set up correctly on my server and I have re-built multiple times while still getting 0B in multiple 100 GB+ folders. The second is it takes a long time to get the folder sizes even with an i7 2600K and WD 4 RE drives even for relatively small folders. The data DrivePool provided before was reasonably accurate, always instantly available, and reflected the cost of duplication, so I was much happier with it. Appreciate your efforts to develop an extremely useful product.

  • edited December 2012 Member
  • I'm in the same boat, using 1.2.4.7226 I turned off windows search service thinking it would speed things up, but then I realized drivepool uses the windows index, so I turned back on and its taking re-indexing hours. I have 29tb of data over 15 drives.

    I'm gonna let it run over night see how far it gets.
  • Same as the above comments.  My Folder sizes under DrivePool 1.2.4.7226 were working earlier, but I think I mistakenly clicked on the Re-Index link on the folders tab and now I've found myself waiting hours with 9 folders still showing 0 B reported and the other four folders of mine showing incomplete totals for now.  My totals populated much quicker before.  I hope this is a one-time wait and that subsequent visits to this page will populate immediately because like this the information is useless.
  • More a question than a comment. I recently made several changes to my server all at the same time (I know - fail). Added another drive bay with 2 new drives, changed my USB 3 controller card, and installed Stablebit Scanner. (I run latest beta of both scanner and drivepool)

    Net result is everythings great except for a) re-indexing times, and b) restart times.

    QUESTION on indexing :- Should both teh drive pool "drive", AND the individual hard drives be set for indexing?

    COMMENT on restart -- I get many alerts on a restart (most annoying is the fact that the Client Update Service fails to start every time, even on a wake from sleep), and it takes almost 10 minutes before everything is settled.

     

    Any help/observations welcome

     

  • Resident Guru
    Re indexing, select only the pool drive, not the physical drives that form it (unless you have non-pool data stored on those drives).

    Re restarting, don't know sorry. Anyone?

  • Thanx. The restart delays may also be related to the re-indexing issue. We'll see.
Sign In or Register to comment.