Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with Google

Our Sites

covecube.com 
community.covecube.com 
blog.covecube.com 
wiki.covecube.com 
bitflock.com 
stablebit.com 

Poll

No poll attached to this discussion.

In order to better support our growing community we've set up a new more powerful forum.


The new forum is at: http://community.covecube.com


The new forum is running IP.Board and will be our primary forum from now on.


This forum is being retired, but will remain online indefinitely in order to preserve its contents. This forum is now read only.


Thank you,

Incomplete File

edited February 2012 in DrivePool
Running M4 5669

Looking in the Service logs I have pages and pages of the following :

I've submitted the logs and a bug report - anyone else encountered this?

DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880202838457
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880202849223
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203190074
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203217605
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203234888
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203243512
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203259738
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203286772
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203312553
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203330991
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203340973
DrivePool.Service.exe Warning 0 [CoveFsPool] Create found an incomplete file. 2012-02-05 09:05:51Z 880203356534

Comments

  • Got a very helpful response from Alex which I expect he will clean-up and post to the blog : 


    I realize that the new M4 duplication process is a bit of a mystery. I plan to do a full write up on the blog once I get a moment.

    First, let's assume you're running with real-time duplication enabled (DrivePool settings). That's the default.

    In short, here's how it works:
    • On a brand new pool, when you copy a file into a duplicated folder, the file is written to 2 drives at the same time in parallel.
    • Whenever you delete / move or perform any I/O on a duplicated file, that I/O is performed, in parallel, over both files.

      This means that the 2 file parts for a single duplicated file are always in sync without any background duplication.
    • Whenever you change the duplication setting of an existing folder, DrivePool needs to go through each file in that folder and make sure that it's duplicated accordingly.

      This is the "Duplicating..." message that you see towards the bottom.

      Any new files placed into that folder get duplicated in real-time, as explained above as soon at you enable duplication, but we need to make sure that any existing files are duplicated appropriately.
    • DrivePool performs a quick sanity check whenever any file opened to make sure that it has the correct number of file parts. E.g. A duplicated file should have 2 file parts, a non-duplicated file should have 1 file part.

      If it finds a file with the incorrect number of file parts it sends a message to the DrivePool system service essentially saying incomplete file found (that's the warning that you see in the log).

      When DrivePool sees this message, if it's not already performing background duplication, it queues a run immediately.

      It is normal to see this in the log for folders that have just had their duplication level changed. That's because the background duplication run is opening each file to query it for duplication status and some of those are "incomplete". The duplication engine ignores this message in these cases because it expects it.
    What is considered consistent?
    • For all file parts to be considered consistent, they must have the same size and modification time.
    • If the modification time is not the same, the background duplication engine performs a full file hash of all the involved file parts to check whether the file contents differ.
    What can go wrong?
    • If a file is open exclusively, background duplication can't work on it so it skips it for this run. Remember, this is background duplication only and doesn't apply to normal pool usage.

      You will see this in the form of a warning at the end of the run.
    • If a file is inconsistent (as outlined above) you will get a "file conflict". This will show up as a list of file parts that don't match. You will be given the option for DrivePool to automatically resolve the conflict by deleting the older file part and restoring duplication using the newer one.

      Alternatively, you can examine the file parts manually by opening the PoolPart... folder in explorer and deleting the one that's incorrect.

      This is a very rare condition that should never happen under normal usage.

      This should only happen if something modifies the files on one of the pool parts and does not change the other copy of the same file, causing a mismatch.
     
Sign In or Register to comment.