Files reappear after deletion....?

TommySharp

I'm running 1.2.2.7145

I have a folder of photos which is duplicated, but I've noticed random instances where I delete a photo but at some point it re-appears again which is driving me nuts!

My pool condition is stuck at "50%", do we think this is causing these types of problems?

Shane

Might be a good idea to run a consistency check on your pool: go to the "Pool" tab of the Dashboard's "Server Folders and Hard Drives" section, select your pool drive, choose "Check duplication consistency" from the task list on the right.

TommySharp

Hmmm says it's already running a check, seems to be constantly stuck in "duplicating" mode.

My one thought was to change all folders to NOT be duplicated and then back to be duplicated one by one, but maybe I'll lose something this way?

Shane

How long after a photo is "deleted" does it reappear?

What other software do you have installed on the server?

If you hover the mouse over the pool condition bar, what message do you see?

TommySharp

Not sure how long before they reappear, a few hours maybe? Definitely overnight.

Only other thing on the server is an IP camera surveillance app.

When I hover over the condition bar it just says "Duplicating..."

Shane

Is it only the one particular folder that is affected?

Is it all deleted photos or only some of them? (saw the "random" bit)

If you re-delete the problem photos, does the deletion stick the second time around (or is that random too)?

Are the photos from that camera app?

TommySharp

I've also noticed it on my Photos folder and also documents.

If i delete the files a second time they re-appear later.... Can't say if it's all of them though.

Might try install the latest version and switch all of my duplication off and then back on...

TommySharp

Wait.....The duplication looks to have stopped and now says this if I hover over it...

"File placement not optimal and needs duplication"

Can now click on the link to check duplication consistency.... Will see what that brings back.

TommySharp

Hmmmmm switching off duplication on all my folders was NOT a good idea...... I've lost a lot of files now!

I have a backup of the server from a month ago so should be able to get most stuff back but still not good!

Shane

No, that's not good. Please check that all of your drives that are supposed to be in the pool are working properly? (although even if they went missing from the pool, normally the pool changes to read-only if that happens)

Shane

I've also submitted this thread to Alex. DrivePool is supposed to "fail safely" by design; turning off duplication should NOT itself cause file loss.

It's possible your files may still be there, but inaccessible via the pool; it might be a good idea to wait until Alex can respond.

Alex

Real-time file duplication work like this:

• Whenever something works over a file in a duplicated folder, and that file is already duplicated (i.e. exists on 2 pool parts), then CoveFS sends all modification I/O requests destined for that file to 2 pool parts at the same time.
  
  This ensures that whatever is trying to modify that file, modifies both parts at the same time.

So what can go wrong here, and where does the 50% pool condition come from?

• With real-time duplication enabled, all the files and sub-folders in every duplicated folder should be duplicated at all times.
  
  CoveFS performs a simple and inexpensive (in terms of performance) sanity check whenever anything opens any file on the pool.
  
  If a file has more than 1 file parts and it's not in a duplicated folder, then CoveFS sends a signal to the DrivePool Service that duplication is inconsistent, and it gives it a file name (or a list of them).
  
  If a file has less than the required number of file parts in a duplicated folder (typically less than 2), then similarly CoveFS sends up a signal to the DrivePool Service that duplication inconsistent at a particular path, or paths.
• The DrivePool service immediately drops the pool condition by 50% and schedules a background duplication check of the entire pool.
  
  Why 50%?
  
  Because this type of error should not be encountered under normal operating conditions, we don't want to make any assumptions about how many files are actually un-protected. For example, what if an entire hard drive just stopped responding, and those files now need to be regenerated elsewhere.
• At this point, DrivePool doesn't issue any "Warnings" to the user, until it actually finds a problem with one of your duplicated files. If there are no problems, the background duplication pass will simply run over the entire pool, re-duplicate any un-duplicated file parts and exit.
• What kind of problems can you have while duplicating?
  
  If there are 2 file parts that are different (by content), then DrivePool will issue a duplication conflict warning.
  
  Other warnings include running out of disk space, not having enough disks to store all your duplicated files, and in fact, any other error that Windows throws our way.
  
  We handle the "known" errors by detecting them and showing the user a duplication warning (next to the pool condition bar - and in the Dashboard Alerts Viewer). Unknown errors also get a similar treatment, but we don't provide suggested resolutions on the duplication warning window.

So in a nutshell, that's how the whole thing works.

You can also look into the logs for a detailed list of paths that CoveFS is sending up to DrivePool when it detects a duplication inconsistency.

In this particular case, it sounds like there may be something wrong with the way the pool parts (or actual disks) are connected to the server or seen by the server. Perhaps the same physical disk is somehow showing up twice (just a theory).

DrivePool does delete duplicated file parts when turning off duplication, but only after absolutely verifying that at least one file part is still on the pool. We do this by making sure that both files exist on separate disks, and are accessible (i.e. No I/O errors).

I've invited Tommy to a remote support session so that I can take a look at what's going on there.

Regards,

TommySharp

I was in the process of building a new server so will just be installing DrivePool on the new box.

Might actually leave the old one running so Alex can log in and take a look some time....