BUILD 3874 ventures into M3.5 territory

Alex · September 2011

DrivePool BETA build 3874 is out:

http://wiki.covecube.com/StableBit_DrivePool

This build starts to feature some of the performance improvements that will be part of M4.

A bit of history

When M3 was released, I said that M4 will be treated differently as far as the development cycle is concerned. Previously the code was branched at a major beta release (M2, M3), and two different versions of DrivePool were being worked on at the same time. The public build, for bug fixes, and an internal build for the next milestone. M4 was not going to be treated the same way. What you'll see with M4 is most of the features being rolled into the M3 code and releaced piece by piece. Once everything is in, we'll call it M4.

The first of these builds arrived today (3874). There was a significant amount of work done in the kernel to lay the foundation for M4 and now it's ready to be tested.

What's new

Visually, what you'll see is a new DrivePool Settings window accessible from the Dashboard tab. In it there will be options to enable the various performance enhancing aspects of the new kernel code. All the options go into effect immediately after clicking OK, except the first one. For the threading option you'll need to restart the Server.

Performance optimizations

Direct file I/O

This is basically a made up term to represent the new layer of code written in the kernel in order to facilitate extremely low overhead I/O. When direct file I/O is utilized, the I/O request will be managed 100% in the OS kernel. In the case of reading and writing, DrivePool will communicate with the NTFS file system to facilitate the completion of the requested operation as quickly as possible.

Direct I/O will consists of various operations, but today DrivePool supports reading and writing files.
Direct file I/O - Writing

When writing duplicated files, DrivePool writes directly to the 2 NTFS files, asynchronously. In other words, for every write it issues 2 write requests to NTFS, in parallel. When both requests complete, the original request is completed.
Direct file I/O - Reading

Direct I/O reading can work in 2 ways, striped and non-striped.

Striped reads are a technique, where for a duplicated file, DrivePool will intelligently balance the reads across the 2 disks holding the file being read in order to maximize performance. Theoretically, you can double your read speed, but the true impact of this needs to be benchmarked.

Non-striped reading will force DrivePool to read from the first disk that it finds. This is the way it has worked in the past.
Fast I/O

Fast I/O is a technique that kernel drivers use to speed up I/O even beyond what Direct I/O can do. In Fast I/O the I/O request take a super fast code path, so that it can be fulfilled immediately. Typically this means reading from the cache, or writing to the cache for later delivery to the disk.

DrivePool 3874 supports both reading and writing with Fast I/O. Fast I/O writing can be turned off individually.

More to come

The directory listing optimization didn't make it into this build, turns out it's more complex than originally thought. But it's doable and you'll see it soon enough. The Directory listing optimization should in theory let you list many thousands of files, very quickly. It will utilize Direct I/O to query all the NTFS volumes at the same time, in parallel, and combine the result for the caller.

Theoretically, it should be faster than a standard NTFS directory listing since we will be querying all the disks in parallel.

And there can be more done in the future in other areas.

How to test

3874 comes with the new file system driver, so you'll have to reboot after the upgrade. But, all of the optimization discussed above are off by default. This was done so as not to cause pain for people who are not expecting the potential instability introduced by the all new kernel code.

You can turn each option on / off in real-time, even during a transfer to see the effect. I expect the Direct I/O will make the most difference, while read striping is mostly a trial run to see how it goes. Fast I/O while it sounds cool might not be as important in today's world of quad cores and i7s.

Edit:

One more thing... Logging is enabled by default in the about box. If anyone wants to conduct a benchmark, make sure to uncheck that as it writes extensive detail on every single I/O request (way beyond what you can visually see in the text log files). We're talking about 50 MB / min worth of rotating binary logs.

hopester · September 2011

Hi Alex,

Installed, rebooted and I'm getting access denied errors on the shares. Will do some more testing to see if new shares are effected or just existing ones.

hopester · September 2011

New shares are ok, any existing shares are the ones effected.

hopester · September 2011

Done a quick test write to a new folder with the new io features turned on and it was a lot faster. Hit the cancel button on the file copy as it was a large file and the copy just continued, seemingly forever. Gave it some time then ended the task via task manager The shares then became inaccessible and would not open and I had to reboot the server.

danivtec · September 2011

I am getting a BSOD with Fast I/O and Direct I/O ticked.

Unticking this seems to stop it happening.

I will try and recreate which I have a couple of times and post logs.

magician94 · September 2011

I am also having a couple of shared folders being no longer accessible after the upgrade. It are not all folders, the majority functions alright. Is there anyway to fix this?

magician94 · September 2011

Found that some folders where also not locally accessible in the share via c:\serverpool. I then dug into the poolparts and checked the folders that were giving errors. All those folder gave the error that the owner could not be determined. By manually re-setting the owner the problem was solved.

DrTone · September 2011

Same problem, my documents folder is now messed up. I cannot get ownership back on it no matter how hard I try.

Update: I see that the permissions are messed up on one of the drives. This folder is duplicated. I've been fixing the ownership/permissions manually, unfortunately there are alot of files and folder.

Part way through the machine blue screened on me.

From the comments above, it might be safer if you guys pull this release until these issue can be looked into.

daveyboy37 · September 2011

Yep same here... Seems to just be documents and a custom made movie folder that will not allow me access. Other default folders and other custom folders are fine.Not through the lauchpad or even when trying to open via RDP.How can i roll back...Are the previous releases still available somewhere?

peste19 · September 2011

before jumping ship, do these errors happen only when you activate these performance features or just by installing it? my trial is about to end and i needed to update to a newer version but didnt want to if its going to affect my shares

thanks in advance

magician94 · September 2011

Here problems are not completely solved yet. I managed to get one share back working correctly. But two shares are still giving me problems. Have to keep on trying i guess. One share even contains files where the fix i did does not work, cannot get/ set permission on those files.

DrTone · September 2011

before jumping ship, do these errors happen only when you activate these performance features or just by installing it? my trial is about to end and i needed to update to a newer version but didnt want to if its going to affect my shares

thanks in advance

I'd wait till the last minute for a bug fix version or specific recommendations from Alex. Don't try the current version.

I got lucky it only screwed up one of my shares and I barely had any data in it.

DrTone · September 2011

Here problems are not completely solved yet. I managed to get one share back working correctly. But two shares are still giving me problems. Have to keep on trying i guess. One share even contains files where the fix i did does not work, cannot get/ set permission on those files.

Are you finding it's only duplicated folders? Also, I had to show hidden system files and fix them as well. The stupid desktop.ini file caused me some grief for a bit.

magician94 · September 2011

I think i am working my way to a working situation. There were some normal files left that did not have a correct owner in some of the folders. This folder is not a duplicated folder.

DrTone · September 2011

I think i am working my way to a working situation. There were some normal files left that did not have a correct owner in some of the folders. This folder is not a duplicated folder.

Yah, sub folder and files all have the ownership and permission messed up. For me it was a duplicated folder and things were messed up on only one drive.

magician94 · September 2011

Now i seem to have fixed everything.

Reset owner recursively on all files.

Re-added proper admin right on files and folders recursively.

Next removed read / write permissions in sharing interface.

After that re-set read / write permissions in interface.

Alex · September 2011

Regarding access denied errors on the shares, in addition to the changes notes above this build also resets the permissions on all the ServerPoolPart folders and the C:\ServerPool\ServerFolders folder.

Those folders serve as the root for the shares on the pool.

So what what exactly changed about the permissions, well they are no longer set to propagate to child sub-folders or files.

What does this mean? Well, when you create a folder on the pool, DrivePool and WHS set the permissions on that folder based on what you selected in the Dashboard. Normally the permissions on each folder are not inherited from the root, which means that each folder has its own separate permissions.

With this change, if your folders were inheriting permissions from the root, and you installed this build, the root now has no permissions to give its children, and therefore you see access denied errors.

----

This means that people who had broken permission settings to begin with now can't access their shares, and people who had their permission set properly would not notice any change.

----

What can we do? I'll try to reproduce the situation and perhaps issue another build to address the broken permissions.

Of course this does not affect the data on the pool just the permissions on the folders.

DrTone · September 2011

Every one of my folders were created via the drivepool add in, so why were the permission broken with my documents share? Of course, it was moved over from the

drive. But I also moved over 3 other folders and they were fine.

As well it turned out to be just one drive of the duplicated folder that was messed up not both.

Alex · September 2011

It looks like the problem is this:

Permissions on some folders were reset to No Access for all users.
Trying to change permissions from folder properties in the DrivePool tab issues an "Access Denied" error.

Does that fit what some of you are seeing?

DrTone · September 2011

Yes, that's what I saw in my case. I couldn't access the share, saw that the permissions were no access for all users and then tried applying my permissions again and got the error.

rfbarwin · September 2011

I have duplication set on the Music, Documents, and Pictures folders. I haven't tried to change folder permissions since installing 3874. The Documents and Picture folders are accessible. The Music folder is not accessible whether logged in as Administrator (using either the Dashboard or RDC) or either user from the Launchpad.

Curiously, I am able to get momentary (22 seconds, then playback stops) access to play several songs via my Samsung Focus using MS Server Solution Phone Connector.

User permissions were set from the Dashboard when I created the user accounts. I'm not using Homegroup.

Thidsa · September 2011

From Alex

>Permissions on some folders were reset to No Access for all users.

No not here. I had a Music folder that had same permissions as before. But was unaccessable..

>Trying to change permissions from folder properties in the DrivePool tab issues an "Access Denied" error.

No, changing permissions from folder properties didnt do anything. ( changed it to no access, and back to read&write) I uninstalled and cleaned c:\programdata\stablebitdrivepool. Reinstalled and then when i set the permissions it worked

Patrickdmorrison · September 2011

I'm getting at least 2-3x more copy speed. Nicely done.

Had an issue with the documents folder being listed outside the pool, but a reboot seemed to fix that.

Alex · September 2011

I've just put up 3923:

http://wiki.covecube.com/StableBit_DrivePool

I've turned off the permissions reset code that was introduced in the last build, we'll re-enable it at a later time.

The Dashboard will now reset permissions on folders that are returning access denied errors. It will now take ownership of the folder and force set the permissions.

Alex · September 2011

I should mention that erasing the files in ProgramData has no affect on folder permissions. DrivePool stores absolutely nothing external about the folders it uses.

The only thing that DrivePool will do is re-create folder links when they're missing. So stopping the DrivePool service, clearing out C:\ServerPool\ServerFolders\, then restarting the service will re-create all the shares with their appropriate permissions.

rjrapson · September 2011

I had this problem with one folder in my pool. Turns out it was spread across all 3 of my pool drives, and I hd to go to each of those drives as administrator on my server, take ownership of the folder and manually set appropriate permissions. I shut down all of the drivepool services while I did that, then when I restarted the services, I was able to access and set the permissions through the dashboard.

Alex · September 2011

I've just put up another build, 3929. No one reported this yet, but there was an important bug fix to Direct file I/O that was returning an invalid handle error under some circumstances, particularly when reading / writing directory alternate streams. DrivePool does this quite a bit.

It was a kernel change so you'll need to reboot after the install.

Vlad_M · September 2011

Hi Alex,

Updated to the new version (3929), it fixed my problems with permissions. Now I have the problem: when Direct file I/O is enabled (as wel as Fast I/O) WHS reboots by itself every 3 - 5 minutes.

I can't upload minidump "Sorry an error occured during the upload.
Upload another file"

File Upload:
Description:	WHS reboots by itself every 10 - 15 minutes when DIrect file I/O enabled (3929).

Vlad.

DrTone · September 2011

I've just put up another build, 3929. No one reported this yet, but there was an important bug fix to Direct file I/O that was returning an invalid handle error under some circumstances, particularly when reading / writing directory alternate streams. DrivePool does this quite a bit.

It was a kernel change so you'll need to reboot after the install.

FYI, it's not prompting to reboot after installation, should it?

sean10780 · September 2011

I have updated to the latest build, but my service is unable to start, even after rebooting, and then uinstalling and reinstall and rebooting again. I contacted support so hopefully alex can have a look rather quickly.

*Edit: I cleared out the C:\ServerPool\ServerFolders.Mount and restarted the service, and it seems to have been resolved.

ArthurHawkins · September 2011

@sean10780 OUch.. I tried that and just lost 400Gb of data.. I was trying to swap out the OS drive with a SSD.. got the OS reloaded and was installing the latest version of DP and couldn't get all my shares back up.. Lost an entire share trying this

Howdy, Stranger!

Categories

Poll

BUILD 3874 ventures into M3.5 territory

Comments