Windows 8 Storage Spaces with DrivePool features

JazJon

http://usingwindowshomeserver.com/2012/01/05/microsoft-talk-storage-spaces-for-windows-8/

Looks interesting, but it's far from ready to test, and they said it won't be as full featured as the old Drive Extender for WHS etc.

I'm sure DrivePool can be tailored to use the new features of Windows 8 in the future.    I wouldn't be surprised if the Storage Spaces is too generic and DrivePool ends up being better in the long run.   Either way, I'm looking forward to M4 since I won't be running Windows 8 on my production WHS 2011 machine any time soon!  (and I doubt will see gold master win 8 in 2012)

I just wanted to share the link to create discussion for now.  The future of innovation is never ending and entertaining. 

Saltydh

Spent a bit of time installing and testing Storage Spaces today.

The biggest problem I see with Storage Spaces is the data storage mechanism is similar to software RAID. Software like DrivePool go to lengths to ensure your data is stored intact in an easily readable format (NTFS). This is not so for Storage spaces which use a propriety data abstraction. Responses to my questioning of the two resiliency modes on the Microsoft blog basically boil Storage spaces down to:

Mode 1 - Data Mirroring: Software RAID 6 level resiliency (Support for 2 drive failures that share data from a single "slab")

Mode 2- Data Parity: Software RAID 5 level resiliency (Single drive failure only)

My #1 requirement for consumer level drive aggregation is: Individual drive information must be readable and fully recoverable in the event of multiple drive failure. I've seen too many situations of failed RAID sets because hot spares or sibling drives fail during recovery, leaving you with not a single piece of recoverable data (short of advanced RAID reconstruction techniques)

Digging further, Storage Space drives have to be "prepared" for use. Which means shifting data somewhere before buidling the storage space and then gradually reintroducing. Atleast DrivePool the data just shifts from one folder to another (under the hood), not one DRIVE to another...

And of course Storage Spaces are only Windows 8 compatible so you can't shift them to Windows 7 or Server 2008 r2 (at time of writing this) and expect to see or read the data.

Finally, recovery tools for Storage Spaces will be similar to advanced RAID reconstruction techniques rather than simplistic NTFS recovery techniques. I'd much rather lose a single corrupt file, even an entire drive of them rather than my entire collection.

Unfortunately I personally feel Microsoft have lost sight of why they removed DriveExtender from WHS and in replacing it for Windows 8 they have lost the original architectural decisions that made DriveExtender useful but more importantly appropriate for a consumer grade drive pooling solution.

StableBit's DrivePool combined with drive parity solutions like FlexRAID will continue to be my choice as they provide an appropriate level of resiliency to failure, without drastically increasing the cost/redundency ratio.

My two cents - I was left wanting but I'm sure many people will find Storage Spaces meet their particular needs. As always, make your own choice for your needs.

Blog referenced here:

http://blogs.msdn.com/b/b8/archive/2012/01/05/virtualizing-storage-for-scale-resiliency-and-efficiency.aspx

Cheers,

Salty.

Bring on M4!!!

JazJon

Thanks for the details!  I agree with you 100% Saltydh, especially your first paragraph.  (spot on, nail on the head)

I'm all about StableBit DrivePool for set it and forget it, yet nearly hassle free should you need to do anything funky.

Bring on M4

Alex

This is a major development.

I knew about storage spaces in Win 8 for a while, but Microsoft's blog post explained a lot.

Obviously, this is a very bad thing for DrivePool. When Microsoft enters a market they dominate because of their sheer size. Plus, it seems like they've really built up an extensive system, so they seem serious about it.

I guess I'll make some quick remarks:

• MS's solution is soft-RAID like and not file based. A core requirement for DrivePool has always been to store your files in plain NTFS format for easy recovery. Black box RAID like solutions are only readable as long as the software functions properly.
• MS's solution recovers from multiple drive failures. DrivePool can store more than two copies of a file without a problem. This has always been in the architecture. It's not exposed in the UI now.
• MS has "parity" redundancy. An interesting idea. Need more information on exactly what this means, how it works, and what the impact is.
• MS can swap drives to a different system, background reconstruction of duplicated parts. DirvePool M4 does this. In addition, your files are stores as standard NTFS files so they are readable on any Windows system for emergency recovery purposes.
• MS can "provision" extra free space. In other words, from what I understand, you will see more free space available than there actually is on the pool, allowing you to "reserve" space for future expansion. I see this as a disadvantage and a completely unnecessary complication.
  
  DrivePool always shows you the actual free space which can dynamically increase / decrease as you add and remove drives from the pool. This is probably a consequence of using a RAID-like system.
• MS has "stand-by" disks. Again, very RAID-like. Need more information on exactly how this works. DrivePool has no stand-by disks. I guess a similar feature in DrivePool would need to mark a disk as suspect and have the balancer move all the files off it.
  
• To add a disk to MS's system, the disk needs to be provisioned in some form and must be wiped. DrivePool doesn't erase / format disks when it adds them. You can even store non-pooled data on a pooled disk.

I'm sure there are advantages / disadvantages on both sides. MS's solution does seem like a very competent and compelling implementation.

Where does this leave DrivePool? What about a Windows 8 port?

DrivePool M4 and eventually 1.0 will get released for WHS 2011 (and similar) in Q1 2012. This will not change.

I'm not sure that there is a need for a port to Win 8 since pooling functionality is essentially built-in. Also, a port to older OSes doesn't make sense as they're going out the door.

Any thoughts?

Philmatic

DriveExtender sucked, and for that exact reason that they chose to ditch the "pseudo file system on top of NTFS approach". The demigrator concept sucked and led to streaming and buffering issues that still persist to this day. The Spaces approach removes all that, at the expense of pure NTFS capability. Spaces is targeted at the enterprise, while DE worked pretty decently for the home user, it couldn't get close to touching a production server.

Storage Spaces is Microsoft's version of Sun's ZFS, that's it.

Honestly, it's the right way to make something scalable, like Microsoft
needs to. I just worry how this will affect DrivePool's development.

joreri508

Microsofts annoucement is consistent with Ballmers message to Terry Walsh in November last year (http://www.engadget.com/2010/11/29/steve-ballmer-looking-into-removal-of-drive-extender-from-window/). If I'm allowed to speculate, either they have licensed the algorithms used by Drobo, or they have implemented this quickly as a copy of Drobo functionality.

What they have announced is very close if not exactly the same as the functionality in a Drobo. However the Drobo is a premium product sold at a high price to media professionals and corporations. It works well as it is a hardware box with no possibility to hack its internal workings. Microsoft Storage Spaces is a software implementation in Windows and we all know how vulnerable Windows is to systematic problems, fiddling with the registry etc.

Product success is all about product positioning. I agree with Philmatic that Spaces can be successful with larger corporations. They have the IT support staff who can manage the complexities. However for the small company or home users, I think Storage Spaces will be too complex to handle. People will not understand the RAID aspect and loose data, and then Storage Spaces will get a bad reputation for the home or small company users. 

Myself, I have eight computers at home, and an extensive DVD and Blu-Ray collection of several thousand movies. I used DriveExtender since 2007 and now Drivepool M3 since last month, to manage the storage on my Intel Atom based SQA-5H WHS (upgraded to WHS 2010) where I have 10 internal and 10 USB external 2TB WD Green HDs. It works very well. Over the five years I have used WHS I have had five hard drive failures and I have been able to save most of my files when I had disk problems as the underlying file system has been NTFS.

To make Drivepool a commercial success (and thereby a successful product for Covecube), you could position it as a solution for home users (as you write WHS 2011) and SME's (Small Business Server) and also try to package it together with Scanner as a solution to sell through hardware vendors.

Tranquil PC in the UK teamed up with Drive Bender (renaming it Monstor). However Tranquil PC have never been good at marketing and nowadays there are several new manuacturers of small and home server hardware, foremost in Germany and the UK. I suggest you contact them and see if you can partner up on them having Scanner and Drivepool pre-installed on the boxes they sell. Then you can carve out a niche for your software and ensure it becomes a surviving solution for years ahead.

Drivepool is so good that I think many proficient home users, such as myself, will have no concern paying well for the software once it hits version 1 status. However to sell only to consumers may not be enough to ensure sucessful product launch, given Microsofts new initiative.

Shane

I'm not sure that there is a need for a port to Win 8 since pooling functionality is essentially built-in. Also, a port to older OSes doesn't make sense as they're going out the door.

I think that depends on how easy or hard it is to port DrivePool to Windows 7. I do tech support for residential and small business; the vast majority stuck with XP, avoiding Vista like the plague, but they like 7. Unless MS plans to kill supply of 7 and/or 8 has a nigh-flawless release, 7 will be the new XP.

You'd also have a niche MS apparently isn't going for ("Windows 7 does not support Storage Spaces and will treat the physical disks just as it would any disk with an unfamiliar partitioning scheme.") and if they are also planning to skip the versions of Server out there that share the same codebase as WHS2011, you have that too. Marketing will be key.

If you can release - and market - a good 7 port before MS releases 8 to the masses, even better. It may become a niche market, but it's a bigger niche than WHS2011 alone.

letho

NTFS is the best windows file system at the moment. Every Windows of the last 10 Years and a lot of the linux OS can read it, so that is the best solution for backup, recovery and usage.For that reason I will use Drivepool for my WHS and for my future PCs (if that PC will ever have more than one harddrive ;-) ).

@Alex: i agree with Shane and joreri508, make a windows 7 build and maybe a build for windows 8 beta, then you can see, how successful it is. But don't forget to make a good advertising.

as you said, for Drivepool 1.0 or later, it is maybe a good feature, if Drivepool is working together with the scanner to discover bad drives and move all the files off it.

PS: M4 looks great.

Alex

Thank you all for your constructive comments. I appreciate it.

A port to Win Vista+ should be very easy.

CoveFS was designed to run on anything Vista+ without changes. The only thing that really needs to be ported would be the management UI, which should be simple. Of course the reality of any port is that some things will need to change and be re-tested, but overall it's very doable.

MaDgamEr

Agreed with letho, you have built an awesome product and we are all eagerly awaiting its "official" release. Yoes M$ is big and bullies everyone around, but your software is so much more than a bulk sized box of band-aids all stuck on top of one another to keep things working... A-la-microsoft way of developing software.

Keep up the amazing work Alex!!

Bill_S

I've been using storage spaces on windows 8 for awhile and write performance is a joke (110MB/s read 20MB/s write max) in windows 8 the same drives and setup running Windows Server 2012 gets better read performance (200MB/s read 26MB/s write) still nothing stellar. To me this implies Microsoft is limiting performance in Windows 8 on purpose 

Shane

I can't imagine any reason MS would limit write performance on Windows 8 on purpose - accidentally, sure, on purpose, no.