Write-once read-often distributed file storage for .NET?

Question

I have a C#/.NET application that needs to store variable-sized binary blobs (10KB - 100MB) in a central repository. MogileFS meets my needs exactly (write-once, highly available, transparent horizontal scalability, optimized for commodity hardware) but the lack of .NET bindings suggests there's some other more .NET-ish way of doing this.

Or should I just be writing those bindings?

Cosgrave

The C# programming language has no constructs to deal with file or database I/O of any kind. Perhaps you're asking if there's a .NET solution for this? – Kozlowski

Yes, thanks. Updated the question to reflect that. – Cosgrave

Note that this question is similar to #804534 but not identical -- I'm looking for file storage rather than key-value storage. – Cosgrave

To me the lack of existing bindings suggests only that no-one has written (and released) such bindings; I am tempted to say the ideal answer here is.... Write the bindings. – Membership

Heck, if the protocol is easy I might see if I can spare a few moments – Membership

Answer 1

You may consider Raven DB for this. It's optimized for a higher read to write ratio than other solutions, and it's .NET specific. The support for distributed scenarios, including sharding, multi-tenancy, etc are baked in.

Raven DB Website

Magazine

Answer 2

I would suggest you look at CloudIQ Storage from Appistry. Its a fully distributed data store that runs on commodity hardware. You can define the number of copies of a file it maintains across nodes on a file by file basis. So if you write a file once, but have many concurrent reads, you can set the value of N (number of copies of a file) to a large number to get greater concurrency.

The system implements a REST based interface for PUT's and GET's, and meta data is returned in JSON, so there are many good libraries available in .NET.

Coprophilous

Answer 3

I'm going to go out on a limb here and suggest MySQL. If you're using MySQL with MyISAM, It would seem to me to fit your needs quite well. If you want highly available, your can use replication to mirror the data across many machines. If you want even more highly available, you can migrate to MySQL Cluster. The .Net bindings for MySQL work quite well. Unless you really need something more specialized, which from your small description, it doesn't seem you do, then MySQL could be up to the task. The advantage of using a standard RDBMS, even when you don't necessarily need all it's features it that it's easy to find developers who already know how to work with it.

Thermit