Shared Storage
We run web applications across multiple servers behind a load balancer to ensure high availability and distribute traffic effectively. When a request comes in, any of these servers might handle it, which creates a challenge: how do we ensure that user-uploaded files, media assets, and other dynamic content are available to whichever server processes the request?
This is where shared storage becomes essential. Without it, a file uploaded through one server wouldn’t be visible to the other servers, leading to inconsistent user experiences - visitors might see content one moment and get a 404 error the next, depending on which server handled their request. Shared storage gives all application servers simultaneous access to the same file system, so uploaded images, documents, and other assets are immediately available across the entire cluster.
We built the shared network storage for freistilbox on MooseFS, an open-source distributed file system.
MooseFS
MooseFS is a distributed file system that spreads data across multiple servers while making it appear as a single storage volume. Think of it as a way to combine the storage capacity of several machines into one large, virtual disk.
Here’s how it works: When you save a file, MooseFS breaks it into chunks and distributes these chunks across different storage servers (called chunkservers). The system keeps track of where each chunk lives using a master server that acts as a directory. This setup brings several key benefits:
High availability: In the freistilbox infrastructure, MooseFS stores three copies of each chunk on separate servers. If one or even two servers fail, your data remains accessible from the remaining copy. This triple redundancy means your applications keep running even during hardware fails.
Scalability: We can expand storage capacity simply by adding more chunkservers. The system automatically starts using the new space without any downtime or complex migration processes.
Performance: Multiple servers can serve different parts of the same file simultaneously, which speeds up both read and write operations. This parallel access becomes particularly valuable when we’re dealing with large files or high traffic loads.
Consistency: All clients see the same file system state at the same time. When you update a file, the changes are immediately visible to all servers and applications accessing that file.
Easy maintenance: We can take individual servers offline for maintenance while the system continues operating. MooseFS automatically routes around unavailable servers and ensures data remains accessible.
For hosting platforms like freistilbox, MooseFS provides a reliable foundation for storing website files and assets. It ensures that customer websites stay online even during hardware failures and makes it straightforward to handle growing storage needs as your customer base expands.