Storage Overview
CobaltCore's cloud storage layer is built on Ceph, a distributed storage system that delivers object, block, and file storage in a single unified platform. The surrounding components handle lifecycle automation, data replication, high-availability quorum, observability, and quota and usage metering - each with a focused responsibility.
Architecture
The storage stack is organized into three layers. See Observability & Audit for metrics, dashboards, and alerting across the stack.
Foundation - Ceph provides the core distributed storage engine. All other components either operate it, extend it, or observe it.
Operations - Rook runs as a Kubernetes operator and manages the full lifecycle of Ceph daemons (monitors, managers, OSDs, MDS, RGW) as containerized workloads. Arbiter extends quorum into stretched cluster topologies by deploying external Ceph monitors that Rook does not manage directly.
Data Services - Chorus provides zero-downtime data replication and migration between object storage systems (S3 and Swift). Liquid-Ceph reports RGW quota, capacity, and usage data to Limes.
Components
| Component | Layer | Role |
|---|---|---|
| Ceph | Foundation | Distributed storage engine - block (RBD), file (CephFS), object (RGW) |
| Rook | Operations | Kubernetes operator for Ceph lifecycle management |
| Arbiter | Operations | External Ceph monitors for quorum in stretched clusters |
| Chorus | Data Services | Zero-downtime object storage replication and migration |
| Liquid-Ceph | Data Services | Limes integration for RGW quota, capacity, and usage metering |
Storage Interfaces
Ceph exposes three storage interfaces that CobaltCore services consume:
- RBD (RADOS Block Device) - thin-provisioned, resizable block volumes used by virtual machines and databases. Striped across OSDs for parallel I/O and backed by RADOS snapshots and replication.
- CephFS - POSIX-compliant distributed filesystem. Metadata is managed by a dedicated MDS cluster; data is striped across OSDs. Supports snapshots, quotas, and multiple active MDS daemons for horizontal metadata scaling.
- RGW (RADOS Gateway) - S3 and Swift-compatible object storage gateway. Supports multi-tenancy, versioning, lifecycle policies, server-side encryption, and multi-site active-active replication.
Component Relationships
High Availability
Ceph achieves HA through monitor quorum (typically 3 or 5 monitors), OSD replication or erasure coding, and MDS standby daemons. In stretched deployments that span two sites, Arbiter deploys a third monitor at a tiebreaker site so that quorum is maintained even if one full site goes offline.
See Also
- Ceph Upstream Architecture Docs
- Rook Documentation
- Observability & Audit - Prometheus metrics, Perses dashboards, and Prysm CLI for the storage stack