Hu's Blog

In this blog series on Redefining Unified Storage, I covered Hitachi Unified Storage (HUS) Dynamic virtual controller, which simplifies the management of LUN assignments and provides enterprise class load balancing. I followed that with the hybrid architecture of HUS file modules, which combine multicore CPUs with FPGAs to increase the throughput performance and scalability of HUS beyond any other unified storage system on the market.  Each of the four major FPGAs in the file module is dedicated to specific functions within the file system, removing contention for resources found in competing architectures.  One of these is the SiliconFS file system FPGA.  SiliconFS is a patented file system technology unique to HUS file module, using an object-based file system at its core to provide faster and more scalable file system operations than traditional file systems. SiliconFS file system in HUS file modules is the same as the object-based file system in our standalone high end Hitachi NAS Platform product family.

The first things you notice about HUS file modules are the high IOPS per node and the capacity that scales up to 8PB usable capacity for the 4 node configuration! The IOPS are due to the hybrid hardware architecture of the file module, which I covered in my previous post. The scalability is due to the design of SiliconFS, which enables this file system to scale way beyond the TB limitations of traditional file systems. File systems that are built on traditional Inode (Index Node) structures are limited in scalability and are time-consuming to search and replicate. The object-based file system of SiliconFS uses Onodes to eliminate these limitations.

The object-based file system in HUS is not to be confused with the object store that is used in our Hitachi Content Platform (HCP), which uses objects for content aware management. HCP has the ability to add custom user-defined metadata, which enables it to understand the content of files to migrate and manage file objects based on a combination of metadata attributes. HUS file modules use objects to abstract and manage file system constructs for internal file system management, snapshots and replication to protect and move objects, and to provide file system scalability and data mobility. HCP can integrate with HUS file module. Based on file attributes such as last date used, HUS File Module can stub a file out to HCP for long term retention and HCP can mange content stored on HUS file module.

What is unique and different about an object-based file system is that all of the file system structures are managed as objects, whether they are file objects, directory objects, or key file system metadata objects (such as the freespace bitmap object or master index object).  Each object has a common structure of the root Onode, which contains the metadata describing the object (traditional file ACL’s, directory information, etc), any number of pointer Onodes (direct or indirect) depending on the object size, and a collection of data blocks.

There are many benefits from the object-based file system structure:

1. Efficiency – The object onode structure enables faster lookups of the location of data blocks, regardless of what type of object it is.  By maintaining a common lookup schema, file system operations are far more efficient.
2. Resiliency – Critical file system objects have dual copies maintained on different physical media, providing the best resilience for the file system infrastructure.
3. Scalability – Each directory object is an independent entity, with the ability to address up to 16 million objects per directory.  Each of those objects could in turn be directory objects pointing to 16 million file objects for directory objects, and so on.  There is no file system level limit (as with a traditional inode table), so scalability is only limited to the maximum 256TB file system size.
4. Portability – The Onode structure provides the ability to separate metadata and data operations, providing feature benefits described below.

Feature Benefits

The object-based filesystem is the foundation for the advanced features available for HUS file module.  While not a complete listing of all of the file module features, these features benefit the most from this unique architecture.

File system Restore – The file system snapshot includes every object in the file system, including critical metadata objects.  This allows the whole file system to be restored instantly and consistently to any version of the snapshot.

Tiered File system – The separation of metadata from data is taken a step further through the use of a tiered file system.  The object structural metadata (root onode, pointer Onodes) are stored on a high speed disk tier such as SSD, while the data blocks are stored on high capacity, more cost effective disk such as NL-SAS.  This gives a significant performance boost to metadata operations, such as file lookups and metadata queries.

File Clones – When a file clone is created, each copy of the clone maintains its own metadata structure, which points to a common set of data blocks.  Independent file level snapshots ensure that no data block is deleted while any copy of the clone exists.  Because each clone is independent, there are no restrictions to creating clones of clones, nor limitations to the number of clones.  File clones can be created up to the 16 million objectsper directory limit.

Replication – HUS file module has multiple replication options available with the BlueArc technology replication feature.  Object-based replication provides the fastest, most efficient replication option by using file system snapshots to instantly find all changed data blocks between the previous and current replication, regardless of whether those blocks are part of file objects, directory objects or file system metadata objects.  Data replication starts far faster, and runs more efficiently than with traditional file-based replication, up to 26 times faster!

This combination of  the HUS file module’s hybrid FPGA architecture, the unique object-based file system and the high performance active/active symmetric architecture of  HUS block module will create a new level of performance and scalability for the explosion of “unstructured data” growth in files and folders, multimedia, SharePoint, medical images, patient records, content, etc. It also provides simpler and faster ways to backup and replicate, and enables an integrated file and content storage solution with active archiving and federated search capability.

Also available on HUS is the new version of Hitachi Command Suite management software, which is more integrated and unified than ever before. It contains a Device Manager GUI, which makes it simpler to implement and use. More information about the unification of management will be provided in my next post “Redefining Unified Storage – Unified Management.”