Data Center Advisors

As my first guest blogger, Matthew O’Keefe, PhD provides the conclusion to his 2 part blog on accelerating NAS with hardware by describing the hardware implementation of hardware pipelining.

Accelerating NAS via Hardware

By Matthew O’Keefe, PhD

The Key to Efficient CIFS and NFS Performance: Pipelining

In my previous blog post, I pointed out that network protocols like NFS and CIFS could, in theory, exploit pipelining to significantly improve NAS system performance. Pipelining allows many operations to proceed in parallel across multiple, independent memory banks and FPGA chips, greatly increasing performance, stability under heavy load and power efficiency. In this post, I’ll talk about how the Hitachi NAS server implements pipelining.

How HNAS Implements Pipelining: SiliconFS Object-based File System

HNAS implements pipelined network file operations via its Silicon File System FPGA-based server architecture. FPGAs are a form of non-custom ASIC (application-specific integrated circuit) that contains programmable logic components called “logic blocks”, and a hierarchy of reconfigurable interconnects that allow the blocks to be “wired together”—somewhat like many (changeable) logic gates that can be inter-wired in (many) different configurations. Logic blocks can be configured to perform complex, combinational functions or merely simple logic functions like AND and XOR. In most FPGAs, the logic blocks also include memory elements, which may be simple flip-flops or more complete blocks of memory.

FPGAs can be used to implement any logical function that a custom ASIC could perform. However, FPGAs cannot achieve quite the same density or performance possible with a custom ASIC, but the ability to update the functionality after shipping, partial re-configuration of a portion of the design and the low non-recurring engineering costs relative to an ASIC design (notwithstanding the generally higher unit cost), offer advantages for many applications, including, as we will see, network file servers.

Mercury uses separate groups of FPGAs to implement network, file system and storage operations, and in particular, the data flow of these operations, as shown in the following Mercury hardware layout diagram.

HNAS architecture includes four FPGA chips: the network interface, disk interface, hardware file system (WFS) and data movement (TFL) FPGAs, all connected to the data movement chip via LVDS (low voltage differential signaling) connections.

A key advantage of this design is the point-to-point relationship between the FPGAs along the pipelines. While traditional computers are filled with shared buses requiring arbitration between processes, this pipeline architecture allows data to transfer between logical blocks in a point-to-point fashion, ensuring no conflicts or bottlenecks. For example, data being processed and transferred from a network process to a file system process is completely independent of all other data transfers, so it would have no impact on data moving to the storage interface. This is vastly different from conventional file servers where all I/O must navigate through shared buses and memory, which can cause significant performance reductions and fluctuations.

The following file system operations are performed directly in HNAS hardware:

• Create/delete (files and directories)
• Read/write (user data and attributes)
• Directory operations (lookup and readdir)
• FS consistency and stable storage
• Snapshots
• Metadata caching
• Free space allocation

Like CPU pipelines, exception conditions (e.g., error handling) and complex (but infrequently used) operations like quotas, file system check, NVRAM replay and management (format, mount, shutdown) are executed outside the pipeline hardware in software. Software (via today’s fastest multi-core CPUs) associated with pipeline operations can also be used to extend the functionality of the fast data pipeline into other areas, including encryption and storage tiering. Hitachi NAS SiliconFS Object-based file system allows customers to exploit custom hardware acceleration for the datapath while riding the price-performance improvements in multi-core CPUs to execute complex, value-add features in software.

HNAS Hardware Architecture: The Results

HNAS pipelined server architecture, implemented via FPGAs, yields outstanding performance (and price/performance metrics) in several dimensions:

• Highest aggregate and most predictable performance — HDS effectively designs directly for the performance it wishes to achieve in the product
• Highest aggregate IOPS of any platform (95,757 SPECsfs2008 NFS throughput with ORT of 1.73 ms, the highest single node throughput of any NAS platform)
• Cheapest IOPS per dollar for any NAS platform
• Largest number of files/directory/overall storage due to high-throughput SiliconFS Object-based file system metadata engines
• Gracefully handles heavy load due to lack of underlying OS bottlenecks (virtual memory, scheduling, thrashing, etc.)
• Highest SPECsfs numbers in the industry (189,994 Spec2008 throughput via a 2-node cluster)
• Lowest power per IOPS

HNAS hardware pipelining strengths allow it to directly address one of the biggest weaknesses found in NAS technology deployments: NAS server proliferation. By using hardware very efficiently, a single NAS server can potentially replace dozens of slower software-based NAS servers.

Conclusion

By integrating a hardware datapath and multi-core CPU system, HNAS delivers the highest performance, most efficient and scalable file server in the market today.