For the first time, this new study byIntel software engineers analyzes the performance impact of using Apache HBase on various modern storage technologies.
Asmore “fast” storage technologies (such as SSD and NVMe SSD) emerge, organizations with big data use caseswant to make better use of them to achieve better throughput and latency. But to this point, there have beenno detailed analyses publishedabout the truesignificance of thatperformance boost, nor about howto best mix fast and “slow” storage to achieve the best balance between performance and cost.
Recently, software engineers in Intel’s Software Solution Groupgroup did a detailed study ofApache HBase write performanceon different storage media. (Results were originally published via the ASF Blog.) In the study, weused the hierarchy storage management support in HDFS, usingYCSB as the benchmark, to store different categories of HBase data on three different storage types: HDD, SSD, and RAMDISK. (HDD is the most popular storage in current use, SATA SSD is faster storage thatis growing in popularity, and RAMDISK wasused to emulate extremely high-performance PCIe SSDs.)
In general, this study tries to answer following questions:
What is the maximum performance a user can achieve by using fast storage? Where are the bottlenecks? What is thebest balance between performance and cost, and how can it be achieved? How can the performance of a cluster with different storage combinations be predicted?We believe that this study provides the first comprehensive and objectiveanalysis of HBase performance on faststorage technology.
Download the full report here .
Jingcheng Du and Wei Zhou are Software Engineers at Intel, and HBase contributors.