Sign Up for the Quarterly Newsletter
Previous Fling
Next Fling
fling logo of HCIBench

HCIBench

version 2.8.1 — March 03, 2023
VMware vSAN

Contributors 6

View All

Comments 1664

  • profile picture of Max Timchenko
  • profile picture of Chen Wei
  • profile picture of Chen Wei
  • profile picture of Max Timchenko
  • profile picture of Chen Wei
  • profile picture of Chen Wei
  • profile picture of Chen Wei
  • profile picture of Muhammad Toffaha
View All
Project Details
Summary Requirements Instructions Changelog Contributors
Discover
Similar Flings
Community
Comments 1664 Bugs 18

Summary

HCIBench stands for "Hyper-converged Infrastructure Benchmark". It's essentially an automation wrapper around the popular and proven open source benchmark tools: Vdbench and Fio that make it easier to automate testing across a HCI cluster. HCIBench aims to simplify and accelerate customer POC performance testing in a consistent and controlled way. The tool fully automates the end-to-end process of deploying test VMs, coordinating workload runs, aggregating test results, performance analysis and collecting necessary data for troubleshooting purposes.

HCIBench is not only a benchmark tool designed for vSAN, but also could be used to evaluate the performance of all kinds of Hyper-Converged Infrastructure Storage in vSphere environment.

Requirements

  • Web Browser:
    IE8+, Firefox or Chrome
  • vSphere 5.5 and later environments for both HCIBench and its client VMs deployment

Instructions

See the HCIBench_2.5_User_Guide.pdf for details.

Changelog

Version 2.8.1

  • Added multi ESXi credential support
  • Exposed compression and deduplication fields to both Vdbench and Fio generation page.
  • Optimized Easy-Run by adding compression/deduplication ratio while testing against vSAN datastore with dd/c enabled.
  • Updated the user manual to 2.8.1
  • Added storage policy recommendation for vSAN ESA in pre-validation
  • Bug fixes

Version 2.8.0

  • Updated fio to 3.30
  • Added latency target parameter for fio
  • Fixed CVE-2021-40438
  • Introduced vSAN support bundle graph for vSAN Debug mode
  • Enhanced resource utilization report
  • Introduced support for testing vSAN 8 ESA
  • Fixed issue for testing on VMC

Version 2.6.1

  • Bug fixes for graphite script when running vdbench
  • Enhanced pre-validation and ip allocation

Version 2.6.0

  • Upgraded tomcat to 8.5.68
  • Support ipv6 for both management and vm networks, with DHCPv6 or link-local mode
  • Support multi-writer disk deployment
  • Enhanced pre-validation by moving to govc
  • Support read/write io_limit specification for fio
  • Fixed issue of running on VMC
  • Enhanced vm support bundle collection in vSAN debug mode
Version 2.5.3
  1. Fixed graphite permission issue which blocked vdbench/fio grafana display
  2. Updated drop cache script to make it compatible with upcoming vSphere
  3. md5sum: 622625cc7a551bd7bf07ff4f19a57a17 HCIBench_2.5.3.ova
Version 2.5.2
  1. fixed testing issue for hybrid cluster
  2. fixed testing issue for the version prior to vSphere 6.7
  3. fixed easy-run issue with self-defined ip range
  4. replaced icon
  5. upgrade httpd to 2.4.46

Version 2.5.1 Update
  • Fixed non vSAN cluster testing issue
  • Fixed pre-validation storage policy issue
  • Added in-place upgrading capability, to do so, you can also upgrade HCIBench to the latest version by running the following command in HCIBench: tdnf install -y git && git clone https://github.com/cwei44/HCIBench.git && sh HCIBench/upgrade.sh
    MD5 Checksum: 1d14426f92b353e90469a8623ade2bc1 HCIBench_2.5.1.ova

Version 2.5.0 Update

  • Added support vSAN HCI Mesh testing, now you can test both local and remote vSAN datastores at the same time
  • Added support local storage including VMFS and vSAN-Direct testing
  • Added vSAN Debug Mode, allow user to collect vm-support bundle and vmkstats automatically when running testing against vSAN
  • Changed guest VMs name convention to {vm_prefix}-{datastore_id}-batch_num-sequence_num
  • Enhanced testing report format
  • Allow user to specify customized IP addresses for guest VMs
  • Allow user to configure CPU and Memory for guest VMs
  • Added best practice and network troubleshooting guide in the user manual
  • Bug fixes
  • MD5 Checksum: 817c2c788364f252e728d4253b3b96da HCIBench_2.5.0.ova

Version 2.4.0 Update

  • Fixed tvm deployment bug when specifying host
  • enabled easy run to support stretched cluster
  • fixed timezone issue on pdf report, and added more vSAN info into PDF report
  • set testname and testcase as variables in grafana
  • added CPU workload into fio config page
  • updated rbvmomi to support vsphere 7.0+
  • enhanced fio and vdbench graphite dashboards
    MD5 Checksum: 0cfd6cc852e33e5ce32022a66539b4c9 HCIBench_2.4.0.ova

Version 2.3.1 Update

  • Fixed static IP setting issue
  • Fixed reuse VMs on multi datastores issue
  • Fixed vm/tvm deployment issue
  • MD5 Checksum: 1b220f22575eacf62a965992a4c916e7 HCIBench_2.3.1.ova

Version 2.3.0 Update

Version 2.2.1 Update

  • Fixed docker volume moving issue
  • MD5 checksum of HCIBench_2.2.1.ova: 1a39c9df7d1485bc06332ae0b9d92ca7

Version 2.2 Update

  • Moved docker volume to sdb to avoid blowing up OS disk
  • Added Fio spreadsheet generator
  • Added DRS warning checkup 
  • Enhanced Grafana to keep all the historical data 
  • Added DNS exception handler 
  • Fixed RAM and PCPU reporting issue 
  • Fixed Vdbench spreadsheet not reporting issue 
  • MD5 checksum of HCIBench_2.2.ova: bb2a77dcf2ecc23b1ec2c30aee9945ec

Version 2.1 Update

  • Switched UI to dark theme 
  • Redesigned VMDK preparation methodology, which can complete much faster using RANDOM on deduped storage 
  • Added VMDK preparation process update 
  • Added Graphite port check into prevalidation 
  • Added vCenter/Host password obfuscation 
  • Added "Delete Guest VM" button 
  • Fixed Grafana display issue 
  • Fixed FIO blank results issue 
  • Bug fixes 
  • MD5 checksum of HCIBench_2.1.ova: d37e6f164ed962a6e7ccbe104ba9eaec

Version 2.0 Update

  • Added fio as an alternative workload generator
  • Added Grafana for workload live monitoring
  • Switched UI to clarity
  • Allow user to select one to four cases while using easy-run
  • Bug fixes
  • MD5 checksum of HCIBench_2.0.ova: ba3c2b06b8c27fb41a1bb68baedb325f 

Version 1.6.8.7 Update

  • Enhanced easy-run, put original 4k,70% read as the first test case, then 4k, 100% read and 256k, 100% write
  • Enhanced tvm deployment validation
  • Added Checksum into easy-run consideration
  • Updated guest VM template with increased ring_pages and disk scheduler
  • Added DNS configuration guidance into welcome message

Version 1.6.8.5 Update

  • Added 2 more test cases into easy-run, 4k 100% random read and 256k 100% sequential write
  • Batch deployment will be involved if deploying more than 8 VMs to speed up deployment process
  • Allow user to choose IP prefix when using static IP
  • Optimized UI to allow user to review the results by single click
  • Fixed regression issue when placing Datacenter/Cluster in the folder

Version 1.6.8.1 Update

  • Fixed regression when datastore is in the datastore folder
  • Avoid checking connection to host directly and use tvm deployment instead
  • Added Vdbench version check in summary script

Version 1.6.8 Update

  • Added resource pool and VM folder fields for VMC environment
  • Fixed easy-run disk size issue
  • Enhanced pre-validation error message handling
  • Changed the names of network interface from "Public Network" to "Management Network", and "Private Network" to "VM Network"

Version 1.6.7.2 Update

  • Enhanced write/read buffer/cache methodology
  • Fixed network ip-prefix selection issue
  • Fixed 95% percentile calculation issue

Version 1.6.7.1 Update

  • Fixed vSAN Performance Diagnostic API call
  • Fixed network validation message not clear issue
  • Fixed setting re-use VMs as default bug in 1.6.7

Version 1.6.7 Update

  • Enabled https instead of http
  • Added storage policy field, user can specify storage policy for the data disks. For this version, storage policy can't be assigned to existing client VMs
  • Enhanced deployment methodology
  • Enhanced vSAN Observer to avoid blow up the memory
  • Enhanced vSAN Performance Diagnostic API call with HCIBench workload configuration included
  • Added timestamp to the testing status
  • Bug fixes

Version 1.6.6. Update

  • Spectre & Meldown patch on both HCIBench VM and Client VM
  • Added client VM prefix field, allow running multiple HCIBench instances against single cluster
  • Attach testing log along with testing results
  • Enabled live vSAN Observer when running testing, using https://HCIBench_IP:8010
  • Updated the drop read/write cache script
  • Added more message info during the testing
  • Bug fixes

Version 1.6.5.2 Update

  • Added case comparisons by generating an XLS file for each test folder
  • Fixed bug when there's white space in datastore name or test name

Version 1.6.5.1 Update

  • Enhanced IP segment selection
  • Set open file limit to 4096
  • Updated vm-tools to the latest version
  • Bug fixes

Version 1.6.5 Update

  • Enhanced 95th percentile calculation.
  • Added Curve and Multi Run calculation.
  • Added SSH Service validation.
  • Replaced DHCP Service with Static IP Service.
  • Added IP conflict check.
  • Fixed bunch of bugs.
  • Change the default client VM RAM from 4GB to 8GB

Version Version 1.6.3 Update

  • Enhanced vSANPerformanceDiagnose function call
  • Enhanced port 443 validation
  • Enhanced results calculation
  • Added host maintenance mode validation
  • Added deployment validation

Version 1.6.2 Update

  • Integrated with vSAN Performance Diagnostic of vSphere_6.5U1/vSAN_6.6.1.
  • Added DHCP Service validation.
  • Added Vdbench workload profile validation.
  • Removed the root password expiration policy.
  • Changed results display to show full file names.
  • Changed easy-run calculation from host basis to disk-group basis.

Version 1.6.1 Update

  • Added network name uniqueness check
  • Changed the "disk warmup" to "Virtual disk preparation" to avoid confusion
  • Changed the pvscsi configuration, when there are more than 4 vmdks per pvscsi, more pvscsi controllers will be added and the vmdks will be evenly distributed
  • Bug fixes

Version 1.6.0.0 Update

  • Added Clear read/write cache option for vSAN.
  • Added Easy Run feature for vSAN, Easy Run can help vSAN user to determine the VMs/VMDKs/size/Disk_Init_Methods and run testing automatically.
  • Added Re-use VMs feature, user is able to re-use the existing client VMs for more tests.
  • Added the 95th percentile of calculation into the results.
  • Had special characters issue resolved and got other bug fixes.

Version 1.5.0.5 Update

  • Increased Timeout value of client VM disk from 30 seconds to 180 seconds.
  • Disabled client VM password expiration.
  • Disabled client VM OS disk fsck.
  • Set Observer interval to 60 seconds to shrink the size of observer data.
  • Fixed PCPU calculation.
  • Created link directory of /opt/automation/logs, user will be able to review the testing logs in http://HCIBENCH/hcibench_logs/
  • Increased the RAM of HCIBench from 4GB to 8GB to avoid running out-of-resource issue.

Version 1.5.0.4 Update

  • Added the checking if user saved the configuration
  • Increased the stack size to 65536 due to "ls too many arguments issue while process PCPU usage"
  • Fixed the typo from "Netowrk" -> "Network" in the deployment page.
  • Added vCenter hostname resolve checking.
  • Fixed the client VM OS VMDK size from 15.5GB to 16GB; changed the vmdk size specification from decimal format to binary format.

Version 1.5.0.3 Update

  • The bug "When tested against a non-vSAN datastore and having 'Directly Deploy on Hosts' checked, the test won't go through after deployment." is fixed.
  • Enhancement: extract storage policy information when doing pre-validation and checks if the deployment size would be too aggressive to vSAN.

Version 1.5.0.2 Update

  • Bug "test would fail if datastores' name contains white space" fixed.

Similar Flings

Oct 10, 2022
fling logo of vSAN Hardware Compatibility List Checker

vSAN Hardware Compatibility List Checker

version 3.0

The VSAN Hardware Compatibility List Checker is a tool that verifies all installed storage adapters against the VSAN supported storage controller list.

Apr 23, 2020
fling logo of vSAN Performance Monitor

vSAN Performance Monitor

version 1.3

The vSAN performance monitor is a monitoring and visualization tool based on vSAN Performance metrics.

performance
Apr 26, 2023
fling logo of vSAN Objects Viewer

vSAN Objects Viewer

version 1.5.7

Compared to existing analyzers, our tool decodes more details on vSAN DOM and LSOM, such as type and status distribution, going beyond hardware information and deep into DOM objects.

troubleshooting