Data Reduction Rate (DRR) Estimator for FlashBlade

Linux
Audience
Public
Product
FlashBlade
FlashArray
Technology Integrations
Linux
Source Type
Documentation

DISCLAIMER:

DRR compression results are Everpure Confidential, may contain 3rd-party calculations, and are provided as a rough approximation of current and future needs. Recommendations are based on customer provided input and reasonable assumptions made by all parties. Actual capacity and performance may vary from the calculated estimates. DRR projections may not reflect the customer's actual growth or future performance and capacity needs. Everpure provides no guarantees or warranties, express or implied, on the accuracy of the estimates provided.

NDA information:

Customers using the DRR Estimator tool need to sign an NDA. Our NDA is available for you to generate off of your customer or partner’s SFDC Account page. Look for “ContractExpress” in the tools across the top. If your customer or partner needs their own NDA reviewed - submit it through the same place and it will come right to one of your legal partners to review! Tip: NDAs, End User Agreements and Partner Agreements are all created from the Account page as they apply to the entire account (versus a specific opportunity).

Note:

Everpure Confidential. Everpure provides no guarantees or warranties, express or implied, on the accuracy of the estimates provided. Actual capacity and performance may vary from the calculated estimates.

Question

What is the Data Reduction Ratio (DRR) Estimator tool, and how is it used?

Answer

DRR Estimator tool is a simple and easy to use tool for estimating data compression rate in a customer environment. The tool provides estimates of data reduction that customers can expect when storing their data on Pure FlashBlade appliances.

It also serves the needs of sales and partners to give their prospective customers a glimpse of compression they can expect on a Pure Flashblade appliance. The DRR Estimator tool can effectively serve as a precursor for using the Pure sizer apparatus. Sales and partners can use realistic information when using the sizer for arriving at suitable configurations for the end-customer.

What does the DRR Estimator tool do?

  1. Processes all accessible files and folders under the given path.

  2. Performs computation of 3 compression values:

    • FlashBlade 1st Generation
    • FB//S200
    • FB//S500

  3. Compression estimates are available in an aggregated manner at multiple levels:

    • DRR estimate of the entire path contents.
    • DRR estimate for each file type.
    • DRR estimate for different file size buckets.

  4. Provides two different compression estimation options:

    • Quick Scan (default): Processes a smaller sample of the file in case of large sized files for compression estimation (and assumes uniform compression across the file). In case of high density folders, a smaller sample set of files is processed for compression estimation.
    • Full Scan: Processes all files completely for compression estimation.

What are the considerations while executing the tool?

  1. DRR Estimator tool can only be executed on a Linux based client.

  2. The tool does not edit any dataset file/folder and only requires read privileges for the mount/folder for its execution. The tool privileges should be able to walk through the dataset directory layout for its execution.

  3. Tool execution will put load on the file system and host while reading files.

  4. Client resource utilization levels may vary depending on the underlying dataset. Executing the tool in non-peak hours is recommended.

  5. Time taken to get the DRR results may vary depending on the dataset.

  6. Inaccessible files and folders are ignored in the compression estimation results.

Tool Recommendations

  1. The following Linux client configuration is recommended for executing the tool:

    • 4 CPU/vCPU

    • 8GB RAM

  2. The DRR Estimator tool is supported on POSIX Linux Operating Systems like CentOS, Ubuntu, etc.

  3. Scanning a small dataset is advised to check resource impact and performance before scanning larger datasets.

  4. Scan a dataset representative of the overall production workload to arrive at informed compression estimations.

License

Distribution and use of the DRR Estimator tool is governed by the Everpure EULA for Plugin / Adaptor / Provider / SDK / Management Pack, which is available here. Licenses for open source software used by DRR Estimator tool can be found in the downloadable .tgz file.

Downloading DRR Tool

Everpure customers and partners can download the DRR tool and user guide by signing in and using the link below (login required):

DRR Estimator tool

Release Date

Download Links

Fingerprints

1.0.1 (GA)

Mar 01, 2023

tar archive:

drr-estimator-tool-1.0.1-linux-x86_64.tgz.zip

tar archive sha256: 62726a46fb9023484c0be0a2eb08f7a86f3caaa9173cd961f8eb49a93427e828

1.0.0 (GA)

Jan 30, 2023

tar archive:

drr-estimator-tool-1.0.0-linux-x86_64.tgz

tar archive sha256: 0b9ad31150d8d52ee3715213318280f494d7423ecd2399d9838a6e5a9af04de3

Examples

  1. Display the help menu:

    
./pure_drr_estimator -h

Pure Data Reduction Rate Estimator
Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html.
The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com

Usage: pure_drr_estimator [OPTIONS]... DIRECTORY

Options:
    -f, --full
            Perform Full Scan and compression processing. Default is Quick Scan.
            NOTE: Full Scan performs the compression on the entire contents of the selected folder. As such, the operation may take time depending on the density and size of the folder.
Common Options:
    -h, --help
            Show this message
    -V, --version
            Show version

Report bugs and find online help at:https://support.purestorage.com/Solutions/Linux/Linux_Reference/Data_Reduction_Rate_(DRR)_Estimator_for_Flashblade

  2. Perform DRR estimation scan of a sample dataset(default is quick scan):

    
./pure_drr_estimator /path/to/directory 

Pure Data Reduction Rate Estimator
Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html.
The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com

Disclaimer: Quick scan DRR results may sometimes be unable to give a correct estimation for certain pre-compressed file types. It is recommended to run a full scan to arrive at accurate estimations

Version: v1.0
Scan performed: Quick Scan
Timestamp: 1673585697 ( 01/12/2023 23:54:00 EST )
Dataset Details:
Total files scanned: 10000
Total folders scanned: 1
Inaccessible/Skipped files: 0
Inaccessible/Skipped folders: 0

Top level Compression Details:
Actual Size         FB-1st-Gen                  FB//S200               FB//S500
1.72 GB             11.22 MB (157.35:1)         1.95 MB (907.84:1)     1.95 MB (907.84:1)

Compression Details by File Size:
Type         Count    Actual Size    FB-1st-Gen                   FB//S200                 FB//S500
<4KB         3000     9.54 MB        165.04 KB (59.17:1)          82.03 KB (119.05:1)      82.03 KB (119.05:1)
4KB-16KB     1000     7.63 MB        80.08 KB (97.56:1)           27.34 KB (285.71:1)      27.34 KB (285.71:1)
16KB-32KB    1000     19.07 MB       156.25 KB (125.00:1)         42.97 KB (454.55:1)      42.97 KB (454.55:1)
32KB-128KB   2000     135.42 MB      923.83 KB (150.11:1)         179.69 KB (771.74:1)     179.69 KB (771.74:1)
128KB-512KB  1000     143.05 MB      940.43 KB (155.76:1)         167.97 KB (872.09:1)     167.97 KB(872.09:1)
>512KB       2000     1.42 GB        9.01 MB (161.06:1)           1.46 MB (996.07:1)       1.46 MB (996.07:1)

Compression Details by File type:
Type     Count    Actual Size        FB-1st-Gen                   FB//S200                 FB//S500
.c       1000     47.68 MB           339.84 KB (143.68:1)         74.22 KB (657.89:1)      74.22 KB (657.89:1)
.cpp     1000     143.05 MB          940.43 KB (155.76:1)         167.97 KB (872.09:1)     167.97 KB (872.09:1)
.doc     1000     3.81 MB            58.59 KB (66.67:1)           27.34 KB (142.86:1)      27.34 KB (142.86:1)
.iso     1000     572.20 MB          3.56 MB (160.90:1)           589.84 KB (993.38:1)     589.84 KB (993.38:1)
.ost     1000     1.91 MB            47.85 KB (40.82:1)           27.34 KB (71.43:1)       27.34 KB (71.43:1)
.ppt     1000     19.07 MB           156.25 KB (125.00:1)         42.97 KB (454.55:1)      42.97 KB (454.55:1)
.pst     1000     87.74 MB           583.98 KB (153.85:1)         105.47 KB (851.85:1)     105.47 KB (851.85:1)
.txt     1000     3.81 MB            58.59 KB (66.67:1)           27.34 KB (142.86:1)      27.34 KB (142.86:1)
.xlx     1000     7.63 MB            80.08 KB (97.56:1)           27.34 KB (285.71:1)      27.34 KB (285.71:1)
.xyz     1000     879.29 MB          5.46 MB (161.16:1)           902.34 KB (997.84:1)     902.34 KB (997.84:1)

Pure DRR Estimator took 17 Seconds

  3. Perform full DRR estimation scan of a sample dataset:

    
./pure_drr_estimator -f /path/to/directory

​​Pure Data Reduction Rate Estimator
Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html.
The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com

Version: v1.0
Scan performed: Full Scan
Timestamp: 1673585943 ( 01/12/2023 23:59:00 EST )
Dataset Details:
Total files scanned: 100000
Total folders scanned: 1
Inaccessible/Skipped files: 0
Inaccessible/Skipped folders: 0

Top level Compression Details:
Actual Size         FB-1st-Gen                  FB//S200                FB//S500
17.25 GB            112.25 MB (157.35:1)        19.45 MB (907.84:1)     19.45 MB (907.84:1)

Compression Details by File Size:
Type          Count     Actual Size     FB-1st-Gen                 FB//S200                   FB//S500
<4KB          30000     95.37 MB        1.61 MB (59.17:1)          820.31 KB (119.05:1)       820.31 KB (119.05:1)
4KB-16KB      10000     76.29 MB        800.78 KB (97.56:1)        273.44 KB (285.71:1)       273.44 KB (285.71:1)
16KB-32KB     10000     190.73 MB       1.53 MB (125.00:1)         429.69 KB (454.55:1)       429.69 KB (454.55:1)
32KB-128KB    20000     1.32 GB         9.02 MB (150.11:1)         1.75 MB (771.74:1)         1.75 MB (771.74:1)
128KB-512KB   10000     1.40 GB         9.18 MB (155.76:1)         1.64 MB (872.09:1)         1.64 MB (872.09:1)
>512KB        20000     14.17 GB        90.12 MB (161.06:1)        14.57 MB (996.07:1)        14.57 MB (996.07:1)

Compression Details by File type:
Type          Count     Actual Size     FB-1st-Gen                  FB//S200                  FB//S500
.c            10000     476.84 MB       3.32 MB (143.68:1)          742.19 KB (657.89:1)      742.19 KB (657.89:1)
.cpp          10000     1.40 GB         9.18 MB (155.76:1)          1.64 MB (872.09:1)        1.64 MB (872.09:1)
.doc          10000     38.15 MB        585.94 KB (66.67:1)         273.44 KB (142.86:1)      273.44 KB (142.86:1)
.iso          10000     5.59 GB         35.56 MB (160.90:1)         5.76 MB (993.38:1)        5.76 MB (993.38:1)
.ost          10000     19.07 MB        478.52 KB (40.82:1)         273.44 KB (71.43:1)       273.44 KB (71.43:1)
.ppt          10000     190.73 MB       1.53 MB (125.00:1)          429.69 KB (454.55:1)      429.69 KB (454.55:1)
.pst          10000     877.38 MB       5.70 MB (153.85:1)          1.03 MB (851.85:1)        1.03 MB (851.85:1)
.txt          10000     38.15 MB        585.94 KB (66.67:1)         273.44 KB (142.86:1)      273.44 KB (142.86:1)
.xlx          10000     76.29 MB        800.78 KB (97.56:1)         273.44 KB (285.71:1)      273.44 KB (285.71:1)
.xyz          10000     8.59 GB         54.56 MB (161.16:1)         8.81 MB (997.84:1)        8.81 MB (997.84:1)

Pure DRR Estimator took 3 Minutes 44 Seconds

Additional Information

  • If you are a registered Everpure user and need assistance with DRR Estimator tool, please contact Everpure Technical Services.

  • If you would like to submit product feedback on DRR Estimator tool, please submit a comment via the Feedback link on this topic.