Dell DR Series Appliance Cleaner Best Practices Dell Engineering June 2016 A Dell Best Practices Document
Revisions Date Description July 2016 Initial release THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND. Copyright © 2016 Dell Inc. All rights reserved. Dell and the Dell logo are trademarks of Dell Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.
Table of contents Revisions ............................................................................................................................................................................................. 2 Intended audience ............................................................................................................................................................................ 4 1 Deduplication and cleaning .....................................................................
Intended audience The DR appliance cleaner is process that is configured to run efficiently and effectively out of the box with no tuning or adjusting required. Only in extreme cases will the DR cleaner possibly may need calibration.
1 Deduplication and cleaning Data deduplication refers to a technique for eliminating redundant data across all files in a data set. Large amounts of data can be significantly reduced saving costs and resources. The DR implements a variable block sliding window deduplication engine to produce industry leading results. Each file stored in a DR appliance is a blockmap consisting of pointers to its chunks of data saved on the filesystem.
This document explains how to determine if the cleaner is keeping up, how much is required for the cleaner to finish, how to adjust the cleaner schedule and runtimes, how to determine what the cleaner is doing and best practices on what to set. Note: To use this document, ensure the DR appliance is running 3.2.0.2 or 3.2.6.1. Build 3.2.0418.1a and future releases can also be used with this document.
2 How to determine if the cleaner is keeping up Dell recommends that a full cleaner pass compete once every 7 days or less. To determine if the cleaner is keeping up, a quick inspection of the cleaner statistics ‘Last Completion time(s)’ variable is described below. 2.1.1 Using the stats --cleaner command The stats --cleaner command displays the current running cleaner progress and the amount of time taken to complete its latest full pass.
2.1.2 Understanding cleaner statistics The ‘Last Completion time(s)’ variable provides the latest full pass cleaner runtime. In Example-1 above, the cleaner completed its latest full pass consuming 258897.00 seconds or 2.99 days. To convert from seconds to days, take the total seconds and divide by 86,400 (60 * 60 * 24 = 86,400). Thus, 258,897 seconds / 86,400 = 2.99 days. Because the cleaner completed its latest job within a week’s time frame (7 days), the cleaner is not falling behind.
3 Cleaner adjustments & best practices Under rare and extreme cases, the DR cleaner may run longer than 7 days. If so, apply the following cleaner best practices in order when adjusting cleaner runtimes. 1. If the cleaner is set to run automatically (default setting) schedule the cleaner to run for 40 or more hours per week. 2. Schedule the cleaner to run in long durations instead of smaller short durations. 3.
2. For each day, apply a cleaner runtime schedule, then click Set Schedule. 3. Inspect the newly configured cleaner schedule. 4. After a week (7 days), run the stats --cleaner command to examine cleaner’s ‘Last Completion Time(s)’ value. If the cleaner completed a full pass within 7 or less days, no more cleaner adjustments are necessary. If the cleaner has not yet completed a full pass or has taken beyond 7 days to complete, increase the cleaner schedule runtimes and repeat this step.
3.1.1 Cleaner considerations 1. Additional cleaner runtime is required as more data is ingested or begins to age out. 2. When the cleaner is triggered to run by manual, scheduled or forced methods, the cleaner is run at the same priority as ingest and replication. 3.1.2 Scenario 1: Cleaner analysis and adjustments Raymond has been successfully running his DR appliance at his computing center for the past 3 years.
Raymond adjusted cleaning schedule successfully completes in 6.78 days which meets Dells recommended full cleaner pass of 7 or less days.
A Additional cleaner commands A.1 Create a cleaner schedule Using the CLI: schedule --add –day --start_time --stop_time -cleaner A.2 Examine the cleaner Using the CLI: schedule --show --cleaner A.
schedule --delete --day --cleaner Other cleaner schedule help is defined by running the following command: Schedule --help A.4 Running the cleaner manually The cleaner can be manually run and is a onetime run and will complete a full cleaner pass very quickly as it does not pause for any DR activities until the cleaning process has completed.
Using the GUI: 15 DR Series Appliance Cleaner Best Practices
B How to run the cleaner There are multiple ways in which the cleaner process is run: Scheduled – The cleaner is triggered to begin processing and continues until the cleaning work has been completed or the scheduled window ends. If cleaning has completed within the scheduled window, it will not start again until the next scheduled window. For large cleaning jobs, the cleaner will span multiple schedule windows picking up where it left off until the cleaning job has completed.
C Estimating cleaning work currently in progress The ‘Estimated Logical Bytes left’ value provides current active cleaner progress. It provides an estimate of how much work is left for the current active cleaner to complete a full pass. When this value reaches 0, the cleaner has completed a full pass.
The DR cleaner operates in three states: Done – When the cleaner is done and there is nothing more to process. Running – The cleaner is currently running. Pending – The cleaner has work to do, but is currently waiting to run. In Example – 2 above, the cleaner is placed in “Pending” mode because there is cleaning yet to be done, and the DR cleaner is outside of its scheduled window of operation and the DR is not idle, but is busy with ingest or replication.
D If the cleaner runs behind When a full cleaner pass runs beyond the 7 days recommended by Dell, the cleaner is considered to be in cleaning debt. For example, if a full cleaner pass completes in 9 days, the cleaner has 2 days of cleaning debt and cleaner runtimes need to be immediately increased. Cleaning debt occurs when the DR ingests more data than can be cleaned. Depending on the rate of cleaner debt accumulation, the DR can quickly fill up eventually halting backups.
Cindy notes that the latest full pass cleaning process required 1,069,260 seconds or 12.38 days (1,069,260 / 86,400) thus, Cindy’s cleaner has fallen behind of 5.3 days (12.38 – 7 = 5.3) in cleaner debt. Cindy must increase her cleaning runtimes such that cleaning debt begins to diminish and eventually achieves a full cleaner pass to complete within 7 days.
Cindy takes note of the value 66,390,640 in the Estimated Logical Bytes left variable and logs it with the date. After making the cleaner adjustments above, Cindy waits 7 days to determine if additional cleaner adjustments are required. She hopes to reduce the accumulated cleaner debt, and the weeks’ worth of aged out data.
Notice that a full cleaner pass completed in ~ 7.85 days (678,927 / 86,400) which means Cindy’s adjusted 90-day cleaner schedule has nearly eliminated the cleaner total run time from 12.38 to 7.85 days. With such big cleaner debt progress and such little cleaner debt remaining, Cindy believes that the current cleaner settings will accomplish Dells recommended full cleaner pass within the next 7 days during next week’s backup cycle.
Cindy then runs the following command: stats --cleaner Cindy has achieved in setting a successful cleaner schedule as her full cleaner pass has completes in 7 days or less. Her full cleaner pass actually completes in 4.76 days (411,480 / 86,400). Cindy will wait another 3-6 months and will reexamine if her cleaning runtimes.
E 24 Cleaner monitoring and adjustment flow chart DR Series Appliance Cleaner Best Practices
