Scheduled Scanning allows you to process new files or existing files based on a schedule. Instead of processing new files as they come in, your workflow may allow them to be scanned once per day (or at some other preferred time interval). For compliance reasons you may be required to scan all of your files on a quarterly basis and Scheduled Scanning will allow you to do that. Whether it is compliance or infrastructure efficiencies and cost optimizations, scheduled scanning provides you the flexibility to determine how you scan your data.
Creating a Schedule¶
Creating a schedule is a simple process. It is made up of two steps: picking the buckets you want included and defining the scan frequency.
Select the S3 buckets to include in this schedule
Leverage the flexible searching capabilities to your advantage if you have a lot of buckets that you need to sort through.
Define the Scan Frequency by filling in the details
Fields Description Schedule Name Name to identify this particular schedule (must be unique) Scan Period Choose what the scan frequency should be from: Hourly, Daily, Weekly, Monthly, Yearly Schedule Description Provide a useful description of what this schedule is doing (optional) Files to Scan This allows you to specify whether this schedule should scan all objects within the bucket or only the new objects since the last scheduled scan. When first creating or re-activating a schedule, the
last scanned valueis set to current UTC time.
- All Files - will scan all objects in the bucket(s) every time the schedule is kicked off
- New Files Since Schedule Creation - Going forward from creation and then from the last schedule execution, this schedule will scan only new objects found in the bucket(s)
- Scan All Files on First or Next Run - this is an add-on to the New Files scan option. This allows you to baseline the bucket(s) by scanning all objects within the bucket(s) the very first time the schedule is created. Note: you can click this again at any time the schedule is active and the very next execution of the schedule will scan all objects again.
Times to Scan Each scan period will allow you to specify options for when to scan within that period.
- Hourly - You can select
Every Half Hour
Note: If the buckets included in this schedule are quite large (>~8 million objects) you may not get find all the new objects in the half hour option
- Daily - You can select every hour of the day or some subset of the day (at noon and midnight for example)
Note: all times are UTC
- Weekly - You can select all days of the week or any subset. For example, every Friday night the schedule would run. Or every Monday, Wednesday and Friday.
- Monthly - You can select any one day of the month. For the last day of the month, select the
Last Day of Monthoption
- Yearly - You can select all months of the year or any subset. For example, if you wanted to perform quarterly scans you could select
Last Day of Monthfor March, June, September and December
A schedule is not activated when you save it (as you can see in the pic above). You have to take the second step from the row actions menu to activate the schedule. You may be prompted to take additional steps for region setup. If you are protecting buckets with a schedule in a region that has never been configured before, you will need to provide networking setup information as part of the schedule activation.
Once the schedule is created you will need to manage. There are 5 main things you can perform on your schedules: activate / deactivate, modify, delete and run it now.
Activating and Deactivating¶
As noticed above, schedules are not created in an
active state. You must activate the schedule using the action menu to the far right side of the schedule row. Simply click the button and select
Activate as seen below.
To deactivate the schedule you click the action menu and select
Deactivate. The deactivate menu option will only appear when the current state of the schedule is
Once a schedule is activated, it will execute at the Next Scheduled Scan time. While a schedule is inactive, it won't scan at all. If you reactivate a schedule that has previously run, its last scan time will be set to current UTC time and move forward as if it were a new schedule. The schedule will not go back to the previously active time.
If a schedule is missed because the console is offline (upgrades, reboots, etc), the schedule will make up for the lost time and scan since the set last scanned time.
You must deactivate a schedule before you can delete it.
You may need to modify an existing schedule in some way: add or take away buckets, modify the scan period, etc. Editing can be triggered by clicking the
Edit button from the action menu.
Make whatever changes are needed and click the
Whether you want to get rid of test schedules or a schedule that is no longer needed, you can easily delete a schedule from the action menu.
Note: the schedule must be inactive before it can be deleted. The delete menu option will not appear until the schedule is in an
If you need to run a schedule immediately or on-demand you can do so with the
Scan Now option in the action menu. If your schedule is an
All Files schedule then for the selected buckets all objects will be scanned. If the schedule is a
New Files schedule then new objects from the last scan date will be scanned.
The provided search field will search anything in the Schedule Name, Schedule Description and Scan Frequency columns. On top of that, you can also search by bucket names and any schedules that match the full or partial bucket name will be displayed.
Example bucket name search:
Per gigabyte pricing varies based on whether the objects are new objects or existing objects. The same applies to scheduled scanning. If the schedule is going back over existing objects, you will get the
retro scanning price. If the schedule covers new objects coming into the bucket(s), then you will be charged the
go forward data price.