Scan Settings

Scan Configuration Overview

There are three configuration adjustments you can make to the Scanning Agents: Tag Name changes, Infected File handling and decisions around which objects to scan (Scan listing / Skip listing). These Scan Behavior Settings modifications will apply to all agents currently running and new ones that spin up going forward no matter the region. Changes made here will not require a reboot for any agent, but could take 30 seconds to take affect. Such as, custom naming the tags we apply to each object as well as providing scan list and skip list functionality for the buckets.

Object Tag Keys

Every file the s3 scanner touches has an AWS Tag applied to it. You can change the default key names if required or desired, but not the values.

Key	Description
scan-result	Identifies whether a file is found to be clean or having issues. Possible values: clean, infected, unscannable, error clean = no issues found with file infected = malware found; note: encrypted and password protected files will also be marked as infected at this time unscannable = access to object issues due to KMS or object greater than 2gb in size and therefore too large to scan error = access to object issues due to cross account permissions
date-scanned	The date and time the object was scanned
message	A description of what has been identified in the file. Only populated for Error or Unscannable results.
virus-name	Name of virus identified
uploaded-by	AWS user identifier

Action for Infected Files

There are 3 main actions you can take with an infected file: move (default), delete and keep.

Move directs the scanner to take the file and place it (copy then delete) in a quarantine bucket. The console creates a quarantine bucket in each region you enable buckets for scanning. The bucket will be named uniquely with the ApplicationID tacked onto it along with the region it was created in. This is the default behavior.

Delete directs the scanner to remove the file entirely.

Keep directs the scanner to leave the file in the bucket it found it. The scanner will still tag the object appropriately.

Private Mirror - Local Signature Updates

In certain situations, you may prefer the scanning agents to retrieve signature updates locally rather than reaching out over the internet. Local updates will allow you to better control and potentially eliminate outbound access for the VPCs hosting the scanning agents. This new option allows you to specify an Amazon S3 bucket in your account for the scanning agents to look to for signature updates. You can get the updates into this bucket however you see fit (sample lambda function provided below). Each time a scanning agent boots it grabs the latest definitions. A running scanning agent will check every 6 hours for new signature definitions.

Note

If you can't wait 6 hours after a new signature update comes out, simply reboot all your agents and they will pick the new updates up immediately.

Sample Lambda Code
The following lambda code can be leveraged to pull the three update files down the Amazon S3 bucket of your choice you indetify in the BUCKET_NAME variable. That is the only modification you would need to make to the code itself. Add a time-based CloudWatch trigger event to it to grab updates on the timing of your choice (hourly, every 4 hours, etc) and modify the timeout to be 3 minutes and you should be all set.

import urllib.request
import boto3
import botocore
# Set to the name of your desired private mirror bucket
BUCKET_NAME = '<specify-bucket-name-here>'
CVD_FILE_NAMES = ['main.cvd', 'daily.cvd', 'bytecode.cvd']
def lambda_handler(event, context):
    s3 = boto3.client('s3')
    try:
        for cvdFileName in CVD_FILE_NAMES :
            tmpFile = f'/tmp/{cvdFileName}'
            urllib.request.urlretrieve(f'https://database.clamav.net/{cvdFileName}', tmpFile)
            response = s3.upload_file(tmpFile, BUCKET_NAME, cvdFileName)
    except Exception as e:
        print(e)
        raise e
    return {
        'statusCode': 200
    }

Note

The signature update process uses the object URL to access the objects. Therefore, the objects must be public so the standard URL can be used to access them.

Bucket Path Definitions - Scan and Skip Lists

Buckets themselves are inherently skipped by the fact they are turned off to start. When you enable a bucket for scanning, you are explicitly marking it to be scanned. When you do this and nothing else, then all objects in the given bucket will be scanned no matter the path inside the bucket. This portion is all handled from the Bucket Protection page.

The Scan List and Skip List located here inside the Agent Configuration page allows you to take this concept a step further by allowing you to apply it to paths (folders) within the buckets. Scan listing and skip listing are opposites of one another. Scan listing a path is explicitly marking that specific path(s) within that bucket to be scanned. All other paths within the bucket will be ignored. You can list as many paths within the bucket as you'd like. For example, you have 5 different paths within the bucket and you want to scan only 3 of them. Simply add the 3 you want scanned to the Scan List. There is no need to add the other 2 paths to the Skip List as they will automatically be skipped. Skip listing a path will stop that defined path(s) from being scanned, but leave all others to be scanned. Depending on how many you want to include versus exclude you can choose which list to leverage. From the previous example, you could have just Skip listed the 2 paths you didn't want to scan which leaves the other 3 to be scanned. With numbers that split you could go either way, but if you have a much larger number of paths, one may become more self-evident.

Special Scan / Skip List Capabilities

The root of the bucket is also a "path" that we define in the settings as an empty path. So, if you'd like to scan or skip list the root along with your paths you can do so.

---

You can use a wild card (*) in the path. This is useful in a repeated sub-path structure where you need to skip a certain folder amongst all those top level folders.

For example, you have a path structure that is Year/Month/scanMe and Year/Month/skipMe where the month reflects each month of the year creating 12 unique paths. Underneath that Month folder you have folders you want scanned or skipped. You can create a path one time to setup scanning in every Month folder like this: Year/*/scanMe. That path will scan the scanMe folder under every month in the bucket without you having to add 12 entries.

This could also be leveraged not just in the path, but down to the object name as well. If you only wanted to scan a certain file type in a given bucket/folder you could put a path with *.jpg.

Note

As you can see, but wasn't spelled out, the * does not mean everything at the level it is placed and below. If you wanted everything underneath Year you could place a path /Year/ and that would do absolutely everything below Year. The wildcard represents the level itself where it is placed only.

---

Similar to wild cards an often used with wild cards, you can specify a global entry or your Scan and Skip Lists. If you have a repeated path structure across all your buckets where being able to define a scan / skip entry that would apply to all, you can select the _GLOBAL_ option in place of a bucket to then define across all buckets.

Let's take a look at an example. The following is an S3 bucket with 4 paths (folders) in it. With the default settings, meaning no paths defined, the root and all 4 folders will be scanned.

First thing you need to do is enter the <bucket name> in the Scan List or Skip List field and click Add Scan list.

You'll get:

Next, you'll add the path you want to scan and click the Add Entry button. *This can be the full multi-depth path.

That's it. You've now identified that the only thing you will scan in the css-protect-01 bucket will be the scan-me folder and nothing else. The steps above can be used for skip listing as well. Look below at the examples.

Default Setting - scan all

Scan list - scan-me folder

Skip list - root and donot-scan-me

Full Rules

Note

You probably wouldn't have a scan list and a skip list entry for the same bucket as we see in this example, but it is possible and I'm sure a scenario could be found to support it.

---

Last update: December 28, 2020