Event Driven Scanning for New Files
Last updated
Last updated
Event driven scanning is where an event, in this case the All object create event, is leveraged on the bucket so any time an object is created/modified within the bucket an event is raised. Antivirus for Amazon S3 places and event destination / handler onto the protected buckets which listen for these events to trigger scanning. This allows Antivirus for Amazon S3 to easily plugin to any existing workflow you have without modifications.
So this looks as follows:
An object is added to a protected bucket
An event is raised and sent to an SNS Topic
The Antivirus for Amazon S3 provides an SQS Queue which subscribes to the Topic
One or more Antivirus for Amazon S3 Agents are monitoring the queue
Entries are pulled from the queue identifying the object to scan. The object is retrieved and scanned
Objects are handled according to the Scan Settings you have set a. All objects are tagged b. Infected files are moved to a quarantine bucket (default behavior)
This flow and behavior is irrespective of region. Amazon S3 buckets have a global view, but are regionally placed. This flow will be performed local to each region you enable buckets for scanning.
After a file has been scanned we will tag the object in the S3 bucket. These tags are how our solution recognizes whether we've previously scanned the file. If an object is tagged and is copied to another protected bucket, our solution will skip over scanning the object.
Here are examples of object tags based on scan results:
When a bucket is protected and event listener (SNS Topic) is added to the bucket. This will send S3 Events to the topic which in turn populates an SQS Queue. From there, everything is scanned in near real-time.
Users or apps upload objects to an S3 bucket protected by our solution.
The S3 bucket has an event listener which pushes a notification to Amazon SNS. SNS pushes a message to an Amazon SQS Queue.
Our Event Agent monitors this queue for objects and will copy the object into itself, performing a scan of the object.
The Event Agent will tag the object with its verdict.
The Event Agent will send scan results to Amazon CloudWatch.
The Event Agent will send problematic scan results to the Console service for surfacing.
(If Proactive Notifications are configured) The Event Agent will forward scan results to Amazon SNS.
(Optional) The Event Agent will move infected files into a Quarantine S3 Bucket
The Two Bucket System (2 Bucket System) allows a customer to physically separate the incoming files from the downstream users of the "production buckets". The separation lasts as long as it takes to scan the files and ensure they are clean. In this way, you can ensure nothing other than clean files makes it into your production buckets and therefore are safe to be consumed.
We have 2 options for a 2 bucket flow: Console approach and Lambda Approach.
The Console approach can be configured in the Console and moves files tagged as Clean to a designated destination bucket. This option is straightforward but cannot be tweaked for additional configurations.
Some customers opt to put a Lambda function into place for more granular control over objects. Moving multiple types of files or moving files to more than one bucket are common use cases for those who opt to utilize a Lambda instead of our console-managed two bucket functionality.
For guided steps on how to setup the 2 Bucket System go here
Users or apps upload objects to an S3 bucket protected by our solution.
The S3 bucket has an event listener which pushes a notification to Amazon SNS. SNS pushes a message to an Amazon SQS Queue.
Our Event Agent monitors this queue for objects and will copy the object into itself, performing a scan of the object.
The Event Agent will tag the object with its verdict.
The Event Agent will send scan results to Amazon CloudWatch.
The Event Agent will send problematic scan results to the Console service for surfacing.
If the Event Agent deems an object as Clean, it will move that object from the staging S3 bucket into the destination S3 bucket.
(If Proactive Notifications are configured) The Event Agent will forward scan results to Amazon SNS.
(Optional) The Event Agent will move infected files into a Quarantine S3 Bucket.
