Conflicted buckets
Amazon S3 buckets have an event system associated with them. There are 15 possible events that can be triggered on an bucket.
Last updated
Amazon S3 buckets have an event system associated with them. There are 15 possible events that can be triggered on an bucket.
Last updated
As of v7.00.000 we automatically use EventBridge to resolve any bucket conflicts.
If you are running an older version of our product you will need to use the instructions in this article to troubleshoot the conflicted bucket.
If you have any questions or need assistance please Contact Us.
There is one particular event, All object create events, that Antivirus for Amazon S3 listens for. Because AWS only allows one event listener per event, this is where the conflict can occur. If the bucket has an event listener already assigned to All object create events then we could have a conflict.
There are 3 types of event listeners than can be assigned to each of the events: SNS Topics, SQS Queues and Lambda Functions. Amazon Simple Notification Service (SNS) is a highly available, durable, secure, fully managed pub/sub messaging service. Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. AWS Lambda lets you run code without provisioning or managing servers.
We reflect these conflicts with colored backgrounds to the rows in the bucket management views. A yellow row represents that your bucket has an SNS Topic on it for the event we care about. It is yellow because we can fix this one automatically as you'll learn below. A red row indicates you have either a Queue or a Lambda assigned to the event. We cannot directly fix those and they will require some intervention on your part. Read on to solve these.
Because SNS is a pub/sub messaging service, which multiple subscribers can in fact subscribe to, we can simply subscribe the Antivirus for Amazon S3 queue to the Topic to receive all the events triggered post setup. We will do this automatically when enabling a bucket in the console. So no real conflict when the event listener is an SNS Topic. We just want to make you aware.
There is a chance multiple publishers push messages to this topic. If that is the case, Antivirus for Amazon S3 will see all messages and attempt to process them. If you have this setup, Contact Us to work through one additional option we can explore.
The simple answer here is: turn these to yellows. Done. Ok, that is simplistic and there are some steps needed here, but it is true that if these buckets only had SNS Topics on them then we all (your queue, your lambda and our Antivirus for Amazon S3) could subscribe to the Topic and each get the event package we needed.
You should take a moment to determine what functions you are performing based on that event trigger to ensure there won't be a conflict between what you are doing and what we are doing. For example, you expect the object to be there, but we found an infection and quarantined it to another bucket. Does this break your flow?
Now let's walk through how we change from red to yellow.
The short answer is we need to place an SNS Topic on the bucket and assign it to the All create object events. Then subscribe your Queue/Lambda that was originally assigned to the event now to the Topic. And then go back to the Antivirus for Amazon S3 console and enable the bucket for scanning. We will automatically subscribe our queue to the topic you specified.
You have two choices when it comes to the SNS Topic: create one from scratch or leverage the one Antivirus for Amazon S3 creates. For simplicity we'll walk through leveraging the Antivirus for Amazon S3 created Topic.
If you have not protected any buckets in the given region prior to these steps the SNS Topic we create will not have been created yet. Protect another bucket, even temporarily, in that region to have the SNS Topic created and then you can proceed.
We'll walk through how to fix a bucket with lambda configured on it. Fixing a red "queue" bucket will be the same process, but for a queue rather than a lambda function. What it looks like in the Antivirus for Amazon S3 console:
What it looks like in the Amazon console:
As you have it configured here, each time a new object is created/modified the All object create events fires and sends the event to a Lambda function. In this case the helloworld lambda function.
First, we'll simply delete the event off of the bucket. Now select the event, click Delete and then click Save.
With this done, if you go back to the Antivirus for Amazon S3 console the bucket will no longer be conflicted.
While here, enable the bucket for scanning to apply the SNS Topic to the bucket.
Which gives you this now:
You're half way there. Next thing to do is subscribe your lambda to the Topic so it continues to receive the new object bucket events. The easiest way to do this is from the Lambda Management page. Navigate to AWS Services-->Lambda--><lambda function name> as seen below.
Delete the S3 Trigger seen there and then click the + Add trigger button and populate the following screen as seen below:
Click the Add button and you will now see the following for your lambda function:
Make sure to click Save on the Lambda page in the upper right corner. Once you do, you will no longer see the S3 Bucket trigger and will be left with just an SNS Trigger as seen below:
You have now made it so your Lambda function will get triggered by the SNS Topic instead of directly from the S3 Bucket.
There is one difference in the JSON the Lambda will receive. The JSON coming directly from the bucket was just that, the package of information related to the object added to the bucket. When coming from the SNS Topic, that same package of information is wrapped in an SNS Topic JSON block as the Message. So you will just need to grab that Message block and you will be right back where you started. You can see the differences here:
Now you are all set! Your Lambda function gets the event, as seen above, and the Antivirus for Amazon S3 SNS Topic also receives the event and can now scan the objects.
One more reminder, if that Lambda was only tied to a single bucket and you only want it to receive events from that single bucket, you may need to take a slightly different approach. As mentioned earlier, this SNS Topic receives events from all enabled buckets within the region. You will be better off creating a unique SNS Topic that only applies to this bucket. Subscribe your Lambda to it just as we did above and then simply enable this bucket from the console. We'll treat it as a yellow bucket and just subscribe our queue to that Topic as well.
