How to Deploy¶
The detailed Deployment Guide has more detail for some topics and information regarding pre-requisites and the long term running of the solution. It is a valuable resource.
Please check it out for more information on the following:
- Total Cost of Ownership - also check out TCO Video
- Recovery discussions
Ensure you have properly subscribed to Cloud Storage Security Antivirus for Amazon S3 before you attempt to deploy the CloudFormation template. If you are not properly subscribed, the deployment will fail to start. Antivirus for Amazon S3 won't run because it will fail the entitlement check.
If you'd like to run the software outside the context of the Amazon Marketplace, please Contact Us to discuss.
Deploying Antivirus for Amazon S3 is accomplished by using a CloudFormation Template that will install the necessary infrastructure components as well as the required roles and permissions. This section will help you fill out and run the CloudFormation Template.
The CloudFormation Template will create the following resources:
ECS Fargate Cluster with 1 Service and Task
(This is used to run the Antivirus for Amazon S3 Management Console)
(This is used to save data for the software)
IAM Roles and Policies
(These are used to run the software)
(Used for user management)
SNS Topic and CloudWatch Log Groups → Streams
(These are used for logging and notification purposes)
(OPTIONAL - Leverage to use your own domain or to abstract the Management Consoles public access a step further. Check out the Deployment Details for more information)
Once running, the Management Console will create the following resources:
Adds 1-2 Services and Tasks (Scanning Agents) to existing region cluster
(This is used to run the scanning agents that process the objects)
Adds 1 ECS Cluster and 1-2 Services and Tasks in each additional region you scan buckets
(This is used to run the scanning agents in new regions)
SNS Topic, SQS Queue, S3 Bucket events, CloudWatch Log Groups → Streams
(These are used to keep track of the object work)
Launching the deployment template from the Marketplace Listing will default you to the
us-east-1 region. If you'd like to deploy in a different region, please ensure to change regions before proceeding.
Supported regions for Console deployment:
us-east-1 (N. Virginia), us-east-2 (Ohio), us-west-1 (California), us-west-2 (Oregon), ap-south-1 (Mumbai), ap-northeast-2 (Seoul), ap-southeast-1 (Singapore), ap-southeast-2 (Sydney), ap-northeast-1 (Tokyo), ca-central-1 (Canada), eu-central-1 (Frankfurt), eu-west-1 (Ireland), eu-west-2 (London), eu-west-3 (Paris), eu-north-1 (Stockholm), me-south-1 (Bahrain), sa-east-1 (Sao Paulo), and GovCloud (West) AWS regions.
Missing regions are where Amazon Cognito is not supported
Step 1 - Quick Create Stack and Specify the Parameters¶
Simple / Default Deployment¶
You must provide values for the fields in red boxes above. You can leave all the default settings in the rest of the fields.
- You must select a
VPCfrom the drop down list for the Management Console to run in.
You must select a
Subnet A IDfrom the drop down list contained in the selected VPC.
The subnets you choose for
Bmust either be able to offer a public IP or be on a network you can access. Without that, the console will run, but not be reachable. This could simply be a public subnet reachable through an Internet Gateway. Or it could be a private subnet that is an extension of your corporate network you access either from your corporate network or through a VPN connection. Or these subnets maybe private subnets fronted by a public Load Balancer. Learn more about this option in the Advanced Deployment Considerations.
You must select a different
Subnet B IDfrom the drop down list contained in the selected VPC.
YOU MUST SELECT A DIFFERENT SUBNET FROM THE ONE SELECTED IN Subnet A ID. If you select the same subnet, stack creation will fail. If you happen to pick the same value for each field, you will get a message as follows, please correct this before moving on.
Ensure both subnets you select belong to the vpc you have selected. It is a best practice to select subnets in different Availability Zones.
You must select a
Console Security Group CIDR Block. Specify your network access or set to "0.0.0.0/0" to access the Antivirus for Amazon S3 Management Console from anywhere.
0.0.0.0/0 is wide open to the world. The application is still protected by SSL, a username and a password, but you may still choose to control this access. This could be your corporate network range or the /32 IP address your specific system is currently assigned. Google "what's my ip" and the first thing returned is what your current IP address is.
Note: If your IP address changes, you may have to manually change the AWS Security Group value to represent the new IP
You must set a valid Email address.
If you do not properly set the above 5 items, the stack creation will fail or you will be unable to access your deployment. If you run into any trouble whatsoever, please Contact Us.
Advanced Deployment Considerations¶
Advanced Considerations - Additional Deployment Options
Filling in the 5 fields discussed above are the minimum fields that have to be set to get the product successfully deployed. For many deployments, the default settings will work well. With that said, the other options in the CloudFormation Template could be beneficial for you and therefore could warrant your attention.
Feel free to SKIP THIS SECTION if you just want to start with the defaults. With two exceptions:
Consider This Now
An AWS Load Balancer can only be leveraged and setup at the time of deployment. If you decide you want a load balancer after deployment, you will have to deploy the solution again, migrate the DynamoDB data over manually and reconfigure bucket protection. It is essentially starting over, but you get to keep your historical data. This is painful, but possible. Please Contact Us if you'd like to talk through this.
Update January 2022
You can now add a load balancer to a deployment without one. And you can remove the load balancer after the fact as well. Simply run a Stack Update and add or remove the load balancer. This must be done manually through the CloudFormation Console.
Resources can only be renamed to match your particular naming scheme at the time of deployment. Changing these values later will result in errors.
Console and Agent Sizing - vCPU and Memory
The default values provided for both the Console and the Agents will meet the needs of the vast majority of customers. If any changes are made initially, you could up the Console settings if you plan to do large sets of Scheduled Scanning or regular assessment reports. You could also adjust the Agents down to 1vCPU and 3GB Memory for similar performance and about half the cost. All of the performance testing we have done has been with 2/4, but we saw very little impact on CPU overhead and memory that reducing both of those should work. Additional testing will come with different configurations in the near future.
If you do decide to make changes for either task, you must specify the proper vCPU:Memory ratio. The memory must be in the range of 2x to 8x of the vCPU.
vCPU = .5, then 1gb <= memValue <= 4gb vCPU = 1, then 3gb <= memValue <= 8gb vCPU = 2, then 4gb <= memValue <= 16gb vCPU = 3, then 6gb <= memValue <= 24gb vCPU = 4, then 8gb <= memValue <= 32gb
The new minimum memory requirement for agents is 3gb. We no longer offer a 2gb memory choice as we saw unstable scanning results with this value.
Agent vCPU and Memory recommendations have changed. We recommend to run the scanning agents with 1 vCPU and 3gb Memory. The defaults have been 2 vCPU and 4gb Memory, but we are finding there is no advantage to those settings at the moment. Switching to 1 and 3 is a worthwhile cost savings while having no impact on performance. In the Performance Throughput Table the ClamAV numbers were produced with 2vCPU and 4gb Mem, but the Sophos numbers were run with 1vCPU and 3gb Mem. In subsequent testing, we saw no noticeable reduction in performance with ClamAV as well.
If the memory specified for the Console and Agent are not within the proper range, you will see the following:
Access to KMS Encryption Keys
The default value is
Yes and this is our preference to simplify scanning for you. In order to scan objects in buckets with a KMS Key assigned, the Scanning Agent Role requires access to the key. This setting gives the scanner role access to all KMS keys, but with only the scope to use with Amazon S3. This is good for when new buckets come online and use new keys for the encryption or you change the keys on existing buckets. The scanning agent will automatically be ready to go without a hitch.
Alternatively, you can select
No and then manually, individually assign keys to the scanner role. Check this trouble shooting writeup for more information. This value can be changed post-deployment as well by following the trouble shooting link.
Auto Assign Public IPs - Console and Scanning Agents
The default value is
Enabled and this will assign public IPs to the Console and Scanning Agent tasks. For the scenarios where you will not be leveraging the public IP or do not want the IP assigned at all, you can switch these values (one or both) to
Disabled and we will not assign a public IP.
Agent Auto-Scaling Configuration
The default selections will set you up to have an scanning agent running 24x7x365. Depending on your workflow and your time-to-verdict requirements, you may not need a scanning agent running all the time. Changing the
Only Run Scanning Agents When Files are in Queue option to Yes will set the scanning agents to shut down completely when there is no work to be done. We call this feature Smart Scan This is great for cost efficiencies, but does slow down how quickly a file is scanned as it takes 60-90 seconds to spin a scanning agent up. Decide what your requirements are and then fill this section out.
When you switch
Only Run ... to Yes, you must change the
Minimum Number of Agents Per Region to 0.
Maximum Number of Running Agents Per Region can be set to any value. It can be left high or reduced down to some smaller number so you know exactly how many agents will spin up at any given time.
Number of Messages in Queue to Trigger Scaling is used to determine when the first or next scanning agent will spin up. If you are in Smart Scan mode then it will determine when the first agent spins up. Most leverage Smart Scan with a value of 1, but some like to let the work build up a bit before they spin a scanning agent up. Choose a number that makes sense for your organization.
Alternatively, you can create Scheduled Scans that allow you to only spin up scanning agents on a schedule basis to scan your objects (all or new).
Note: all of these settings can be changed after the fact from our Management Console Agent Settings page
Optional Load Balancer Configuration
By default, the stack will deploy without the use of a Load Balancer. Instead, we register your console IP with a subdomain tied to the
cloudstoragesecapp.com domain in a Route53 hosted zone inside a Cloud Storage Security account. There are times when this will not work for your use case. Two such scenarios are: you want to use your own DNS and SSL cert or you want to deploy in a private subnet behind a NAT. These are both great reasons to leverage a load balancer.
There are 4 fields you must provide values for in order to use a load balancer
Use a Load Balancer for the Console- change this to Yes
SSL Certificate ARN- specify the ARN to a certificate accessible in-region
Load Balancer Subnet A ID- specify the Subnet ID found in the AWS console
Load Balancer Subnet B ID- specify a different Subnet ID found in the AWS console
Load Balancer subnets and the Console subnets must be in the same Availability Zones (this is an AWS requirement). For example, the Console gets put in private-subnet1 in AZ-a and private-subnet2 in AZ-b, then the load balancer should be placed in public-subnet1 in AZ-a and public-subnet2 in AZ-b. If the subnets do not match availability zones the load balancer will not be able to communicate with the console
The remaining fields are optional. If you happen to be using Route53 for your domain, then you can leverage these fields to setup the DNS value (although this can be done after deployment sa well). If you are managing DNS in some other way, you can skip these fields and setup DNS how you normally would to point at the Load Balancer URL.
Register a subdomain on Route53- change this to Yes
Hosted Zone Name- specify the base domain value
Subdomain- specify the unique meaningful name to access the Antivirus for Amazon S3 application
Info Opt-Out- do not send any DNS info and version information to us
Optional Local Image Repository
If you have a requirement to host the repo for the container images, you can specify an
account number in this field to instruct the Console and Scanning Agents where to look to retrieve their images. Simply create the repos in your own account, name them as required (cloudstoragesecurity/console and cloudstoragesecurity/agent), download the images from our repos and place them into your own repo. When the console or scanning agents boot up they will look to your local repo for updates.
You are now responsible for ensuring your local repos get updated. The Antivirus for Amazon S3 solution will now only look to your repo for updates.
Optional AWS Resource Renaming
Do not change after the initial deployment. If you feel you need to rename any resources after deployment, please Contact Us. There are ways to rename most services, but you need to be aware of the impacts.
By default, we uniquely name each resource we deploy with
CloudStorageSec-. We often include the resource type (i.e. Queue) and append the unique application ID. When customers have a formal naming standard they would like to see our resource naming follow suite. Leveraging the fields in this section will allow you to rename all resources. Be aware, you should do this at the time of deployment. It can be done after, but it has to be done with care.
We are calling these "prefix naming" because we must still append the unique
appID to each resource.
Step 2 - Review CloudStorageSec-AV-for-S3¶
I acknowledge that AWS CloudFormation might create IAM resources with custom names. under
Create Stack at the bottom of the screen.
Step 3 - Trouble Shoot if needed¶
The stack you just created,
CloudStorageSec-AV-for-S3 will have a status of CREATE_IN_PROGRESS. Wait for this to change to CREATE_COMPLETE.
If the stack creation fails, click here to open the Trouble Shooting section.
Step 4 - Console Access¶
Once the stack creation completes, click the
Output tab. The top row in the table will have the URL for the Antivirus for Amazon S3 Management Console. It will be in the format of https://<accountID-appID>.cloudstoragesecapp.com.
It may take a few minutes for DNS to propagate. If you are unable to load the URL provided in the email (and/or CloudFormation Outputs) then wait a few minutes. If it continues to be an issue, check the console access trouble shooting to see if it is another issue.
If you chose to use a load balancer, the first field will be called
LBWebAddress instead of
You have completed creating the stack for Cloud Storage Security Antivirus for Amazon S3. Go to the How to Configure section to start setting up the anti-malware detection.