This article details the AWS CloudFormation building blocks to deploy a containerised application using the AWS Elastic Container Service (ECS). I use this method to deploy this vary website which was initially running in ECS using an on-demand instance deployed the old fashion way (with many mouse clicks and typing). With this CloudFormation template the entire stack can be created from a single command aws cloudformation create-stack…! and completely blown away and stood up again with minimal effort. The following borrows from an existing CloudFormation template you can find in the References at the bottom of the page.
My main goals when creating the template were to
- achieve infrastructure as code
- reduce the operational cost by using Spot instances
- ensuring the website can auto-heal when said Spot instances are terminated
- brew my coffee. OK, maybe not brew my coffee but with all that free time I have while CloudFormation does everything for me I can take care of that myself
At a high level, the following things are being created and configured by the Cloudformation template which I have broken up in the following text. For the complete template, jump to the bottom of the page.
- Create a VPC and basic networking namely an Internet Gateway, public subnet, and route table
- Create an Elastic IP and associate it to an Elastic Network Interface - this will facilitate reuse of the same public IP when the Spot instances terminate
- Create security groups for Web and SSH access and access from the EC2 instances to EFS
- Create an EFS and a mount point
- Create an ECS Task Definition which defines the required containers and their configurations - this includes mounting the EFS volume into the NGINX container so it can access the Let’s Encrypt files (see omission notes below)
- Create an ECS Cluster and Service
- Create an Auto Scaling Target and Auto Scaling Group which maintains 1 Spot instance for ECS
- Create a Launch Template which configures the Spot Instance/s
- Create the roles required for everything to work correctly
- Create an SNS topic for email notifications when events occur in the Auto Scaling Group
What’s omitted from the template.
- Some super-secret environment variables defined in the AWS Secrets Manager
- The initial S3 bucket which contains Let’s Encrypt configuration files used by the NGINX container that are seeded into EFS when the CloudFormation Template is run for the first time
Caveats.
- I intend only to maintain a single EC2 instance running at a time, and so the template and the way it works are configured as such
- I already had a working Let’s Encrypt configuration and certificates which I used to ‘pre-load’ the S3 seed bucket
For reference, the website has the following container components.
Standard Boiler Plate
The usual boilerplate data for any CloudFormation template.
1
2
|
AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation Template for jasonneurohr.com on ECS
|
Mappings
In the mappings collection is an ecsOptimizedAmi declaration which indicates what AMI’s should be used; specifically this has an ECS optimised AMI in the ap-southeast-2 region. The list of ECS optimised AMI’s can be found here.
1
2
3
4
|
Mappings:
ecsOptimizedAmi:
ap-southeast-2:
AMI: ami-0c7dea114481e059d
|
Outputs
In the outputs, collection are references to several objects, nothing fancy.
1
2
3
4
5
6
7
8
9
10
11
12
13
|
Outputs:
awsRegionName:
Description: The name of the AWS Region your template was launched in
Value: !Ref AWS::Region
websiteVpc:
Description: A reference to the created VPC
Value: !Ref websiteVpc
publicSubnet1:
Description: A reference to the public subnet in the 1st Availability Zone
Value: !Ref publicSubnet1
efsFs:
Description: Reference ID for the EFS
Value: !Ref efsFs
|
Parameters
The parameters collection defines several required parameters required to be either passed to CloudFormation or accept the default where specified. Of note are keyName which defines the EC2 keypair to use for the created instances and asgNotificationEp which is the email address that will receive Auto Scaling Group notifications.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
Parameters:
ecsClusterTargetCapacity:
Default: 1
Description: Number of EC2 Spot instances to initially launch in the ECS cluster
Type: Number
instanceType:
AllowedValues:
- t3.small
Default: t3.small
Description: EC2 instance type to use for ECS cluster
Type: String
keyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the EC2 instances
Type: AWS::EC2::KeyPair::KeyName
sourceCidr:
Default: 0.0.0.0/0
Description: Optional - CIDR/IP range for instance ssh access - defaults to 0.0.0.0/0
Type: String
environmentName:
Description: An environment name that will be prefixed to resource names
Type: String
Default: website-ecs
vpcCidr:
Description: Please enter the IP range (CIDR notation) for this VPC
Type: String
Default: 10.192.0.0/16
publicSubnet1Cidr:
Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
Type: String
Default: 10.192.10.0/24
asgNotificationEp:
Description: The email address to receive notifications for the Auto Scaling Group
Type: String
|
Resources
The resources collection is where the magic happens.
VPC and VPC Networking
The first block of YAML below does the following
- Creates a VPC containing
- Creates an Internet Gateway
- Creates a public subnet
- Creates a route table and default route
- Creates a CloudWatch Log Group
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
|
Resources:
websiteVpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref vpcCidr
EnableDnsHostnames: true
EnableDnsSupport: true
InstanceTenancy: default
Tags:
- Key: Name
Value: !Ref environmentName
internetGateway:
Type: AWS::EC2::InternetGateway
DependsOn:
- websiteVpc
Properties:
Tags:
- Key: Name
Value: !Ref environmentName
internetGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
DependsOn:
- websiteVpc
- internetGateway
Properties:
InternetGatewayId: !Ref internetGateway
VpcId: !Ref websiteVpc
publicSubnet1:
Type: AWS::EC2::Subnet
DependsOn:
- websiteVpc
Properties:
VpcId: !Ref websiteVpc
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref publicSubnet1Cidr
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${environmentName} Public Subnet (AZ1)
publicRouteTable:
Type: AWS::EC2::RouteTable
DependsOn:
- websiteVpc
Properties:
VpcId: !Ref websiteVpc
Tags:
- Key: Name
Value: !Sub ${environmentName} Public Route Table
defaultPublicRoute:
Type: AWS::EC2::Route
DependsOn:
- internetGatewayAttachment
Properties:
RouteTableId: !Ref publicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref internetGateway
publicSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref publicRouteTable
SubnetId: !Ref publicSubnet1
cloudWatchLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
RetentionInDays: 7
|
Elastic IP and Elastic Network Interface
The next portion of the resources collection creates an Elastic IP (EIP) and an Elastic Network Interface (ENI). These are used later in the LaunchTemplate which facilitates EC2 instances maintaining the same IP when they are replaced.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
elasticIp:
Type: AWS::EC2::EIP
DependsOn:
- internetGatewayAttachment
Properties:
Domain: vpc
elasticNetworkInterface:
Type: AWS::EC2::NetworkInterface
Properties:
Description: ENI for spot instance
SourceDestCheck: true
SubnetId: !Ref publicSubnet1
GroupSet:
- !Ref ecsInstanceSecurityGroup
Tags:
- Key: Name
Value: !Ref environmentName
elasticIpAssociation:
Type: AWS::EC2::EIPAssociation
DependsOn:
- elasticIp
- elasticNetworkInterface
Properties:
AllocationId: !GetAtt elasticIp.AllocationId
NetworkInterfaceId: !Ref elasticNetworkInterface
|
Security Groups
The next portion of the template creates the required security groups for access to the EC2 instance and access from EC2 to the Elastic File System (EFS) created later.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
|
ecsInstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- websiteVpc
Properties:
GroupName: ecs-access-from-internet
GroupDescription: Web and SSH from anywhere to ECS Auto Scaling Instances
SecurityGroupIngress:
- CidrIp: !Ref sourceCidr
FromPort: 22
IpProtocol: tcp
ToPort: 22
- CidrIp: !Ref sourceCidr
FromPort: 443
IpProtocol: tcp
ToPort: 443
- CidrIp: !Ref sourceCidr
FromPort: 80
IpProtocol: tcp
ToPort: 80
VpcId: !Ref websiteVpc
efsSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- websiteVpc
Properties:
GroupName: efs-access-from-ecs
GroupDescription: Security Group for EFS access from ECS
SecurityGroupIngress:
- IpProtocol: tcp
SourceSecurityGroupId: !Ref ecsInstanceSecurityGroup
FromPort: 2049
ToPort: 2049
VpcId: !Ref websiteVpc
|
Elastic File System
The Elastic File System (EFS) and EFS mount point are then created. The Let’s Encrypt configuration directory is stored in EFS and referenced in the Launch Template user data defined later such that if an EC2 instance gets terminated and replaced it can come back online without human intervention by being remounted at the host and attached to the NGINX container inside ECS.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
efsFs:
Type: AWS::EFS::FileSystem
Properties:
Encrypted: false
FileSystemTags:
- Key: Name
Value: !Ref environmentName
PerformanceMode: generalPurpose
ThroughputMode: bursting
efsMountPoint:
Type: AWS::EFS::MountTarget
DependsOn:
- efsFs
Properties:
FileSystemId: !Ref efsFs
SecurityGroups:
- !Ref efsSecurityGroup
SubnetId: !Ref publicSubnet1
|
Elastic Container Service
The next collection defines the Elastic Container Service (ECS) infrastructure, specifically
- the ECS Task Definition which includes the EFS volume to be mounted into the NGINX container
- the ECS Cluster
- the ECS Service
The ECS Service defines one desired instance of the Task Definition should be running. It also stipulates a MinimumHealthyPercent of 0 and a MaximumPercent of 100 which facilitates an Azure DevOps pipeline instructing ECS to force a new deployment of the task.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
|
ecsTaskDefinition:
Type: AWS::ECS::TaskDefinition
DependsOn:
- efsFs
- efsMountPoint
Properties:
ContainerDefinitions:
- Name: nginx
Hostname: nginx
Cpu: 128
Memory: 256
MemoryReservation: 256
Image: jasonneurohr/private:nginx
Essential: true
PortMappings:
- ContainerPort: 80
HostPort: 80
Protocol: tcp
- ContainerPort: 443
HostPort: 443
Protocol: tcp
Links:
- web:web
MountPoints:
- ContainerPath: /etc/letsencrypt
SourceVolume: nginx-letsencrypt
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
- Name: web
Hostname: web
Cpu: 256
Memory: 256
MemoryReservation: 256
Image: jasonneurohr/private:web
Essential: true
PortMappings:
- ContainerPort: 5000
HostPort: 5000
Protocol: tcp
Links:
- webapi:webapi
- contactsvc:contactsvc
- notifysvc:notifysvc
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
Environment:
- Name: ApiLocation
Value: http://webapi:55000
- Name: contactsvcEndpoint
Value: contactsvc:5001
- Name: notifysvcEndpoint
Value: notifysvc:5002
- Name: SendGridApiKey
Value: asdf
- Name: webapi
Hostname: webapi
Cpu: 128
Memory: 256
MemoryReservation: 256
Image: jasonneurohr/private:webapi
Essential: true
PortMappings:
- ContainerPort: 55000
HostPort: 55000
Protocol: tcp
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
Environment:
- Name: dbName
Value: asdf
- Name: dbServer
Value: asdf
- Name: dbUser
Value: asdf
Secrets:
- Name: dbPassword
ValueFrom: asdf
- Name: notifysvc
Hostname: notifysvc
Cpu: 128
Memory: 128
Image: jasonneurohr/private:notifysvc
Essential: true
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
- Name: contactsvc
Hostname: contactsvc
Cpu: 128
Memory: 128
Image: jasonneurohr/private:contactsvc
Essential: true
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
ExecutionRoleArn: arn:aws:iam::asdf:role/ecsTaskExecutionRole
Family: website-ecs
RequiresCompatibilities:
- EC2
Tags:
- Key: Name
Value: !Ref environmentName
Volumes:
- Name: nginx-letsencrypt
Host:
SourcePath: /mnt/efs
ecsCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: website-cluster
Tags:
- Key: Name
Value: !Ref environmentName
ecsService:
Type: AWS::ECS::Service
DependsOn:
- ecsAutoScalingGroup
Properties:
Cluster: !Ref ecsCluster
DeploymentConfiguration:
MaximumPercent: 100
MinimumHealthyPercent: 0
DesiredCount: 1
EnableECSManagedTags: true
LaunchType: EC2
PlacementStrategies:
- Field: attribute:ecs.availability-zone
Type: spread
- Field: instanceId
Type: spread
SchedulingStrategy: REPLICA
ServiceName: website-service
Tags:
- Key: Name
Value: !Ref environmentName
TaskDefinition: !Ref ecsTaskDefinition
|
ECS Auto Scaling
An ApplicationAutoScaling collection defines a MaxCapacity of one and a MinCapacity of zero in line with the service definition. This facilitates auto-scaling of the ECS service though only to one right now.
1
2
3
4
5
6
7
8
9
10
11
12
13
|
ecsAutoScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
DependsOn:
- autoScalingRole
- ecsCluster
- ecsService
Properties:
MaxCapacity: 1
MinCapacity: 0
ResourceId: !Join [ "/", [ "service", !Ref ecsCluster, !GetAtt ecsService.Name ] ]
RoleARN: !GetAtt autoScalingRole.Arn
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
|
Launch Template
The Launch Template collection is where the Spot price is defined and includes user data to configure several things. Of note is the use of the ENI created above and the last several lines which setup the EFS volume on the instance for use in the NGINX container.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
|
websiteLaunchTemplate:
Type: AWS::EC2::LaunchTemplate
DependsOn:
- elasticNetworkInterface
- instanceProfile
- ecsCluster
Properties:
LaunchTemplateData:
InstanceType: !Ref instanceType
KeyName: !Ref keyName
CreditSpecification:
CpuCredits: standard
ImageId:
Fn::FindInMap:
- ecsOptimizedAmi
- Ref: AWS::Region
- AMI
IamInstanceProfile:
Arn: !GetAtt instanceProfile.Arn
NetworkInterfaces:
- NetworkInterfaceId: !Ref elasticNetworkInterface
DeviceIndex: 0
InstanceMarketOptions:
MarketType: spot
SpotOptions:
InstanceInterruptionBehavior: terminate
MaxPrice: 0.01
SpotInstanceType: one-time
UserData:
Fn::Base64: !Sub |
#!/bin/bash
export PATH=/usr/local/bin:$PATH
yum -y --security update
yum -y install jq
easy_install pip
pip install awscli
aws configure set default.region ${AWS::Region}
echo ECS_CLUSTER=${ecsCluster} >> /etc/ecs/ecs.config
echo ECS_BACKEND_HOST= >> /etc/ecs/ecs.config
cat <<EOF > /tmp/awslogs.conf
[general]
state_file = /var/awslogs/state/agent-state
[/var/log/dmesg]
file = /var/log/dmesg
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/dmesg
initial_position = start_of_file
[/var/log/messages]
file = /var/log/messages
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/messages
datetime_format = %b %d %H:%M:%S
initial_position = start_of_file
[/var/log/docker]
file = /var/log/docker
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/docker
datetime_format = %Y-%m-%dT%H:%M:%S.%f
initial_position = start_of_file
[/var/log/ecs/ecs-init.log]
file = /var/log/ecs/ecs-init.log.*
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/ecs/ecs-init.log
datetime_format = %Y-%m-%dT%H:%M:%SZ
initial_position = start_of_file
[/var/log/ecs/ecs-agent.log]
file = /var/log/ecs/ecs-agent.log.*
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/ecs/ecs-agent.log
datetime_format = %Y-%m-%dT%H:%M:%SZ
initial_position = start_of_file
[/var/log/ecs/audit.log]
file = /var/log/ecs/audit.log.*
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/ecs/audit.log
datetime_format = %Y-%m-%dT%H:%M:%SZ
initial_position = start_of_file
EOF
cd /tmp && curl -sO https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py
python /tmp/awslogs-agent-setup.py -n -r ${AWS::Region} -c /tmp/awslogs.conf
cat <<EOF > /etc/init/cloudwatch-logs-start.conf
description "Configure and start CloudWatch Logs agent on Amazon ECS container instance"
author "Amazon Web Services"
start on started ecs
script
exec 2>>/var/log/cloudwatch-logs-start.log
set -x
until curl -s http://localhost:51678/v1/metadata; do sleep 1; done
ECS_CLUSTER=\$(curl -s http://localhost:51678/v1/metadata | jq .Cluster | tr -d \")
CONTAINER_INSTANCE=\$(curl -s http://localhost:51678/v1/metadata | jq .ContainerInstanceArn | tr -d \")
sed -i "s|%ECS_CLUSTER|\$ECS_CLUSTER|g" /var/awslogs/etc/awslogs.conf
sed -i "s|%CONTAINER_INSTANCE|\$CONTAINER_INSTANCE|g" /var/awslogs/etc/awslogs.conf
chkconfig awslogs on
service awslogs start
end script
EOF
cat <<EOF > /etc/init/spot-instance-termination-notice-handler.conf
description "Start spot instance termination handler monitoring script"
author "Amazon Web Services"
start on started ecs
script
echo \$\$ > /var/run/spot-instance-termination-notice-handler.pid
exec /usr/local/bin/spot-instance-termination-notice-handler.sh
end script
pre-start script
logger "[spot-instance-termination-notice-handler.sh]: spot instance termination notice handler started"
end script
EOF
cat <<EOF > /usr/local/bin/spot-instance-termination-notice-handler.sh
#!/bin/bash
while sleep 5; do
if [ -z \$(curl -Isf http://169.254.169.254/latest/meta-data/spot/termination-time)];
then
/bin/false
else
logger "[spot-instance-termination-notice-handler.sh]: spot instance termination notice detected"
STATUS=DRAINING
ECS_CLUSTER=\$(curl -s http://localhost:51678/v1/metadata | jq .Cluster | tr -d \")
CONTAINER_INSTANCE=\$(curl -s http://localhost:51678/v1/metadata | jq .ContainerInstanceArn | tr -d \")
logger "[spot-instance-termination-notice-handler.sh]: putting instance in state $STATUS"
logger "[spot-instance-termination-notice-handler.sh]: running: /bin/aws ecs update-container-instances-state --cluster \$ECS_CLUSTER --container-instances $CONTAINER_INSTANCE --status \$STATUS"
/bin/aws ecs update-container-instances-state --cluster $ECS_CLUSTER --container-instances \$CONTAINER_INSTANCE --status \$STATUS
logger "[spot-instance-termination-notice-handler.sh]: running: \"/bin/aws sns publish --topic-arn ${snsTopicForSpotInstanceMonitorScript} --message \"Spot instance termination notice detected. Details: cluster: \$ECS_CLUSTER, container_instance: \$CONTAINER_INSTANCE. Putting instance in state \$STATUS.\""
/bin/aws sns publish --topic-arn ${snsTopicForSpotInstanceMonitorScript} --message "Spot instance termination notice detected. Details: cluster: \$ECS_CLUSTER, container_instance: \$CONTAINER_INSTANCE. Putting instance in state \$STATUS."
logger "[spot-instance-termination-notice-handler.sh]: putting myself to sleep..."
sleep 120
fi
done
EOF
chmod +x /usr/local/bin/spot-instance-termination-notice-handler.sh
mkdir /mnt/efs
yum install -y amazon-efs-utils
cp /etc/fstab /etc/fstab.bak
echo "${efsFs}:/ /mnt/efs efs defaults,_netdev 0 0" | tee -a /etc/fstab
mount -a
# aws s3 cp s3://temp-jasonneurohr/letsencrypt /mnt/efs/ --recursive
|
Auto Scaling Group
The next collection defines the Auto Scaling Group, which maintains one EC2 instance and uses the above Launch Template. It also specifies the SNS Topic, which will receive notifications.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
ecsAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
DependsOn:
- efsMountPoint
- snsTopicForAutoScalingGroup
Properties:
AutoScalingGroupName: ecs-website
AvailabilityZones:
- !Select [ 0, !GetAZs '' ]
Cooldown: 300
DesiredCapacity: 1
HealthCheckGracePeriod: 0
HealthCheckType: EC2
LaunchTemplate:
LaunchTemplateId: !Ref websiteLaunchTemplate
Version: !GetAtt websiteLaunchTemplate.LatestVersionNumber
MaxSize: 1
MinSize: 1
NotificationConfigurations:
- TopicARN: !Ref snsTopicForAutoScalingGroup
NotificationTypes:
- autoscaling:EC2_INSTANCE_LAUNCH
- autoscaling:EC2_INSTANCE_LAUNCH_ERROR
- autoscaling:EC2_INSTANCE_TERMINATE
- autoscaling:EC2_INSTANCE_TERMINATE_ERROR
- autoscaling:TEST_NOTIFICATION
Tags:
- Key: Name
Value: !Ref environmentName
PropagateAtLaunch: true
TerminationPolicies:
- Default
|
Roles
Roles are setup were required for various actions in ECS, auto-scaling, CloudWatch, and SNS.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
|
autoScalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ["sts:AssumeRole"]
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- "application-autoscaling:*"
- "cloudwatch:DescribeAlarms"
- "cloudwatch:PutMetricAlarm"
- "ecs:DescribeServices"
- "ecs:UpdateService"
Resource: "*"
instanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Version: 2012-10-17
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- ecs:UpdateContainerInstancesState
Effect: Allow
Resource: '*'
Version: 2012-10-17
PolicyName: ecsUpdateContainerInstancesStatePolicy
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- logs:DescribeLogStreams
Effect: Allow
Resource: arn:aws:logs:*:*:*
Version: 2012-10-17
PolicyName: cloudWatchLogsPolicy
- PolicyName: ec2-read-temp-jasonneurohr-s3-bucket
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- s3:Get*
- s3:List*
Resource:
- arn:aws:s3:::temp-jasonneurohr
- arn:aws:s3:::temp-jasonneurohr/*
- PolicyDocument:
Statement:
- Action:
- sns:Publish
Effect: Allow
Resource: !Ref snsTopicForSpotInstanceMonitorScript
Version: 2012-10-17
PolicyName: snsPublishPolicy
instanceProfile:
Type: AWS::IAM::InstanceProfile
DependsOn:
- instanceRole
Properties:
Path: /
Roles:
- Ref: instanceRole
snsTopicForAutoScalingGroup:
Type: AWS::SNS::Topic
Properties:
DisplayName: ecs-website-autoscalingroup
Subscription:
- Endpoint: !Ref asgNotificationEp
Protocol: email
TopicName: ecs-website-autoscalingroup
|
Simple Notification Service
An SNS collection is created which will send Auto Scaling event notifications to the email address provided as a parameter to CloudFormation.
1
2
3
4
5
6
7
8
|
snsTopicForSpotInstanceMonitorScript:
Type: AWS::SNS::Topic
Properties:
DisplayName: ecs-website-spotinstancemonitor
Subscription:
- Endpoint: !Ref asgNotificationEp
Protocol: email
TopicName: ecs-website-spotinstancemonitor
|
The complete CloudFormation Template YAML is located below for reference.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
|
AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation Template for jasonneurohr.com on ECS
Mappings:
# https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html
ecsOptimizedAmi:
ap-southeast-2:
AMI: ami-0c7dea114481e059d
Outputs:
awsRegionName:
Description: The name of the AWS Region your template was launched in
Value: !Ref AWS::Region
websiteVpc:
Description: A reference to the created VPC
Value: !Ref websiteVpc
publicSubnet1:
Description: A reference to the public subnet in the 1st Availability Zone
Value: !Ref publicSubnet1
efsFs:
Description: Reference ID for the EFS stack
Value: !Ref efsFs
Parameters:
ecsClusterTargetCapacity:
Default: 1
Description: Number of EC2 Spot instances to initially launch in the ECS cluster
Type: Number
instanceType:
AllowedValues:
- t3.small
Default: t3.small
Description: EC2 instance type to use for ECS cluster
Type: String
keyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the EC2 instances
Type: AWS::EC2::KeyPair::KeyName
sourceCidr:
Default: 0.0.0.0/0
Description: Optional - CIDR/IP range for instance ssh access - defaults to 0.0.0.0/0
Type: String
environmentName:
Description: An environment name that will be prefixed to resource names
Type: String
Default: website-ecs
vpcCidr:
Description: Please enter the IP range (CIDR notation) for this VPC
Type: String
Default: 10.192.0.0/16
publicSubnet1Cidr:
Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
Type: String
Default: 10.192.10.0/24
asgNotificationEp:
Description: The email address to receive notifications for the Auto Scaling Group
Type: String
Resources:
websiteVpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref vpcCidr
EnableDnsHostnames: true
EnableDnsSupport: true
InstanceTenancy: default
Tags:
- Key: Name
Value: !Ref environmentName
internetGateway:
Type: AWS::EC2::InternetGateway
DependsOn:
- websiteVpc
Properties:
Tags:
- Key: Name
Value: !Ref environmentName
internetGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
DependsOn:
- websiteVpc
- internetGateway
Properties:
InternetGatewayId: !Ref internetGateway
VpcId: !Ref websiteVpc
publicSubnet1:
Type: AWS::EC2::Subnet
DependsOn:
- websiteVpc
Properties:
VpcId: !Ref websiteVpc
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref publicSubnet1Cidr
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${environmentName} Public Subnet (AZ1)
publicRouteTable:
Type: AWS::EC2::RouteTable
DependsOn:
- websiteVpc
Properties:
VpcId: !Ref websiteVpc
Tags:
- Key: Name
Value: !Sub ${environmentName} Public Route Table
defaultPublicRoute:
Type: AWS::EC2::Route
DependsOn:
- internetGatewayAttachment
Properties:
RouteTableId: !Ref publicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref internetGateway
publicSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref publicRouteTable
SubnetId: !Ref publicSubnet1
elasticIp:
Type: AWS::EC2::EIP
DependsOn:
- internetGatewayAttachment
Properties:
Domain: vpc
elasticNetworkInterface:
Type: AWS::EC2::NetworkInterface
Properties:
Description: ENI for spot instance
SourceDestCheck: true
SubnetId: !Ref publicSubnet1
GroupSet:
- !Ref ecsInstanceSecurityGroup
Tags:
- Key: Name
Value: !Ref environmentName
elasticIpAssociation:
Type: AWS::EC2::EIPAssociation
DependsOn:
- elasticIp
- elasticNetworkInterface
Properties:
AllocationId: !GetAtt elasticIp.AllocationId
NetworkInterfaceId: !Ref elasticNetworkInterface
ecsInstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- websiteVpc
Properties:
GroupName: ecs-access-from-internet
GroupDescription: Web and SSH from anywhere to ECS Auto Scaling Instances
SecurityGroupIngress:
- CidrIp: !Ref sourceCidr
FromPort: 22
IpProtocol: tcp
ToPort: 22
- CidrIp: !Ref sourceCidr
FromPort: 443
IpProtocol: tcp
ToPort: 443
- CidrIp: !Ref sourceCidr
FromPort: 80
IpProtocol: tcp
ToPort: 80
VpcId: !Ref websiteVpc
efsSecurityGroup:
Type: AWS::EC2::SecurityGroup
DependsOn:
- websiteVpc
Properties:
GroupName: efs-access-from-ecs
GroupDescription: Security Group for EFS access from ECS
SecurityGroupIngress:
- IpProtocol: tcp
SourceSecurityGroupId: !Ref ecsInstanceSecurityGroup
FromPort: 2049
ToPort: 2049
VpcId: !Ref websiteVpc
efsFs:
Type: AWS::EFS::FileSystem
Properties:
Encrypted: false
FileSystemTags:
- Key: Name
Value: !Ref environmentName
PerformanceMode: generalPurpose
ThroughputMode: bursting
efsMountPoint:
Type: AWS::EFS::MountTarget
DependsOn:
- efsFs
Properties:
FileSystemId: !Ref efsFs
SecurityGroups:
- !Ref efsSecurityGroup
SubnetId: !Ref publicSubnet1
ecsTaskDefinition:
Type: AWS::ECS::TaskDefinition
DependsOn:
- efsFs
- efsMountPoint
Properties:
ContainerDefinitions:
- Name: nginx
Hostname: nginx
Cpu: 128
Memory: 256
MemoryReservation: 256
Image: jasonneurohr/private:nginx
Essential: true
PortMappings:
- ContainerPort: 80
HostPort: 80
Protocol: tcp
- ContainerPort: 443
HostPort: 443
Protocol: tcp
Links:
- web:web
MountPoints:
- ContainerPath: /etc/letsencrypt
SourceVolume: nginx-letsencrypt
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
- Name: web
Hostname: web
Cpu: 256
Memory: 256
MemoryReservation: 256
Image: jasonneurohr/private:web
Essential: true
PortMappings:
- ContainerPort: 5000
HostPort: 5000
Protocol: tcp
Links:
- webapi:webapi
- contactsvc:contactsvc
- notifysvc:notifysvc
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
Environment:
- Name: ApiLocation
Value: http://webapi:55000
- Name: contactsvcEndpoint
Value: contactsvc:5001
- Name: notifysvcEndpoint
Value: notifysvc:5002
- Name: SendGridApiKey
Value: asdf
- Name: webapi
Hostname: webapi
Cpu: 128
Memory: 256
MemoryReservation: 256
Image: jasonneurohr/private:webapi
Essential: true
PortMappings:
- ContainerPort: 55000
HostPort: 55000
Protocol: tcp
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
Environment:
- Name: dbName
Value: asdf
- Name: dbServer
Value: asdf
- Name: dbUser
Value: asdf
Secrets:
- Name: dbPassword
ValueFrom: asdf
- Name: notifysvc
Hostname: notifysvc
Cpu: 128
Memory: 128
Image: jasonneurohr/private:notifysvc
Essential: true
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
- Name: contactsvc
Hostname: contactsvc
Cpu: 128
Memory: 128
Image: jasonneurohr/private:contactsvc
Essential: true
RepositoryCredentials:
CredentialsParameter: asdf
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: /ecs/website
awslogs-region: ap-southeast-2
awslogs-stream-prefix: ecs
ExecutionRoleArn: arn:aws:iam::asdf:role/ecsTaskExecutionRole
Family: website-ecs
RequiresCompatibilities:
- EC2
Tags:
- Key: Name
Value: !Ref environmentName
Volumes:
- Name: nginx-letsencrypt
Host:
SourcePath: /mnt/efs
ecsCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: website-cluster
Tags:
- Key: Name
Value: !Ref environmentName
ecsService:
Type: AWS::ECS::Service
DependsOn:
- ecsAutoScalingGroup
Properties:
Cluster: !Ref ecsCluster
DeploymentConfiguration:
MaximumPercent: 100
MinimumHealthyPercent: 0
DesiredCount: 1
EnableECSManagedTags: true
LaunchType: EC2
PlacementStrategies:
- Field: attribute:ecs.availability-zone
Type: spread
- Field: instanceId
Type: spread
SchedulingStrategy: REPLICA
ServiceName: website-service
Tags:
- Key: Name
Value: !Ref environmentName
TaskDefinition: !Ref ecsTaskDefinition
ecsAutoScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
DependsOn:
- autoScalingRole
- ecsCluster
- ecsService
Properties:
MaxCapacity: 1
MinCapacity: 0
ResourceId: !Join [ "/", [ "service", !Ref ecsCluster, !GetAtt ecsService.Name ] ]
RoleARN: !GetAtt autoScalingRole.Arn
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
cloudWatchLogsGroup:
Type: AWS::Logs::LogGroup
Properties:
RetentionInDays: 7
websiteLaunchTemplate:
Type: AWS::EC2::LaunchTemplate
DependsOn:
- elasticNetworkInterface
- instanceProfile
- ecsCluster
Properties:
LaunchTemplateData:
InstanceType: !Ref instanceType
KeyName: !Ref keyName
CreditSpecification:
CpuCredits: standard
ImageId:
Fn::FindInMap:
- ecsOptimizedAmi
- Ref: AWS::Region
- AMI
IamInstanceProfile:
Arn: !GetAtt instanceProfile.Arn
NetworkInterfaces:
- NetworkInterfaceId: !Ref elasticNetworkInterface
DeviceIndex: 0
InstanceMarketOptions:
MarketType: spot
SpotOptions:
InstanceInterruptionBehavior: terminate
MaxPrice: 0.01
SpotInstanceType: one-time
UserData:
Fn::Base64: !Sub |
#!/bin/bash
export PATH=/usr/local/bin:$PATH
yum -y --security update
yum -y install jq
easy_install pip
pip install awscli
aws configure set default.region ${AWS::Region}
echo ECS_CLUSTER=${ecsCluster} >> /etc/ecs/ecs.config
echo ECS_BACKEND_HOST= >> /etc/ecs/ecs.config
cat <<EOF > /tmp/awslogs.conf
[general]
state_file = /var/awslogs/state/agent-state
[/var/log/dmesg]
file = /var/log/dmesg
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/dmesg
initial_position = start_of_file
[/var/log/messages]
file = /var/log/messages
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/messages
datetime_format = %b %d %H:%M:%S
initial_position = start_of_file
[/var/log/docker]
file = /var/log/docker
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/docker
datetime_format = %Y-%m-%dT%H:%M:%S.%f
initial_position = start_of_file
[/var/log/ecs/ecs-init.log]
file = /var/log/ecs/ecs-init.log.*
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/ecs/ecs-init.log
datetime_format = %Y-%m-%dT%H:%M:%SZ
initial_position = start_of_file
[/var/log/ecs/ecs-agent.log]
file = /var/log/ecs/ecs-agent.log.*
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/ecs/ecs-agent.log
datetime_format = %Y-%m-%dT%H:%M:%SZ
initial_position = start_of_file
[/var/log/ecs/audit.log]
file = /var/log/ecs/audit.log.*
log_group_name = ${cloudWatchLogsGroup}
log_stream_name = %ECS_CLUSTER/%CONTAINER_INSTANCE/var/log/ecs/audit.log
datetime_format = %Y-%m-%dT%H:%M:%SZ
initial_position = start_of_file
EOF
cd /tmp && curl -sO https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py
python /tmp/awslogs-agent-setup.py -n -r ${AWS::Region} -c /tmp/awslogs.conf
cat <<EOF > /etc/init/cloudwatch-logs-start.conf
description "Configure and start CloudWatch Logs agent on Amazon ECS container instance"
author "Amazon Web Services"
start on started ecs
script
exec 2>>/var/log/cloudwatch-logs-start.log
set -x
until curl -s http://localhost:51678/v1/metadata; do sleep 1; done
ECS_CLUSTER=\$(curl -s http://localhost:51678/v1/metadata | jq .Cluster | tr -d \")
CONTAINER_INSTANCE=\$(curl -s http://localhost:51678/v1/metadata | jq .ContainerInstanceArn | tr -d \")
sed -i "s|%ECS_CLUSTER|\$ECS_CLUSTER|g" /var/awslogs/etc/awslogs.conf
sed -i "s|%CONTAINER_INSTANCE|\$CONTAINER_INSTANCE|g" /var/awslogs/etc/awslogs.conf
chkconfig awslogs on
service awslogs start
end script
EOF
cat <<EOF > /etc/init/spot-instance-termination-notice-handler.conf
description "Start spot instance termination handler monitoring script"
author "Amazon Web Services"
start on started ecs
script
echo \$\$ > /var/run/spot-instance-termination-notice-handler.pid
exec /usr/local/bin/spot-instance-termination-notice-handler.sh
end script
pre-start script
logger "[spot-instance-termination-notice-handler.sh]: spot instance termination notice handler started"
end script
EOF
cat <<EOF > /usr/local/bin/spot-instance-termination-notice-handler.sh
#!/bin/bash
while sleep 5; do
if [ -z \$(curl -Isf http://169.254.169.254/latest/meta-data/spot/termination-time)];
then
/bin/false
else
logger "[spot-instance-termination-notice-handler.sh]: spot instance termination notice detected"
STATUS=DRAINING
ECS_CLUSTER=\$(curl -s http://localhost:51678/v1/metadata | jq .Cluster | tr -d \")
CONTAINER_INSTANCE=\$(curl -s http://localhost:51678/v1/metadata | jq .ContainerInstanceArn | tr -d \")
logger "[spot-instance-termination-notice-handler.sh]: putting instance in state $STATUS"
logger "[spot-instance-termination-notice-handler.sh]: running: /bin/aws ecs update-container-instances-state --cluster \$ECS_CLUSTER --container-instances $CONTAINER_INSTANCE --status \$STATUS"
/bin/aws ecs update-container-instances-state --cluster $ECS_CLUSTER --container-instances \$CONTAINER_INSTANCE --status \$STATUS
logger "[spot-instance-termination-notice-handler.sh]: running: \"/bin/aws sns publish --topic-arn ${snsTopicForSpotInstanceMonitorScript} --message \"Spot instance termination notice detected. Details: cluster: \$ECS_CLUSTER, container_instance: \$CONTAINER_INSTANCE. Putting instance in state \$STATUS.\""
/bin/aws sns publish --topic-arn ${snsTopicForSpotInstanceMonitorScript} --message "Spot instance termination notice detected. Details: cluster: \$ECS_CLUSTER, container_instance: \$CONTAINER_INSTANCE. Putting instance in state \$STATUS."
logger "[spot-instance-termination-notice-handler.sh]: putting myself to sleep..."
sleep 120
fi
done
EOF
chmod +x /usr/local/bin/spot-instance-termination-notice-handler.sh
mkdir /mnt/efs
yum install -y amazon-efs-utils
cp /etc/fstab /etc/fstab.bak
echo "${efsFs}:/ /mnt/efs efs defaults,_netdev 0 0" | tee -a /etc/fstab
mount -a
# aws s3 cp s3://temp-jasonneurohr/letsencrypt /mnt/efs/ --recursive
# add to the bottom of the launch template user data for initial EFS seeing only
# aws s3 cp s3://temp-jasonneurohr/letsencrypt /mnt/efs/ --recursive
ecsAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
DependsOn:
- efsMountPoint
- snsTopicForAutoScalingGroup
Properties:
AutoScalingGroupName: ecs-website
AvailabilityZones:
- !Select [ 0, !GetAZs '' ]
Cooldown: 300
DesiredCapacity: 1
HealthCheckGracePeriod: 0
HealthCheckType: EC2
LaunchTemplate:
LaunchTemplateId: !Ref websiteLaunchTemplate
Version: !GetAtt websiteLaunchTemplate.LatestVersionNumber
MaxSize: 1
MinSize: 1
NotificationConfigurations:
- TopicARN: !Ref snsTopicForAutoScalingGroup
NotificationTypes:
- autoscaling:EC2_INSTANCE_LAUNCH
- autoscaling:EC2_INSTANCE_LAUNCH_ERROR
- autoscaling:EC2_INSTANCE_TERMINATE
- autoscaling:EC2_INSTANCE_TERMINATE_ERROR
- autoscaling:TEST_NOTIFICATION
Tags:
- Key: Name
Value: !Ref environmentName
PropagateAtLaunch: true
TerminationPolicies:
- Default
autoScalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ["sts:AssumeRole"]
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- "application-autoscaling:*"
- "cloudwatch:DescribeAlarms"
- "cloudwatch:PutMetricAlarm"
- "ecs:DescribeServices"
- "ecs:UpdateService"
Resource: "*"
instanceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Version: 2012-10-17
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- ecs:UpdateContainerInstancesState
Effect: Allow
Resource: '*'
Version: 2012-10-17
PolicyName: ecsUpdateContainerInstancesStatePolicy
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- logs:DescribeLogStreams
Effect: Allow
Resource: arn:aws:logs:*:*:*
Version: 2012-10-17
PolicyName: cloudWatchLogsPolicy
- PolicyName: ec2-read-temp-jasonneurohr-s3-bucket
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- s3:Get*
- s3:List*
Resource:
- arn:aws:s3:::temp-jasonneurohr
- arn:aws:s3:::temp-jasonneurohr/*
- PolicyDocument:
Statement:
- Action:
- sns:Publish
Effect: Allow
Resource: !Ref snsTopicForSpotInstanceMonitorScript
Version: 2012-10-17
PolicyName: snsPublishPolicy
instanceProfile:
Type: AWS::IAM::InstanceProfile
DependsOn:
- instanceRole
Properties:
Path: /
Roles:
- Ref: instanceRole
snsTopicForAutoScalingGroup:
Type: AWS::SNS::Topic
Properties:
DisplayName: ecs-website-autoscalingroup
Subscription:
- Endpoint: !Ref asgNotificationEp
Protocol: email
TopicName: ecs-website-autoscalingroup
snsTopicForSpotInstanceMonitorScript:
Type: AWS::SNS::Topic
Properties:
DisplayName: ecs-website-spotinstancemonitor
Subscription:
- Endpoint: !Ref asgNotificationEp
Protocol: email
TopicName: ecs-website-spotinstancemonitor
|
References