Introduction to AWS IaaS Solutions
Introduction to AWS
Today's systems administrators need to acquire and strengthen their skills on public cloud platforms. Amazon Web Services (AWS) began as an infrastructure to run the Amazon.com website and, as of this writing, has since grown to be the largest public cloud provider.
AWS provides a wide variety of service offerings from Infrastructure as a Service (IaaS) through to the application and Platform as a Service (PaaS) and nearly everything in between, which offers alternatives to running infrastructure on-premises.
AWS services are available on demand throughout the world, which makes it a compelling place to run infrastructure and applications. You might already have some familiarity with AWS, which is fine; this guide is geared toward folks who are early in their AWS journey or those looking to solidify their understanding of AWS IaaS solutions for compute, block storage, and networking.
AWS Command-Line Interface Installation
You will be using the AWS command-line interface (CLI) along with the AWS console for the examples in this guide. You can find CLI installation instructions online.
In this chapter, we begin our journey by exploring the AWS public cloud platform with a focus on the IaaS features. We cover general architectural features of the AWS cloud including geographic regions and availability zones. This will give you a comprehensive understanding of the basics needed to deploy your IaaS workloads on AWS.
A full glossary of AWS terms is available in the additional resources online.
Regions
AWS infrastructure is comprised of many services available in many areas of the world known as Regions. These Regions provide geographic availability with close proximity for low-latency access. AWS also provides the GovCloud region, which is a specialty region for government agencies and provides additional compliance and security requirements.
Each Region is located within a country's boundary to ensure protection by any regulatory requirement for geo-locality of workloads, data, and services. Some Regions might also require special access such as Asia Pacific (Osaka) due to country-specific regulations.
Edge connectivity is provided globally, which also gives servicefocused access to features like the Content Delivery Network (CDN), Domain Name System (DNS) using Route 53, Identity and Access Management (IAM), and others. This ensures that you and your customers have rapid access to the resources as well as geographic availability in the case of loss of access to a particular Region.
Regions are names identified by a two-letter country code (e.g., US, EU, CA, CN), a general location (e.g., East, West, Central), and a numeric marker; for example:
- US-East (North Virginia) Region: us-east-1
- US West (Oregon) Region: us-west-2
- EU (Ireland) Region: eu -west 1
- AWS GovCloud (US): us-gov-west-1
It is helpful to know the Region names and their programmatic short name when using the AWS CLI or other systems that deploy and manage AWS infrastructure. You will see references throughout this guide for the AWS CLI and links to more resources for other configuration management and Infrastructure as Code (IaC) tools (e.g., Terraform, RackN, Chef, Puppet, Ansible).
Availability Zones
Within each of the AWS Regions are physically separated and isolated datacenters known as Availability Zones (AZs) which you can see illustrated in Figure 1-1. Each AZ has high-speed and lowlatency networking within a Region and is described as being within a metropolitan distance (approximately 100 km) to provide low enough latency for replication of services and data while also providing protection against a significant business disruption such as power grid failure or some other localized outage.
Figure 1-1. Logical view of AZs within Regions
AWS does not publish the physical locations or proximity between datacenters or any technical specifications on the hardware environments. It can be possible to have a single AZ span more than one datacenter; however, there will not be two AZs sharing the same datacenter infrastructure.
Network Access for AWS Infrastructure
Administrative access to AWS is available anywhere using either the AWS console in a web browser or using the AWS CLI, which can each be used on a variety of devices. Network access for the applications and specific resources within your AWS environment is what you must design for to use the services that you create.
There are three methods that you can use to access your AWS infrastructure and services:
Internet
Open connectivity via direct network across the internet using an Internet Gateway within your AWS environment
Virtual Private Network (VPN)
Software or hardware VPN with an endpoint on-premises and continuous or on-demand tunnel access to your AWS environment using your own VPN devices
Direct Connect
Dedicated hardwire access to the AWS network which is available through networking and cloud service providers for highspeed, low-latency, and routed access directly to your AWS Region
These options are not mutually exclusive. Your requirements for access will vary from application to application and service to service. Lots of AWS services are used directly over the internet with public/private key, AWS credentials, and other access-level controls. This reduces the need for dedicated networks for many customers.
Design decisions around internet access, workload placement, and data locality are important because you might require subnet accessibility, internet gateways, IP routing, and other infrastructure configuration from Region to Region.
Direct Connect is ideal for organizations that want bidirectional direct access to network resources. You can use this for database replication services, disaster recovery, replication between onpremises datacenters and the AWS cloud, data warehouse accessibility, and much more.
Just imagine that you want to have your data on-premises but have applications running on AWS to access that data. Using direct network access, caching services, and other options now opens the door to exciting hybrid deployments for real-time applications.
Design Patterns for Availability with AWS
The best way to ensure availability is to take advantage of existing AWS services and its resilient infrastructure. Certain trade-offs must occur when you design for resiliency because of cost and performance. As we get further into this guide, we explore more of the service-specific deployment patterns.
Here are some key tips for designing for availability for core services:
Think globally, act locally
Just like the earth-friendly phrase goes, you should utilize services with global availability but be mindful of where your customers and users access the environment. Make use of CDNs, caching, and cross-Region services where possible for the best consumer experience.
Use multiple AZs
They are called "Availability Zones" for a reason. Utilize more than one AZ within your Regions for safety. Designing your network strategy must include this or else you might bump into network addressing challenges as you try to expand later.
Cross-region deployments
For broad availability, use services that can span Regions as well as the AZs within them. Treat a Region like you would a Metropolitan Area Network and build applications to be able to be run and recovered across Regions.
Back up your data and configuration
Cloud services can be distributed and have high availability, but that does not ensure the backup of resources. Backups are needed for either point-in-time recovery or complete loss recovery after a significant disruption. Even replication will replicate errors and data loss, leaving your team with only backups and snapshots as recovery options.
You will find there are fewer limitations on architecture than there are on your budget. Resiliency is available at nearly every layer of the stack provided that you can budget for it. The value of ondemand infrastructure is that you can scale as needed and design for these burst patterns.
You must design your networking strategy in advance of these bursts and expansions. Early decisions about network addressing within and across your AZs and Regions can affect growth and expansion.
AWS services have quotas and some upper-bound technical limits. For example, you can have no more than five Virtual Private Clouds (VPCs) per Region by default. You can order increases, which also increases dependencies, such as an increase to the number of inter‐net gateways per Region. Hard limit examples include 500 security groups per VPC, and 16 security groups per network interface. All service limits and quotas are available in the AWS documentation.
Conclusion
Now that you have a good understanding of the general AWS infrastructure, we now move on to the AWS VPC environment. This is the basis of access control, networking, and network security for the AWS IaaS platform where you will be launching your compute and storage resources.
Basic Networking and Security with Amazon Web Services Virtual Private Cloud
In this chapter, we explore some of the foundational features of the Amazon Web Services (AWS) Virtual Private Cloud (VPC) platform. You will learn about networking and security options and see practical use with an example cloud instance configuration. These are important to understand when bringing your cloud workloads online. There is the potential to expose data and services to the public-facing internet, which also opens the door to vulnerability and attack.
Understanding VPC features and how to configure one step-by-step is important if you are studying for the AWS Solutions Architect Associate exam. It is important in general for your AWS product knowledge, but as many exam-related resources indicate, VPC knowledge might be heavily featured in the certification process.
What Is VPC?
VPC is the logical construct that gives your workloads a common networking and security boundary. Every AWS Region you launch a workload into has a default VPC to allow for immediate use of compute and storage resources without the need to set up a specific VPC.
Creating a VPC is free. The only costs that come up are when you create a Network Address Translation (NAT) gateway or deploy resources within the VPC. There is also an option to create a gateway-type VPC endpoint, which uses internal networking to reach other AWS resources (e.g., Amazon Simple Storage Service [Amazon S3] object storage and Amazon Relational Database Service [RDS] database services). This is handy as you grow your AWS usage across services and want to avoid accessing other AWS-hosted data over the public internet, which incurs networking fees.
Features and capabilities for VPC include the following:
Public subnets
The ability to create one or more public-facing subnets for internet access using an internet gateway for both.
Private subnets
The ability to create one or more private subnets for internal communications between VPC resources using both.
VPC Peering
The option to connect multiple VPCs as a single routed private network, which you can do between VPCs in a Region, across Regions, and even between AWS accounts.
It is recommended to set up your resources within a VPC to pool together the active environment and the supporting networking and security configuration. You can create all of your access rules, networking policies, and give out access in granular ways to the VPC and below with the Identity and Access Management (IAM) service. We implement our example here using root or a full administrative access profile.
Tagging is also your friend for many reasons. Tagging every resource and workload to identify it (production, test, dev, web server, db server, owner, etc.) makes other administrative processes easier in future. You can tag nearly every object in AWS.
Core Networking and Security on AWS
VPC networking and security are a fundamental part of your AWS Infrastructure as a Service (IaaS) design and day-to-day operations. Networking and security are paired up here because they often share the common goal of securing and isolating network access to your AWS resources. The VPC construct is a way to have a set of access rules and logical networks that can be shared by multiple AWS resources. We cover an example application that uses compute, storage, and networking to illustrate a real use case.
Access to your resources can be protected by Secure Shell (SSH) key pairs, which we use for the example applications. Be sure to set up your first key pair in the us-west-2 region if you would like to follow along with the examples.
Instructions are available in the AWS user guide for creating and uploading your key pair.
VPC Subnets
You have four choices when creating your VPC:
VPC with single public subnet
Defaults with /16 network and a /24 subnet using Elastic IP for public IP addressing. This is good for general purpose, publicfacing web servers and for simplified deployments.
VPC with public and private subnets
Defaults with /16 network and two /24 subnets using Elastic IP on public and NAT for private IP addressing. This is great for adding application servers or other backend resources that can be on private subnets with access to frontend servers on the private VPC network.
VPC with public and private subnets and hardware VPN access
Same public/private as the aforementioned public and private subnets with IPsec VPN tunnel from the private subnets to your corporate network. Now you can extend your on-premises environment to share data and application access across a hybrid environment.
VPC with a private subnet only and hardware VPN access
No public subnets, with an IPsec VPN tunnel to your corporate network. This is a great way to use burstable resources on AWS that are able to access your on-premises environment (e.g., batch processing and AI applications).
Note that in each of these scenarios, there are usage charges for both the VPN and NAT on VPC. Our example deployment will use a VPC with single public subnet, as shown in Figure 2-1, because they are just getting started with their AWS usage and have no need for VPN or private subnets.
Figure 2-1. Example VPC with single public subnet
If you choose to use private subnets for your architecture, you need NAT access to those private subnets, which you can do with an Elastic IP (must be preallocated before VPC creation) or by using an EC2-based NAT instance. You choose this during the VPC wizard or you can configure it manually if you build your VPC from scratch or after initial deployment at any time.
Your choice of subnetting is one to spend some time on. Especially when it comes to your subnet allocation. Even with the use of private IP subnets (10.0.0.0/8, 172.16.0.0/16, 192.168.0.0/24), you are likely to find yourself running into colliding IP address ranges if you are not careful. Each private subnet in an Availability Zone (AZ) will require you to select from the parent range created for your VPC.
Here's an example for IPv4 subnet of a VPC with one public subnet and two private subnets: VPC IPv4 CIDR Block
10.0.0.0/16 (65,531 available IP addresses) IPv4 public subnet CIDR Block
10.0.0.0/24 (251 available IP addresses) IPv4 private subnet one CIDR Block
10.0.1.0/24 (251 available IP addresses)
IPv4 private subnet two CIDR Block
10.0.2.0/24 (251 available IP addresses)
Notice that we have given private IP addresses to the "public" subnet because these are the private interface access addresses used for inter-instance and intra-VPC communication. As instances are brought online, you can also assign them a public IP address given by AWS.
Instances are created with private IP addresses automatically. You also can opt to have a public IP address and AWS-assigned Domain Name System (DNS) entry on an interface at launch. This public IP address is not persistent and can change when the instance restarts. We look at Elastic IP options further into the chapter, which helps with this issue of nonpersistent IP addresses.
Security Groups
VPC Security Groups are stateful policies that allow inbound and outbound network traffic to your EC2 instances. You can apply Security Groups to your EC2 instances and modify them in real time at any time. Inbound rules are defined by port/protocol plus the source network and a description, as shown in Figure 2-2. Notice the Source can be custom, anywhere, or the "My IP" option which detects your IP from the browser and assigns it to the rule.
Figure 2-2. Inbound rules on a Security Group
Use the "Anywhere" option as a target/source carefully. It is ideal to use as narrow a network range as possible when creating rules, as well, such as specifying RDP from a particular IP address with a /32 for the CIDR subnet (e.g., 204.195.21.134/32). Granular network access can help in reducing the exposure and risk for your AWS workloads.
You can assign multiple Security Groups to each instance. This is helpful if you have a global rule for allowing SSH or Internet Control Message Protocol (ICMP) that you want to have on the entire VPC along with specific instance-related rules. Figure 2-3 shows how you can use the AWS console to attach multiple Security Groups to your instances.
Figure 2-3. Choosing multiple Security Groups
The most permissive of the cumulative rules applies when multiple Security Groups or rules are applied to an instance.
Example: one rule for inbound SSH from only one IP address and another inbound SSH rule from Anywhere will result in allowing SSH from Anywhere. Each EC2 instance has the cumulative inbound rules visible from the Description tab, as shown in Figure 2-4, along with which Security Group the rule comes from.
Figure 2-4. View inbound rules and Security Groups
Elastic IPs
You can assign a persistent public IP address to your EC2 instance by using an Elastic IP (EIP). This is free for each provisioned instance or charged at 0.01$ per hour for disassociated EIPs. Even an instance with a public IP address assigned at launch is a dynamic and ephemeral IP. Choosing an EIP means that you will have it consistently for your account and can create a DNS association for that resource now.
You are allowed to map only one EIP to a network interface. You can assign multiple network interfaces to a single instance as a way to allow multiple EIPs to be used. Examples where this can be used is a web server with multiple sites, each assigned to a specific IP address and DNS name.
The default quota of EIPs for each account is five per region. This is important to know so that you can request increases if needed to meet your EIP requirements.
AWS CLI Command Basics
You will notice consistent patterns with the AWS CLI for command structure. This is how it breaks down:
aws [options] <command> <subcommand> [parameters]
Example: give a list of EC2 instances:
aws ec2 describe-instances
Example: create a VPC routing table:
aws ec2 create-route-table --vpc-id vpc-006960a6c4d805f10
Commands use verbs, which include create, describe, associate, attach, delete, detach, import, modify, and others. The full CLI structure and command reference is available online in the AWS documentation reference page.
Deployment Example: Web Application
For this example, your customer is the Utility Muffin Research
Kitchen (UMRK) company, which needs a basic website to display its supply catalog at http://supplies.utilitymuffinresearchkitchen.com. The company will use this website to run its custom web application code that is built for a web server that is certified on Amazon Linux.
UMRK uses an on-premises local balancer to route traffic to the web servers and will be using two web servers in AWS to distribute the load. It will want to keep all information together in a geographic region while ensuring availability by spreading the servers across two AZs, as shown in Figure 2-5.
The UMRK operations team needs HTTP and SSH access to each instance to be able to display the website and to manage the web server configuration and code. UMRK is located in Blue Ash, Ohio, with their primary distributors in Cincinnati, so they will choose to deploy into the US (Ohio) Region (us-east-2).
Figure 2-5. UMRK's architecture diagram
Deploying the UMRK VPC
Let's walk through the setup process here for configuring the UMRK VPC based on the requirements that we just defined:
- VPC will be in us-east-2 (Ohio)
- SSH key must be uploaded in advance
- Two IPv4 public subnets will be used for resiliency
To begin, go to the VPC service in your AWS web console, which brings you to the default page and features a prominent Launch VPC Wizard button similar to that shown in Figure 2-6. Click the button to get started with your new VPC.
Figure 2-6. Launch the Create VPC Wizard
The four options in Figure 2-7 show the various configurations for a VPC that we discussed earlier in this chapter; for this example, choose Single Public Subnet.
Figure 2-7. Choose the Single Public Subnet option
The UMRK requirements are for a limited number of workloads for now. A good practice is to name your resources (e.g., subnets) with meaningful and readable names. Lots of areas within the AWS console and AWS CLI will include the description and name, so this makes future administration a little easier. Figure 2-8 shows the wizard filled in for the UMRK example using a 10.0.0.0/16 VPC network and a 10.0.0.0/24 public subnet allocation.
It is also ideal to set the Availability Zone manually. This way you know precisely where the first AZ is and which additional AZs to use if you want more resiliency. AWS automatically creates a Fully Qualified Domain Name (FQDN) for your resource, which will display after it's created.
Figure 2-8. Entering your UMRK VPC details
At this point you have your VPC created with a public subnet.
Choose the Subnets option in the VPC console and follow along with Figure 2-9 to add the second subnet with another /24 IP range within the 10.0.0.0/16, which is the network range for the VPC. Let's use 10.0.3.0/24 and call this umrk-public-2c to make it easy to know which subnet it's on.
Figure 2-9. Create the second public subnet
Now you have your VPC created with two public subnets. Notice that the networking defaults to /16 and /24, which might work for your environment. There are no security groups in the new VPC by design. You must create new ones directly or when you create your EC2 instances.
Using AWS CLI to Create a VPC
The CLI process is a bit more involved than some of the other examples. This gives you an idea of all that is happening when you use the VPC wizard.
The example commands here are using sample IDs, which you will need to replace with those for your environment.
Following this sequence of CLI commands will create a VPC:
Create your VPC aws ec2 create-vpc --cidr-block 10.0.0.0/16
Create your first subnet
aws ec2 create-subnet --vpc-id vpc-006960a6c4d805f10 --cidr-block 10.0.0.0/24 --availability-zone us-east-2a
Create your second subnet
aws ec2 create-subnet --vpc-id vpc-006960a6c4d805f10 --cidr-block 10.0.3.0/24 --availability-zone us-east-2c
Create your internet gateway aws ec2 create-internet-gateway
Attach your internet gateway
aws ec2 attach-internet-gateway --internet-gateway-id igw-04a86ecc70949724d --vpc-id vpc-006960a6c4d805f10
Create a VPC routing table
aws ec2 create-route-table --vpc-id vpc-006960a6c4d805f10
Create a default route to move traffic to the internet gateway
aws ec2 create-route
--route-table-id rtb-01b04201fe9909f9b
--destination-cidr-block 0.0.0.0/0
--gateway-id igw-04a86ecc70949724d
Add a route to your first subnet
aws ec2 associate-route-table
--subnet-id subnet-01789b9231f45eb62
--route-table rtb-01b04201fe9909f9b
Add a route to your second subnet
aws ec2 associate-route-table
--subnet-id subnet-03c4b8c88971c3097
--route-table rtb-01b04201fe9909f9b
Keep track of the IDs that are generated from the output of each command because they are needed for commands further into the process. Output from each command is in JSON format. You can also pull the data from the AWS console if needed.
Generating Elastic IPs
The last step you have in your UMRK configuration for VPC is to generate two EIPs to ensure a persistent address. In the VPC console, choose the Elastic IPs sidebar menu option and then use the Allocate New Address button to open the dialog box shown in Figure 2-10.
Figure 2-10. Allocating a new Elastic IP address in your VPC
The AWS CLI command to allocate your IP address is a simple oneliner:
aws ec2 allocate-address --domain "vpc" --region us-east-2
Allocate your two EIPs for the example applications and then you are ready for launching your UMRK instances in the next chapter. You will recall from earlier that there is a quota limit of five EIPs per Region per customer by default which might trigger the need for you to request an increase.
Design Patterns for Availability with AWS VPC
Designing with operational patterns in at the outset ensures a greater chance of success for both security and networking in your VPC. Here are some important tips to follow:
Security taxonomy is important
A good use of global, instance-specific, and shared Security Groups helps to reduce repetitive rule creation and confusion. You should regularly revisit your Security Groups to determine whether they match your operational model.
Audit your Security Groups regularly
With multiple rules in place and the most permissive of them applying, it is easy to lose track of what is actually allowed or blocked as your environment scales. It is a good practice to regularly audit, review, and test your access rules.
Subnetting matters
Network assignment in your VPC should be done with some planning. The allocation must align from the top-level subnet through to the AZs to allow for enough addresses for all potential instances. This might also include working with NAT and VPN back to your private infrastructure, which can create routing challenges.
Conclusion
This exercise took us through creating a new VPC using the built-in wizard, creating a new subnet, and learning how Security Groups will apply to a VPC. Now that you have taken in a lot of information about the AWS VPC configuration, it's time to move on to adding compute resources into your newly launched VPC.
Amazon Web Services Elastic Compute Cloud
In this chapter, we examine the Amazon Web Services (AWS) Elastic Compute Cloud (EC2) using practical examples of deploying an EC2 instance, taking snapshots, creating images, and operational tasks through both the web console and the AWS command-line interface (CLI). These are important tasks that will be familiar to what you might have done in a virtualization environment.
EC2 Fundamentals
Working with Infrastructure as a Service (IaaS) implementations on AWS means using EC2 for your compute platform. This is where you build virtual machines (VMs) that can run nearly any operating system (OS). EC2 runs on shared underlying infrastructure by default but with strict security boundaries protecting resources from reaching one another between customers.
You do have the option to run dedicated instances that are allocated onto physical hosts, which are only provisioned with your EC2 workloads. Another option is to choose dedicated hosts. This lets you have more control at the physical layer and also lets you run licensed applications that have limitations around binding to specific CPUs. You might find this on-premises today if you run Microsoft Windows Server, Microsoft SQL Server, Oracle SQL Server, or other similarly licensed software.
You choose EC2 instance configuration and size based on family type (e.g., T3, C5, R5, H1) and then by what you might call "t-shirt size" (e.g., small, medium, large, xlarge, and 2xlarge). The instance type is not just about size, but also about the capabilities that it presents and, of course, the price per hour.
Scaling up your instances within the family type adjusts the virtual memory and virtual CPU allocated to the instance. It is important to note that changes to the instance size or family will require a restart of the instance to apply the new configuration.
General purpose (T3, T2, M5, M4)
Various types in the family for general or "bursty" workloads that might not map to another instance type.
Compute optimized (C5, C4)
Designed for more compute-centric workloads.
Memory optimized (R5, R4, X1e, X1, z1d)
Higher-speed access to memory and higher memory to CPU ratio.
Accelerated computing (P3, P2, G3, F1)
GPU-accessible option for workloads that can make use of the enhanced parallel processing power.
Storage optimized (H1, I3, D2)
Low-latency and high-speed access to the storage subsystem for read/write–intensive workloads.
Reserved Instances
Reserved instances are available to buy as a precommit for one or three years with deep discounts (63% at three-year commitment) off of the hourly on-demand pricing. This is ideal for instances that will be online for a long period. You can pay fully upfront (largest discount), partial upfront (moderate discount), or no upfront (lowest discount) for an instance type.
Reserved purchases are made for a specific instance type in a specific Region, and the reserved discount is applied to active instances that match the size and location. This introduces the challenge of matching your active workloads to the best possible discounts.
Understanding Amazon Machine Images
You can save instances as an Amazon Machine Image (AMI) or launch one from there. This is helpful to speed the creation of your EC2 instances from either the AWS Marketplace or your own library of images.
AMIs are available from a catalog of default images that are either AWS Marketplace AMIs, Community AMIs, or from your own catalog of AMIs, which you can create from your EC2 instances. Keeping your instances as an AMI is a handy way to clone as templates, which makes it easy to launch multiple versions with prebuilt configuration and software installed.
A great example would be when you configure your application server to have specific libraries, security lockdowns, log export configuration, and some custom code that your dev team needs. Creating an AMI takes the fully configured live EC2 instance and makes it your base image, which you can use to launch other EC2 instances. It is similar to using VM templates in a virtualization stack.
Example: Deploying the UMRK Web Servers
The UMRK team needs two web servers, each deployed into a separate Availability Zone (AZ) within the same Region. Your task also gives you a chance to launch an EC2 instance running Amazon Linux. The Amazon Linux is a lightweight, secure, and versatile Linux derivative built and maintained by the AWS team, and it has many operational similarities to CentOS.
You start in the AWS EC2 console by launching a new instance through the wizard, as depicted in Figure 3-1, which takes you through the steps.
Figure 3-1. Start the EC2 wizard by choosing an image
There are a few details to choose from in the third step of the wizard, which include those highlighted via arrows in Figure 3-2. You must choose to assign a network that is associated to your Virtual Private Cloud (VPC) to prevent ending up in the default VPC. Your subnet will be chosen from the ones you created in Chapter 2. This example for UMRK disables the public IP in favor of using an Elastic IP (EIP).
Figure 3-2. Choose the instance details
IAM roles allow for service-to-service and more granular administrative access to resources. We are not covering IAM roles in this guide, so for this example, we are not using an IAM role.
It's a good idea to select the "Protect against accidental termination" checkbox to reduce the risk of accidentally terminating and losing your instance. You can disable termination protection later in the EC2 instance details.
Set "Shutdown behavior" for the UMRK example to Stop rather than Terminate to ensure that we don't kill the instance if it is stopped.
The requirements for UMRK include application code, which the developers will want on persistent storage. Figure 3-3 shows choosing a 20 GB volume for the example (default is 8 GB). You should also choose the General Purpose (SSD) option for Volume Type. More detail on the volume types is available in the next chapter.
Figure 3-3. Assigning a 20 GB root volume
Tagging resources helps to identify them visually and programmatically. This helps for things like grouping, searching, chargeback/ showback ownership, and also helps for many third-party products that you might use with AWS that use tags. Figure 3-4 shows three tags that assign an owner, an application, and an environment label on both the EC2 instance and the associated volume.
Figure 3-4. Assigning your EC2 instance tags
Every instance creation wizard will default to creating a generically named new rule. You should assign a meaningful name similar to what Figure 3-5 illustrates. You also can add existing Security Groups, which is what you will use in the UMRK example when creating your second instance.
Figure 3-5. Configuring the UMRK security group rules
The following CLI examples show you how to deploy an EC2 instance using the AWS CLI to perform the same task, which requires only the AMI ID of the image from which to launch. You will use the ami-0b59bfac6be064b78 from the same example used in the AWS Console earlier in Chapter 2. You also need to know the name of the SSH key you have uploaded to the Region to which you are deploying.
Get your SSH key name aws ec2 describe-key-pairs
Create Security Group
aws ec2 create-security-group --description "UMRK Blue Ash" --group-name "UMRK HTTP and
SSH Group" --vpc-id vpc-006960a6c4d805f10
Create a rule to allow HTTP on TCP
aws ec2 authorize-security-group-ingress --group-id sg-01a304e61f20427d9 --protocol tcp --port 80 --cidr 0.0.0.0/0
Create a rule to allow HTTPS on TCP
aws ec2 authorize-security-group-ingress --group-id sg-01a304e61f20427d9 --protocol tcp --port 443 --cidr 0.0.0.0/0
Create a rule to allow SSH from a single IP address
aws ec2 authorize-security-group-ingress
--group-id sg-01a304e61f20427d9 --protocol tcp
--port 22 --cidr 71.125.28.141/32
Launch EC2 instance
aws ec2 run-instances --image-id ami-0b59bfac6be064b78
--subnet-id subnet-0b37218d7745e9b86
--key-name discoposse-macbook
--instance-type t2.micro
--security-group-ids sg-03b44aa884493f3f8
--block-device-mappings
DeviceName=/dev/sdh,Ebs={VolumeSize=20}
--tag-specifications 'ResourceType=instance,
Tags=[{Key=owner,Value=WebOps},
{Key=application,Value="umrk web"},
{Key=environment,Value=production}]'
That takes care of the launch of the EC2 instance, attaching the Security Group, and creating the 20 GB root volume.
Creating the Second UMRK EC2 Instance
You create the second instance similarly except on the second subnet that you created when configuring the VPC, and you use the existing Security Group to assign the access rules instead of creating a separate one. The AWS CLI command will require a different alternate subnet ID, as well.
Associating Your Elastic IP Addresses
The final step for enabling public access on a persistent IP address is to associate your EIP from the VPC console. Figure 3-6 shows the association wizard which maps the available EIP to an instance and assigns it to a network interface.
Use the Reassociation checkbox if you want to force the EIP to be associated to this instance even if it is already associated to another resource. In this case, it's good to leave this unchecked to make sure the EIP is free to be used.
Figure 3-6. Associating an available EIP to an EC2 interface
The AWS CLI command for this step requires the allocation-id from your EIP, the instance ID from your EC2 instance, and the private IP address that you are going to assign the EIP to on your internal network, plus your Region. Here is an example:
aws associate-address
--allocation-id "eipalloc-06ecee9d2670d11e1"
--instance-id "i-01f0026ea5396f197"
--no-allow-reassociation --private-ip-address "10.0.3.179"
--region us-east-2
Conclusion
Now that you have learned about EC2 compute and the basics of spinning up your cloud instances within a VPC, you are ready to learn about how persistent block storage is provided on AWS.
Amazon Web Services Elastic Block Storage
This chapter explores the Amazon Web Services (AWS) Elastic Block Storage (EBS) platform. You will learn which types of storage are available, compare the cost and performance impact with EBS storage types, and look at practical examples of operational tasks with EBS using both the web console and the AWS command-line interface (CLI).
Block storage for EC2 instances and other systems needing block volumes will be provided by EBS. You can attach these volumes to instances, as well as clone, archive, and even detach and reattach them to other machines. Block storage is needed in many cases for Infrastructure as a Service (IaaS) instances. It's important that you understand the costs and performance impact of choices for EBS tiers.
Storage Tiers/Types in EBS
EBS volumes are all scalable to 16 TB at a maximum, and each type comes in four storage performance tiers, which vary in features and functionality. Choosing which type is important because this affects the cost, performance, and scaling of the storage, plus it must match the EC2 instance type to ensure that you are getting the most out of your storage.
gp2
EBS General Purpose SSD—default when launching EC2 instances—good for boot volumes, general mid-performance volumes, and active data volumes.
io1
Provisioned IOPS SSD—guaranteed input/output operations per second (IOPS) with a maximum of 32 K IOPS—good for database or high-volume data storage.
st1
Throughput Optimized HDD—high throughput with maximum 500 IOPS—good for lower-access frequencies, but larger files are good. Can be used for some boot volumes on lowerdemand instances.
sc1
Cold HDD—"slow and low" with a maximum of 250 IOPS— good for archival data that must be on block storage.
You can make storage tier changes after initial deployment. This is important because you might find that your compute workloads require different performance and throughput over time. Making the initial move to high-performance storage is a pricey way to avoid the problem of changing storage tiers down the road.
EBS storage is charged at an hourly rate per gigabyte per month for the allocated amount. This is sometimes confusing because many cloud resources are thought to be pay-for-what-you-use when it is actually pay-for-what-you-allocate. A 200 GB EBS volume with 5 GB of data will be charged as 200 GB per month for the Region in which it is located.
Understanding EBS Snapshots
EBS has the ability to create point-in-time incremental snapshots of an existing EBS volume. These snapshots are stored only as the incremental changes (see Figure 4-1) since the last snapshot was taken, which helps reduces the cost and size. You can spawn another volume or instance using a snapshot, which creates a fully inflated live storage volume from the cumulative snapshots.
Figure 4-1. Snapshot example of an EBS Volume
Taking a snapshot is done easily from the EBS view, as shown in Figure 4-2, by selecting a volume and using the Create Snapshot action. You are prompted for a snapshot description and then can view active snapshots in the EBS view under snapshots.
Figure 4-2. Creating an EBS snapshot in the AWS console
The associated AWS CLI is quite simple, needing only the volumeid and a description, which is more for readability and searchability later.
aws ec2 create-snapshot --description "umrk pre-launch web snapshot" --volume-id vol-024e8fc8350b3add0
Managing the UMRK EBS Volumes
Your UMRK web servers must each have additional EBS volumes, which will store the data used by the WebOps team to carry out future upgrades of its custom applications.
Open the EC2 console and then click the Create Volume button located in the Elastic Block Store | Volumes section of the page. Follow the prompts shown in Figure 4-3 to choose a size (100 GB in the example), the Availability Zone (must be located with the EC2 instance due to iSCSI network requirements), and create any tags that are relevant for the volume.
Figure 4-3. Creating the EBS volume in the AWS console
The AWS CLI to do this same task is shown in the following code example, which shows your AZ, volume size, volume type, and the tags:
aws ec2 create-volume --availability-zone us-east-2c
--size 109 --volume-type gp2
--tag-specifications 'ResourceType=volume,
Tags=[{Key=owner,Value=WebOps},
{Key=application,Value="umrkweb"},
{Key=environment,Value=production}]'
Figure 4-4 demonstrates attaching the volume, which you do in the Volumes section of the EC2 console.
Figure 4-4. Attaching the new EBS volume to an EC2 instance
The associated AWS CLI requires only your device name, volumeid of the EBS Volume and the instance-id for your EC2 instance.
aws ec2 attach-volume --device /dev/sdf --volume-id vol-041b8dcb66cdbb40b --instance-id i-01f0026ea5396f197
You now have your new volume created, presented, and attached to the EC2 instance.
Design and Operational Patterns for Availability Using EBS
With EBS, there are not as many design patterns as there are operational patterns to consider, aside from those that require understanding the storage needs of your cloud workloads:
Choose your default carefully
The default is gp2 for new instances, which might be more than you require for I/O and also for cost. You can carry out storage-tier migrations nondisruptively, which is helpful to reduce outages during future changes.
Manage snapshots carefully
Snapshots are not backups! Be sure that you understand the point-in-time nature and what you want to use snapshots for. You should use them for short-term protection ideally and then remove them to reduce overhead and cost.
Backup your volumes and instances
Data is still vulnerable to loss, corruption, and external access or even malware. It is critical that you have a backup and recovery strategy for your AWS data and applications. Contact your data protection vendor to make sure that you are licensed and able to protect your cloud assets.
Beware of cost versus performance decisions
Choosing lower-cost volumes can affect performance but choosing higher-throughput volumes could be expensive and unnecessary. Performance will also change over time with application and consumption pattern changes. These are not onetime decisions.
Match storage to instance capabilities
Make sure that your high-throughput storage is attached to the right instance type to fully use the performance features (e.g., EBS optimized).
Conclusion
Volume management is quite simple on AWS through the AWS console as well as the CLI. You now have completed the requirements for our UMRK example and have a solid foundation to start operating your own AWS instances and volumes.