Automating AWS services with Python BOTO3 Programming
In this article, we will look at using Python scripts to communicate with Amazon Web Services (AWS) providing infrastructure. We will learn how to configure a workstation with Python and the Boto3 library. Then you’ll learn how to create and manage programming with following:
- Using virtual machines in elastic computing.
- Simple storage EC2 instance
- Relational Database Service (RDS)
Need for AWS automation:
A lot of newbies face difficulties when moving their resources to AWS. To accomplish any project, the functioning of all facilities and thorough infrastructure analysis become essential. Running DevOps activities frequently complicates the problem even more. This could require a lot of investment in human resources, and the resulting chance of human mistakes is unavoidable.
By automation, AWS solves those issues. Using language- libraries that are easily incorporated and used in simple scripts is handy when a person has to perform an operation that requires a lot of manual effort. In the world that is rapidly moving towards technological advances, it is not wise to perform tedious manual operations in old school. For example, if you copied a large number of snapshots from one region to another, and midway realized you’ve also copied snapshots of unwanted volumes which can prove to be a costly mistake. To prevent these errors and needless headaches, the responsibility can be shifted to a simple script that takes care of the operations involved.
Specifications
Before we get started, there are a few things you’ll need to set up:
You need to setup an AWS services account with admin privileges or power users privileges. Since in this exercise we will be making, changing, and removing stuff, the account should be a sandbox account that has no access to production VMs, files, or databases.
- Connection to a Linux shell system with active connection to the Internet.
- Some experience working with the Python command line interface and the Bash command line.
Configuration:
Let’s customize our workstation with Python, Boto3 and the AWS services CLI app. While the focus of this article is on using Python, we will need to set up a few items using the AWS services CLI tool.
Once we are set up on the command line with our software, we will go to the AWS services console to set up a user and offer permissions to access the services we need to communicate with.
Python and Pip
First, check if you already have Python enabled. You can do this by typing your shell into which python. If Python is enabled the response will be the executable path to the Python. If Python is not available, go to the Python.org website to find out how to download and install Python for your specific operating system.
For this we shall use Python 2.7.10. Check out your Python version by typing python -V. Your update will work well, as long as the version is 2.6 or higher.
The next thing we need is pip, the package manager at Python. Installing the Boto3 library and the AWS services CLI application we’ll use pip. By typing out which pip you can search for pip. If pip is enabled the path to the executable pip will be the answer. If pip is not installed follow the instructions for installing pip on your device at pip.pypa.io.
Check your pip edition by clicking “pip -V” Your pip update should be either 9.0.1 or newer.
Now we can install the packages needed for our scripts to access AWS services with Python and pip installed.
pip install AWS cli boto3 -U — ignore-installed six
If you are interested to Learn Python you can enroll for free live demo python Online Training
AWS services CLI Tool and Boto3 to install the AWS services CLI and Boto3 using the pip command:
Boto lets you write scripts to automate stuff like starting AWS services EC2 instances. Boto is a Python script that provides Amazon Web Services (AWS ) programmatic connectivity. It is a python script to get ec2 instance details.
AWS services provides a variety of dynamically scalable cloud services including core computing infrastructure, Elastic Compute Cloud (EC2), as well as multiple storage solutions, load balancers, and DNS. Such services can be managed either through the AWS services console, or through the broad AWS services API. But there are a range of libraries for third-party use of this API. We have Boto for Python which allows you to write handy scripts to automate simple things like starting or stopping instances of EC2 or taking daily snapshots of your servers.
You may get the fundamentals down or brush up on some more advanced skills, depending on your level of Python training experience. Whatever stage you are at, we deliver a comprehensive set of courses to help you train at your own speed and time. For Beginners Learning Path new programmers will move straight into Python. Programmers with some experience can use the Python Programming Learning Route to firm up their coding chops. You can train with scenario-based learning paths and thousands of lessons, quizzes, tests with industry experts at Python online training.
Installing Boto on Linux (CentOS)
1. For install Python and the Python pip module installer, you must first use the yum module manager: yum install python-devel python-pip
2. We will now install the Boto package via pip:
boto 3 python-pip update.
Finally, we can check if everything successfully on our Linux box:
Configuring AWS services credentials
On initialization, the Boto library searches for configuration files in the following locations (in this order):
/etc / boto.cfg — for site-wide settings for all users on this computer
~/.boto — for user-specific settings
Use cases
Let’s start with a simple scenario: starting an EC2 instance.
#/python/usr/bin
import boto.ec2
conn = boto.ec2.connect_to_region(“us-west-2”)
conn.run_instances(
‘ami-6ac2a85a’,
key_name=’nitheesh_oregon’,
instance_type=’t1.micro’,
security_groups=[‘nitheesh_oregon’]
)
The above python script connects to the us-west area of Amazon and begins an instance by using AMI
I d: ami-6ac2a85a.
Stop Scheduling.
Let’s say you now want to avoid the instances in your test environment every day at a certain time. Run:
import boto.ec2 conn = boto.ec2.
connect to region(“us-west-2) “
conn.stop instances(instance ids=[‘instance-id-1’,’instance-id-2]) ‘
By creating a cron job you can schedule this script to run every day at a given time.
Invalidation in CloudFront.
AWS services does not provide CloudFront with command line tools. So if you want to invalidate objects, you need to sign in to the console and add objects to invalidate them. It’s a tiresome and really long process. Instead you can use the boto library:
import boto
dist_id = ‘xxxxxxxxxxx’
invalidation_path = ‘/home/ec2-user/invalid.txt’
def main():
paths = open(invalidation_path,”r+”)
conn = boto.connect_cloudfront()
inval_req = conn.create_invalidation_request(dist_id, paths)
print inval_req
touch = open(invalidation_path,”w”)
touch.write(“”)
if __name__ == ‘__main__’:
main()
Boto-supported AWS services:
Currently, Boto supports more than 50 Amazon services running the entire spectrum from storage, database, program, and payments and billing. You can find a full and current list on the Python.org website.
Boto Updates
Boto3 is Boto’s new update, which is considered as the Amazon Software Developers Kit (SDK) by Python. Some of the key ways Boto3 differs from the original Boto is that the new edition is not hand-coded and is thus kept continuously up-to-date for the benefit of users.
Conclusion
Boto python helps you to write scripts to handle complex AWS setups. This also provides support for other public services such as Google Cloud, in addition to private cloud platforms such as Eucalyptus, OpenStack, and OpenNebula.
