Detailed Notes
We start a new series of tutorials as we build out a backup system that drops archive files into Amazon's S3 and related storage. This episode gets us started with the Boto3 library and AWS command line interface configuration. We wrap it up with a simple script to list the S3 buckets on our account.
You can find out more through our online classes at https://school.develpreneur.com and register for free. Registration will add you to our email list and you will periodically receive coupons for courses as well as notifications of the latest releases.
Transcript Text
[Music] well hello and welcome back we are embarking on something a little different uh this time we're going to spend a little bit of time we're building a script that is in Python we're going to utilize Amazon AWS or S3 and all of those related buckets and what we're going to do is it's really going to be something rather simple utility where we can take a file that is a backup uh eventually we're going to actually build this it's going to go you point it to a folder it's going to grab all that stuff it's going to uh you compress it all together throw it into effectively a you know a tar file if you know anything about that and then we'll shove it out to S3 and then we'll also have the ability to pull those things back down a lot of this is related to using Amazon's CLI their command line interface and a thing called Bodo boo3 which is a library for Python and we'll go ahead and get started and just go as far as we can with each of these episodes this one really going to get set up so the first thing you need to do with this uh the photo is you're going to have to pip 3 install it it's just B3 and it will take a second to do it I and by a second I mean a couple of them and then we're ready to go so what I've got here is I think it's called this yep what I've got here is a really simple script and I grabbed this from uh some examples that are out there if you go out and we'll get a uh I'll have a list out there link out in the notes for this let's do this um and uh there's a lot of great examples uh the first we're going to do is we're just going to go out to our buckets if you don't know much about S3 it is uh such it's a file storage system uh it's I think I forget what the three s's are it's a simple something storage and what it does is it allows us to put files out there in folders and then we can always retrieve them back via essentially URL and there's some other things we can do around a permissions and such so we're going to do first we're just going to do a super simple client that allows us to connect to our account and then pull a directory of the uh of the folders that we have oops go to this one because it's easier to read and they're called buckets our folders are essentially our folders are called buckets and so all we're going to do is we're going to connect uh to S3 and this can go to uh other AWS Services as well so we're keeping it oh it's simple storage service that's what it was um so we're going to keep it you know sort of sticking to that for now uh we may Branch out in future items or future episodes so right now it's just going to come in we just print a little line and then we're just going to walk through each buckets and each bucket and print the bucket name so it's fairly straightforward um and all we need really all we need here is the boo3 because you're going to see we're using it to connect and uh then we're just going to get all the buckets and then we're just printing it out so I've got a few other things floating in there at Imports that we will use later but not right now um let me get out of this so now if I take this little script that I wrote we're going to find that oops it's not python it's Python 3 so if I try to run it this probably what's going to happen so we've we've installed Bodo 3 and basically what's going to end up is you're going to get some variation of access toide and the reason that is is because we need to have the command line interface set up in order to do this and so order to use the python Library we need to have the command line interface set up now if you come to this link uh which I will throw that in there as well uh this is a c my downloads and it's really simple it's just amazon.com uh aws.amazon.com CLI and then it will do a you've got a little getting started over here uh you can basically grab it for Windows for Mac uh Linux Amazon Linux so we're going to do for this one we're going to do the Linux installer and so I can actually do let's see if this is updating I don't have to worry about this uh so I'm going to come in and I'm just going to pull it down it used to be that there was a package for it apparently so let's go set this up in our server uh that's good enough place for it okay got it and then I just want to unzip it so I need to get uh to do this I need to get unzip so I'm going to do unzip on dbn I can go out there basically if you just go Google unzip dbn1 or what whatever your operating your Linux operating system is you can do that you may already have unzip available uh in this case I did not so now I uh pulled that out and I'm assuming I got it all uh should be all there interesting enough okay and then I should be able to once I unzip it do a AWS [Music] install oh for some reason it did not unzip right uh let's see oh I did not unzip it that's why uh unzip there we go I installed it I forgot to unzip it now we'll go in and then we're just going to run this uh install so now I'm going to have the Amazon command line interface will be in installed uh there we go so now I should be able to do uh it's I believe it's AWS C well let's go back and look so it's just a I need to do a configure which is AWS configure and what it's going to do is going to ask me for a key and a a password and so what I need for that is I need to go over and we'll just do this if I want to do um AWS access key let's call it S3 we'll do that and it'll take us security credentials and this will walk you through setting those up uh so you'll basically have to go get yours you'll need to log into your account so that you can go uh with your access keys and you'll be able to see where they're at and where you're able to set those up I'm not going to go into that because your account may look a little bit different and I don't want to expose my my keys uh so I'm going to jump back in a second I'm going to go ahead and enter my two uh if you need something if you need to reach out to me definitely email them and we can always walk you through it uh but I'm going to go ahead and enter these keys and I'll be right back it's going to ask you a couple of things it's going to ask you a default region and some things like that you don't need those uh what that's going to do is that's going to set everything up for you so that once you've got your B3 installed plus your um your command line your CLI configured then what you should get and this is where you be able to test it is if you run that back up you should be able to see that hey wh that it connected and it's going to give you some buckets now in this case I've got four buckets in there and so it's just going to list through each of those and so I've got a very simple uh sort of hello world uh program going on here [Music]
Transcript Segments
[Music]
well hello and welcome back we are
embarking on something a little
different uh this time we're going to
spend a little bit of time we're
building a script that is in Python
we're going to utilize Amazon
AWS or S3 and all of those related
buckets and what we're going to do is
it's really going to be something rather
simple utility where we can take a file
that is a
backup uh eventually we're going to
actually build this it's going to go you
point it to a folder it's going to grab
all that stuff it's going to uh you
compress it all together throw it into
effectively a you know a tar file if you
know anything about that and then we'll
shove it out to S3 and then we'll also
have the ability to pull those things
back down a lot of this is related to
using Amazon's CLI their command line
interface and a thing called Bodo
boo3 which is a library for Python and
we'll go ahead and get started and just
go as far as we can with each of these
episodes this one really going to get
set up so the first thing you need to do
with this uh the photo is you're going
to have to pip 3 install it it's just
B3 and it will take a second to do it I
and by a second I mean a couple of
them and then we're ready to
go so what I've got here
is I think it's called this yep what
I've got here is a really simple
script and I grabbed this
from uh some examples that are out there
if you go out and we'll get a uh I'll
have a list out there link out in the
notes for
this let's do this
um and uh there's a lot of great
examples uh the first we're going to do
is we're just going to go out to our
buckets if you don't know much about
S3 it is uh such it's a file storage
system uh it's I think I forget what the
three s's are it's a simple something
storage and what it does is it allows us
to put files out there in folders and
then we can always retrieve them back
via essentially URL and there's some
other things we can do around a
permissions and such so we're going to
do first we're just going to do a super
simple client that allows us to connect
to our
account and then pull a directory of the
uh of the folders that we have oops go
to this one because it's easier to read
and they're called buckets our folders
are essentially our folders are called
buckets and so all we're going to do is
we're going to connect uh to S3 and this
can go to uh other AWS Services as well
so we're keeping it oh it's simple
storage service that's what it was um so
we're going to keep it you know sort of
sticking to that for now uh we may
Branch out in future items or future
episodes so right now it's just going to
come in we just print a little line and
then we're just going to walk through
each buckets and each bucket and print
the bucket name so it's fairly
straightforward um and all we need
really all we need here is the boo3
because you're going to see we're using
it to connect and uh then we're just
going to get all the buckets and then
we're just printing it out so I've got a
few other things floating in there at
Imports that we will use later but not
right now um let me get out of
this so now if I take this little script
that I wrote we're going to
find that oops it's not python it's
Python 3 so if I try to run it this
probably what's going to happen so we've
we've installed Bodo 3 and basically
what's going to end up is you're going
to get some variation of access toide
and the reason that is is because we
need to have the command line interface
set up in order to do this and so order
to use the python Library we need to
have the command line interface set up
now if you come to this link uh which I
will throw that in there as
well uh this is a c my
downloads and it's really simple it's
just amazon.com uh aws.amazon.com
CLI and then it will do a you've got a
little getting started over here uh you
can basically grab it for Windows for
Mac uh Linux Amazon Linux so we're going
to do for this one we're going to do the
Linux
installer
and so I can actually do let's see if
this is updating I don't have to worry
about
this uh so I'm going to come in and I'm
just going to pull it down it used to be
that there was a package for it
apparently so let's go set this up in
our
server uh that's good enough place for
it okay got
it and then I just want to unzip it so I
need to get uh to do this I need to get
unzip so I'm going to do unzip on dbn I
can go out there basically if you just
go Google unzip dbn1 or what whatever
your operating your Linux operating
system is you can do that you may
already have unzip available uh in this
case I did not so now I uh pulled that
out
and I'm assuming I got it all uh should
be all
there interesting enough
okay and then I should be able to once I
unzip
it do a AWS
[Music]
install oh for some reason it did not
unzip right uh let's see oh I did not
unzip it that's why uh unzip there we
go I installed it I forgot to unzip it
now we'll go in and then we're just
going to run this uh install so now I'm
going to have the Amazon command line
interface will be in installed uh there
we go so now I should be able to do uh
it's I believe it's AWS
C well let's go back and look so it's
just a I need to do a configure which is
AWS
configure and what it's going to do is
going to ask me for a key and
a a password and so what I need for that
is I need to go over and we'll just do
this if I want to do um
AWS access key let's call it S3 we'll do
that and it'll take us security
credentials and this will walk you
through setting those up uh so you'll
basically have to go get yours you'll
need to log into your account so that
you can go uh with your access keys and
you'll be able to see where they're at
and where you're able to set those up
I'm not going to go into that because
your account may look a little bit
different and I don't want to expose my
my keys uh so I'm going to jump back in
a second I'm going to go ahead and enter
my two uh if you need something if you
need to reach out to me definitely email
them and we can always walk you through
it uh but I'm going to go ahead and
enter these keys and I'll be right back
it's going to ask you a couple of things
it's going to ask you a default region
and some things like that you don't need
those uh what that's going to do is
that's going to set everything up for
you so that once you've got your B3
installed plus your um your command line
your CLI configured then what you should
get and this is where you be able to
test it is if you run that back up you
should be able to see that hey wh that
it connected and it's going to give you
some buckets now in this case I've got
four buckets in there and so it's just
going to list through each of those and
so I've got a very simple uh sort of
hello world uh program going on
here
[Music]