Detailed Notes
We continue the series of tutorials as we build out a backup system that drops archive files into Amazon's S3 and related storage. We step back a bit to look at packaging files into a compressed zip file and then storing it in the cloud in S3.
You can learn more through our online classes at https://school.develpreneur.com and register for free. Registration will add you to our email list, and you will periodically receive coupons for courses and notifications of the latest releases.
Transcript Text
[Music] well hello and welcome back we are continuing working on our little uh backup script type thing and uh this episode we're going to give ourselves something a little more useful for the backup so what we're going to do is we're going to create something that is a uh to zip some stuff up so what we're going to do is we're going to use uh sh util which is basically shell utils this is super simple uh because what we're going to do is we're going to then do uh let's do this uh we're going to create a method that is going to be uh ZIP and store and it is going to be a directory and a bucket name and what we're going to do is we're going to make an archive out of the directory um oh let's give it a backup name let's do that uh backup name and we're going to create a zip and then what we're going to do is we're basically going to turn around and do a uh what do we call that save um oh upload so this going be upload file to S3 and that one takes a path and a bucket so it's going to be the backup name and then the uh bucket name so let's see how this works uh oops and then zip and store let's see we're going to have one more of these and this one's going to be uh we'll call this archive and let's do it it's going to be the local directory and the bucket name if we take those so if um we're going to create another operation here it's going to be archive it's going to be zip and store and it is going to be uh how do we do that call zip in store or we call it store yeah store directory backup bucket okay so directory is name Target is the bucket actually let's do this that's going to be the file name we'll call that archive name and for right now we're going to default the bucket which is going to be this and this will just save us dealing with that so we come in here and we're going to do zip and store so now what we're going to do is let's take uh we're going to take this folder including its temp and we're going to store it so we're going to do uh what do we call this bucket project archive and local directory is going to be let's just just do this users we're going to do the fully path one S3 backup and then I want the archive name which will be uh test backup that and we'll just call it that because I think we'll do it I think it'll tack the zip on we're going to find out if we do that what is it going to say uh no such file or directed test backup so if we look here oh it does test backup. zip did it zip it up uh it did so we're going to get rid of him and it needs to be uh let's see when it does the [Music] upload uh we do zip where did I do that so if we do store and zip what it needs to do then is because we created a zip file file based off of that backup name then it needs to be backup name plus. zip and so now if we run it boom it's uploaded and if we wanted to pull it back down so let's take uh let's do this uh we're going to do the download where did I have that I've got the somewhere back in my history there we go so we're going to do it into temp two and we are going to download everything into that so now if we look and we go to Temp two we can see there is our test backup do I have unzip I do um and it's going to unload it unzip it here and we can see now that we've got wh that we've got our uh tempar we've got it with all of its little stuff so we have created a uh backup uh let's go ah a and get some of this stuff cleaned up so we created ourselves a backup uh for uh we go in we give it a folder it's going to back that thing up and then it's going to uh unzip those folders for and also note from the backup that it came at the um at that level so sometimes if you do tar or something like that you may run into this but so here uh let's see that's archive it started at that S3 backup level when I unzipped it that's where it was at so if I wanted to recreate it it would be I'd have to create that folder S3 backup and then I would unzip my files into to it and then boom we are off and running so we now have a a way to back up our files and store those out in S3 now one of the things we probably want to do is we want to look at the ability to uh we have download we have upload we also want to be able to delete so we're going to do delete file S3 and it's going to be a file name and a bucket and let's take a look at an example for that so what we're going to do is doing our handy dandy pulling some stuff out of the documentation yet again uh what we're going to do is oops I want to do this because I don't think I have client error set up nope oh I do okay so I can leave that um so delete fall S3 we're going to give it a key and a bucket name so that means whoa come on down here this is going to look very similar to let's say upload so here if he is delete and he's going to be delete file S3 we're going to give it a name and a Target and again like most of these it's just going to be delete object so we're going to give it a bucket we're going to give it a key which is the file name uh remember that could be we're assuming it's a file name in our case but it could be something else so uh then we're going to delete it so let's go look at at our bucket contents first and uh uh let's get rid of test backup and if we do let's just run it uh so if we do Python 3 bucket then we've got so these are the guys we've got so let's take this one and we're going to get rid of him get rid of him first so what we want to do is our action is delete the name is that and the bucket name is this which again we could simplify but we'll keep it we'll send that so now if we do delete then uh did it tell us it didn't really tell us anything useful oh because we quit so let's get rid of that and that and now run it and it's going to say deleted object oh it didn't pull that stuff around properly uh why where's deleted deleted deleted deleted let's just do this and instead of that that's going to be key do a little bit of fixing here so we'll just uh make sure that it brings that in right it's going to be bucket names we don't need those okay I don't want spotlights search that and we'll just keep it here and this will say could not delete couldn't delete object there we go okay so now if we come back down here and we do our handy dandy list bucket contents we should not see archived anymore and we do not let's like CL that list up a little bit uh where is list that's list buckets list bucket content let's just do ah we'll do the we don't need the object so we're going to just do the file key so if we look we see we do not have it now let's get rid of test backup as well uh so if we go to delete the only thing we need to change here is test backup. zip and oh because I'm still giving my list let's actually run it and now we deleted it from that bucket and so now if we' List It then now we've only got our one file left so we have uh let's do this we'll do that and that just gives us something to work with so now we have put together quite a few little utilities we can delete files we can add files we can zip stuff up so we're getting pretty close to our little uh our archiver type thing so we'll do a little more work with that next time around and uh be able to like put quite a bit into this little script and now suddenly you've got a nice little Backup Tool that you can push stuff out to you know get it off of your server or your local machine and store it out somewhere else where it's protected there are much more things we can do with it but we're going to just go with that for now I give you some good little examples as always if you have any questions give us uh shoot us an email at info@ developer and other than that go out there and have yourself a great day great week and we will talk to you next time well hello this is Rob from develop andur building better developers and we have a mastermind class that will be starting or actually several Mastermind classes starting in 2024 check us out at technology mastermind2 we've got all the information you need including some pricing early bird discounts up to 50% off but those are things that you're going to have to act soon there is an application process so go ahead and sign up today thanks a lot
Transcript Segments
[Music]
well hello and welcome back we are
continuing working on our little uh
backup script type thing and uh this
episode we're going to give ourselves
something a little more useful for the
backup so what we're going to do is
we're going to create something that is
a uh to zip some stuff up so what we're
going to do is we're going to use uh sh
util which is basically shell utils this
is super simple uh because what we're
going to do is we're going to then do uh
let's do
this uh we're going to create a method
that is going to be uh
ZIP and
store and it is going to be a
directory and a bucket
name and what we're going to do is we're
going to make an archive out of the
directory um oh let's give it a backup
name let's do that uh backup
name
and we're going to create a zip and then
what we're going to do is we're
basically going to turn around and do a
uh what do we call that
save um oh
upload so this going be upload file to
S3 and that one
takes a path and a bucket so it's going
to be the backup
name and then
the
uh bucket
name so let's see how this works uh oops
and then zip and
store let's see we're going to have one
more of these and this one's going to be
uh we'll call this
archive and let's do
it it's going to be the local directory
and the bucket name if we take those
so if um we're going to create another
operation here it's going to be
archive it's going to be zip and
store and it is going to
be uh how do we do
that call zip in store or we call it
store yeah store directory backup bucket
okay so directory is
name Target is the bucket actually let's
do
this that's going to be the file
name we'll call that archive
name and for right now we're going to
default the bucket which is going to be
this and this will just save us dealing
with
that so we come in here and we're going
to do zip and store so now what we're
going to
do is let's
take uh we're going to take this folder
including its temp and we're going to
store it so we're going to
do uh what do we call this bucket
project
archive and local directory is going to
be let's just just do this
users we're going to do the fully path
one S3
backup and then I want the archive name
which will be uh test backup that and
we'll just call it that because I think
we'll do it I think it'll tack the zip
on we're going to find out if we do that
what is it going to say uh no such file
or directed test backup so if we look
here oh it does test backup.
zip did it zip it up uh it
did so we're going to get rid of
him and it needs to
be uh let's
see when it does the
[Music]
upload uh we do
zip where did I do that so if we do
store and zip what it needs to do then
is because we created a zip file file
based off of that backup name then it
needs to be backup name
plus. zip and so now if we run
it boom it's
uploaded and if we wanted to pull it
back down so let's
take uh let's do this uh we're going to
do the
download where did I have that I've got
the somewhere back in my history there
we go so we're going to do it into temp
two and we are going to download
everything into that so now if we
look and we go to Temp two we can see
there is our test backup do I have unzip
I
do
um and it's going to unload it unzip it
here and we can see now that we've got
wh that we've got
our uh
tempar we've got it with all of its
little stuff so we have
created a uh
backup uh let's go ah a and get some of
this stuff cleaned
up so we created ourselves a backup uh
for uh we go in we give it a folder it's
going to back that thing up and then
it's going to uh unzip those folders for
and also note from the backup that it
came at the um at that level so
sometimes if you do tar or something
like that you may run into this but so
here uh let's see that's
archive it started at that S3 backup
level when I unzipped it that's where it
was at so if I wanted to recreate it it
would be I'd have to create that folder
S3 backup and then I would unzip my
files into to it and then boom we are
off and running so we now have a a way
to back up our files and store those out
in S3 now one of the things we probably
want to do is we want to look at the
ability
to uh we have download we have upload we
also want to be able
to delete so we're going to
do delete file
S3 and it's going to be a file
name and a
bucket and let's take a look at an
example for that so what we're going to
do is doing our handy dandy pulling some
stuff out of the documentation yet again
uh what we're going to do is oops I want
to do this because I don't think I have
client error set up nope oh I do okay so
I can leave
that um so delete fall S3 we're going to
give it a key and a bucket name so that
means whoa come on down
here this is going to look very similar
to let's
say
upload so
here if he is
delete
and he's going to
be delete file S3 we're going to give it
a name and a
Target and again like most of these it's
just going to be delete object so we're
going to give it a bucket we're going to
give it a key which is the file name uh
remember that could be we're assuming
it's a file name in our case but it
could be something else so uh then we're
going to delete it so let's
go look at at our bucket contents
first and
uh uh let's get rid of test
backup and if we do let's just run it uh
so if we do Python 3
bucket then we've got so these are the
guys we've got so let's take this one
and we're going to get rid of him get
rid of him
first so what we want to do is our
action is
delete the name is that and the bucket
name is
this which again we could simplify but
we'll keep it we'll send that so now if
we do
delete
then uh did it tell us it didn't really
tell us anything useful oh because we
quit so let's get rid of that and
that and now run it and it's going to
say deleted object oh it didn't pull
that stuff around
properly uh why where's deleted deleted
deleted
deleted
let's just do
this and instead of that that's going to
be
key do a little bit of fixing here so
we'll just uh make sure that it brings
that in
right it's going to be bucket names we
don't need
those okay I don't want spotlights
search
that
and we'll just keep it here and this
will say could not
delete couldn't delete
object there we go okay so now if we
come back down here and we do our handy
dandy list bucket
contents we should not see archived
anymore and we do not let's like CL that
list up a little bit uh where is list
that's list buckets list bucket content
let's just do ah we'll do the we don't
need the object so we're going to just
do the file
key so if we look we see we do not have
it now let's get rid of test backup as
well uh so if we go to delete the only
thing we need to change
here
is test
backup.
zip and
oh because I'm still giving my
list let's actually run
it and now we deleted it from that
bucket and so now if we' List
It
then now we've only got our one file
left so we
have uh let's do
this we'll do that and that just gives
us something to work with so now we have
put together quite a few little
utilities we can delete files we can add
files we can zip stuff up so we're
getting pretty close to our little uh
our archiver type thing so we'll do a
little more work with that next time
around and uh be able to like put quite
a bit into this little script and now
suddenly you've got a nice little Backup
Tool that you can push stuff out to you
know get it off of your server or your
local machine and store it out somewhere
else where it's protected there are much
more things we can do with it but we're
going to just go with that for now I
give you some good little examples as
always if you have any questions give us
uh shoot us an email at info@ developer
and other than that go out there and
have yourself a great day great week and
we will talk to you next time well hello
this is Rob from develop andur building
better developers and we have a
mastermind class that will be starting
or actually several Mastermind classes
starting in 2024 check us out at
technology
mastermind2 we've got all the
information you need including some
pricing early bird discounts up to 50%
off but those are things that you're
going to have to act soon there is an
application process
so go ahead and sign up today thanks a
lot