Detailed Notes
This episode looks into File IO (reading, writing, and appending) in Python as we continue moving through topics on the certification syllabus
Useful Links: https://docs.python.org/3/library/io.html https://realpython.com/read-write-files-python/ https://medium.com/swlh/python-stringio-and-bytesio-compared-with-open-c0e99b9def31
Learn more about the overall certification and syllabus here: https://pythoninstitute.org/certification/pcap-certification-associate/pcap-exam-syllabus/
Github repository for code examples: https://github.com/robbroadhead/PythonCertificationSeries
Transcript Text
[Music] well hello and welcome back we are continuing our series of uh tutorials basically we're going through our pursuit of python certification gone through a lot of different topics and today we are going to go through file io and let me clean up a couple just random stuff that's open so there's going to be a couple of files that are going to be of note in our repository if you're looking at some of the uh some of the examples you're following along we'll have files.py but there's also going to be a file we create and there's also which should already exist there's going to be a readme.md file and we're using that just for examples you can adjust those however you need so first thing you need to do in dealing with a file is you have to be able to open it you want to open it and do something with it and python it's pretty easy open is just a file name a mode and then optionally you have an encoding so i can do this or this one would be valid now the r is for read there's a w for right and then you can also do a b for binary but let's let's sort of look at what we've got so far so let's say we open this and i'm just going to do a little uh it's a nice little snippet or whatever so i'm going to open the file i'm going to read the first line of the file and then while that line is not an empty line i'm just going to print it and then i'm going to read the next line of the file and i'm going to close it because once you open it you want to close it otherwise depending on how things go you could have resource locks and stuff like that now typically what's going to happen let me actually get rid of that first typically what's going to happen so let's just do this i'm going to run this one through so if i go here so the readme let me go over that so the readme.md file let's look at that whoop there we go um here oh it's doing it as a markup but kind of like that there we go so that's all it is it's pretty straightforward and what you'll see here is it has the that tag and it comes and does that now note that it's actually doing the um when i do a print that print has embedded in there basically we can't see it it has the uh the line the line terminator end of line so if i come in here um well that's because i'm doing a print so it's going to do that every time if i did that without the line feed and i could pull that off i could come in and i could actually replace the the line feed in it and then i would be okay let's see if i do now another one i should be sure is i can do there's a limit i can do so if i do a limit let's just say 10. now you'll see here it's going to read but it's only going to go ten characters out one two three four five six seven eight nine ten so it's only gonna read that it's only gonna pull that line out to a certain number of characters but notice that it's still if you look here like so oh so the first one i did i read the line but then i come into the loop and these are all at a length of 10 and so it's not actually it's limiting it so it's either going to be in this case either 10 characters or the end of line which can be valuable at times where it's like okay i want to read but if i hit the end of a line then i'm just gonna you know cut it off there which would be fairly useful if you run into things like if you're doing like some comma separated value or some sort of file separator some character separated text file or something like that so we can do that we can read we do read line we can also do let's just go ahead and do this as we do a straight read now this one's going to do by number of characters so if i do read then it goes all the way it's going to go all the way out um let me do this let's just change that so we do both of them read and so now when i do it i'm not getting that extra line when i do that realign it ends up slapping essentially a l end of line character on it when i do a read then it's just gonna straight through and dump that thing out and so here i could actually do it like this and i'm gonna get the same thing because it just reads the whole file at this point and let me get that back oh let's do that and let's do it this way and then i'm gonna do it this way we'll get all of those examples in there so let's do it let's read it 15 at a time so let's make sure all of those oh let's do this um so note also uh let's see file a time and this one will be just read complete file let's do it that way so now if we look now this is an interesting one because what you're going to see here is not what you probably expected before when i did all these things they work fine but here like read 15 there's nothing read the complete file nothing and that's because f that little pointer to the file has gone all the way through so i can come up here and i can do um and i get to actually have a lot of different options i can do here as you can see i've got a lot of your normal functions available but what i can do is i can come in and i can open it again or i can do close and then open it and let me go and get rid of the encoding so you can just see the difference uh here and so now we're going to see them and notice that there's no real difference in what i'm seeing because this is utf-8 in general so if i don't close it that file pointer has gotten to the end the first time through and now it's not going to do anything for me any good it's you know it's not terribly useful so if i close and reopen it that's uh that's going to probably be the easiest way to deal with it particularly when you focus on the the functions that we're going to have within the um within the certification now another thing i can do uh well okay so we've got redline and we've got read let's go ahead and close that file and now what i can do is i'm going to do f dot write [Music] this is a line this is a second line and close it actually i'm not going to close it so let's do this first so let's go through here oh and run it and it says create a simple file well did it create the file yes it is let's see what it says so notice that it ran these lines together even though those separate writes then there was issues with that now that f-close doesn't isn't really needed in this one exactly because once python once that the script completes the application is done it closes all the file handles however if i do this and i try to work with it within the application before it's closed out that before a close has either been explicitly like here called or implicitly because the script is done there's gonna be issues with it i'm not gonna be able to find that file it's not gonna be in the state that i think it is so what i can do here is i can do this oh there we go i'm gonna run that and now let's look at that file and notice i've got this is line and this is second line now notice that what i've got here is test.text has been rewritten when i did that open it was rewritten but if i come in and do this oh i'm sorry i can't do both so i can't do right i can do append so it's an a um let's do this this is create a simple file let's do this let's do and to file and so i've got to close it so i can reopen it in a pin mode and now if i run it uh let's see oh did i print it out oh i'm sorry i did so i did it right right print print pen to sip file do that there we go and so now if i cat there we go so now i've got it so in this script so now you can see where i hit this twice when this script runs i blow away the file that exists this time because i open it for write not for append this time i come in and i do a an append and now i can do all of those same things in a binary fashion so let me first do because we mentioned that so let's do this so let's go ahead and do this readme except for open it in binary and i'm going to do read and let's just do this now let's see what it looks like if i open up binary mode [Music] oh there now here we see it's got this little b because basically it took all of that stuff and now i can see my line feeds so this isn't a stream or it's not doing the printable carry it's just taking the whole thing so now i could take let's see do i have i don't have like a good binary sitting around but um i can open things like executables or other binary files that also means that i can do let me just do this like so if i do input text and then i just insert one two three four five six seven eight nine ten or zero uh well i'm gonna let first let's see what happens when i do a read line and so here when i do the binary and i do the reline it still recognizes my line but it's a binary string and it brings that end line in there that line feed in there i can also do that so like if i do let's take the same thing and let's do let's see what happens if i do the same thing and i create a simple i'm just going to call it a b file instead of typing binary well shoot i probably should just for note sake and i'm gonna do b so let's see what that looks like oh so here we go binary mode does not take an encoding argument oops sorry i forgot about that i do that because binary is straight binary encoding is for strings and so here what i'm going to see in the simple binary file when i do that byteslate object like object is required not a string so i can't send it a string in this case i could say um let's just call it temp equals this and write temp and let's do it again temp equals this and let's just bail out so let's not append to it at this point we'll just do that open it right so now we run it we're gonna see again it's not a string even though it's writing temp it's a string it's not a bytes object so instead i can do this and make it a bite [Music] and now it runs and so i've got this simple binary file created and if i do what did i call that test.text let's see it looks basically the same but now it's saying that this is bytes because i can also do the same thing and so now i could take a temp equals let's just get rid of this part and come to come three four and five against six so temp's now an array but it still doesn't like it as a list so i'd have to convert that over i don't think i can do it this simply i don't know because it's going to do that so if i do it like this it's going to treat it essentially it's going to say those are hard bites so it's going to yeah so if i test that yeah so i just have that it's a direct version of that but i could do bytes um i can do a byte array and and things like that where um i can write those out and i can write characters as needed into it so now i can actually get pretty specific with what i do um let's just do it same thing let's just do like temp equals 12. and let's whoop let's see what it says when i set an integer note integer doesn't count so i have to get it converted to bytes so i could probably do [Music] this two bytes and i need to give it a length so let's say a length of one and byte order i don't think i need that so let's see so if i do that oh i do need bite order and i don't know my bite order offhand let's just see what that does but what i can do is i can go in i'm not going to get too deep into it and this one uh mostly because of where i'm at oh it's either little or big oh big indian or little indian which there we go and so now oh if i look it's what is that test.text yes and so that's it's writing that character so i need something that's like let's do 68 oh and that's just one that's the problem so it's like this two let's see what that does there okay so now it's creating it's a capital d because that is if you do a 68 the ascii code 68 it's a d so it's converting that in for us as we look at the file but it's actually um you see this like null d as it's putting that space before it so you can play around with that and do binary reads and writes i think that was everything i really wanted to mess with you can also read into so i can do uh which basically is just going to push one value into another so if i do this i'm going to open it for read binary temp equals f dot read into oops it's not that way [Music] hmm uh let me do it this way i'm gonna copy and paste some stuff because it's easier that way so file name equals just that text so i have to create a byte array and oops i got import os because this is where i get obviously os.path and so i'm going to get the size of the file i'm going to open it as a read into i'm going to open it as f so this one's a little different instead of doing assigning it and then doing something with it i'm just going to say hey i'm going to grab it and i'm going to read into my buffer the file and then i'm gonna do print buffer and it's like i like it because it's a byte array i think no it does and so it just tells me that that is a bite array if i do a string i don't know if i can do a string of a bible right so find out in a minute uh yep so it doesn't really do much for me again uh where was that up here you can see so it's really it's that hex value uh it's a capital d because the ascii of 68 is a capital d so it's actually i'm seeing it converted into that string because of the output we're seeing here i think that's enough for now we've gone pretty deep into files it should allow you the basics of reading and writing and doing some stuff with them and understand what you need for this certification so we'll uh we'll call it a an episode for here and go out there and have yourself a great day a great week and we will talk to you next time you
Transcript Segments
[Music]
well hello and welcome back we are
continuing our series of
uh tutorials basically we're going
through our pursuit of python
certification
gone through a lot of different topics
and today we are going to go through
file io and let me clean up a couple
just random stuff that's open
so there's going to be a couple of files
that are going to be of note
in our repository if you're looking at
some of the
uh some of the examples you're following
along
we'll have files.py but there's also
going to be a file we create
and there's also which should already
exist there's going to be a readme.md
file and we're using that just for
examples you can
adjust those however you need
so first thing you need to do in dealing
with a file
is you have to be able to open it
you want to open it and do something
with it
and python it's pretty easy
open
is just a file name
a mode and then optionally you have an
encoding
so i can do this or
this one would be valid
now the r is for read
there's a w
for right
and then you can also do
a b for binary
but let's let's sort of look at what
we've got so far so let's say we open
this
and i'm just going to do a little uh
it's a nice little snippet or whatever
so i'm going to open the file
i'm going to read the first line of the
file
and then while that line is not an empty
line i'm just going to print it
and then i'm going to read the next line
of the file and i'm going to close it
because once you open it you want to
close it otherwise depending on how
things go you could have resource locks
and stuff like that
now typically what's going to happen let
me actually get rid of that first
typically what's going to happen so
let's just do this
i'm going to run this one through
so if i go here
so the readme let me go over that so the
readme.md file let's look at that
whoop
there we go
um here oh it's doing it as a markup but
kind of like that there we go so that's
all it is it's pretty straightforward
and what you'll see here is
it has the
that tag and it comes and does that now
note that it's actually doing the um
when i do a print
that print has
embedded in there basically we can't see
it
it has the uh the line the
line terminator end of line so if i come
in here
um
well that's because i'm doing a print so
it's going to do that every time
if i did that without the line feed
and i could pull that off i could come
in
and i could actually replace
the
the line feed in it
and then i would be okay let's see if i
do
now another one
i should be sure is i can do there's a
limit i can do so if i do a limit let's
just say 10.
now you'll see here it's going to read
but it's only going to go
ten characters out one two three four
five six seven eight nine ten
so it's only gonna read that it's only
gonna pull that line
out to a certain number of characters
but notice that it's still
if you look here like so
oh
so the first one i did i read the line
but then i come into the loop
and these are all at a length of 10
and so
it's
not actually it's limiting it so it's
either going to be in this case either
10 characters or
the end of line
which can be valuable at times where
it's like okay i want to read
but if i hit the end of a line then i'm
just gonna you know cut it off there
which would be fairly useful if you run
into things like if you're doing like
some comma separated value or some sort
of
file separator
some character separated
text file or something like that
so we can do that we can read we do read
line we can also do
let's just go ahead and do this as we do
a straight read now this one's going to
do by number of characters
so if i do read
then it goes all the way
it's going to go all the way out um let
me do
this
let's just change that so we do both of
them read
and so now
when i do it i'm not getting
that extra line when i do that realign
it ends up slapping essentially a
l end of line character on it when i do
a read then it's just gonna straight
through and dump that thing out and so
here
i could actually do it like this
and i'm gonna get the same thing because
it just reads the whole file at this
point
and let me get that back
oh let's do that
and let's do it this way
and then i'm gonna do it this way we'll
get all of those examples in there
so let's do it let's read it 15 at a
time
so let's make sure all of those oh let's
do this um
so
note also
uh let's see
file
a
time
and this one will be just read
complete file let's do it that way
so now if we
look now
this is an interesting one
because what you're going to see here
is not what you probably expected
before when i did all these things they
work fine but here like read 15 there's
nothing read the complete file nothing
and that's because
f
that little pointer to the file has gone
all the way through
so i can come up here
and i can do
um
and i get to actually have a lot of
different options i can do here as you
can see i've got a lot of your normal
functions available
but what i can do is i can come in and i
can open it again
or
i can do close
and then open it
and let me go and get rid of the
encoding so you can just see the
difference
uh here
and so now we're going to see them
and
notice that there's no real difference
in what i'm seeing because this is utf-8
in general
so if i don't close it
that
file pointer has gotten to the end the
first time through
and now it's not going to do anything
for me any good it's you know it's not
terribly useful
so if i close and reopen it
that's uh that's going to probably be
the easiest way to deal with it
particularly when you focus on the
the functions that we're going to have
within
the um
within the certification
now another thing i can do
uh well okay so we've got redline and
we've got read
let's go ahead
and
close that file
and now
what i can do
is i'm going to do f dot write
[Music]
this is a
line this is a second line
and close it
actually i'm not going to close it so
let's do this first
so let's go through here
oh and run it and it says create a
simple file well did it create the file
yes it is let's see what it says
so
notice that it ran these lines together
even though those separate
writes
then there was issues with that
now
that f-close
doesn't isn't really needed
in this one exactly because once
python
once that the script completes the
application is done it closes all the
file handles
however if i do this and i try to work
with it within the application before
it's closed out that before a close has
either been explicitly like here called
or implicitly because the script is done
there's gonna be issues with it i'm not
gonna be able to find that file it's not
gonna be in the state that i think it is
so what i can do here is i can do this
oh
there we go i'm gonna run that and now
let's look at that file
and notice i've got this is line and
this is second line now notice
that what i've got here
is
test.text
has been rewritten when i did that open
it was rewritten
but if i come in
and do this
oh i'm sorry i can't do both so i can't
do right i can do append so it's an a
um let's do this
this is create a simple file
let's do this let's do
and to file
and so i've got to close it so i can
reopen it in a pin mode
and now
if i run it
uh let's see oh did i print it out
oh i'm sorry i did so i did it right
right
print print pen to sip file
do that
there we go and so now
if i cat there we go
so now i've got it so in this script so
now you can see where i hit this
twice when this script runs
i blow away the file that exists this
time because i open it for write not for
append
this time i come in and i do a an append
and now i can do all of those same
things
in a binary fashion
so let me first do
because we mentioned that so let's do
this
so let's go ahead and do this readme
except for open it in binary
and i'm going to do
read
and let's just do this
now let's see what it looks like if i
open up binary mode
[Music]
oh
there
now here we see it's got this little b
because basically it took all of that
stuff and now i can see
my line feeds
so this isn't a stream or it's not doing
the printable carry it's just taking the
whole thing
so now i could take
let's see
do i
have i don't have like a good binary
sitting around but
um i can open things like
executables or other binary files that
also means that i can do
let me just do this like so if i do
input
text
and then i just insert one two three
four five six seven eight nine ten or
zero
uh well i'm gonna let first let's see
what happens when i do a read
line
and so here
when i do the binary and i do the reline
it still recognizes my line
but
it's a binary string and it brings that
end line in there that line feed in
there
i can also do that
so like if i do let's take the same
thing and let's do
let's see what happens if i do the same
thing and i create a simple
i'm just going to call it a b file
instead of typing binary
well shoot i probably should just for
note sake
and i'm gonna do b
so let's see what that looks like
oh so here we go
binary mode does not take an encoding
argument oops sorry i forgot about that
i do that
because binary is straight binary
encoding is for strings
and so here what i'm going to see in the
simple binary file
when i do that
byteslate object like object is required
not a string so i can't send it a string
in this case
i could say um
let's just call it temp
equals
this
and write temp
and let's do it again temp equals this
and let's just bail out so let's not
append to it at this point we'll just do
that open it right so now
we run it we're gonna see again it's not
a string even though it's writing temp
it's a string it's not a bytes object
so instead
i can do this
and make it a bite
[Music]
and now it runs and so i've got this
simple binary file
created and if i do what did i call that
test.text
let's see it looks basically the same
but now it's saying that this is bytes
because i can also do
the same thing
and so now i could take a temp equals
let's just get rid of this part
and come to come three four and five
against six so temp's now an array
but it still doesn't like it as a list
so i'd have to convert that over
i don't think i can do it this simply
i don't know because it's going to do
that so
if i do it like this it's going to treat
it essentially it's going to say those
are hard bites so it's going to
yeah
so if i test that yeah so i just have
that
it's a direct version of that but i
could do bytes um
i can do a byte array and and things
like that where
um
i can write those out and i can write
characters as needed into it so now i
can actually get pretty specific with
what i do um let's just do it same thing
let's just do like temp equals 12.
and let's whoop let's see what it says
when i set an integer note
integer doesn't count so i have to get
it converted to bytes
so i could probably do
[Music]
this
two bytes
and i need to give it a length so let's
say a length of
one
and byte order i don't think i need that
so let's see
so if i do that
oh i do need bite order
and
i don't know my bite order offhand
let's just see what that does
but what i can do is i can go in i'm not
going to get too deep into it and this
one uh mostly because of where i'm at oh
it's either little or big oh big indian
or little indian
which
there we go
and so now
oh if i look
it's what is that test.text
yes
and so
that's it's writing that character so i
need something that's like
let's do 68
oh and that's just one that's the
problem so it's like this two
let's see what that does
there okay
so now
it's creating it's a capital d because
that is if you do a 68
the ascii code 68 it's a d so it's
converting that in for us as we look at
the file
but it's actually
um
you see this like null d as it's putting
that space before it
so you can play around with that and do
binary reads and writes i think that was
everything i really wanted to mess with
you can also read into
so i can do
uh
which basically is just going to push
one value into another
so if i do this
i'm going to open it for read binary
temp
equals f dot read
into
oops it's not that way
[Music]
hmm
uh let me do it this way
i'm gonna copy and paste some stuff
because it's easier that way so file
name equals
just that text so i have to create a
byte array
and oops i got import os
because this is
where i get obviously os.path
and so i'm going to get the size of the
file
i'm going to open it as a read into
i'm going to open it as f
so this one's a little different instead
of doing
assigning it and then doing something
with it i'm just going to say hey i'm
going to grab it
and i'm going to read
into my buffer the file
and then i'm gonna do
print buffer
and it's like i like it because it's a
byte array
i think
no it does and so it just tells me that
that is a bite array if i do a string i
don't know if i can do a string of a
bible right so find out in a minute
uh
yep
so it doesn't really do much for me
again
uh where was that up here
you can see so it's really it's that hex
value
uh it's a capital d because
the ascii of 68 is a capital d so it's
actually i'm seeing it
converted into that string because of
the output we're seeing here
i think that's enough for now we've gone
pretty deep into files it should allow
you the basics of reading and writing
and doing some stuff with them and
understand what you need for this
certification so
we'll uh we'll call it a an episode for
here
and go out there and have yourself a
great day a great week and we will talk
to you
next time
you