Detailed Notes
DevOps is a hot topic in recent years. However, it is often misunderstood or ill-defined. This presentation tries to correct that. We look at its short history and how DevOps is used. Likewise, we look at the pros and cons involved when your team embraces this facet of software development.
DevOps Is a Steadily Growing Area While a young way to look at a part of the software development process, it is still rapidly growing. The tools available to assist in this area grow daily. Also, the growth of the Cloud and systems deployed there make DevOps more and more a necessity for developing modern solutions.
Transcript Text
[Music] so focus today in this presentation is devops it is definitely a buzzword something we've heard about a lot depending on what size company you're in but even even mid and even small size companies there is a sense of devops in a lot of cases and particularly when you get into the world of cloud it really you'll find as we're talking through it it is well suited to a lot of the cloud related features the pros and cons of working in the cloud so we're going to give a nice little overview of that today it's definitely a an area that you can go deeper in it's like like agile it's one of these things that it's there are a lot of pieces so you can say you do it but you may or may not do different pieces of it you may do it differently uh it's definitely just we're gonna hit sort of the tip of the iceberg to try to get an idea of what they are and what things fall into the under the umbrella of devops and so we'll start with just a sort of a brief little sort of definition a little bit of the history of it which we'll find is not very deep or long uh what does it provide why who cares basically about this devops thing what does it do for us and then talk about some of the practices the methods that are that fall under that umbrella that are devops practices and methods how you how it fits in with some of the various methodologies that are out there and then sort of a pros and cons benefits and concerns about devops and what you what you get out of it and what you need to be worried about with it so definition um and this came unfortunately i forgot the link to where this came from but i think it was off basically on wikipedia but um the word devops it's a combination of just development and operations so it's as you would sort of guess what it means that's what it is it's the development and operations and it is where you have a collaboration between the developers and the operation side of software development developers are the ones that are building software operations are the ones that are basically deploying maintaining doing upgrades and patches and things of that nature they sometimes it's configuration management back in the day when we think about the waterfall life cycle software development life cycle we talk about a there's a deployment step that's typically the operations type people are taking that that's what they own in its broadest meaning it is a philosophy that promotes better communication and collaboration between these teams there have been historically it's almost like a throw it over the wall approach where the developers do their stuff they say all right i've got it on a you know development machine or a testing machine or something like that and now we're going to send it off the operations people and they promote it to production and they have to deal with it doesn't work super well doing that so devops is a way to try to clean up that relationship and make it smoother and along the way do some other things like remove you know make it less error prone make it more streamlined scalable things of that nature it also covers the term devops also covers culture changes that is how we change looking at the development team and the operations team in some cases they've been seen as vastly different groups that don't really you know back that communication thing they don't really talk they don't do much together they don't they don't interact much and or when they do it's you know through a simple email or document or something like that this turns us in devops has more idea of like bringing those groups together so there's much more cohesion from the beginning of creating software all the way through deploying it and testing it and putting it out to production making sure that we've got the platforms and the places that we're putting these things the you know the servers and the resources match appropriately and so that devops is sort of the uh they help look at that and help the development team do a better job of moving their code out to [Music] testing and users things of that nature but this because of all this devops includes there's different roles there's different approaches there's different views there's different best practices and all of this will sort of like i said we'll touch on there's a you could spend days there's devops conferences and such out there you could spend a lot of time talking about it we're gonna try to cover it in a very short period of time so brief history which you'll find there there isn't much more than a brief history of devops it was a it's a term that was essentially created in 2009 by patrick dubois it was actually through a conference it was the dev ops conference and from there uh it quickly has grown into there's devops became something that was like oh that's a neat idea to this is something we really want to use it was embraced within the first few years by the itil champions people that are into processes and reproducing software development tasks and promotion and deployment tasks it's if you want to think of sort of a [Music] a core document about devops is there's a fictional book called the phoenix project that came out in 2013 and it definitely it gives you sort of the instead of the theory it's hey here's how this would look it's more of an uh an application view of devops and how that should work in the i guess in the real world it's really been it's been embraced and sometimes is intertwined with continuous integration content continuous deployment because definitely that you really almost need devops to do that so that you've got people that are that own the cicd part of development as opposed to like coders that are or you know software developers that are writing stuff you have they just don't have the time or the bandwidth a lot of times to do the integration and deployment piece and so that's where devops has become more and more of a almost a need and when you look at agile and rapid turnarounds if you look at cloud infrastructures as code and things like that there were a lot of changes in the last almost 15 years now that have really helped push devops to the forefront and made it much more valuable much more useful if you go back 20 or 30 years ago when you had essentially zero virtual machines there was it was a different approach and you can't you know like you can't use source code to create a physical hard server if you've got vms you can you can spin stuff up and down based on code so there's things like that that just the progression of technology has allowed devops to become a thing and a very useful one and something that we probably do more often than we think so with that what is what does devops give us how does that help us in our software development tasks now devops although it is not the same as continuous integration and deployment there's it it enables that it makes that something that we can do and it's really almost it really is also goal of devops is to do that is to have that quick and steady and continuous flow of code into uh some sort of a production type or a usable environment it may not be production it could be test it could be some sort of staging environment and something along those lines and so when you have that when you have things like either instantaneous or daily builds and deployments then you're by nature going to have smaller and more easily reversible changes to the code you're going to be doing smaller blocks of work and you're going to be validating that within the bigger system on a regular basis so if you do that if you do daily builds and with daily builds it's implying that there's also a social regression test going on and things like that so code is committed validated maybe reviewed there's a regression test it goes through then it gets pushed out and if not if it fails anywhere then you get a message back to so you know that your little bit of code whatever you worked on today didn't work however you know you may get more or less details than that but you know what what broke you know it was the work you did today and not necessarily the work you did in the last three months or something like that or i guess it's the work you've committed today so it could have been work you've been working on for a while but typically because of how devops pushes us as a development teams we're more likely to do frequent more frequent commits smaller sections of code much more defined and then that makes it easy to reverse you know easy to pull it back out and also easier to debug and that leads to shorter cycle times from requirements all the way up to live software because much like we've talked about when we talk about agile and sprints and ideally you're releasing software at the end of every sprint or you know every other sprinter every third sprint or whatever it is but there's regular deployment there's regular software made available to users ci cd it's an even shorter cycle time so you may see you may have requirements and you may see live software that's you know some pieces of functionality that are showing up you know within days it gives us devops gives us silos for development and operations and really highlights that those exist when we talked way back when we talked about the waterfall approach the waterfall methodology that was one of the things is you had these silos for testing for development when you got to deployment it was a little murky because it depended on the size of software depending on the environment and a lot of times you would have the developers or the testers would also be essentially the operations piece or in some cases operations wasn't even considered until after the fact so you would go build software put it out there to users and then say oh crud i gotta have a i'm gonna have some sort of support desk or something like that and it wasn't it wasn't really considered it wasn't definitely was not part of development it was just sort of this other thing that was like a attack on add-on thing we do after the software is done and what devops gives us is highlighting that these are the roles that we need this is uh these are skill sets that we need this is experience that we need in these various areas we we're comfortable with yeah we need developers but there's also operations that needs to be there and we need to account for that with the with some of the things that devops gives us the ability to move quickly from like a development environment to a staging environment or testing environment production we have more realistic testing opportunities instead of developers just sort of creating their own little uh development sandbox and they're playing around and doing stuff and their code's not necessarily all together and you know up to date with everybody else's and their test data may be completely wonky now you've got these environments that you can actually make them more real for lack of a better term but put actual maybe something that is much closer to mimicking the what the production environment is going to look like as far as like including size of data if you're a developer and you've got to enter in data you're not going to enter into more than a few records because it just takes too much time but if you've got a team that's either you know maybe the operation side is has some data load scripts or you just have a lot of people working on the system you're going to get more data and it's going to be different from what you would do as a developer and so you have better testing opportunities you get to test more against something that that looks real when you're doing it using devops a key thing to hold this is that then starts it it gets us closer to solving the it works on my machine problem which you see a lot of times with developers where they they write their code they do a build that works for them they can show you it works for them they can show you exactly how it it is but then as soon as you put it to somebody else's machine as soon as you put it to the test environment it breaks and because without devops we don't necessarily know what the differences are between the development chain and the qa machine and development in particular because it developers are famous for having a whole bunch of stuff on their uh on their machine in their environment that is not going to be there even when you go to the test or the stage environment and those things can you know those can impact whether something works or not so if you don't really if you don't know what you have if you can't define in detail the differences in the environments it's that much harder to figure out why it doesn't work the flip side if you've got you know like go to the nicest thing if you've got a script that builds your development environment then that same script should be able to point to your qa environment and build that and so you know that the environments are for all intents and purposes identical and that way if it works on the development machine then it needs to work you know you know it's going to work on the qa machine or vice versa or at the very least you know that it's not a difference in the machine in the in the environment itself or at least you've got a better starting point i guess so let's talk about some of the practices and methods i've actually touched on some of these already but i think it's good to you know walk through them and and this may have even been one of those things that's sort of out of place in the presentation but i think there were it helps to give you an idea of what the value is of devops a little bit and at a high level before we get into some of the specifics so some specific things that you're going to find in devops one is build automation i've talked about this already uh which is basically combined with infrastructure as code is devops is going to automate as many things and script out as many things as possible when you think of you know if you want to do a comparison think about testing is that there are manual testers and there's automated testing and as you grow systems you want more and more to be automated testing because you just it's not feasible to do manual testing with the uh the frequency and the size that you want devops is the same thing when you get into operations where you're building out dozens maybe hundreds maybe thousands of machines or devices or deploying out to them [Music] you're dealing with you know maybe as many projects millions of lines of code and all these different complex build strategies and integrations and things like that you want to automate it it's not something where you want somebody you know they have to run this script and they have to type this thing in they have to build this and do this other thing you want that done as a script and there are a lot of tools that help us do this uh you can look at hudson hudson and jenkins and things like that that have been around for a while that provide us these ci cd tools continuous integration and employment ways to commit software commit code changes and then have a process somewhere that's watching that says hey i have a commit i'm gonna do a build whatever that looks like i'm gonna you know check out the latest code if i if there's merges or something maybe it does it maybe it kicks back to somebody that says hey this requires a manual merge if it's gonna do the build it's going to run through some sort of test to say yep the build was successful we built what we wanted to and it roughly works and then push it out to you know deploy it out to some target whether that's uh it could be a development machine it could be qa or even production you
Transcript Segments
[Music]
so focus today in this presentation is
devops it is definitely a buzzword
something we've heard about a lot
depending on what
size company you're in
but even
even mid and even small size companies
there is
a
sense of devops in a lot of cases
and particularly when you get into the
world of cloud
it really
you'll find as we're talking through it
it is well suited to a lot of the cloud
related
features the pros and cons of working in
the cloud
so we're going to
give a nice little overview of that
today it's definitely a
an area that you can go deeper in it's
like
like agile it's one of these things that
it's
there are a lot of pieces so you can say
you do it but you may or may not do
different pieces of it you may do it
differently
uh it's definitely just we're gonna hit
sort of the tip of the iceberg to try to
get an idea of what they are and
what
things fall into the
under the umbrella of devops
and so we'll start with just a sort of a
brief little sort of definition
a little bit of the history of it which
we'll find is not very deep or long uh
what does it provide why who cares
basically about this devops thing what
does it do for us and then talk about
some of the practices the methods that
are
that fall under that umbrella that are
devops practices and methods
how you
how it fits in with some of the various
methodologies that are out there
and then sort of a pros and cons
benefits and concerns about
devops and what you what you get out of
it and what you need to be worried about
with it
so definition
um and this came unfortunately i forgot
the link to where this came from but i
think it was off basically on wikipedia
but um the word devops it's a
combination of just development and
operations so it's as you would sort of
guess what it means that's what it is
it's the development and operations and
it is where you have a collaboration
between the developers
and the operation side of software
development developers are the ones that
are building software operations are the
ones that are basically deploying
maintaining doing upgrades and patches
and things of that nature
they
sometimes it's configuration management
back in the day
when we think about the waterfall life
cycle software development life cycle
we talk about a there's a deployment
step that's typically
the operations type people are taking
that that's what they own
in its broadest meaning it is a
philosophy that promotes better
communication and collaboration between
these teams
there have been
historically
it's almost like a throw it over the
wall approach where the developers do
their stuff they say all right i've got
it on a you know development machine or
a testing machine or something like that
and now we're going to send it off the
operations people
and they
promote it to production and they have
to deal with it
doesn't work super well doing that
so devops is a way to try to clean up
that relationship and make it smoother
and along the way do some other things
like remove
you know make it less error prone
make it more
streamlined scalable things of that
nature
it also covers
the term devops also covers culture
changes
that is
how we change looking at the development
team and the operations team in some
cases they've been seen as
vastly different
groups that don't really you know back
that communication thing they don't
really talk they don't do much together
they don't they don't interact much
and or when they do it's you know
through a simple email or document or
something like that
this turns us in devops has more idea of
like bringing those
groups together so there's much more
cohesion from
the beginning of creating software all
the way through deploying it and testing
it and putting it out to production
making sure that we've got
the platforms and the places that we're
putting these things the you know the
servers and the resources
match
appropriately
and so that devops is sort of the
uh
they help look at that
and help the development team
do a better job of moving their code out
to
[Music]
testing and users
things of that nature but this because
of all this
devops includes there's different roles
there's different approaches there's
different views there's different best
practices
and all of this will sort of like i said
we'll touch on there's a you could spend
days there's devops conferences and such
out there you could spend a lot of time
talking about it we're gonna
try to cover it in a very short period
of time
so brief history which you'll find there
there isn't much more than a brief
history of devops
it was a it's a term that was
essentially created in 2009 by patrick
dubois it was actually
through a conference it was the dev ops
conference
and from there
uh it quickly has grown into
there's
devops became something that was like oh
that's a neat idea to this is something
we really want to use
it was embraced within the first few
years by the itil
champions people that are into processes
and reproducing
software development tasks and promotion
and deployment tasks
it's
if you want to think of sort of a
[Music]
a core document about devops
is there's a fictional book called the
phoenix project that came out in 2013
and it
definitely it gives you sort of the
instead of the theory it's hey here's
how this would look it's more of an uh
an application view
of
devops and how that should work in the i
guess in the real world
it's really been
it's been embraced and sometimes is
intertwined with continuous integration
content continuous deployment
because definitely that
you really almost need devops to do that
so that you've got people that are
that own the cicd part of development as
opposed to like coders that are or you
know software developers that are
writing stuff you have they just don't
have the time or the bandwidth a lot of
times to do
the integration and deployment piece and
so that's where devops has become more
and more of a
almost a need
and when you look at agile and rapid
turnarounds if you look at
cloud infrastructures
as code and things like that
there were a lot of changes in the last
almost 15 years now
that have really helped
push devops to the forefront and
made it much more valuable
much more
useful if you go back 20 or 30 years ago
when you had
essentially zero virtual machines
there was it was a different
approach and you can't you know like you
can't use
source code to create a physical hard
server
if you've got vms you can you can spin
stuff up and down based on code so
there's things like that that
just the progression of technology has
allowed devops to become
a thing and a very useful one and
something that we probably do more often
than we think
so with that what is what does devops
give us how does that help us in our
software development tasks
now
devops although it is not the same as
continuous integration and deployment
there's
it it enables that
it makes that something that we can do
and
it's really almost it really is also
goal of devops is to do
that is to have that
quick and steady and
continuous flow of code into
uh some sort of a production type or a
usable environment it may not be
production it could be test it could be
some sort of staging environment and
something along those lines
and so when you have that
when you have things like either
instantaneous or daily builds
and deployments then you're by nature
going to have smaller and more easily
reversible changes to the code
you're going to be doing smaller blocks
of work
and you're going to
be validating that within the bigger
system on a regular basis
so if you do that if you do daily builds
and with daily builds it's implying that
there's also
a social regression test going on and
things like that so code is committed
validated maybe reviewed
there's a regression test it goes
through
then it gets pushed out
and if not if it fails anywhere then you
get a message back to so you know that
your little bit of code whatever you
worked on today
didn't work
however you know you may get more or
less details than that but you know what
what broke you know it was the work you
did today and not necessarily the work
you did in the last three months or
something like that or i guess it's the
work you've committed today so it could
have been work you've been working on
for a while but
typically because of how
devops
pushes us as a development teams
we're more likely to do
frequent more frequent commits smaller
sections of code much more
defined and then that makes it easy to
reverse you know easy to pull it back
out and also easier to debug
and that leads to shorter cycle times
from requirements all the way up to live
software
because much like we've talked about
when we talk about agile and
sprints
and ideally you're releasing software at
the end of every sprint or you know
every other sprinter every third sprint
or whatever it is
but there's regular deployment there's
regular software made available to users
ci cd
it's an even shorter cycle time
so
you may see you may have requirements
and you may see live software that's you
know some pieces of functionality that
are showing up you know within days
it gives us
devops gives us silos for development
and operations and really
highlights that those exist when we
talked way back when we talked about the
waterfall
approach the waterfall methodology
that was one of the things is you had
these silos for
testing for development
when you got to deployment it was a
little murky because it depended on the
size of software depending on the
environment and a lot of times you would
have the developers or the testers would
also be
essentially the operations piece
or
in some cases operations wasn't even
considered until after the fact so you
would go build software put it out there
to users and then say oh crud i gotta
have a i'm gonna have some sort of
support desk or something like that
and it wasn't
it wasn't really considered it wasn't
definitely was not part of development
it was just sort of this other thing
that was like a attack on add-on thing
we do after the software is done
and
what devops gives us is highlighting
that these are the roles that we need
this is uh these are skill sets that we
need this is experience that we need in
these various areas we we're comfortable
with yeah we need developers but there's
also operations that needs to be there
and we need to
account for that
with the
with some of the things that devops
gives us the ability to move
quickly from
like a development environment to a
staging environment or testing
environment production
we have more
realistic testing opportunities
instead of
developers just sort of creating their
own little uh development sandbox and
they're playing around and doing stuff
and their code's not necessarily all
together and you know up to date with
everybody else's
and their test data may be completely
wonky
now you've got these environments that
you can actually
make them more
real for lack of a better term
but put actual maybe something that is
much closer to mimicking
the
what the production environment is going
to look like as far as like including
size of data if you're a developer and
you've got to enter in data you're not
going to enter into more than a few
records because it just takes too much
time
but if you've got a team
that's either you know maybe the
operation side is has some data load
scripts or
you just have a lot of people working on
the system you're going to get more data
and it's going to be different from what
you would do as a developer and so you
have better testing opportunities you
get to test more against something that
that looks real when you're doing it
using devops
a key thing to hold this is that then
starts it it gets us closer to solving
the it works on my machine problem which
you see a lot of times with developers
where they they write their code they do
a build that works for them they can
show you it works for them they can show
you exactly how it it is but then as
soon as you put it to somebody else's
machine as soon as you put it to the
test environment it breaks
and because
without devops
we don't necessarily know what the
differences are between the development
chain and the qa machine and
development in particular because it
developers are famous for having a whole
bunch of stuff on their
uh on their machine in their environment
that is not going to be there even when
you go to the test or the stage
environment
and those things can
you know those can impact whether
something works or not
so if you don't really if you don't know
what you have if you can't define
in detail the differences in the
environments it's that much harder to
figure out
why it doesn't work
the flip side if you've got
you know like
go to the nicest thing if you've got a
script that builds your
development environment then that same
script should be able to point to your
qa environment and build that and so you
know that the environments are
for all intents and purposes identical
and that way if it works on the
development machine
then it needs to work you know you know
it's going to work on the qa machine or
vice versa or at the very least
you know that it's not a difference in
the machine in the in the environment
itself
or at least you've got a better starting
point i guess
so let's talk about some of the
practices and methods i've actually
touched on some of these already
but i think it's good to you know walk
through them and and this may have even
been one of those things that's sort of
out of place in the presentation but
i think there were it helps to
give you an idea of what the value is of
devops a little bit and at a high level
before we get into some of the
specifics
so
some
specific things that you're going to
find in
devops one is build automation i've
talked about this already uh which is
basically combined with infrastructure
as code
is devops is going to automate as many
things and script out as many things as
possible
when you think of you know if you want
to do a comparison think about testing
is that there are manual testers
and there's automated testing and
as you grow systems you want more and
more to be automated testing because you
just
it's not feasible
to do
manual testing with the uh the frequency
and the size that you want
devops is the same thing
when you get into operations where
you're building out
dozens maybe hundreds maybe thousands of
machines or devices or deploying out to
them
[Music]
you're dealing with you know maybe
as many projects
millions of lines of code and all these
different complex build strategies and
integrations and things like that
you want to automate it it's not
something where you want somebody you
know they have to run this script and
they have to type this thing in they
have to build this and do this other
thing
you want that
done as a script
and there are a lot of tools that help
us do this
uh you can look at hudson hudson and
jenkins and things like that that have
been around for a while that
provide us these ci cd tools continuous
integration and employment ways to
commit software commit code changes
and then have a process somewhere that's
watching that says hey i have a commit
i'm gonna do a build whatever that looks
like i'm gonna you know check out the
latest code
if i if there's merges or something
maybe it does it maybe it kicks back to
somebody that says hey this requires a
manual merge
if it's gonna do the build it's going to
run through some sort of test
to say yep the build was successful we
built what we wanted to and it roughly
works
and then push it out to you know deploy
it out to some target whether that's uh
it could be a development machine it
could be
qa or even production
you