Disaster Recovery Plan: Protecting Your Business from Unexpected Threats

Welcome to Building Better Developers, the Develop-a-Nor podcast, where we work on getting better step by step, professionally and personally. Let's get started. Well, hello and welcome back. We are continuing our season of Building Better Businesses, but we are actually Building Better Developers. Welcome to the Develop-a-Nor podcast. And this episode, When Disaster Strikes, How Are You Going to Recover? That's where we're going with this one. But first, I'm going to introduce myself. My name is Rob Broadhead. I am one of the founders of Develop-a-Nor, Building Better Developers. Also a founder of RB Consulting, where we, way before the disaster strike, sometimes actually the disaster, it feels like a disaster because you have this technology sprawl and all of the pain and the sweat, blood, sweat and tears that went into it. And you've got all this nice stuff and you've spent all this money and you've invested in technology or you're thinking about investing in technology and it's just a little too much. And we sit down with you and we walk through what does your business do? What is your secret sauce? What is the special recipe that you have for your customers? And then we craft a special recipe of technology for you to serve your customers and your business so that you can go out there and actually do the things you want to do. We do this through simplification, automation, integration, even innovation. We may even help you build a custom solution for you that is not going to take the billions of dollars that you may think it does. And we can get it down to something that's going to work with your budget, your time and help you grow so that this is going to be something that will work for you today, tomorrow, six months from now, 10 years from now. Well, maybe not 10, but we're going to get it stretched as far out as we can. Good thing, bad thing. I have like the quintessential good thing, bad thing this week of like, I went and got a house on the market. A couple of days later, got a cash offer. Awesome. Surprisingly, like the whole thing was better than we thought, faster than we expected. The bad side of that is they want us out in two weeks. As we close in two weeks, so it is just like, just busier than a one-legged man in an ass kicking contest, as I used to say. So it is a good and a bad if there ever was one. But another good thing is that Michael is here with me so we can do yet another episode and he's going to introduce himself. Hey everyone, my name is Michael Mollasch. I'm one of the co-founders of DeveloperNURB, Building Better Developers, this season Building Better Businesses. I'm also the founder of Envision QA, where through quality assurance testing, we come in and walk you through your software sprawl. We figure out your problems and from a user's perspective, we walk through all your processes, all those little nuances that you do with your current software or your current systems and help you identify what it is that you have. Is it working for you? Do you need something custom or do you just need to go buy a tool like maybe Microsoft Office or some other type of profiling tool? We help you with these things. We help make technology work for you, not you work for technology. Good and bad. Good, getting a lot done. It's getting warmer. Spring is almost here, getting the yard ready for the springtime flowers gardening. Bad side, the pollen is here. My allergies are already starting to get bad and my car is starting to turn yellow. So that's where I'm ready for spring to get here, the flowers to come out and be nice and be done. This is that time of year where I am miserable because it's like popping allergy pills every day. I know the feeling. I'm a Claritin bro, I guess in that sense or Allegra or all of the other Zyrtex and all of those. I have to cycle through them. Now, let's talk about disasters and more specifically disaster recovery. This is an area and I think we, I know even large businesses that I've worked with that their disaster recovery plan is very minimal. Now it has definitely gotten better in the last 10 to 20 years, particularly with cloud services and software as a service and some of those things. We are more likely to have just offset, just sort of push off to somebody else the concerns about disaster recovery. Because now if we've got everything in the cloud, we don't have to worry about our data center getting struck or worse, what used to be somebody spills coffee on the server back in the closet that we call a server room. But there used to be things like offsite backups and all these things we did that now it just sort of comes with software as a service. It comes with cloud services. But there still needs to be an intentionality to that. And that's what I talk a little bit about in disaster recovery. Now first, it is the most basics of these is that you have whether you're Windows or Apple or whatever your primary system is, whatever your main provider, if it's Google, if you're like a Google Docs kind of company and use Google Sheets and all that and Google Drive or where they use like a maybe like Dropbox for showing a lot of stuff or maybe use Windows in their Windows environment or their Azure environment, those kinds of things. You still need to make sure because a lot of these will back stuff up to the cloud. And you want to make sure that you've got like the right level of membership or subscription or whatever it is to the services so that your stuff can, you can put your stuff out there that you need. But then you also need to make sure that your intention about where you put it so that it does get backed up and put out on the server on the regular basis. Things like Google Drive, Dropbox and Microsoft's that I just lost the name of it off the top of my head. Those services, they start with a specific folder basically. And the things that are in those folders just automatically get sucked up and synced and it makes your life a lot easier. However, you need to make sure you do this, that you take advantage of this, that you store stuff on there. Now a lot of like the nice thing and I'm normally not a huge Windows fan, but one of the nice things about the Windows stuff is that it will just your folder, your desktop and that it will pull that stuff in by default. It will just store that off out on your your OneDrive and everything's good. But you do need to be cognizant about that and what you put there so that you're making sure that stuff gets backed up. But also there's a lot of things that we do that are not particularly a part of that that are not our normal documents. And so one of the things I think you should consider is on a regular basis and we've talked about that just along the same lines of making sure that you've got your licenses up to date and things like that. So make sure that you have a backup of your software, of the applications that you use, especially these days because a lot of what we get is digital. We go download it, we do whatever we need to. So we need to make sure that if something happens to that desktop, that laptop, your phone, that if you go to another one, that you have all of the things in place so you know where to like download the software you use. You need you know where to, you know how to reconnect to things. How do you log in? How do you have the right IDs? And this is not just you. This is for your business. So this is where you put things in place. You have processes and procedures. Even if you're a very small business to say, okay, this is where we put our business documents. This is how we store them. This is how often we do backups. And I would do them even outside of those software as a services because you never know where like Amazon's going to like there have been Amazon's gone down, Microsoft's gone down, Dropbox has gone down. These things have gone down. And so you want to be able to access your stuff if you need to. This can be as simple as having like a nice little thumb drive or an external drive because these USB drives these days are, you know, it's maybe for a couple hundred bucks, you can have like four or five terabytes of data that you can store. I've gotten so I've got one that's just uses Microsoft or Microsoft Apple's time machine. I just put all the stuff out there so I can have a backup all the time on this thing and I can take it wherever I want. I don't care. And it's always there with me and I've got all kinds of data on it. But what you want to do is think through what is the worst case to get started on this. Like what's the worst that can happen? What you know, it depends on how you work, but it's like what if I don't have internet connection? What if my desktop gets destroyed or my laptop gets destroyed? How do I continue serving the business that I need to? Now we're going to talk a little bit in the next episode more about like business continuity and some things during this. But disaster recovery is really more about if I were like just maybe it's as simple as if I unplug all of my servers and devices and I plug them back in, will they come back up and everything connect as it needs to be? And it's things like doing failover. So if I've got a somewhere I've got a software as a service or I've got a let's say I've got a database that I use for my business. If that site goes down, if that location goes down, if I've got let's say I'm using Dropbox or Amazon, Dropbox like they hide you from a little bit, but especially if you're using Amazon or Azure or Google Cloud and those things, they're going to have regions where your servers exist. And so this is where you think on the bigger side on disaster recovery. It's like what if the East Coast of the US gets wiped out in storms or something like that and it goes down? How are my servers still going to be valid? Can I move my business and everything to the West Coast, for example? Now these may seem too rare to take to think through, but it only takes one. If it happens once every five years, but during that five years you have a down period where your business disappears effectively for five days. What does that do to you? That may not be a big thing if you're side hustle, but as you get further along, what does that do to you? And particularly it's things like the things that I think are most important with these is like what happens to payroll? What if I can't access? Will payroll still work? Will my people get paid? Will I get paid? Will money flow? The things that are the heart and soul of your business, are those going to be able to still recover and move forward even though there's been a nuclear war or a meteor hit or something like that? In some cases the act of God kinds of things that go on. What happens there and how do I go back and restore where I was? And this is actually just one last thing. I'm going to throw it to Michael because I've talked too long on this at this point. But the other thing is with the extra lens of there are a lot of cyber attacks that go on where people get locked out of their systems. And so those are probably even more likely than these act of God things that I've mentioned. But they are the same kind of thing. What if somebody hacks your system, gets a hold of everything? How do you go back to a point before you were hacked and restart your business basically and not have to pay whatever their fines are? And I'm going to not make you pay a fine though and throw this over to Michael for a little bit and see what your thoughts are on some of these and what are some of your experiences with disaster recovery? So we worked at a company years ago. Before you came on board, one of the things that first kind of turned me on to disaster recovery was the fact that the company's server room was destroyed in a fire because the server room was next door to a dry cleaners or to a restaurant that had a kitchen fire and literally wiped out our server room. And it took them months to recover because their backups weren't anywhere else. It was just there at that one particular location. So the whole idea of disaster recovery preparedness is basically to make sure that you have a way to stand up your business again in case something happens. And banks have been doing this for years. Hospitals have been doing this for years pre-cloud and a lot of them still do this. But what they do is they back up their systems once a day, once a week. They're critical information daily. Major scale backups weekly, monthly. It's probably a more aggressive backup. And what they do with those is they take those offsite. So they actually go put them in lock boxes, secure places that can actually handle extreme weather, fire, flooding. So if anything happens to the business, they can go offsite, take those tapes, rebuild the server room or whatever they have to do, pop it back in and they've got their data back. Now they may be down for a couple of days, but at least they are able to bring things back up. Now over the years, as Rob mentioned, we now have cloud services. We have better external backups through USB drives, external hard drives. So we don't have to necessarily rely on the slow tape backups anymore. We could even have remote hostings. So you could go with cloud services or you could even go with offsite storage, essentially have your own virtual cloud, get your own, like hire your own service rooms, go buy a rack at one offsite server and just have your backups go there. They're still secure. They're not in the cloud. And this is especially important for those of you that have critical information like PII, banking information, financial information, those things you kind of want to keep secure and a lot of cloud services aren't HIPAA or SOX compliant. A lot of them are, but you have to pay for it. So you have options. These are just some of the things you can do. One of the things I like to do is I keep an offshore backup because I've got two locations. And what I also do is locally, like Rob mentioned, he uses like time machine backups and backs up his machines to external drives. I also there the external or the home business options for backups have gotten better. You can actually now buy a personal NAS for about three hundred dollars and that gives you up to 10 terabytes of backup, RAID one, where you can have a perfect image. And if one of the drives fail, you just pop it up, take the drive out, pop another one in. Boom, you're good to go. Now, I still back that up once a month and I take that offsite, but it's right here. It's not in the cloud. All my stuff's private. It's protected. Now, if both places get destroyed, I do have some virtual information for critical business infrastructure needs in the cloud in Dropbox. So what you want to do is you kind of want to look at what you have and break it out, diversify, put it in places where it makes sense. So critical business information. So this was something that I never really thought of. But like Rob mentioned, what if the banks go down? You know, where's your money? Well, this actually happened to me recently where my account got hit for fraud by my own bank doing something with my bank. I was buying a service through them to set something up and they locked me out for a week. I was panicked because I physically had to go into location. The banker tried to work it through it with me. And here I am two months later, almost three months later, and I find out it's still not done. My account is still got certain lockdowns in it. So I'm now looking at how can I disaster recover my bank? So I need to diversify my banks even to have certain critical information at one bank, other financial information at another bank. So if one goes down, I still have access to money somewhere else. So that's just one thing that a lot of people don't think about. The other thing, and this was actually rather interesting, probably about a decade ago, my mother had a house fire. My mom was self-employed, running flea markets. All her tax information and personal information went up in the fire. All of it was gone. We had to go to the accountant and the accountant had to kind of backdate all her notes to put together a tax history. So your financial information, all your files for your business, keep paper copies, but also scan those copies as soon as you get them into some type of cloud environment or digital environment and get those somewhere safe. Be it a lockbox at a bank or somewhere else, but make sure that you keep those separate. You can also do that for your home stuff, too. I recommend that because, you know, if your social security guard goes up, it's a pain in the ass to get another one. So little things like this you don't think about daily, but these are things that if disaster strikes, you are going to be in a whole lot of pain trying to get your life back together. So decrease that now by being prepared, by getting things together. That's the whole point of this disaster recovery. And one last thing I'll note is Rob mentioned, you know, backup those online systems like websites, you know, any services you have. A customer of mine just recently had their website hacked, and even though they were doing daily backups, the backups had exceeded the period from when the site was hacked to the backup. So all they had were bad backups. So I recommend if you do stand up any type of software, a mobile app, be it a website, whatever software is perfect for this. The moment you go live, have a snapshot, a backup, put it in a container, whatever, but then store that somewhere else and say, do not touch, get it off your system, get it where no one can touch it and put it where you will always find it. So if disaster strikes, you can go pull that out, plug it in, restore your site within hours, even minutes, and be back online and not have your business be down. There's a couple of key things there that I think we should, people miss out is that you do need to test your disaster recovery. What Michael says is a perfect example is that periodically you need to make sure that you do have backup so that your backup process is not an issue. I may have shared this before, but this actually is something that goes way, way back. As I had a, this back when I was in college, I would back up my senior project, backed it up religiously, like I think daily. The problem I had was that the drive, and this was back in the world of like hard disks and floppy disks and stuff like that, the drive I had was slowly failing. So every time it wrote to a disk, it was at a slightly different speed, a slightly different way that it did it so that it was literally not readable by any device within about a week. And so I was sitting there backing all this stuff up and I could always like back it up and I could go look at it and see that, oh, the files are right there. But even on the same machine a week later, the drive that had written it was now moved as like degraded to it could read it anymore either. So I had all these backups and I thought I was doing great, but I didn't test it often enough. I didn't use another device often enough. I ended up losing everything. The only thing we had was we had actually had hard printouts of our code and had to from those recode the entire solution. And it just happened to be that like it was fortunate that we had that kind of a backup. Somewhere since there was a couple of years ago that there were a whole bunch of companies that were, well, actually, and I can't think of the name of them right now. Oh, I think it's Parler was a social site. They went out there and they did their thing and they have this great growing community and they were locked into a single vendor. They worked with and I'll just throw all the names out there. You can find them, but they worked with Amazon. They used Amazon cloud services and they breached the service contract with Amazon. Something they did breach the service contract and so Amazon shut them down. This is a, I guess at that point, multimillion or more, maybe hundreds of millions company that now is shut down and they have no access to their stuff. Their customers have no access to them. And this is where not only do you want to test your disaster recovery, but you want to make sure that you do this on a regular basis, that you have multiple disaster recovery sites. Don't have just one. Like Michael says, you have maybe something local, then you have something off site, and you have something even further out. Now, how you do that may be, there's a cost effectiveness to it, but verify that your disaster recovery plan works. Like I said, the way that you do this sometimes, a lot of the big businesses do it. If they've got a data center and they've got a disaster recovery plan, they will test it by, they will go in there and they will just turn the switch off on the data center and say, okay, now what happens? Does everything fail over properly? All that kind of stuff. Does the business then sort of like spin back up a second later or a minute later or an hour later, whatever it is, to recover from that disaster? And so you need to think about how that could work, what that would look like to you, and create a test plan and test your disaster recovery. I would say at least once a year, a lot of times, like every six months is a pretty good way to do it because you want to make sure before it happens that you know how to recover from it, that you know how to ensure that you haven't lost the things that you've lost and be properly prepared. All of this being what it is, I'm going to throw out our standard. Go ahead, if you have questions, if you have comments, if you have suggestions, if you have war stories from disasters that have hit you, shoot us an email at info at developerneur.com. And also reach out to us on the developerneur.com site. You can leave us a contact form. But before I go any further, I'm going to throw it back to Michael for a second. I think you have something to add. You forget the challenge. Did I forget the challenge? Yes, I did. But see, this is a good example of disaster recovery. My brain just shut down, but I have a backup and I just tested it. So there you go. I could say that that was planned, but I would be lying if I did. So that is, though, what the challenge is. Actually, it's a two-parter. It is create a disaster recovery plan if you don't have one. It's like, what does it look like? What do I do? Do your backups, whatever you need to do, and then test it. As part of that, to complete this challenge, to complete this task, is test your disaster recovery. For example, a lot of times I'm very much driven by one... I've got two machines, but I've got one particular laptop. Very often, that is where I go to do all of my business. And so the way for me to do it is I just put that laptop away and I go try to... I'll have another laptop or I'll sit on desktop or I'll do something to say, okay, can I do the critical things? So I don't have to do... That's really nice because I pretend the laptop doesn't exist and when I'm done with my disaster recovery test, wow! I pretend that it exists again and it's right there. Things like that can make it very easy, especially as a side hustle kind of business. And as you're growing, that is where you want to go. As you want to start from the very start being able to do this, that's just going to allow you to build the processes, the procedures, and all the steps that will make that work even as your business grows. Now, we will wrap up because we have challenged you. You can also leave us feedback wherever you get your podcast. You can go out to developernor.com channel on YouTube. If you want to check us out there, leave us comments. Lots of content out there. Lots of content on the Developer Norse site. If you've struggled to develop on our site, we are in talks amongst ourselves basically of doing some upgrades there. It's been around for a while. We're going to do some server changes and such. So it also might be down a little bit here and there, but I'm trying to get that... It's like it's one of those things that you like, your server is starting to get older and older. Those squirrels that are running around or the hamsters running around their wheels are getting a little old, a little slow. So do a couple updates and things like that and see what we can do to bring it up into at least this century. That being said, go out there and have yourself a great day, a great week, and we will talk to you next time. Thank you for listening to Building Better Developers, the Developer Nor podcast. You can subscribe on Apple Podcasts, Stitcher, Amazon, anywhere that you can find podcasts. We are there. And remember, just a little bit of effort every day ends up adding into great momentum and great success.