Dealing with Legacy Code - When to Let Go and Start Anew

Welcome to Building Better Developers, the Developer podcast, where we work on getting better step by step professionally and personally. Let's get started. Hello and welcome back. We are continuing our discussions. Last time we got into some stuff that was more like living a better life. This time it's more about saving customers from certain death. It may not be that bad, but it feels like it. If you have ever been in that situation, and the situation we're going to talk about today is legacy code. When does it need to be buried in the pyramids in Egypt? It's like, when has it gone on too long? More importantly, when has it been too long since you touched that legacy code? When did you let stuff run too long and now it needs to be replaced? First off, because I don't want to be amiss about this, I want to introduce myself. My name is Rob Brodhead. I am one of the founders of Develop-a-Nor slash Building Better Developers. On the other side is Michael. Go ahead and introduce yourself. Hey everyone. My name is Michael Molasse. I'm also one of the co-founders of Develop-a-Nor. We got into this, for those of you, and I was totally remiss the last episode I forgot. I didn't give any of the cool links. Go back to that episode and just click on all the links and all that kind of stuff. You normally would like our email address and all that. Hopefully, I'll remember this time. Those of you that are watching this, which if you're not, we also have a YouTube channel. You can go check that out. Check that out. Develop-a-Nor, we have all this stuff getting posted twice a week, just like we do the podcast. We just have bonus material. Okay, you do have to look at our horrible faces, but other than that, you have bonus material. We were talking before this that Michael hit a spot right now with a customer that really triggered me in a sense because I've lived this same thing. It's this customer that they've got a product. It's not a bad product. It works. They like it. All they really need to do is update it. I will start with an experience I had and then throw it over to Michael. I ran into this same thing. It's not been a couple of years. I'll get a little technical. I apologize. I won't get too technical, so you won't need to be resuscitated at some point. They had a really cool custom application that they had built. It was built on the Eclipse foundation, the RCF, and all of its stuff that was there. When it was built, it even had Adobe's flash. It had some flash in there. It had a whole bunch of stuff, and it was really nice. It worked. The downside is when we touched it, it was in 2021, 22, the last time a developer had touched the code was in 2014. They had some developers that were in. They were consultants. They had built this thing. They'd made a couple of fixes and stuff like that. They put it on some machines, installed it on the machines, rode off into the sunset. Nobody knows what happened to them, basically. They didn't need them because everything worked. It all worked. Fine. Everything was awesome. The problem is, and actually I've seen now a couple of customers that have had something like this where it works long enough, the underlying technology starts to disappear. For example, some of the stuff that they had machines that were running it on that had to upgrade. When they did the upgrade, they could not run the software anymore. They slowly went from, I think originally they had like eight or 10 machines, down to seven or eight, two or three, and they were down to one machine when I was introduced to this. They said they did have source code, which basically they had everything that had been dumped on a machine, and they had one working machine that was not virtual or anything. It was a machine that if somebody smells coffee on it, game over. They said, hey, we would like you to upgrade this because it was way back, the Java versions it was on, the stuff was on was not working. Things were starting to fail and they knew it and they needed to get modern enough to just keep it going. They literally wanted almost, and I think I can't remember, zero or very close to zero changes. The only thing was they knew right away when we got into our first little, I think we did like a intro project. We did 10 or 20 hours, looked at their stuff, said, let's assess it and see what you got. Right away, one of the things I said is like, all of this flash stuff, that stuff doesn't even exist anymore. Basically, we can't find stuff that's going to, we're not going to be able to take modern stuff to go connect to that old technology. I said, if we do, there's like a whole, you could do it, but it's a whole bunch of crap you're going to have to go through that you'll have to break the app anyways. And they said, that's okay. That stuff we didn't really need. It's like, okay, cool. We'll cut that out. I looked at the rest and I was like, okay, it's going to take a while. We're going to have to get this thing built. We're going to have to, and it's on a, you know, almost 10 year old version of Eclipse, basically, we have the base foundation of it. And so we've got to upgrade that. We're going to have to upgrade stuff. And basically just said, we'll give it a shot. Let's see what we can do. Maybe we'll luck out, but I know that there are watershed releases that we're going to have to do some code changes and we're going to have to like, switch out some libraries and things like that. Long story short, I know too late, but long story short, we spent, I think we spent probably close to six months going through this, trying to upgrade it, trying to as much as possible, like just shepherd this thing into that next version. And we ended up getting into and doing this and all the different libraries that had to be updated and managed and stuff like that. We basically got into something where we couldn't upgrade without rewriting the core JDBC stuff that it was using. The core database connection stuff it was using did not exist or was not compatible. Essentially, it had stopped being worked on and had been renamed and redone to a point where we were going to have to rebuild huge pieces of it. Large portions of code was going to have to be recoded at which point, which is where we finally get to the point for this episode, is got to a point where it's like, okay, you're going to have to rewrite it. As in, yes, you can try to build that same thing again, but the effort, the time, the cost, the risk is we're going to have to rewrite it. And I had several conversations with them and their developers that they had and said, you have this thing that you cannot support. You don't have anybody. You don't have anybody that knows it. You don't have anybody that knows any piece of it. You have developers and resources that can do something modern and they have lots of ideas of how they wanted to rebuild it. And basically what we came to, I said, we'll help you if you want. We can rebuild it. We can redesign it and do what you want to, but it needs to be. It has to die. You cannot move that forward. We can move it forward on that platform, but nobody does that anymore. Anybody that's doing what you wanted to do today would do, we gave them like a half dozen. These are the different ways they would do it. That was great at the time. And this is where we had to really convince them, not convince them, but agree with them and say, what you have is great. You've got an awesome tool. It was an awesome tool 10 years ago and now it is not. It's like your horse and buggy rock. But that was 200 years ago and now horse and buggies are no where, they're useless on a road. So time to move on to a bicycle or whatever it is. So that was my experience was that we really got in this situation where it was, and we may have gone too far in trying to keep their stuff. And I think that's where the discussion becomes because we really did get to a point where while we were going through it and we were, I was warning them along the way on a regular basis saying this is costly. This is taking a lot of work. And this means just because of the amount of stuff we're having to touch, you're going to have to go through the testing again. You're going to have to spend time on it. It's probably going to break stuff that hasn't been broken before. We're not even really, at this point we were like, we're not even learning the code. We're just learning the background and the libraries and making sure those all sync. And this is becoming more of an issue as we have these grander applications like going to React and Node and all this stuff where there's all these little libraries that have to go inside and you're going to run into those situations yourself, I think, where you look at it and say, yes, we can find a way maybe to make this work and to limp it forward or to upgrade it or to maintain it. Or it's either too old or it could be just too poorly written. So the cost of building something new, building it from scratch or roughly building it from scratch is equal to or less than going with what you had. Bonus is that when you come to the, if you've built from scratch or you restart, now you have a new modern solution. And I've done this several times for customers where it's not only that, but now we have unit tests involved with it. We have documentation. You have people you can get a hold of that can help other developers. If you want to move on to another team or you can go to our team and we can support it and enhance it. So that's it in a very big nutshell. How does that sort of apply to and where does that have you thinking about based on your current customer and share however many details you feel like makes sense to do it with that as well? So it's an interesting conundrum. So you and I have been around software since pre-Windows days. I mean, we had to deal with DOS old Epsodic or old VT-100 machines back in the day. I feel old saying that. But I mean, not even the day. I mean, we're talking like the 80s, the 90s when people built software, it was more of a waterfall approach. They started with an idea and then they come out this big monolithic application that gets released and they go through a couple of tweaks and everything gets stable. It's released. It's done. There's no more software, right? Because back then we didn't have the internet. So you once you release software out into the wild, the software is out in the wild. People bought the software, they own it, they ran it. If the software was great, they ran it. That's what they use to run their business. My particular client is in the healthcare sector and they're working on an IBM, well, what essentially is an IBM iSeries, but it's actually an older AS400. It is an old green screen Epsodic machine that IBM put out years ago for things like healthcare and they're a beast. They're really good at what they do. They protect your patient information. You just go in, it's a bunch of green screen, bunch of little menu commands. It's not point and click. It's all keyboard driven. You print, it works. Problem being monolithic softwares or software these days that gets packaged and delivered. Today we tend to hear things like software as a service, continuous deployments, continuous integration. That stifly stems from the fact that that is one, keeping your software current. It's keeping it up to date, constantly addressing bugs over and over. So there's always someone working in the system. The system's always been tweaked. It's almost always staying somewhat current or at least able to run on the current operating Another project I worked on similar to this wasn't so much a 400 application, but it was actually a Windows 95 slash DOS application. So if you worked on back when they had Windows 95, even Windows 98 still supported DOS. If you wrote a DOS based application, you could still run it on DOS, Windows 31, Windows 95, Windows 98. After Windows 98, your software was dead in the water. You had to essentially go back, rebuild the software in the current technologies for Windows 2000 and up because it was all the NT architecture and TFS, not FAT 13 or FAT 32. So you had to go through this process. The problem is it doesn't make sense to try to convert some of those applications because who wants write in basic anymore? I mean, you literally are going to have to dig, look for that needle in the haystack because there's that few developers that do that. Now granted, DOS is now an open source. First, it's now in the public domain and hey, we have people playing around with it again, but truthfully, will it ever be a business level application operating system? Probably not. So you're going to run into these problems and especially with the way technology is going and systems are being built, we have to keep up with this. We have to look at changing this. The problem you run into is if you have someone that's been working with the piece of software for more than five, maybe 10 years, they purchased it from a company, you run into situations where the company is not in business anymore. You can't get updates to the software anymore because of time, technology changes like Rob said or heck, it's a mobile app and the mobile devices are now all 64 bit and your application was compiled in 32 bit. You can't port over. You have to get this source code, recompile it and hope it works for the 64 bit operating system. But there you go. Oh, you may not even have the source code. You might be, what is it, Interplay or Wizard of the Coast and we can't get Ice One Dale 2 anymore because they lost the source code. They were able to do all the other enhanced additions because they found all the code on a computer except Ice One Dale 2. So as far as lessons learned, we constantly have to watch and keep up with current technology, but we do need to look at and figure out how to help our customers move along. In this particular case, I've been doing the assessment now for about two weeks and we thought that they had a current enough AS400 release build that we could port them up except we found out there are two versions back from where IBM made a change that they have to now recompile the source to change something for the new operating system levels. If they don't have the source, we're dead. We can't go forward. So now we have to figure out how to get the data out of the system so that maybe we can do something else. I think that's where you sort of draw a line with customers and that's where with the conversations I've had, there's the two sides of that. Can we get our... If we build... When you get them to even the idea of, okay, right, we build a new thing. We build it from scratch or something. We don't use what we have anymore. It's almost always, can I get my data out? Or at least having that conversation of does that data need to come forward? Because that can be a very big thing. Moving that data forward, particularly if you want to fix it, if you want to modernize the design of the solution, which often is what you want to do. Because usually the thing was built and designed for a very different system from what we have today. And so you have different concerns, different integrations, different user experience, all of the stuff around that. And different ways even to deliver reports and things of that nature. So you do have to think about it is what happens to the data, to the history of having that application there. It's like if you want to do it on a very simple, think about it. Like if you played a game back in the day and you had saves for that game, can you move those saves to your new game? Logical question, people ask it all the time. It means a lot more to quote move your saves from an old system if you're a multi-billion dollar business. Because you need to take that data, that customer data, that sales data, all of that stuff, most likely you need to bring forward. And so that needs to be part of the discussion is what happens? It's sort of like a two for there as well. What happens if that system dies? What happens if that machine, if you can't get into that system anymore through the application? Do you have a way to get to the data other than through the application? That may be a red flag that if you don't, you need to look very quickly at how are we going to get the data out of that application? It may be virtually impossible. It may be something where they have to go in and manually are going to have to go to every page and have somebody type it all in somewhere else. I've seen some really nasty, we'll call them data migration processes like that, where it's literally somebody is sitting there and they bring up a screen and then they swivel over to another one and they just are entering the data back and forth. They're literally hand moving the data. That can be pretty nasty. And that's where you want you don't want your customer to get to that point. Sometimes it is the solution is just as much. We're going to keep you limping along. But the first thing we're going to do is we're going to find a way to connect hoses and integrations to your system to pull that data out so it doesn't get lost. Even if we put it in just some random data store that we can, as long as we can get it back out, that's key. And then we can always come back later and we can build a new front end or whatever we need to do. So I think it is part of your conversation needs to be not only the cost of maintaining what they've got, the risk of losing what they have, including their data. And then what's what is that going to cost them? And then if we build a new system, does it have to support that legacy system, the legacy data in some way, form or fashion? And that may include, and I've seen running to these where there's it has to have the old user experience, basically, because of for whatever reason, that is the only way that you can work with the data the way it is. But now in the new system, they bought the sort of bit the bullet and said, all right, that's our legacy experience. We have to maintain it. But we're going to embrace all of the things we can do and make improvements. So now you may have, I've seen it where you have customers before 2020 has a whole different experience than customers that have come in since then, because they said, well, we're just going to start pulling new data. All the new customer data, sales data is going to come in through this new system. And we're just going to have maybe even as bad as just a screen scraper, which may be something that you guys run into to go hit the old system, which is some way to start moving that data and eventually hopefully automate it. So you don't literally have to have somebody sitting there and keying it from one machine to another. Some of the things I've thought. So more thoughts on that? Yeah, it's interesting you pointed that out, because some of the things that have come up recently is I uncovered that there's a couple of integration points with this system, with applications that are also no longer supported or around anymore. And then there's another piece within healthcare, you have to essentially, you know, you have to bill your customers while they have an exchange to go through and that they have a file that goes to the exchange that does all that. And then they manually enter that back into the system when they get the information back. The problem is they don't know what that is. And I have not found clear documentation yet as to what that is. I suspect I know what it is. But some of the verbiage that we've used from modern medical systems did not ring a bell. So that was a warning flag, whereas it's like, OK, we need to figure out what to do quickly. I knew this was urgent, which is why I took the project to help them. But in the same sense, I'm glad we went this approach because they really did not understand what they had or what they needed to get from A to B. They just knew that, you know, someone spills coffee on this, we're dead in the water because they only have one machine and there is no replacement for it. And that I really hate having to have part of those. I love those projects and hate those projects. And what I hate about them is the ones where it's you get into the the initial discussions about the project and they can't tell you anything about it. They can do it. Here's some code. OK, great. And sometimes here's all of the code. Great. If I want to look through a million lines of code, I'm not going to be able I mean, I can get a gist of how like, it's sort of this and it's sort of that. I can look at the data, the database model and say, OK, it's sort of this and sort of that. But I have to have some context even there. And usually it's going to take you think about if you're going to read if it's read the manual and somebody gives you a 10,000 page manual, it's going to take a long time to go through that. And that's roughly what they're doing when you get just piles and piles of code. You're talking about it takes a long time to do. And yeah, you can sort of like look through it and you get an idea and you can you can get a feel for it's good or it's bad or it's really confusing or yeah, it seems to be pretty good. But you're not digging for skeletons at that point. You're just like, you know, blowing across large portions of systems. You're sort of looking at stuff. Yeah, this makes sense. But you're not, you know, walking through it line by line by any means. And so those are the hard projects because you come in and you say, all right, well, you don't know what you've got. So we're going to spend, you know, it's basically you you pick a point and you say, this is what it's going to be. We're going to spend X amount of time and we're going to we're going to do some research and we're going to give you back what we've got in that amount of time. And it's like I said, it's really tough when you get into that and you get done with it and you go, now what we need is another block of time. And we're going to have to dig deeper because we've got all of these red flags that have popped up and we have to actually start looking deeper into them. We thought we may be able to get there, but here's the concerns. And if sometimes I've had a few where it's like, OK, first round, here's five concerns. Next round, here's 10 more concerns. By the time you're getting into it and very quickly, where it's the point of say, OK, you have a lot of issues. And again, that's where it I think that's where you have to have those conversations. And you say, OK, the good news is we're here and we have some paths forward. The bad news is, is none of the paths forward are going to be cheap, easy, quick, or maybe even include your existing system. You know, we're going to get data out of it the best we can, but it may be that thing cannot move forward. And I think that's part of what you want to do is that when you're doing those kinds of assessments, it's just like anything else is go hit the high risk stuff first. Look at the things that are critical or that would cripple the system if those exist. And it's things like, do you have the source code? Do you do you have documentation? Do you do you have a build process that you know that works? Sometimes it's do you have this on multiple machines? When was the last time you freshly installed it on a machine? Things like that, which like you shorten that down to it only got on a one machine. They know they're in trouble. They know that can't die. So they say, oh, well, big problem. Sirens are going off. We've got to do something quickly. And so that is leverage, at least then to say, OK, this is important. This is critical to your business. And in this case, you know, probably even more so because it's dealing with health care information and billing and all these kinds of things. So it's their business and probably their patients as well to at least a medium to a very great extent. So then it's like, all right, we got to get in here and we've got to figure out. Now it's like if they're in health care, it's basically that's like you've got somebody that's just hit the table in an emergency room. They're bleeding out. What do you do? And it gets into some of the other conversations we've had where there's like there may be a level of triage there and it may be something where it's like, OK, we're we're stable. The patient is stable right now, but they could blow up at any time. And so we need to make sure that we are prepared for that and as fast as possible, start doing the we'll call it the healing process of their system or the replacing process of their system so we can get them moving forward. And this is where it does become I've found this personal thing is I found that it is much easier to do that in like little bites. Sometimes it's you almost have to take the whole thing. But if you can find a way to do little bite size chunks so that you can show ready, steady progress, so you can start sucking data out of the existing system and start replacing functionality so people can use the new system, even while the old system exists, then I think you build some you build some trust, you build some confidence and for yourself, it helps you build knowledge about those systems. The new one, which you should know because you're building it in the old one, which you don't know squat about because you just got handed a mess. And I said the last place we worked together. That was exactly what I got. They sent me a hard drive, couldn't send it to anyone. They sent me a hard drive that had stuff in it. They said, here you go, go figure it out. That's how bad it can be. So parting thoughts. Yeah. So like Rob said, you know, it can be bad. But the thing is, don't always go in that it has to be rewrite. First go in at least the way I'm handling this or I've handled this before is I'm going in and assessing what the problem is. What is the overlying problem? Then I'm trying to understand the customer needs for this problem. You know, is it life threatening? Is it mission critical? Is it an, oh my God, moment. And then from there, walk back, you know, what is it that the business uses this for? How do they use this? And ultimately that's gotten me to where it is to the fact that, okay, they are in as much trouble as we suspected. Thankfully, it won't affect the patient so much in this situation because thankfully we determined that it's all their billing, all their insurance information, all that's on this box, all their schedule. Scheduling, if that were to go down, they could still use Microsoft calendar. It's the billing information that they're going to lose. And there's some more disconnects with that that I'm uncovering. But you do, you have to treat it like an onion. You got to peel the layers back. You got to slowly get into it and you may never get to the core. You may get to a certain point where it's rotted. It's like, oh, okay, now what do I do? Well, now you either have to go build or grow a new onion or replace it with something else. And I think that's a great stopping point for us at this point, partially because we are out of time. But I think this is one of those things that we do run into, it's especially now because there's been such an advancement of technology in the last five or ten years. And it feels like we've said that for a long time, but particularly in the last five years where remote and a lot of other things have become a lot more reality. The ability to communicate, the world has shrunk and all those kinds of things. And then there's been a lot of advancements in the internet area that things that didn't even exist, possibilities you had in the past that were never going to happen, they were dreams, now are very easy to do and very commonplace. I think this is something for us all to keep in mind is that, yeah, you don't want to fix something that's not broken, but you also want to make sure that you're properly maintaining things and updating them and keeping them current enough so that if there is a point where you have to make a big change, you're not having to essentially go back to the drawing board to rewrite it. By all means, have repositories and things like that where you're keeping track of your source code, document things, add comments, that kind of stuff so that you're helping your organization, your company, your customer out whenever somebody does come back and need to take a look at that application. If you have anything that you would like to throw at us as far as some of your experiences or if there's some applications you want us to look at, we can even do that. You can reach us at info at developerneur.com. You can check us out on the developerneur site out on Facebook. We've got stuff out on LinkedIn, the developerneur site. We've got forms. We've got blog posts. We've got the podcast. We've got the YouTube channel. We've got a lot of different ways you can reach us, a lot of different topics that we've covered. Always looking forward to any kind of information we get from you guys, feedback to just let us know where do we want to go? What do you guys want to hear? What do you like? What don't you like? If you don't like how we look, sorry, can't do anything about that. I guess we can cover our face or something like that. We'll use some filters because that's all getting good. We'll just have a little AI people that act for us and we'll just talk behind the scenes. All right. I digress. That being said, we're going to wrap this one up. We're not done with the season. We are still going to just continue talking our way through the things we're running into and the best ways that we can grow as developers and also help our customers be a blessing to them and to those that work around us. That being said, go out there and have yourself a great day, a great week, and we will talk to you next time. Thank you for listening to Building Better Developers, the Develop-a-Nor podcast. You can subscribe on Apple Podcasts, Stitcher, Amazon, anywhere that you can find podcasts. We are there. And remember, just a little bit of effort every day ends up adding into great momentum and great success.