{"id":12251,"date":"2019-08-27T16:10:48","date_gmt":"2019-08-27T13:10:48","guid":{"rendered":"https:\/\/mozaicworks.com\/?p=10837"},"modified":"2022-02-01T18:42:52","modified_gmt":"2022-02-01T16:42:52","slug":"a-new-method-for-fast-refactoring-of-legacy-code","status":"publish","type":"post","link":"https:\/\/mozaicworks.com\/software-engineering\/a-new-method-for-fast-refactoring-of-legacy-code","title":{"rendered":"A new method for fast refactoring of legacy code"},"content":{"rendered":"\n<p>In this article, I will present a method that I&#8217;ve tried in a few codebases in compiled languages for safely and quickly refactoring untested code. First, we will discuss the main problem we are trying to solve, quickly introduce the techniques coined by Michael Feathers, then discuss some shortcomings of the existing techniques, and finally describe the proposed technique.<\/p>\n\n\n\n<p>Briefly, the technique I&#8217;ve been experimenting with implies refactoring first towards pure functions using safe, mechanical refactoring steps, then testing the pure functions by quickly writing data-driven tests, property-based tests or by using the golden master technique, and finally refactoring the pure function towards the desired design. <\/p>\n\n\n\n<!--more-->\n\n\n\n<p>Please feel free to skip to the section &#8220;A New  Method&#8221; if you are familiar with the legacy code problem and techniques.  <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The Problem<\/h2>\n\n\n\n<p>Many software projects go through a cycle like the following:<\/p>\n\n\n\n<ul><li>In the beginning: &#8220;we&#8217;ll write the best code we can, and as fast as we can&#8221;<\/li><li>Few months later: &#8220;we need to ship now, let&#8217;s make things work&#8221;<\/li><li>A year or so later: &#8220;we can&#8217;t touch that part of code anymore because it might blow up&#8221;<\/li><li>Few years later: &#8220;it&#8217;s very slow to add more features because of the existing code&#8221;<\/li><\/ul>\n\n\n\n<p>You probably recognized the technical debt \/ legacy code problem. Once you&#8217;ve reached the final point, the options are limited. You can either <\/p>\n\n\n\n<ul><li>rewrite the project (extremely risky)<\/li><li>live with the low productivity (bad for business)<\/li><li>try to throw more people at the problem (and prepare to fail due to <a href=\"https:\/\/www.wikiwand.com\/en\/Brooks%27s_law\">Brooks&#8217;s law<\/a>)<\/li><li>refactor the existing code to allow faster development<\/li><\/ul>\n\n\n\n<p>But here&#8217;s the rub: refactoring involves changing code, and changing code can create even more problems. Also, given that you&#8217;ve written code that&#8217;s hard to change until now, what makes you think that you can suddenly create code that&#8217;s easier to change?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Existing Solutions<\/h2>\n\n\n\n<p>Fortunately, Michael Feathers did the heavy lifting for us and found ways to safely refactor existing code. If you don&#8217;t know how, go and read his book <a href=\"https:\/\/www.amazon.com\/Working-Effectively-Legacy-Michael-Feathers\/dp\/0131177052\">&#8220;Working Effectively with Legacy Code&#8221;<\/a>. Then practice the techniques, at a <a href=\"\/public-trainings-and-workshops\/working-fast-and-safe-with-existing-code-workshop\/\">workshop<\/a> or at a <a href=\"\/public-trainings-and-workshops\/internal-legacy-code-retreat\/\">Legacy Coderetreat<\/a>. Only then, maybe you can start trying to apply the techniques on your code.<\/p>\n\n\n\n<p>The basic technique goes as follows:<\/p>\n\n\n\n<ul><li>Pick a section of the code<\/li><li>Write characterization tests (i.e. tests that define how the code works now). You may need to do some small changes in the code to allow writing the tests using seams.<\/li><li>Refactor the code while using the characterization tests to preserve existing behavior.<\/li><\/ul>\n\n\n\n<p>This solution works very well once you master the techniques. However, it has one problem: it&#8217;s quite slow and tedious. Sure, that should be expected &#8211; cleaning up a mess is rarely fast or easy. But the business often doesn&#8217;t have time to invest into the clean up.<\/p>\n\n\n\n<p>Maybe there&#8217;s another way to do the same thing?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A New Method<\/h2>\n\n\n\n<p>I&#8217;ve been pondering this problem for a long time. At the same time, I learned more and more about functional programming. Therefore, I&#8217;m proposing a new method that involves pure functions and passing functions as arguments, and that I believe to be faster.<\/p>\n\n\n\n<p>Before we move on, I&#8217;d like to make it clear that, while I&#8217;ve played with this method in various code samples, I can&#8217;t claim that it&#8217;s fully studied and perfect. I plan to try it out with more people, and see what I can learn from it. I believe however that it&#8217;s promising enough to be described.<\/p>\n\n\n\n<p>The method has three steps:<\/p>\n\n\n\n<ol><li>Refactor the selected code towards pure functions, using mechanical, safe refactorings and no tests<\/li><li>Write data-driven tests or property-based tests for the pure functions<\/li><li>Refactor the pure functions towards your desired design paradigm using the tests.<\/li><\/ol>\n\n\n\n<p> Let&#8217;s define a few terms, and move on to describe each step.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is a pure function?<\/h2>\n\n\n\n<p>Before we move on, let&#8217;s define pure functions. A pure function is a function that returns the same output values when receiving the same input values, and changes nothing in the program state. For example the following function is pure:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>int add(int first, int second){\n    return first + second;\n};<\/code><\/pre>\n\n\n\n<p>while the following function is not pure:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>int add(int first, int second){\n    first += second; \/\/ this changes the value of first\n    return first;\n};<\/code><\/pre>\n\n\n\n<p>As you can see, pure functions cannot be dependent on I\/O or on time, they cannot change the parameters they receive, and they are very predictable.<\/p>\n\n\n\n<p>So how can we take advantage of pure functions?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">It&#8217;s Pure Functions All the Way Down<\/h2>\n\n\n\n<p>I will postulate that <strong>any non-trivial program can be written as a set of pure functions combined with a few mutable functions<\/strong>. <\/p>\n\n\n\n<p>For example, if your program is a web application, all the code that writes or reads from the database, all the code that creates the response and reads the request, and all the code that writes log files can be encapsulated in a few mutable functions. Everything else is easily written with pure functions.<\/p>\n\n\n\n<p>If your program is a game, all the code that interacts with the graphical card, all the code that reads the player actions, and all the code that saves or loads the game is mutable. Everything else can be easily written with pure functions.<\/p>\n\n\n\n<p>A more interesting effect is that <strong>this rule applies at different levels<\/strong>. It can apply to a class, it can apply to a module, to a set of classes, or to a method. The only time it fails is if we try to apply it on a very simple I\/O method.<\/p>\n\n\n\n<p>That&#8217;s very powerful: it means that <strong>the pure functions representation of the program can be used no matter where we start or how large the code is<\/strong>. This is the first part of the puzzle.<\/p>\n\n\n\n<p>The second part of the puzzle is: how can we safely refactor any code towards pure functions? <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Phase 1: Refactor towards pure functions<\/h2>\n\n\n\n<p>The basic technique is the following:<\/p>\n\n\n\n<ul><li>select one or more lines of code<\/li><li>extract method from them<\/li><li>make the method static<\/li><li>use the compiler to identify mutable code (NB: I&#8217;ve tried this with compiled languages until now)<\/li><li>for each piece of mutable code, either extract a new function that you pass as a parameter, or extract data that you pass as a parameter. When in doubt, prefer functions.<\/li><\/ul>\n\n\n\n<p>The end result is a pure function that receives a lot of parameters, either other functions or data parameters. Either way, we have refactored the function towards a <strong>pure function with all dependencies injected<\/strong>. This makes the function testable. <\/p>\n\n\n\n<p>At this point, you can either move to phase 2, or refactor the function to reduce its parameters.<\/p>\n\n\n\n<p>From my experience, <strong>each of these steps is mechanical and safe to do with modern IDEs<\/strong> (or with vim).<\/p>\n\n\n\n<p>It&#8217;s time to write some tests.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Phase 2: Write tests for the pure function<\/h2>\n\n\n\n<p>At this point, we have at least one pure function we can test. Since the function doesn&#8217;t change its parameters, and returns the same outputs for the same inputs, we can use data-driven tests. In fact, the function is equivalent with a very large data table whose last column represents the output.<\/p>\n\n\n\n<p>Moreover, the input data can be generated. We can use property-based testing and\/or the golden master technique to take advantage of input data generators, thus making the process faster.<\/p>\n\n\n\n<p> As for the functions injected as variables, we can use in our tests stubbing or, for some of the I\/O code, mocking.<\/p>\n\n\n\n<p>This leads us to phase 3.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Phase 3: Refactor the pure function<\/h2>\n\n\n\n<p>Once covered by tests, our pure function can be easily refactored. The functions passed as parameters can be turned into interfaces that are injected. The function can be split, and the resulting functions moved into their own classes. Some of the input parameters can be passed to a constructor, while others remain function parameters. etc.<\/p>\n\n\n\n<p>Or, we can go all in functional, extracting lambdas, composing functions, and using partial application to remove duplication. That&#8217;s up to you.<\/p>\n\n\n\n<p>This takes us to a conclusion.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>I have briefly presented in this article a new method to refactor existing, untested code, by refactoring first towards pure functions, then covering with data-driven or property-based tests, and finally refactoring the pure function towards the end goal. There are many more techniques that I found while trying this method, but for brevity I decided not to include them.<\/p>\n\n\n\n<p>I have also made a few claims that require more investigation and experiments. Can an average programmer, once taught the basic techniques, apply them without changing the behavior of the code? Does this work for any type of code? etc.<\/p>\n\n\n\n<p>I can only hope that you find this method interesting, and decide to try it out or ask questions about it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">An Example<\/h2>\n\n\n\n<p>For brevity, I have avoided a particular example in this article. If enough people are interested, I will create a few examples to showcase the technique. Until then, an example in C++ is detailed in my latest book <a href=\"https:\/\/www.amazon.com\/Hands-Functional-Programming-Practical-Production\/dp\/1789807336\/\">&#8220;Hands-on Functional Programming with C++&#8221;<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><a href=\"https:\/\/www.amazon.com\/Hands-Functional-Programming-Practical-Production\/dp\/1789807336\/\"><img decoding=\"async\" src=\"https:\/\/cdn.mozaicworks.com\/wp-content\/uploads\/2019\/08\/functional-programming-c-cover-700x864.jpeg\" alt=\"\" class=\"wp-image-10841\"\/><\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>In this article, I will present a method that I&#8217;ve tried in a few codebases in compiled languages for safely and quickly refactoring untested code. First, we will discuss the main problem we are trying to solve, quickly introduce the techniques coined by Michael Feathers, then discuss some shortcomings of the existing techniques, and finally [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"footnotes":""},"categories":[1126],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/posts\/12251"}],"collection":[{"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/comments?post=12251"}],"version-history":[{"count":1,"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/posts\/12251\/revisions"}],"predecessor-version":[{"id":14346,"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/posts\/12251\/revisions\/14346"}],"wp:attachment":[{"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/media?parent=12251"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/categories?post=12251"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mozaicworks.com\/wp-json\/wp\/v2\/tags?post=12251"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}