Creating A Fortran Unit Testing Framework (part 1)

I’ve decided to develop a simple unit testing framework for Fortran[1]. The next several posts will document how the library is built, what decisions I’m faced with, and how they are resolved. At the end of the series, we will have a library that we can use to help us to develop new code, and to modernize existing code.

Here’s the problem that started the whole thing off. I am working on a series of simulations of the distribution of a statistic. We have derived the asymptotic distribution of the statistic, and want some idea of what sample size is sufficiently large to allow us to use the result. The distribution in question involves the sum of weighted chi-squared variates. This isn’t simple (or obvious) to compute, and there aren’t easily available, ready-made implementations to grab off the shelf and use. Moreover, I don’t much care about how to compute this quantity. It is a necessary piece of a larger puzzle, but not a piece I care much about.

Doing the obvious web searches yielded a couple of good alternatives. One in particular on Statlib seemed to be just perfect solution. The problem was that the original was in ALGOL or PASCAL, neither of which I work in on a daily basis. I could do the translation into something else (Fortran or Java or C# or any of the other, more modern languages that I know), but I am very leery of introducing a bug during the translation. A bit more web work revealed that the author of the original article had already done a translation into Fortran. So I downloaded it, happy to be able to move on to my real question. But, when I opened up the file, I found that it was written in an old version of Fortran, one that my compiler wasn’t going to compile without a lot of complaints, if it compiled it at all. So, same problem, different language. SPAM!

Modernizing Old (“Legacy”) Code

There is a lot of legacy code out there, much of it written in ancient versions of Fortran. I have code that I want to use that is written in FORTRAN 66, and there is even FORTRAN IV (or FORTRAN II!) floating around. A quick Google search also reveals a small but ongoing problem of being handed code that works but is written in an archaic version of Fortran.

As my experience demonstrates, finding one of these examples is always a pain. What starts as joy over finding a subroutine or library that does what you want turns into despair as your realize that you can’t use it. At this point, you know you have to convert it to a more modern version or the language (Fortran 95 or 2003) to incorporate into your program. Or, if you want to translate it into another language, you have the same problems, plus the differences between how one does things in Fortran and how they are done in your language of choice.

In general, these algorithms are sound, but the code often won’t even compile under a modern compiler. For example, the venerable Hollerith constant was deprecated in Fortran 90, and removed from the language entirely in Fortran 95.

So what to do? One possibility (if the code is based on an article in a professional journal or conference paper) is to roll up your sleeves, dig into the equations/derivations and write a program in your language of choice. While this certainly works, it defeats the whole benefit of finding the solution to our problem. In general, this solution requires that we deeply understand the logic underlying the original implementation, which is a lot of work that feels unnecessary, and is usually tangential to the real goal of our study.

The other alternative is to re-write the code in a modern version of the language. This always has the risk of introducing an error into the code. Because we intend to treat this code pretty much as a black box (we send values in, and new values come out, and we don’t much want to know how it does its work). We don’t want to burrow into the logic.

This is a well-known problem in professional (as opposed to academic/scientific) computing. Companies rely on code written years (or in some cases, decades) before. Often these programs are key to the business’ functioning, perhaps handing all of its billing, or being the central to a key inventory control system. The function often isn’t sexy, but it is vital to the company’s survival. Those who are old enough to have paid any attention during the late 1990’s will recall the “Y2K” problem. This was the classic maintenance nightmare. Code that was often decades old had to understood and modified. The original programmers had often retired (or died).

The defnitive book on dealing with this problem is "Working Effectively with Legacy Code" by Michael Feathers. It is brilliant, and well worth 10 times whatever Addison-Wesley is charging for it. In a nutshell, Feathers describes legacy code as code without tests. The ideal is to have a blanket of tests completely cover the code we want to change. Each test is small, and test a relatively small portion of the code. These are called "unit tests." We also want tests of the overall function ("functional tests"), but unit tests give us a fine-grained picture of what the code should do. Feathers' book is dedicated to ways that we can create unit tests.

Refactoring

The goal of modernizing any program is to change its internal structure (e.g., to update to a more modern version of a language). But as we do this restructuring, we want to insure that we don't change the function of the code. This process of changing the internal state while maintaining the external behavior is called "refactoring." One of the key ideas in refactoring is that we make very small changes, and verify that we haven't broken anything after each step. To do this, we need a good set of unit tests.

But why is this important? With a good set of unit tests, we can safely make changes to the code. If our change introduces an error, running the unit tests will let us know; a passing test will now fail. Because we have only made a small change since the last time we ran the tests, we know exactly what the change was that introduced the error. Fixing the error we introduced will make all of tests pass again.

To summarize:

1. We have legacy code that we want to convert to a modern version of the same language (Fortran)
2. Modernizing the code is a form or refactoring. We want to change the internal
3. To refactor the code with confidence, we need to create a good set of unit tests. Each unit test examines a small piece of functionality; the set of tests provides good coverage of the all of the functionality of the legacy code.
4. In refactoring, we proceed by making a series of very small changes to the code. After each, we run the tests. If we introduce a bug, one or more of the tests will fail. Because we have made only small changes, we can quickly identify the bug. Once we've fixed it, all of the tests will pass again.
5. My web surfing hasn't turned up a good unti testing framework for old Fortran, so I'm going to write one. I'll document the development here, describing the decisions made and why I made them.

Next time, I will take a look at existing unit testing frameworks for Fortran, and start developing my framework.

[1] Fortran v. FORTRAN: The original name of the language was "FORTRAN", an abbreviation fo FORmula TRANslation. The name was written all in capital letters. As of Fortran 90, the name of the language has been changed to "Fortran", with only the first letter capitalized. I'll use Fortran when speaking generically about the language. When I'm speaking of a specific version, I will use the correct capitalization for that version.