Testing Software Code Is Challenging, Even With Agile. Need a Practical Guide?
New software engineers and technicians involved in code testing should read this resource.
September 1, 2021
Decades ago, around the time of the Y2K bug, several software books emerged whose aim was to improve the software development process for engineers and technicians. I remember one popular book by Steve McConnel: Software Project Survival Guide (SPSG).
The SPSG was a hit because it presented the software development process in straightforward and practical terms. For example, test code consists of three elements: unit, integration, and system testing. During system testing, it recommended independent testers, not the same developers that created the programs. Further, tests are needed to cover 100 percent software’s functionality. There were practical suggestions on the ratio of testers to developers:
Microsoft: 1 tester / 1 developer
NASA: Life critical software, 10 testers / 1 developer
Small, in-house business systems: 1 tester / 4 developers
Of course, system test is just one of the many test types needed to produce good software systems. But let’s stay focused on the system as deals with the top-level concerning in testing.
Fast-forward to today. Much has changed in the realm of software development and testing – think Agile systems. But new and even forgotten best practices in testing have also emerged that are especially valuable for the younger software developer. Consider a recent book by two tech experts with decades of experience in the industry (PayPal, LinkedIn, WePay, Cloudera, Twitter): Chris Riccomini and Dmitriy Ryaboy, “The Missing README: A Guide for the New Software Engineer.”
Software Testing Today
In their chapter on testing, the authors understood that “many software projects were missing certain flavors of tests, while others were inconsistent about the separation—intermingling “unit” and “integration” tests. It’s important to know what these categories mean and the trade-offs between them.”
The authors note that successful software developers must make real-world pragmatic testing decisions. These practitioners shouldn’t get hung up on naming and categorization, and should also refrain from passing judgment if the test setup is not quite right.
The testing chapter is full of useful information relevant to today’s modern systems. For example, the authors suggest using injectable clocks rather than static time methods so testers can control the timing that the code sees in a test. This will help avoid the problem of a code segment that waits 500ms for something. A test will pass if the code runs in 499ms but fail when it runs in 501ms. This is a problem.
ASimpleThrottler Ruby class illustrates the problem. SimpleThrottler invokes a throttle method when the operation count exceeds a threshold, but the clock is not injectable.
+++++++++++++++
class SimpleThrottler
def initialize(max_per_sec=1000)
@max_per_sec = max_per_sec
@last_sec = Time.now.to_i
@count_this_sec = 0