7 Metrics for Configuration Management

Years ago, a company might have released a software suite and then proverbially kicked back in a chair with its feet on a desk basking in celebration.

Suffice it to say that the software world moves much faster today. It seems as though there are some companies that push out new updates every few days. And thanks to microservices architecture and the DevOps Mindset, there are many companies that are constantly updating their software or at least some feature in it.

Pumping out release after release isn’t easy. With so many moving parts and so much riding on each new update, companies need to do everything within their power to ensure that releases are well-received by users.

That starts with getting their development house in order through a process known as configuration management.

What Is Configuration Management?

Configuration management is the process in which organizations and development teams oversee new software updates to ensure they are working as designed when bugs are fixed, new features are introduced and old features are decommissioned.

Thanks to configuration management, organizations can gain full visibility into the development lifecycle and easily identify errors that may need to be fixed.

If you’re thinking about implementing configuration management, or CMDB*, at your organization, that’s great news. But like anything else, you can’t just expect configuration management to solve all of your problems on its own. You need the right approach.

What is a CMDB

A CMDB (Configuration Management Database) is a database that stores information about the IT assets and infrastructure of an organization. It is used to manage and track the configuration items (CIs) in an organization’s IT environment, including hardware, software, network devices, and other components.

A CMDB can include detailed information about the relationships between CIs, such as dependencies, dependencies, and interdependencies. This information can be used to help organizations plan changes to their IT environment, understand the impact of changes on other components, and maintain the integrity and availability of their IT services.

In addition to maintaining information about CIs, a CMDB can also be used to track service requests, incidents, problems, and changes related to the IT environment. It can help organizations improve their IT service management processes and make better decisions about their IT investments.

With that in mind, let’s take a look at seven different configuration management metrics you can track to increase the chances that your initiatives help you achieve results. Keep track of these metrics and work hard to improve them over time, and you’ll build better applications that are better received by your users.

1. Frequency of Updates

Some companies are perfectly fine with shipping updates once a quarter or even once a year. Other companies pride themselves on pumping out new updates every month, and some might aim to release even more new software packages than that.

Every software company has unique goals. It might not matter how regularly your software is updated, but it might matter how consistently it is. Your users will expect at least some rhyme and reason to the number of updates you pump out.

Keeping track of the frequency of updates metric will help you make sure you are meeting your company’s goals and satisfying customer expectations. If you’re not shipping releases as frequently as you’d like, you might want to drill deeper and find out why.

2. Release Downtime Metrics

We all know how applications should work. When they don’t work as designed, we’re unable to get things done quickly. Depending on how bad the problem gets, it’s easy to get frustrated to the point a user starts thinking about whether they should find a substitute solution.

End users depend on your software. For a business user, that might mean a platform they use to store information and communicate with colleagues. It might mean a place they store code for a developer. And for a regular customer it might be a social network they use every day to meet new people.

Whatever the case may be, the moment you are unable to meet user expectations might be the moment your users begin an exodus.

Worse than that, downtime can be prohibitively expensive. In fact, a recent Gartner report found that downtime can cost as much as $540,000 per hour.

Keeping track of how much downtime you incur (if any) while a new update is released can help you maintain positive and productive user experiences. In the event there is downtime during a new release, you can quickly identify what happened and take steps to reduce the chances it happens again.

Add it all up, and keeping tabs of this metric can help you provide better experiences while increasing profitability.

3. Average Number of Errors

In a perfect world, your developers would write flawless code every day, and each new release would ship with perfect code. But we live in the real world where people do make mistakes.

Of course, it’s in your best interest to work as hard as you can to keep those mistakes down to a minimum. By keeping track of the average number of errors in each new software release, you can identify areas in your workflow that could be improved. This may help you catch mistakes earlier in the process.

For example, you might realize that a new adding a new tool to your DevOps team’s arsenal can help your release smoother updates every time.

At the very least, tracking this metric provides an easy mechanism to determine whether your team is trending in the right direction, i.e., making fewer errors as time goes on.

4. Code Lines Per Update

The point of writing is to convey a point to your readers. Unless the author is getting paid per-word, writers should state their case in as little words as possible. The question is what day is it today? It’s not do you have any idea as to which 24-hour period we are currently in the middle of?

In the world of software development, the same maxim holds true. You don’t need 100 lines of code when a single line will do the same trick.

Keeping track of code lines per update can help you ensure that you are writing software efficiently. Depending on what your team’s workflows are like, you may be able to identify individual developers who are writing too many lines of code and have the more efficient coders give them a few pointers.

5. Rework Metrics

How many files does your team rework each month?

Developers don’t come cheap. The last thing you want to do is pay them to do the same work over and over again—whether that’s because someone did it incorrectly in the first place or because your team is struggling to communicate effectively.

Tracking rework metrics can help you make sure that the percent of rework your team does each month doesn’t increase in perpetuity. On the flipside, you may also be able to identify what you are doing that is decreasing rework. With that information on hand, you may be able to bake additional efficiencies into your development processes.

6. Frequently Changing Files

Track this metric to determine whether certain files are changing too frequently. If you find out that certain files are changing with each update, you may need to look into the issue a bit.

For example, you can determine why certain files are changing so often. Maybe it’s because developers aren’t sure of the requirements. Maybe it’s because there’s an issue with your testing and QA approach.

Whatever the case may be, this metric can help you add additional efficiencies into your development processes by reducing or eliminating duplicative work and rewriting inefficient code blocks as needed.

7. Root Causes for Late Delivery

As you optimize your release management workflows, everything should get more and more predictable.

Yet nobody can predict the future and nobody’s perfect. So things will invariably not go according to plan every now and again.

Configuration management lets you drill down into the root causes for late delivery.

Fingers crossed that you never run into any errors that slow down your releases. But in the event you do miss some deadlines, you may be able to start detecting a pattern as to why you are unable to meet them.

Armed with that information, you can begin working backward to identify what is causing delays and what you need to do to prevent that from happening in the future.

Are You Ready to Start Using Configuration Management?

Is your development team reaching its full potential and doing its best work? If not, it may be time to get started with configuration management. That way, you’ll be able to delight customers by meeting their expectations while avoiding downtime and increasing profitability.

And the best part? With the right tools in place, configuration management can largely be automated.

To learn more about how your DevOps team can integrate configuration management into their workflows to build better software more efficiently, take a look at Enov8.

ITIL 4.0: What Has Changed?

It’s hard to imagine a world that existed without technology. Yet it wasn’t so long ago when things like computers and the internet were brand-new and seemingly futuristic concepts. As computing infrastructure became increasingly widespread in the 1980s, the government of the United Kingdom issued a set of recommended standards that IT teams should follow because it realized that, at the time, everyone was just doing their own thing.

Shortly thereafter, the first iteration of Information Technology Infrastructure Library (ITIL) emerged, called the Government Information Technology Infrastructure Management (GITIM). These guidelines outlined a set of practices, processes, and policies organizations could follow to ensure their IT infrastructure was set up in such a way to support their business needs. The ITIL standards were inspired by the process-based management teachings of productivity and management guru W. Edwards Deming.

What Is ITIL?

Life would be difficult if it were impossible to learn from other people and we had to figure everything out by ourselves. Good thing that’s not the case.

At a very basic level, ITIL is a framework that outlines the best practice for delivering IT services throughout the entire lifecycle. Organizations that follow this framework put themselves in a great position to stay on the cutting edge of technology and leverage the latest tools and philosophies that drive leading innovators forward today. They are also able to respond to incidents faster and enact change management initiatives with more success.

At a high level, there are five core components of ITIL 4:

Service value chain.
Practices.
Guiding principles.
Governance.
Continual improvement.

Now that we’ve got our definitions locked down, let’s shift our attention to the pros and cons of enacting ITIL at your organization.

What Are the Pros of ITIL?

ITIL is popular for good reason. The framework helps organizations big and small optimize their IT infrastructure. It also helps them secure their networks and realize productivity gains.

More specifically, ITIL enables organizations to:

Keep IT aligned with business needs, ensuring that the right infrastructure is in place for the task at hand. For example, a team that has a mobile workforce should leverage cloud platforms that enable employees to work productively from any connected device.
Delight customers and strengthen user experiences by improving the delivery of IT services and maintaining a network and infrastructure that works as designed and meets modern expectations.
Reduce IT costs and eliminate unnecessary expenditures by ensuring that IT infrastructure is optimized and efficient. For example, if you’re storing petabytes of duplicative data for no reason, best practices would tell you that you need to do a lot of culling to save on storage costs.
Gain more visibility into IT expenses and infrastructure to better understand your network and detect inefficiencies that can be improved. For example, if your software development team has recently started using containers to build applications, you might not need to run as many virtual machines anymore, which drain more computing resources.
Increase uptime and availability due to increased resiliency and robust disaster recovery and business continuity plans. This is a big deal because downtime can be prohibitively expensive, depending on the scale of your organization. Just ask Amazon.
Future-proof tech infrastructure to support agile workflows and adaptability in an era where customer needs shift overnight and competitors are always just a few taps of a smartphone away.

What Are the Cons of ITIL?

But like everything else, ITIL by itself is not a panacea. You can’t just hire some consultant who will preach the virtues of ITIL and expect to transform your IT operations overnight.

While the benefits of the framework speak for themselves, you need to be realistic about shifting to a new approach to IT management. However, with the right approach—which includes training, patience, and reasonable expectations—your organization stands to benefit significantly by adopting ITIL.

How Has ITIL Changed Over the Years?

ITIL initially emerged because more and more organizations were using new technologies but nobody really knew how to manage them effectively. Companies were largely using technology because they could—not because they were making strategic investments to support their customers and business needs. The initial iteration of ITIL found that most companies had the same requirements and needs for their IT networks, regardless of size or industry.

At the turn of the millennium, the second iteration of ITIL came online. In large part, this version consolidated and simplified the teachings and documentation from the inaugural ITIL framework.

In May 2007, ITIL 3 came to the surface. This third iteration included a set of five reference books called Service Strategy, Service Design, Service Transition, Service Operation, and Continual Service Improvement. ITIL 3 picked up where ITIL 2 left off, further consolidating the framework to make it easier for organizations to implement.

Four years later, ITIL 3 was revised once more, primarily to maintain consistency as technology evolved.

Introducing ITIL 4

Fast forward to 2019, and the most recent version, ITIL 4, is where we’re at today. Quite simply, ITIL 4 was issued to align the standards with the agile and DevOps workflows that have grown to dominate technology teams over the last several years. ITIL 4 includes two core components: the four dimensions model and the service value system.

At a high level, ITIL 4 represents more of a change in approach and philosophy than a change in content. Just as software teams adopt agile and DevOps workflows, IT must adopt a similar mindset if they wish to keep pace and support accelerated innovation. At the end of the day, IT is a cornerstone of the success of the modern organization. It’s imperative that IT support the new way of working if an organization wishes to reach its full potential.

How Have Agile and DevOps Impacted ITIL 4?

In the past, software teams would build monolithic applications and release maybe once a year. Today’s leading software development teams have embraced agile development and DevOps workflows. Slowly but surely, monthly releases are becoming the norm. Development is becoming more collaborative, too, with both colleagues and users steering the product roadmap.

ITIL 4 recognizes and supports this new way of working with new core messages:

Focus on value.
Start where you are.
Progress iteratively with feedback.
Collaborate and promote visibility.
Think and work holistically.
Keep it simple and practical.
Optimize and automate.

Where Does Your Organization Stand?

If your company hasn’t yet implemented ITIL, what are you waiting for?

Whether you’re a startup or your organization has been around forever, ITIL serves as a guiding framework. Follow it and it enables you to protect your networks, support your developers, and delight your customers.

And what exactly is the alternative, anyway? Running your IT department like the Wild West?

With so much on the line, you can’t afford that risk. So become an ITIL-driven organization. That way, you’ll get the peace of mind that comes with knowing your networks and infrastructure are secure and support innovation and agility.

What’s not to like?

Software Testing Anti Patterns

Since the dawn of computers, we’ve always had to test software. Over the course of several decades, the discipline of software testing has seen many best practices and patterns. Unfortunately, there are also several anti patterns that are present in many companies.

An anti pattern is a pattern of activities that tries to solve a certain problem but is actually counter-productive. It either doesn’t solve the problem, makes it worse, or creates new problems. In this article, I’ll sum up some common testing anti patterns.

Only Involving Testers Afterwards

Many companies only involve the testers when the developers decide a feature is done. The requirements go to the developers, who change the code to implement the requested feature. The updated application is then “thrown over the wall” to the testers. They will then use the requirements to construct test cases. After going through the test cases, the testers will often find all sorts of issues so that the developers need to revisit the new features. This has a detrimental effect on productivity and morale.

Such an approach to testing is used in many companies, even those that talk about modern practices like Agile and DevOps. However, “throwing things over the wall” without input from the next step goes against the spirit of Agile and DevOps. The idea is to have all disciplines work together towards a common goal.

Testing is about getting feedback, regardless of whether it is automated testing or not. So of course you have to test after the feature has been developed. But that doesn’t mean you can’t involve your QA team earlier in the process.

Having testers involved in defining requirements, identifying use cases, and writing tests is a way to catch edge cases early and leads to quality tests.

Not Automating When You Can

Tests that run by the click of a button are a huge time saver, and as such they also save money. Any sufficiently large application can have hundreds or even thousands of automated tests. You can’t achieve efficient software delivery if you’re testing all this manually. It would simply take too much time.

One alternative I’ve seen is to stop testing finished features. But due to the nature of software, existing features that used to work can easily break because of a change to another feature. That’s why it pays off to keep verifying that what used to work still works now.

The better alternative to manual testing is to automate as many tests as you can. There are many tools to help you automate your tests. From the low level of separate pieces of code (unit tests) over the integration of these pieces (integration tests) to full-blown end-to-end tests.

As a tester, you should encourage the whole team to be involved in manual testing. It will encourage them to write code that is fit for automated tests. Help developers write and maintain automated tests. Help them identify test cases.

Expecting to Automate Everything

As a counterargument to my previous point, be wary of trying to automate every aspect of testing. Manual testing can still have its place in a world where everything is increasingly automated.

Some things could be too hard or too much work to automate. Other scenarios may be so rare that it isn’t worth automating, especially if the consequences of an issue are acceptable.

Another thing you can’t expect to automate is exploratory testing. Exploratory testing is where testers use their experience and creativity to test the application. This allows the testers to learn about the application and generate new tests from this process. Indeed, in the words of software engineering professor Cem Kaner, the idea behind exploratory testing is that “test-related learning, test design, test execution, and test result interpretation [are] mutually supportive activities that run in parallel throughout the project.”

Lack of Test Environment Management

Test Environment Management spans a broad range of activities. The idea is to provide and maintain a stable environment that can be used for testing.

Typically, we call such an environment a testing or staging environment. It’s the environment where testers or product owners can test the application and any new features that the developers have delivered.

However, if such an environment isn’t managed well, it can lead to a very inefficient software delivery process. Examples are:

● Confusion over which features have already been deployed to the test environment.

● The test environment is missing certain critical pieces or external integrations so that not everything can be tested.

● The hardware differs significantly from the production environment.

● The test environment isn’t configured correctly.

● Lack of quality data to test with.

Such factors can lead to a back and forth discussion between testers, management, and developers. Bugs may go unnoticed or reported bugs may not be bugs at all. Use cases may be hard to test and bugs reported in production hard to reproduce.

Without a good test environment management, you will be wasting time and losing money.

Unsecured Test Data

Most applications need a set of data to test certain scenarios. Not all data is created equal though. With modern privacy laws, you want to avoid using real user data. Both developers and testers often have to dig in the data of the test environment to see what is causing certain behavior. This means reading what could be personally identifying information (PII). If this is data from real users, you might be violating certain laws.

Moreover, if your software integrates with other systems, the data may flow away from your system to a point where it is out of your control. Maybe even to another company. This is not something you want to do with real people’s data. Security breaches can lead to severe public image and financial losses or fines.

So you want either made up data or obfuscated / secured data. But you also want to make sure that the data is still relevant and valid in the context of your application. One possible solution to this is to generate the data your tests need as part of your tests.

Not Teaching Developers

The whole team owns the quality of the software. Pair with developers and teach them the techniques so that they can test the features as they finish them.

This is especially important in teams that (aspire to) have a high level of agility. If you want to continuously deploy small features, the team will have to continuously test the application. This includes developers, instead of having them wait for the testers.

In such a case, the role of testers becomes more of a coaching role.

If testers and developers don’t work together closely, both will have negative feelings for each other. Developers will see the testers as a factor blocking them from moving fast. Testers will have little faith in the capacity of the developers to deliver quality software.

In fact, both are right. If the two groups don’t collaborate, precious time and effort will be lost in testing a feature, fixing bugs, and testing the feature again. If the developers know what will be tested, they can anticipate the different test cases and write the code accordingly. They might even automate the test cases, which is a win for testers and developers.

Streamline Your Testing!

The major theme in this article is one of collaboration. Testers and developers (and other disciplines) should work together so that the software can be tested with the least amount of effort. This leads to a more efficient testing process, fewer bugs, and a faster delivery cycle. Top that off with good test environment management (which is also a collaborative effort) and secure data, and you have a winning testing process.

Category Archives: Test Environments