Home » Archives for TestEnvMgt » Page 20

Why Map Your IT Environments?

Running a high-functioning IT team or tech company requires you to be clear in your mind where you want to take your team. If you’re not clear about that, then just like Alice in the quote above, it doesn’t matter which way you go—or, in the context of the increasingly complex tech ecosystem, it doesn’t matter which methodology or tools you adopt. Then you end up implementing this technology or that methodology halfheartedly, which leads to you switching to new technology and methodology, and the cycle repeats. This leads to a form of techno-methodology whiplash for your team. Is that what you want for your team? I hope not.

Know Your Destination, Know Your Landscape

What the Cheshire Cat didn’t point out is that for most of us dealing with complex situations, knowing the destination isn’t enough. We need to know the landscape to plot our way to success. In this article, I will cover the top four reasons why you need to properly map your IT and test environment to bring your team to perform at a high-functioning level.

One View to See It All

When you map your IT and test environment, you essentially establish the landscape of the situation. A good map lets you bring together various priorities and interests of your team and organization in a single view. The benefits of doing so can’t be underestimated. Miller’s law states that the average human mind can hold only about seven things at any one time. Without a map to oversee the entire landscape, how could you possibly navigate your team around risks of deployment, development, and the day-to-day running of the IT and test environments?

In addition, you can build a map that contains multiple levels. Imagine that at the organization overview you map out the various key structures, such as business, ops, IT environment, and test environment. Then you can drill in further by adding in the substructures, such as system instances, applications, data, and infrastructure. All these structures and substructures will interact among themselves, which is why you need to add in the relationships among these structures, the projects, and the teams in your organization.

Now imagine you have this map right now. Wouldn’t that make it a lot easier to think about your decisions and weigh your options? You can almost literally trace how a possible solution would impact which system and which team—so before you even encounter objections, you can anticipate them. That’s the power of a single view of your landscape captured in a map.

Spotting Existing Gaps and New Opportunities

When you have a map, the map almost immediately shows you some low-hanging fruits to pick. Existing gaps and opportunities to improve your existing operations show themselves easily. These low-hanging fruits can give you some quick wins for you and your organization.

Some typical quick wins would be:

Identify waste and save costs. For example, you may identify system instances being maintained but not used.
Identify underutilized resources and consolidate them. This happens quite frequently as well. For example, you have a bunch of system instances that constantly have low utilization. You can decide to consolidate them to bring about a better return on your expenditure on these resources.
Identify undersized systems or applications and reallocate buffer resources. Once you reduce waste and free up resources targeting underutilized resources, you can deploy some of these freed-up resources at the undersized systems. Typically, people would complain that these undersized systems were constantly stretched and not enough resources could be spared due to budget. In other words, you can help reallocate your resources better simply by having this map.
Identify the high-growth areas and enable them to grow faster. With a map, you can view how certain systems or applications are growing quickly because they are driven by fast-growing demand. When you can link these high-growth areas with how they help with organization, you will be able to convince management how adding more budget makes business sense. Or you can redeploy resources from other structures facing slowing growth. In either case, a map bolsters the strength of your decision.

Streamline and Simplify Processes

Everyone has a story about dealing with silly, ridiculous bureaucratic processes. However, as a civilization, progress means more processes are needed for things to run smoothly. Running your IT and test environment successfully means having good processes to ensure things run smoothly. Think Value Stream Mapping.The key is to know when these processes become less effective or even outright unnecessary. Then you need to retire or remodel your processes. The key, therefore, is to discover these increasingly ineffective processes and nip them in the bud.

So, study the stats from your troubleshooting and logs and add those to your map. Talk to your various teams from business and customer support. Add anecdotes in as well. In a single view, you would be able to allow both data and personal stories to drive your decision on how to simplify running your IT and test environments. Streamlining and pruning away processes that used to be (but are no longer) necessary would release more resources back to your budget. This kick-starts a virtuous cycle as freed-up resources can then be redeployed for growing opportunities.

Better Impact Analysis and Scenario Planning

Once you take advantage of the single view to quickly exploit new opportunities, uncover waste, increase better utilization of resources via reallocation, and streamline processes, you have established credibility about mapping. Imagine earning all that success without even using the methodological or technological fad of the day.

Now it’s time for the exciting stuff—planning the future. Once again, the mapping will help greatly. You can plan several scenarios and strategies in a playbook and then check them against the map. The check would involve some kind of impact analysis. The scenario planning exercise is widely used by some of the top-performing organizations in the world. Having a map of your IT and test environment improves the effectiveness and efficiency of the exercise. No more guessing about potential impact of brainstormed strategies for future scenarios; you can immediately check and verify obvious drawbacks and benefits. Scenario planning is better because impact analysis becomes better with a map of your environments.

Conclusion

In Enterprise IT intelligence, “environment mapping” represents a highly beneficial and foundational exercise all IT teams and tech companies should perform at least once every quarter or so. It provides high visibility to the many interrelated structures and their relations in your organization. It is not easy to discern these structures and their relations without the map. The increase in visibility delivers great benefits. Agility, smooth delivery, greater collaboration, and good operational and business decision-making all flow from the greater visibility of the landscape surrounding your team and organization. Buy-in becomes simpler when everybody can be on the same page—and when everybody is looking at the same map as well.

The importance of mapping your environments is key to your organization’s success. Bear in mind that maps are imperfect, but they are still very useful. Mapping helps you and your team become better at your jobs simply because you did the exercise of mapping. The exercise surfaces the differences in the thinking between the members in your team. Therefore, don’t wait until you come up with the perfect map. Your team automatically becomes better with more practice mapping. Your team and your organization will thank you for that when they start to see the uptick in results.

Just Enough ITSM (or ITSM for Non-Production)

We’ve all experienced the frustration that comes from too much or too little service management in your test environment. Lately, the DevOps engineer in me has been thinking about how we end up in one of those states. How can we get just enough service management in non-production environments?

Production environments require more service than non-prod environments. But we shouldn’t throw the baby out with the bathwater when it comes to service management in non-prod. I’m a software developer who practices DevOps, so I do a lot of work involving operations, deployment, and automation. I interface with many groups to achieve a good workflow within the organization.

Operations and development often have contradictory goals. Fortunately, we can all find common ground by working together. Understanding each other’s needs and goals through communication is the key to success!

What Are Non-Production Environments?

We use non-production environments for development, testing, and demonstrations. It’s best to keep them as independent as possible to avoid any crosstalk. We wouldn’t want issues in one environment to affect any of the others.

These environments’ users are often internal—for the most part, we’re talking about developers, testers, and stakeholders. It’s safe to assume that anyone in the company is a potential user. It’s also safe to assume that anyone providing a service to the company might have access to non-production environments. But there could also be external users accessing these environments, perhaps for testing purposes.

Unless you have the environment in question tightly controlled, you may not know who those users are. That’s a big problem. It’s important to understand who’s using which environments in case someone inadvertently has access to unauthorized information. Or maybe you just need to know who needs to stay informed about changes or outages in a specific environment.

That’s where service management comes in. The next section explains how bad things can be when there is no service management in non-production. This exercise should be fun…or it might make you queasy. Better have a seat and buckle up just in case!

When You Have Zero Service Management in Non-Prod

Let’s call this the state of anarchy. Here’s what it looks like:

Servers are running haywire and no one knows it.
Patches are missing.
Security holes abound!
The network is barely serviceable.

Can anyone even use this environment? How did it get like this, anyway? I have a couple of theories…

Evolutionary Chaos: This model was chaos from the start. Someone set up an environment for testing an app a long time ago. It did its job and was later repurposed. Then, it got repurposed again. And again. Eventually, it started to grow hair. Then an arm sprouted out of its back. Then it grew an extra leg. Suddenly, it began to “self-organize.” Now it seems to have a mind of its own. It grew out of chaos!
Entropic Chaos: Entropy is always at play. It takes work to keep it from causing decay. In this theory, things were great in the past. But over time, service management became less and less of a priority for this environment. Entropy won the day, and the situation degraded into chaos.

However the environment got into its current chaotic state, the outcomes are the same. Issues are resolved slowly (if at all). Time is wasted digging up information or piecing it together. Data becomes lost, corrupted, and insecure. Owning chaos is a burden and a huge risk in many respects. We don’t want to end up here!

If you’ve made it this far and still have your lunch in tow, you’re past the worst of it. You can uncover your eyes, but be wary! Next, we’re going to look at a wholly buckled down environment and how it can go wrong in other ways.

When You Have Too Much ITSM in Non-Prod

It’s better to have too much service management than not enough. But it’s still not ideal. For one thing, it’s wasteful. For another, it causes morale to suffer. Granted, it’s reasonable to default to production-level service management at first. But staying on default is a symptom of a big problem—communications breakdown. And the root cause of having too much ITSM is due in part to human nature and in part to organizational legacy.

Here are my two theories on how organizations end up here:

Single-Moded Process: Service delivery, operations, and all other departments focused on service management are hell-bent on making sure the customer is absolutely satisfied with their service. Going the extra mile to make sure the customer is happy is a good thing! Operations folks are trained on production-level service management, so their priority is to keep the trains running. With this in mind, operations management systems are set up for production environments. It’s easiest to use that same default everywhere. For better or worse, every environment is treated like a production environment!
Fractured Organization: Organizations are sub-divided into functional groups. When these groups aren’t aligned to a shared purpose, they’ll align to their own purposes. They even end up competing with each other. They’ll center up on their own aims, tossing aside the needs of others.

How You Know When There’s a Problem

The fractured organization theory may explain what happened to a friend of mine recently. Let’s call him Fabian.

Fabian was the on-call engineer this past June. The overnight support team woke him up several nights in a row for irrelevant issues in the development environment. He brought this up to operations, who were responsible for managing the alert system. Unfortunately, the ops engineer was not sympathetic to his concerns in the slightest. Instead, the ops guy put it upon Fabian to tell him what the alert system should do. That’s understandable, but Fabian had no information to that aim. The ops guy wouldn’t share anything with Fabian or collaborate with him on putting a plan together.

This story illustrates a misalignment between operations and development. Problems like this crop up all over the place. Usually, we can remedy or even avoid these situations by taking just a bit more time to understand the other side.

The four theories I’ve presented tell us about extremes. And yes, these extremes push the boundaries and aren’t likely to occur. Still, an organization sitting somewhere in the middle may not have the right service management in non-production. As we’ve seen with Fabian’s story, this is often an issue of misaligned goals.

So how do we get to just enough service management? Maybe the answers lay in what’s working so well for DevOps! Let’s see how.

Just Enough Service Management

IT teams have members with specialties suited to their functional area. Operations folks keep the wheels turning. QA makes sure the applications behave as promised. There are several other specialties—networking, security, and development are just a few examples. Ideally, all of these teams interact and work together toward a well-functioning IT department. But it doesn’t just happen. It takes some key ingredients.

Leadership

Working together effectively takes good leadership. Leadership happens at all levels in an organization. Remember, a leader is a person, not a role.

Shared Vision

It’s also critical to have a shared vision and shared goals. Creating a shared vision is part of being a leader. Here are a few points to remember about vision:

A shared vision creates alignment.
The vision should be exciting to everyone.
You have to do some selling to get everyone aligned with the vision.

Your vision for the test environment could be something like: “Our test environment will be a well-oiled machine.” Use metaphors like “Smooth Operators” or “Pit Crew” to convey the right modes of thinking.

Open Communications

Keep communications open and honest. Open, honest communications can be one of the most significant challenges you’ll face in implementing the right amount of service management. Many of us have a hard time being honest for fear of looking weak in the eyes of others. That fear is difficult to overcome, especially in an environment where we don’t feel safe and secure. Managers have the vital task of creating an environment where employees feel safe and able to communicate openly. Trust is essential to success.

One Last Look

Getting the wrong amount of service management in any environment is a problem. Too little opens up all kinds of risks. Too much ITSM results in wasted time and resources. In this post, I presented four theories for how an organization might end up with the wrong amount of service management in non-prod and discussed what changes you can make to correct that.

ITSM doesn’t happen in a bubble. It takes alignment between many stakeholders. There are three main things we can do to get alignment: wear your leader hat, share the vision, and converse honestly. You can accomplish any goal when you’re set up to win—even with something as challenging as achieving just enough service management.

The 8 Dimensions of the EMMI (Environment Management Maturity Index)

If your interested in IT & Test Environments Management then you have probably heard of the Environment Management Maturity Index (EMMI), the de-facto standard for measuring ones Test Environment Management capability.

If not then let me summarize: the EMMi is a maturity index that provides you with a standard frame of reference to help you assess your strengths, weaknesses, opportunities, and threats.

A powerful tool for assessing your environment and operational capability across your enterprise and help you quickly opportunities to improve.

As shown in the diagram, the EMMI does this by scoring you on eight key performance areas (KPAs). Today, I’ve decided to dive deeper into each of those key performance areas so that you can make a well-informed assessment.

KPA 1: Environment Knowledge Management

First up is environment knowledge management. This refers to your ability to understand how your projects move through all your environments, including development, testing, staging, demoing, and production.

However, this is about more than just one software team. This is about understanding how your systems are connected in each environment across multiple software systems and business units. You will likely need a few models of both low-level relationships and higher-level connections of your systems to gain a strong understanding.

When you know how your software systems are connected as they move through environments, you can avoid many problems. You reduce the risk of disruption when a team needs to release to a new environment. For example, if your billing system is dependent upon your product catalog and the product team releases a new version to QA, you may suddenly see network timeouts when you call the service. That timeout is probably due to a performance bug. If you understand how these systems are connected in QA and if you know the process well, you’ll avoid hours or days of triaging, trying to figure out why your tests are intermittently failing.

KPA 2: Environment Demand Awareness

Next up we have environment demand awareness. This is not about how much load is on your environments. It’s about why you have those environments. Ideally, you should know who’s using them and why. Some environments may have obvious uses, like development. However, other uses may be surprising.

Take QA for example. I was once on an engagement where we developers thought it was our job to test out new features before we released to production. So we kept changing the setup to suit our needs. Eventually, a flock of business analysts came our way, yelling and waving their arms for us to stop. It turned out that many of our customers used QA to test out significant pieces of data before they staged into production, and we were deleting their hard work. Knowing who’s using your environments and why will prevent these kinds of things from happening.

When you know who’s placing demands on your environments, you can also plan better. You may know of a new group of users coming in the pipeline. Or perhaps your environment is taking a hit from many users at once. If you realize you have two different sets of users in that environment, you can split that environment. You can even tailor each environment depending on those users’ needs.

KPA 3: Environment Planning & Coordination

Once you know who’s connected to your environments as well as who’s making demands of them, you can plan for their needs and yours. It’s key to be able to consistently plan and roll out environmental changes to meet upcoming milestones across your enterprise.

Imagine if one of the product team members decided to load test their catalog system and generated five million fake products in their QA environment. This ripples forth to your QA, and none of the purchasing testers can actually do any work. This in turns clogs up their deployments and delays your ability to launch. We can avoid these types of problems with good planning and coordination.

It’s also important that your planning and coordination is consistent across teams. When you have a consistent process, all the teams will know when to share knowledge and when to synchronize efforts.

KPA 4: Environment IT Service Management

It’s not enough to deliver and manage your environments. Since you have users who demand these environments, we need to put on our customer service hat and support their ongoing use. We should diligently manage incidents, changes, and releases to ensure our users are getting what they need. If we neglect the ongoing support and operations of our users in these environments, the piling amount of incidents and user demand may threaten to overwhelm us.

When we spin up a new environment, we need to ensure the appropriate teams own it end to end. They need to have the necessary tooling and operational support to maintain this environment for its entire lifetime. This means well-understood communication on incident resolution and criticality. And it means well-understood processes to manage changing environmental needs.

KPA 5: Application Release Operations

Alright, this one gets a little tricky. It’s healthy to have consistent and repeatable processes across your enterprise for releasing applications. But it’s an easy risk to read this and interpret it as “standardize your deploys.” I want to be clear: application release and deploys are not the same things.

Your deploys are all about getting packaged source code to the right place. But application release is about exposing new functionality to customers. At the lowest maturity, this happens only during deployment. But with mature teams, we can use tooling and processes to separate the idea of deploying code from activating it for customers.

This means you want to ensure your software teams are equipped to continually deliver code to production and to do it in as automated a fashion as possible. Once your teams are doing this, we can shift our focus to how to activate—or release—this code to our customers. There are many tools to help you make this change. It’s this process that you want to standardize across your organization. That way, customers know what to expect, and they’ll understand how to check if new features have arrived.

KPA 6: Data Release & Privacy Operations

Let’s talk about another key performance area: data release.

Data release across your environments is just as important as application release. But it’s often neglected. Each application team ideally owns its own data, but teams need to be explicit how they manage that across their environments.

Time for another story. I knew a team that was quickly delivering high-value financial software, but they depended on a few backend services. Some of these services had a data refresh that occurred once a quarter or so. However, they didn’t make this known to the team, so the team had set up their QA environment with a test bed of data to give them a speedy turnaround time on user stories. This data refresh hit them like a punch in the gut. It killed their velocity for weeks.

It’s healthy to avoid such problems in your enterprise. We want to ensure data release processes are well known and consistent across teams. It’s also a good idea to automate as much as possible to ensure this consistency stays intact, letting our teams work on more valuable efforts.

KPA 7: Infrastructure & Cloud Release Operations

In the same vein as data releases, infrastructure releases have an indirect but profound impact on your teams’ applications. How you handle your infrastructure has a ripple effect across multiple applications. If managed well, you can provide a cushion of protection for software systems to run and fail in isolation. If mismanaged, it can bring down a whole ecosystem of applications.

One would think I’d be out of stories by now, but I have another: I was on an engagement at a Fortune 10 company that, as far as I know, is still mismanaging their infrastructure releases. They built an in-house cloud platform from the ground up, but they didn’t consider their environmental demand, nor did they create an automated and repeatable system. They instead created a system that requires every application team on it to move every few months. And every move brings with it different problems. They provide no tooling to automate this move. At one point, they would consistently lose a data center every week for three weeks straight. Not only was the platform unstable but it also actively hampered application teams from delivering because they were too busy migrating their infrastructure.

There are many tools to help us manage this effectively. We can take advantage of external cloud platforms. We can practice infrastructure as code principles. Also, we can use configuration management tools to ensure our environments are consistent and we can always go back to a fresh state.

Think of your infrastructure releases as a bed frame, and you want your software teams to feel like they are lying on a comfortable mattress, not a bed of rocks.

KPA 8: Status Accounting & Reporting

Complex systems are quickly becoming table stakes in the world of IT. This complexity makes it harder and more valuable to stay on top of your system health and behavior. Yet the faster you can make decisions about your systems and react to problems, the more competitive you will be.

Throughout your teams, you want to ensure you have ways of understanding team health. That way, you can support troubled areas. You want to monitor system health so that you can triage and fix defects before your customers even know. And you want to get real-time data on your system behavior so that you can react faster than your competitors and get new features out quickly.

This is connected with the infrastructure release key performance area, as you want to equip your software teams with standard tooling to accomplish all of this. The more consistent your tooling, the more you can aggregate data and see behavior across multiple systems.

Multi-Dimensional Success

Getting a handle on these key performance areas across your organization is a potentially tough but worthy endeavor. Mismanaging any of these will cause pain, but handling them well will create a cohesive, value-focused set of teams.

Ready to take the next step? If you’re feeling confident about your environments or you’re just curious, go ahead and calculate your environmental maturity. The results will give you insight into what area most needs your attention.

Author Archive: TestEnvMgt