Baselining

Baselining is the term used to describe the process of measuring specific elements of any given entity in order to determine many attributes of that entity in real-time. In the networking, communications and security worlds this means testing, listing and reporting procedures specifically tailored to provide greater insight into that entities capability.

Whenever baselining it is essential that we scrutinize specific parameters, attributes, performance and stability in isolation as well as a part of the whole. When measuring a parameter in isolation we gain insight into the various aspects of that parameter, which indicate the capabilities of the entity in regards to that parameter.

When scrutinizing those same parameters from the perspective of multiple entity behaviors and interactions we are able to see the effects that other entities have upon the first entity's parameter. This way, we can quantify the “impact” that a given element has upon the whole.

From a network/security perspective, the parameters that we are interested in include physical connectivity, idle network utilization, normal network utilization, stress loaded network utilization, maximum network utilization, protocol usage, mean network throughput, underutilized network resources and network security events. Baselining and benchmarking test regimes must be customizable to suit these objectives as and when necessary.

Once collected it is time to structure and analyse the data. Do not forget to pay close attention to the relationships that may or may not exist between the various elements and entities involved during the testing and listing phases of your baselining suite.

You also need to pay close attention to the identification any anomalies, adverse conditions and results as well as any vulnerabilities and compatibility issues. With a baseline established, we can now use this information to perform assessment scenarios detailing the likely outcomes based upon our baseline data.

In addition, we can also extrapolate from our baseline to making assessments and “judgment calls” (not guesses) about the likely impacts factors such as changes to user behavior patterns and the effects that introducing new technologies into our network might have.

This will assist in the planning, timing and deployment of future upgrades such that when the time for an upgrade arrives not only can we identify it but we also have contingency plans ready regardless of scenario or a plan for all possibilities if you like. Now we can begin to look at the role that benchmarking has to play in this.

Benchmarking

Benchmarking is very similar to baselining but where baselining is more about networks and making comparisons of a system or network with itself benchmarking is about making comparisons between different systems.

Baselining will tell us that our network is not performing as it usually does or making deductive analysis that the network is coming under loads that considered to be stretching our network towards its upper end in terms of performance in servicing the needs of the network's users. In this way, we can say that an upgrade is necessary.

Benchmarking on the other hand will tell us the performance of one component relative to another. Do this by running a set of tests that produce definitive quantitative results on different components. We compare these two values one against the other.

If component one scores 88 and component two scores, 108 we can definitively say that component one is not performing as well as component two. For example benchmarking will enable us to determine which is the faster of two CPUs.

Normality

Benchmarking and baselining are tools that should be used in determining your “normal” state. Underestimating their value is a trap that we may all fall into from time to time. The objectives of these tools will vary from scenario to scenario, as will their installation, configuration and maintenance.

We will now look at some of the ways that we all can put these tools to good use.

Determination of “Normality”

The first thing that you should do is to benchmark and baseline your systems, networks, communications and security structures to learn what is “normal”. Here is a list of guidelines that will help you to design and build your own baselining and benchmarking test regimes.

1. Define Mission Objectives

   Clarify and decide on your objectives. In this way, you will be able to eliminate many tools as being unsuitable in this instance even though they may well be the ultimate tool for some other network. This alone does not mean they are suited to your requirements.

2. Research

   Always research the topic. Build a list of currently available tools. Not all are free but many very good tools are (e.g. Belarc Advisor and Everest). Determine the tools that are best for you. Bear in mind that you will not always run the entire test suite of tools every time. Don't forget the tools that came with your operating system and applications.

3. Tools

   Building your own suite of tools is of enormous benefit as this allows you to tailor your test suite to your current circumstances. Read the manufacturer's documentation to ascertain how to use the tool and for which situations that the tool has been built to test/benchmark.

4. Test Suite

   Use a set containing multiple tools (suite). Do not rely on just one tool. Continually evaluate your test suite. Regularly check for updates applicable to your test suite. Locate and evaluate new network, communications and security test technologies.

5. Repetition

   Plan and design your baselining, benchmarking and testing regimes such that they can run as a loop. This will allow you to run your test suite multiple times with as little direct input from you as possible.

6. Test Regularly

   Networks have a habit of changing. Many elements of networks, communications and security structures will evolve and adapt to new technologies and challenges. Heightened security levels are most sustainable when perpetually maintained.

7. Test Frequently

   By frequently conducting your test regime you will be able to get an “early warning” of impending doom. Your test regime is often one of the first indicators that a piece of hardware or software is about to break or that a new security threat exists.

8. Test Randomly

   Conduct randomly timed testing procedures will allow you to test your equipment under as wide a range of “normal” operating environmental conditions, loads and usage patterns as possible. It is impossible to predict every conceivable possibility so test frequently and you will pick up aberrant anomalies.

9. Penetration Testing

   Develop security oriented test regimes that include as many of the “hacker tools” that a hacker might employ in attempting to penetrate your security defenses. This way you can determine your state of readiness to defend against these threats at least.

10. Documentation

    As per usual always thoroughly document your procedures and develop appropriate training and testing Policies suited to your network and systems.