And more importantly, what isn’t a compliance framework? (Just because the authors call something a framework, doesn’t mean it is actually a framework.)
What is a Compliance Framework?
Why do Compliance Frameworks Exist?
The Definition of a Compliance Framework
Tracking Authority Documents
Identifying the Mandates
Identifying Auditable Mandates
Mapping Mandates to Like Mandates
Transformation and Harmonization
Standardizing Audit Questions
Which of these are frameworks (and which are not)?
Closed Versus Open Frameworks
Compliance simply means following the rules that are set by people other than ourselves. In more specific terms, compliance is ensuring that the requirements of laws, regulations, industry codes, and organizational doctrines are met. This also applies to contractual arrangements to which the business process is subject. Most organizations fall under multiple Authority Documents (laws, regulations, standards, audit guides, etc.). Someone in your organization is bound to say “hey, figure out what we need to do to comply with these various documents”.
Introducing multiple Authority Documents
Once those multiple Authority Documents are examined, of course they are going to have many mandates in them, some of those Mandates seemingly (but not word-for-word) the same.
Oh boy, multiple (and maybe overlapping) mandates
Here’s what we know so far:
That means that you are going to want to achieve three things:
How do you do that? There are three ways of ways of doing it. The first is to just wing it and hope nobody notices. Seriously. We see this done all the time. Not a great idea. The second is to pick a known standard, such as an international standard like one of the ISO standards, an audit standard like CobiT, or a national standard like those the NIST 800-53 standard as the cornerstone, and interpret every mandate according to that cornerstone. The third is to follow a compliance framework designed specifically to integrate multiple documents into a cohesive whole. The first two methods don’t work.
Let’s start with the definition of framework (that applies here, as opposed to creating an actual building);
a basic structure underlying a system, concept, or text. So, a framework is a structure, a schema, method of organization and configuration to accomplish something. When dealing with compliance frameworks, that structure and schema focuses on aggregation of compliance rules, first and foremost. Once aggregation and harmonization are complete, those requirements need a structure for integration in to the organization’s processes as well. Therefore, we can define compliance frameworks as such: A compliance framework is a structured set of guidelines to aggregate and harmonize, then integrate, all compliance requirements applicable to an organization. In other words, a complianceframework is a methodology for compiling multiple authority documents into a cohesive whole. It provides a structure for harmonization. It provides a structure for integration. The goal of any compliance framework is to reduce the burden of following multiple guidelines by finding commonality between mandates and then maintaining accordance with those common mandates using the organization’s compliance processes and tools. We now have the defining requirements for compliance frameworks:
|identify Mandates||Define rules to extract Mandates from Citations within Authority Documents|
|map Mandates to like Mandates||Map Mandates from all Citations to other Mandates like them|
|standardize audits||Leverage a standardized structure for auditing the implementation of the selected set of mapped Mandates|
Let’s look at each of the defining requirements of a compliance framework, in more depth.
Authority Documents are comprised of Citations, some of which have Mandates, with some of those Mandates being auditable. Let’s break this sentence down into its constituent parts and examine it.
|Authority Documents||Anything from laws through regulations, safe harbors, standards, self-regulatory body guidelines, etc.|
|have Citations||The basic unit of content, such as numbered sections or paragraphs of documents.|
|that have Mandates||The actual “go and do this” parts of each Citation.|
|and some of those Mandates are auditable||Some mandates are informational, others are auditable.|
The first identifying characteristic, therefore, has a structure and schema for tracking Authority Documents, extracting their Citations, deriving the Mandates from those Citations, and providing audit questions that match those mandates.
First and foremost, there needs to be some structure and rules for tracking the sources of the Citations and Mandates.
Tracking Authority Documents
This can be as simple as the framework providing a register of all of the Authority Document sources, to a full-fledged data structure for Authority Document library management.
Citations live inside of each of the Authority Documents. There compliance framework needs to provide a methodology for Citation selection, i.e., which Citations in an Authority Document can be ignored, and which must be included into the organization’s list of Citations to work with.
Citations from an Authority Document
Not only does the compliance framework need to provide a structure by which to extract and include Citations, it should also provide a structure to identify and track each Citation. Only some documents will have well-structured formats that number their Citations. Others merely put them onto a page and require some form of paragraph and page or line identification.
A Mandate is a declarative independent clause (a predicate and a subject) that says “do this” or “do that” or “test that this was done”. Mandates must be extracted from each Citation (or at least each Citation that forms a complete sentence). Therefore, the compliance framework must have guidelines and methods for Mandate identification, extraction, and tracking. At bare minimum it must provide guidance and methodologies for how each Citation’s primary verbs are extracted from the Citation;
as well as guidance and methodologies for how each Citation’s primary nouns are extracted.
Not all mandates are auditable. Most of them are, but some of them are written so vaguely that they are open to incredible interpretation and therefore auditing them is a waste of time. But for the 95% of Mandates that are auditable, the compliance framework should provide a methodology for identifying which Mandates are auditable.
Remember that the primary goal of any compliance framework is to reduce the burden of following multiple guidelines by finding commonality between Mandates. Once Mandates have been identified and extracted from Citations, the compliance framework must provide a suite of rules and methods for determining their commonality. This has been called crosswalking, harmonizing, mapping or unifying one regulation or standard to another. For the sake of sanity, we are going to call this process, in general, mapping because crosswalking and harmonizing are both a part of mapping and unifying is too broad. There are four basic methodologies to achieve the goals of defining what data matches and what data doesn’t;
Each of these are combined into either crosswalking or harmonization models. Any form of mapping or unifying compliance must first have a set of relationship rules. Even the most basic system must have a set of rules to map single data sets to differing data sets.
Crosswalking’s goal is to determine if concept 1 matches concept 2. But it doesn’t end there. Crosswalking is a one-to-one task. Given 4 concepts, crosswalking must be performed 6 times to determine which concepts match (or don’t match) each other.
Crosswalking is task intensive. The calculation below shows that 50 concepts creates 1,225 crosswalking tasks. 100 concepts crosswalked to each other creates 4,950 tasks. Too many tasks to be truly leverageable as a standalone methodology.
Crosswalking Tasks = (N*N-1)/2
Unlike crosswalking where each Mandate is determined to be like any other, transformation strips each Mandate down into its declarative kernel (predicate subject pair) and connects the Mandate to a matching, stripped down version of itself. Transformation is a one-to-one ratio of concepts to concepts being transformed as shown below.
Transformation of a Mandate
Once a concept has been transformed, the original concept is mapped to it. Harmonization is the process of either crosswalking new concepts to the transformed one or creating additional transformed concepts if no crosswalk exists.
Therefore, each concept is either crosswalked to an existing harmonized man-in-the-middle concept, or a new harmonization takes place. Given 4 concepts that match the same transformed concept, there would only be 5 tasks. The number of additional tasks depends upon how many new transformations have to take place. Whatever the number, it is surely less than (N*N-1)/2.
Most Authority Documents don’t write their own audit questions. The PCI DSS is one of the very few that does. So did the FTC’s Red Flag Rules guidance. Other than that, almost nothing. Most audit questions are created by either a working group in a framework committee or (worse) audit management software teams (where did you think they came from, the audit fairy?). Only one framework to date has a published standard on methodologies and structures for creating audit questions – the Unified Compliance Framework. Audit Questions to meet ABA requirements, must be asked in Yes/No format and involve either examination, interviews, observation, or testing. There are two types of auditing methodology that a compliance framework might provide – the simple format and the evidential-based format. Simple audit questions are often stated as yes or no questions.
Evidential-based audit questions provide a methodology to answer the question as well as force the organization being audited to rely on evidence in order to come to the conclusion.
Evidential-based audit questions format the question method to the subject being audited. How do you employ these additional elements of evidence? By formatting the audit question methods into test, observe, examine, and interview type questions. Evidential items to support the answers can then be linked to each question.
We now know that to be considered a framework the document or structure must provide these three things:
|Identify Mandates||Define rules to extract Mandates from Citations within Authority Documents|
|Map Mandates to like Mandates||Map Mandates from all Citations to other Mandates like them|
|Standardize audits||Leverage a standardized structure for auditing the implementation of the selected set of mapped Mandates|
Here is a list of documents people frequently think are frameworks, but are not:
These documents, while they might even say framework in the title, are written without regard to Mandate extraction, harmonization, or audit standardization. While PCI-DSS does have its own audit guidelines, that doesn’t make it a framework. Here is a short list of widely known frameworks:
Each of these frameworks were built by extracting Mandates from source Authority Documents (and have rules for doing so), harmonizing those Mandates into the framework, and then creating audit guides from the resulting de-duplicated harmonized controls.
Closed frameworks, such as HITRUST and Shared Assessments provide content, but they
Open frameworks, such as the Unified Compliance Framework and the proposed NISTIR 8204 provide either content as well as
The structure of most closed frameworks is simple. They provide tracking for Authority Documents, their Citations, and audit questions.
Closed framework structure
NIST’s proposed 8204, while an open structure, provides only elements for Issuers (where Authority Documents can be found), the Authority Documents, and their Citations. It doesn’t, at this writing, provide a structure for audit questions.
NISTIR 8204 structure
The Unified Compliance Framework has a very complex structure that stretches from tracking Issues through Authority Documents, Citations, Common Controls, a Dictionary, and various role, asset, record, activity, events, and audit questions.
Unified Compliance Framework structure