Education

Home » Education » What is a Compliance Framework?

What is a Compliance Framework?

A compliance framework is a structured set of guidelines to aggregate and harmonize, then integrate, all compliance requirements applicable to an organization.

And more importantly, what isn’t a compliance framework? (Just because the authors call something a framework, doesn’t mean it is actually a framework.)


Contents

What is a Compliance Framework?
Why do Compliance Frameworks Exist?
The Definition of a Compliance Framework
Identifying Mandates
Tracking Authority Documents
Extracting Citations
Identifying the Mandates
Identifying Auditable Mandates
Mapping Mandates to Like Mandates
Crosswalking
Transformation and Harmonization
Onus probandi – the burden of proof
Bonus - Standardizing Audit Questions
Which of these are frameworks (and which are not)?
Closed Versus Open Frameworks
Structure


Why Do Compliance Frameworks Exist?

Compliance simply means following the rules that are set by people other than ourselves. In more specif-ic terms, compliance is ensuring that the requirements of laws, regulations, industry codes, and organizational doctrines are met. This also applies to contractual arrangements to which the business process is subject. Most organizations fall under multiple Authority Documents (laws, regulations, standards, audit guides, etc.). Someone in your organization is bound to say “hey, figure out what we need to do to com-ply with these various documents”.

Multiple Authority Documents

Introducing multiple Authority Documents

Once those multiple Authority Documents are examined, of course they are going to have many mandates in them, some of those Mandates seemingly (but not word-for-word) the same.

Examples of multiple mandates

Oh boy, multiple (and maybe overlapping) mandates

Here’s what we know so far:

    • you’ll have multiple documents you need to be in compliance with,
    • that have many mandates per document that you’ll need to interpret and apply, and
    • somebody is going to want you to self-assert or audit you to provide you’ve implemented methods and practices to comply with those mandates.

That means that you are going to want to achieve three things:

  1. Identify the Mandates within the Citations of each Authority document.
  2. Map Mandates to like Mandates (to eliminate duplication).
  3. Create an audit methodology to prove you’ve implemented the mandates.

How do you do that?

There are three ways of ways of doing it.

  1. The first is to just wing it and hope nobody notices. Seriously. We see this done all the time. Not a great idea.
  2. The second is to pick a known standard, such as an international standard like one of the ISO stand-ards, an audit standard like CobiT, a national standard like those the NIST 800-53 standard, or even something made up (like Secure Controls Framework) as the cornerstone and interpret every man-date according to that cornerstone.
  3. The third is to follow a compliance framework designed specifically to integrate multiple documents into a cohesive whole.

The first two methods don’t work.


The Definition of a Compliance Framework

Let’s start with the definition of framework (that applies here, as opposed to creating an actual building); a basic structure underlying a system, concept, or text. So, a framework is a structure, a schema, method of organization and configuration to accomplish something.

When dealing with compliance frameworks, that structure and schema focus on the aggregation of com-pliance rules, first and foremost. Once identification and harmonization are complete, those require-ments need a structure for integration into the organization’s processes as well. Therefore, we can define compliance frameworks as such:

A compliance framework is a structured set of guidelines to aggregate and harmonize, then inte-grate, all compliance requirements applicable to an organization.

In other words, a compliance framework is a methodology for compiling multiple authority docu-ments into a cohesive whole.

It provides a structure for identifying Mandates within Citations.

It provides a structure and methodology for harmonization.

It provides the structure and proof to support the veracity of the identification and harmonization.

These are the defining requirements:

1 Identify Mandates1a Provide a structure for identifying source documents.
1b Provide a structure for identifying Citations within those documents.
1c Provide a structure for identifying Mandates within Citations.
1d Provide a structure for linking a Mandate’s predicates and subjects to their situational definitions.
2 Map Mandates to like Mandates or a reference control2 Provide a structure for measuring correlation between Mandates or a reference control.
Provide proof of identification and mapping3a Provide the necessary data structures such as JSON-LD for encod-ing the tagging, dictionary linking, harmonization, and audit trails for change management that are machine readable.
3b Embed data structures and subsequent data into the identification and harmonization of each record.

Let’s look at each of the defining requirements of a compliance framework, in more depth.


Identifying Mandates

Authority Documents are comprised of Citations, some of which have Mandates, with some of those Mandates being auditable. Let’s break this sentence down into its constituent parts and examine it.

Authority DocumentsAnything from laws through regulations, safe harbors, standards, self-regulatory body guidelines, etc.
have CitationsThe basic unit of content, such as numbered sections or paragraphs of documents.
that have MandatesThe actual “go and do this” parts of each Citation.
and some of those Mandates are auditableSome mandates are informational, others are auditable.

The first identifying characteristic, therefore, has a structure and schema for tracking Authority Docu-ments, extracting their Citations, deriving the Mandates from those Citations, and providing audit questions that match those mandates.


Tracking Authority Documents

First and foremost, there needs to be some structure and rules for tracking the sources of the Citations and Mandates.

Tracking authority documents

Tracking Authority Documents

This can be as simple as the framework providing a register of all of the Authority Document sources, to a full-fledged data structure for Authority Document library management.


Extracting Citations

Citations live inside of each of the Authority Documents. There compliance framework needs to provide a methodology for Citation selection, i.e., which Citations in an Authority Document can be ignored, and which must be included into the organization’s list of Citations to work with.

Citations from Authority Documents

Citations from an Authority Document

Not only does the compliance framework need to provide a structure by which to extract and include Ci-tations, it should also provide a structure to identify and track each Citation. Only some documents will have well-structured formats that number their Citations. Others merely put them onto a page and re-quire some form of paragraph and page or line identification.


Identifying the Mandates

A Mandate is a declarative independent clause (a predicate and a subject) that says “do this” or “do that” or “test that this was done”. Mandates must be extracted from each Citation (or at least each Citation that forms a complete sen-tence). Therefore, the compliance framework must have guidelines and methods for Mandate identifica-tion, extraction, and tracking.

At bare minimum it must provide guidance and methodologies for how each Citation’s primary verbs are extracted from the Citation;

Analyzing verbs

Analyzing verbs

as well as guidance and methodologies for how each Citation’s primary nouns are extracted.

Analyzing Nouns

Analyzing nouns


Identifying Auditable Mandates

Not all mandates are auditable. Most of them are, but some of them are written so vaguely that they are open to incredible interpretation and therefore auditing them is a waste of time. But for the 95% of Man-dates that are auditable, the compliance framework should provide a methodology for identifying which Mandates are auditable.


Mapping Mandates to like Mandates

Remember that the primary goal of any compliance framework is to reduce the burden of following mul-tiple guidelines by finding commonality between Mandates. Once Mandates have been identified and ex-tracted from Citations, the compliance framework must provide a suite of rules and methods for deter-mining their commonality.

This has been called crosswalking, harmonizing, mapping or unifying one regulation or standard to another. For the sake of sanity, we are going to call this process, in general, mapping because crosswalking and harmonizing are both a part of mapping and unifying is too broad. There are four basic methodologies to achieve the goals of defining what data matches and what data doesn’t;

  1. semantic mappings,
  2. crosswalking rules,
  3. transformation, and
  4. harmonization.

Each of these are combined into either crosswalking or harmonization models. Any form of mapping or unifying compliance must first have a set of relationship rules. Even the most basic system must have a set of rules to map single data sets to differing data sets.


Crosswalking

Crosswalking’s goal is to determine if concept 1 matches concept 2. But it doesn’t end there. Crosswalking is a one-to-one task. Given 4 concepts, crosswalking must be performed 6 times to determine which concepts match (or don’t match) each other.

Crosswalking

Crosswalking

Crosswalking is task intensive. The calculation below shows that 50 concepts creates 1,225 crosswalking tasks. 100 concepts crosswalked to each other creates 4,950 tasks. Too many tasks to be truly leverageable as a standalone methodology.

Crosswalking Tasks = (N*N-1)/2


Transformation and Harmonization

Unlike crosswalking where each Mandate is determined to be like any other, transformation strips each Mandate down into its declarative kernel (predicate subject pair) and connects the Mandate to a match-ing, stripped down version of itself. Transformation is a one-to-one ratio of concepts to concepts being transformed as shown below.

Transformation of a mandate

Transformation of a Mandate

Once a concept has been transformed, the original concept is mapped to it. Harmonization is the process of either crosswalking new concepts to the transformed one or creating additional transformed concepts if no crosswalk exists.

Harmonization

Harmonization

Therefore, each concept is either crosswalked to an existing harmonized man-in-the-middle concept, or a new harmonization takes place. Given 4 concepts that match the same transformed concept, there would only be 5 tasks. The number of additional tasks depends upon how many new transformations have to take place. Whatever the number, it is surely less than (N*N-1)/2.


Onus probandi – the burden of proof

The third defining requirement of a compliance frameworks is to provide proof of identification and mapping. In Latin, and in the field of law, onus probandi means that there is a burden of proof to be made to support an argument.

Proof is the difference between opinion and truth. There’s a great movie that is focused on the difference between presenting a brilliant outcome and supporting the proof that the outcome is viable – The Man who knew Infinityi. It is the story about Indian mathematics genius Srinivasa Ramanujan who shocked and surprised the English mathematical establishment at the start of the 20th century by the depth and originality of his research in additive number theory. The problem was, he just stated his results without explanation or any kind of proof. A major theme of the movie is this idea of a mathematical ‘proof’. He comes to Trinity College in Cambridge to work with G. H. Hardy who repeatedly emphasizes the im-portance of providing logically sound arguments and proofs of Ramanujan’s mathematical statements. Only with proof can one be sure statements are true beyond mere examples and can be trusted forever.

If compliance frameworks are going to make the statement (the argument) that X Citation maps to Y reference control, they need to prove it. Without proof, the statement X Citation maps to Y reference controls is merely an opinion.

What is onus probandi?

Major thanks to my co-Founder and brilliant attorney Marcelo Halpern, here’s what he has to say about onus probandi. As he states it, “here’s how Black’s Law Dictionary defines “burden of proof” (as linked from “onus probandi”)ii:

burden of proof (18c) 1. A party's duty to prove a disputed assertion or charge; a proposition re-garding which of two contending litigants loses when there is no evidence on a question or when the answer is simply too difficult to find. • The burden of proof includes both the burden of persuasion and the burden of production. — Also termed evidentiary burden; evidential burden; onus probandi. See SHIFTING THE BURDEN OF PROOF. Cf. STANDARD OF PROOF. 2. Loosely, BURDEN OF PERSUASION. — Abbr. BOP.

In the past the term ‘burden of proof’ has been used in two different senses. (1) The burden of going forward with the evidence. The party having this burden must introduce some evidence if he wishes to get a certain issue into the case. If he introduces enough evidence to require consideration of this issue, this burden has been met. (2) Burden of proof in the sense of carrying the risk of nonpersua-sion. The one who has this burden stands to lose if his evidence fails to convince the jury — or the judge in a nonjury trial. The present trend is to use the term ‘burden of proof’ only with this second meaning …” Rollin M. Perkins & Ronald N. Boyce, Criminal Law 78 (3d ed. 1982).

The expression ‘burden of proof’ is tricky because it has been used by courts and writers to mean var-ious things. Strictly speaking, burden of proof denotes the duty of establishing by a fair preponder-ance of the evidence the truth of the operative facts upon which the issue at hand is made to turn by substantive law. Burden of proof is sometimes used in a secondary sense to mean the burden of go-ing forward with the evidence. In this sense it is sometimes said that a party has the burden of coun-tering with evidence a prima facie case made against that party.” William D. Hawkland, Uniform Commercial Code Series§ 2A-516:08 (1984).

He goes on to point out that there is also the concept of “Standard of Proof” which is defined asiii:

standard of proof (1857) The degree or level of proof demanded in a specific case, such as “beyond a reasonable doubt” or “by a preponderance of the evidence”; a rule about the quality of the evidence that a party must bring forward to prevail. — Also termed degree of proof. See BURDEN OF PERSUASION. Cf. BURDEN OF PROOF.

And looking at it from the “proof” side, there’s the concept of Due Proofiv:

due proof (16c) Sufficient and properly submitted evidence to produce a result or support a conclu-sion, such as an entitlement to benefits supported by an insurance policy. • The evidence need not be the best proof possible. Metropolitan Life Ins. Co. v. Frisch, 65 N.E. 2d 852, 855 (Ind. App. 1946).

Clearly, if any person or organization is going to present a harmonized compliance framework, it is upon them to carry the burden of proof that the Citations do in fact correlate to the reference framework’s controls.

How is the burden of proof communicated?

We know that at minimum, each compliance framework will have three data elements to it:

  1. the reference controls and
  2. the Authority Documents from which
  3. Citations are drawn.

In its simplest form, these three data elements can be connected as a model to imply proof.

In its most complete form, these three data elements should be connected as a schema that demostrates proof.

In between the two is the methodology that is used to connect each of the elements together that supply the proof.

In order to understand a compliance framework’s proof, you need to examine its data structure to ensure that the data structure can even provide the proof.


Bonus - Standardizing Audit Questions

Most Authority Documents don’t write their own audit questions (and neither do most frameworks). The PCI DSS is one of the very few that does. So did the FTC’s Red Flag Rules guidance. Other than that, almost nothing.

Most audit questions are created by either a working group in a framework committee or (worse) audit management software teams (where did you think they came from, the audit fairy?).

Only one framework to date has a published standard on methodologies and structures for creating audit questions – the Unified Compliance Framework. Audit Questions to meet ABA requirements, must be asked in Yes/No format and involve either examination, interviews, observation, or testing.

There are two types of auditing methodology that a compliance framework might provide – the simple format and the evidential-based format.

Simple audit questions are often stated as yes or no questions.

  • Are vendor-supplied defaults always changed before installing a system on the network?
  • Have configuration standards been developed for all system components?

Evidential-based audit questions provide a methodology to answer the question as well as force the organization being audited to rely on evidence in order to come to the conclusion.

  • Test a sample of All Assets to ensure the configuration item of Default configuration settings is "Account and password settings do not match vendor supplied defaults." Is this configured correctly?
  • Examine the System Security Configuration standard. Does it ensure that it covers all known configuration items and system components?

Evidential-based audit questions format the question method to the subject being audited. How do you employ these additional elements of evidence? By formatting the audit question methods into test, observe, examine, and interview type questions. Evidential items to support the answers can then be linked to each question.

  • Test applies to testing systems, testing computations, etc.
  • Observe applies to watching processes happen.
  • Examine applies to records or assets that can be scrutinized.
  • Interview is reserved for speaking to individuals or groups.

Which of these are frameworks (and which are not)?

We now know that to be considered a framework the document or structure must provide these three things:

1 Identify Mandates1a Provide a structure for identifying source documents.
1b Provide a structure for identifying Citations within those docu-ments.
1c Provide a structure for identifying Mandates within Citations.
1d Provide a structure for linking a Mandate’s predicates and subjects to their situational definitions.
2 Map Mandates to like Mandates or a reference control 2 Provide a structure for measuring correlation between Mandates or a reference control.
3 Provide proof of identification and mapping3a Provide the necessary data structures such as JSON-LD for encod-ing the tagging, dictionary linking, harmonization, and audit trails for change management that are machine readable.
3b Embed data structures and subsequent data into the identification and harmonization of each record.

Here is a list of documents people frequently think are frameworks, but are not:

  • CobiT
  • NIST Cybersecurity Framework
  • CIS Controls
  • NIST 800-53
  • ISO 27000 series

These documents, while one might even say framework in the title, are written as suites of mandates without reference to, or harmonization with, any other sets of Authority Documents. In other words, they were written as standalone documents – and nothing is wrong with that. It’s just that they aren’t frameworks.

Here is a short list of widely known frameworks that are frameworks:

  • AICPA Trust Services Criteria
  • HITRUST
  • PCI-DSS
  • Shared Assessments
  • Unified Compliance Framework
  • Secure Controls Framework
  • Nymity Privacy Framework
  • Proposed NISTIR 8204 framework for comparing frameworks

PCI-DSS is a framework, but it is also an anomaly in the group. PCI-DSS is a framework because it maps its own documents to itself. Initially, there were only the controls within the main document (PCI-DSS). Then it broke itself down into sub-documents, A through F, mapping each sub-document to the primary one. From there, it has created extended guidance for automated terminals, networking, etc., all mapping back to the core set of PCI-DSS controls.

Each of the other frameworks were built by extracting Mandates from source Authority Documents (and have rules for doing so), harmonizing those Mandates into the framework, and then sometimes creating audit guides from the resulting de-duplicated harmonized controls.

Can internal controls be considered a compliance framework?

In a word – yes.

In a sentence – if you use your internal controls as reference controls and map Citations in Authority Documents to them, then yes.

In another sentence – if you use something like the Unified Compliance Framework as the reference con-trols and you map your internal controls to them, then you are still including your internal controls in the compliance framework, so yes.

We will spend time showing you how to do establish, implement, and maintain your internal controls as your own, bespoke, compliance framework.


Closed versus open frameworks

Closed frameworks, such as HITRUST and Shared Assessments provide content, but they

  • have no published standard on mandate extraction, harmonization, or audit question creation;
  • apply the controls as provided by the framework and wait for the committee to update the framework; and
  • provide no published methodologies for tailoring and “self-reduction” of mandates
  • provide audit questions but no published methodologies for how those questions were created.

Open frameworks, such as the Unified Compliance Framework and the proposed NISTIR 8204 provide either content as well as

  • published standards on mandate extraction, harmonization, audit question creation;
  • the framework itself provides mapping tools to extend the framework by anyone trained to do so; and
  • published methodologies and tools to tailor the mandates to only those necessary for each organization


Structure

The structure of most closed frameworks is simple. They provide tracking for Authority Documents, their Citations, and audit questions.

Closed Framework Structure

Closed framework structure

NIST’s proposed 8204, while an open structure, provides only elements for Issuers (where Authority Documents can be found), the Authority Documents, and their Citations. It doesn’t, at this writing, provide a structure for audit questions.

NISTIR 8204 Structure

NISTIR 8204 structure

The Unified Compliance Framework has a very complex structure that stretches from tracking Issues through Authority Documents, Citations, Common Controls, a Dictionary, and various role, asset, record, activity, events, and audit questions.

Unified Compliance Framework Structure

Unified Compliance Framework structure