Part 2 specifies normalization steps that transform clauses into binary risk features, using policy‑driven rules and feature stores so deterministic risk engines can score contracts consistently and explain decisions.
In Part 1, we designed an architecture that ingests contracts, extracts clause spans, classifies clause types, and persists them in a clause‑centric data store. This already matches the ingestion and classification stages of many contract risk scoring platforms and workflows.
Now we must bridge the last mile: transforming each clause into a normalized feature vector and ultimately into binary risk signals such as liability_cap_above_policy, dpa_missing, or termination_for_convenience_high_risk. This is where deterministic, rule‑based engines like Legiscope’s rule framework or enterprise compliance scoring models operate.
First, normalize the text to remove noise while preserving semantic content:
import re
def canonicalize_text(text: str) -> str:
# Remove page headers/footers markers you detect in parsing
text = re.sub(r"\n?Page \d+ of \d+\n?", " ", text, flags=re.IGNORECASE)
# Normalize whitespace
text = re.sub(r"\s+", " ", text).strip()
# Normalize quotes and dashes
text = text.replace("“", "\"").replace("”", "\"").replace("’", "'")
text = text.replace("–", "-").replace("—", "-")
# Normalize spelled-out numerals for common patterns (e.g. "twelve (12)" -> "12")
text = re.sub(r"\b([A-Za-z]+)\s*\(\s*(\d+)\s*\)", r"\2", text)
return text
Real‑world systems often embed this logic in their document processing pipelines before clause classification to improve model consistency.
Risk signals must be from your firm’s perspective, not neutral. Professional services risk engines typically provide “party‑aware” analysis, framing risk based on whether you are supplier, customer, or borrower.
You want to map textual references (“Supplier”, “Customer”, “Vendor”, “Controller”, “Processor”) into abstract roles our_party and counterparty, and encode which direction rights and obligations flow. For example:
ROLE_ALIASES = {
"vendor": "supplier",
"provider": "supplier",
"service provider": "supplier",
"client": "customer",
"licensee": "customer",
"licensor": "supplier"
}
def normalize_party_label(raw_label: str) -> str:
key = raw_label.lower().strip()
return ROLE_ALIASES.get(key, key)
def derive_perspective(clause_text: str, our_role: str) -> dict:
# our_role: "supplier" or "customer" decided at contract-level
#
# NOTE: In a true enterprise pipeline, party and perspective are first inferred by
# Legal NER + Coreference Resolution models (e.g., contract-party NER and relation
# extraction pipelines), and this deterministic logic acts as a validation/parsing
# layer on top of those model outputs.[web:81][web:84][web:89]
# Hard-coded patterns like the ones below are for illustration and for schema-level # sanity checks, not the primary source of truth for party roles.
lower = clause_text.lower()
direction = "neutral"
if "supplier shall not be liable" in lower and our_role == "supplier":
direction = "protects_us"
elif "customer shall indemnify" in lower and our_role == "customer":
direction = "creates_our_obligation"
# ... more patterns, or ML-based role assignment ...
return {
"our_role": our_role,
"direction": direction
}
Party‑aware perspective is essential when computing features like “liability cap too low in our favor” vs “liability cap too high against us”, which several risk scoring solutions expose as separate signals.
Legal clauses express amounts and durations in many formats (“twelve months”, “1 year”, “36 calendar months”, “USD 100,000”, “one hundred thousand (100,000) US Dollars”). Risk frameworks compare these values against policy thresholds: e.g., liability cap must be ≥ 12 months of fees, payment term ≤ 45 days, cure period ≥ 30 days.
import regex as re
from decimal import Decimal
CURRENCY_SYMBOLS = {
"$": "USD",
"€": "EUR",
"£": "GBP",
"₹": "INR"
}
DURATION_PATTERNS = [
(re.compile(r"(\d+)\s+year"), 12),
(re.compile(r"(\d+)\s+month"), 1),
(re.compile(r"(\d+)\s+day"), 1/30)
]
def normalize_amount(text: str) -> dict | None:
# Super simplified: "$100,000" or "USD 100,000"
m = re.search(r"(?P<currency>USD|EUR|GBP|INR|\$|€|£|₹)\s*(?P<amount>\d[\d,\.]*)", text)
if not m:
return None
raw_currency = m.group("currency")
currency = CURRENCY_SYMBOLS.get(raw_currency, raw_currency)
amount = Decimal(m.group("amount").replace(",", ""))
return {"currency": currency, "amount": amount}
def normalize_duration_months(text: str) -> float | None:
for pattern, factor in DURATION_PATTERNS:
m = pattern.search(text.lower())
if m:
value = int(m.group(1))
return value * factor
return None
Industrial implementations combine regex, statistical models, and LLM extraction to populate numeric attributes for clauses like liability caps, termination notice periods, renewal windows, and payment terms.
Next, convert normalized text and numeric values into semantic attributes tied to your clause taxonomy. This is where you capture concepts like:
Platforms that auto‑tag clauses and compute risk often describe this as “clause tagging + parameter extraction”, with tags for clause type and attributes for business impact. Research systems recommend hybrid architectures that use classifier outputs plus retrieval over precedent clauses to populate such attributes reliably.
def extract_liability_attributes(clause_text: str) -> dict:
text = clause_text.lower()
attrs = {
"has_cap": "not exceed" in text or "shall not exceed" in text,
"cap_unlimited": "unlimited liability" in text or "without cap" in text,
"includes_indirect_damages": "indirect" in text and "consequential" in text and "excluded" not in text,
"excludes_confidentiality_breach": "except for breach of confidentiality" in text,
"mentions_fees": "fees" in text or "charges" in text,
}
return attrs
The union of numeric normalization and semantic attributes becomes your normalized clause representation.
Now we map normalized attributes to binary risk signals using a policy engine. Deterministic, rule‑based risk engines are widely used in contract risk scoring platforms because they guarantee repeatable, auditable outputs. Many combine these rules with ML‑based risk scores, but the rules remain the backbone for high‑stakes decisions.
# policies/liability_cap.yaml
policy_id: LIAB_CAP_STANDARD
applies_to_clause_type: LIABILITY_CAP > AGGREGATE
our_role: customer
rules:
- id: LCAP_UNCAPPED
condition: "cap_unlimited == True"
risk_flag: "liability_cap_unlimited"
severity: "HIGH"
- id: LCAP_TOO_LOW
condition: "cap_multiple < 1.0"
risk_flag: "liability_cap_below_12m_fees"
severity: "HIGH"
- id: LCAP_OK
condition: "cap_multiple >= 1.0 and cap_multiple <= 3.0"
risk_flag: "liability_cap_within_standard"
severity: "LOW"
- id: LCAP_TOO_HIGH
condition: "cap_multiple > 3.0"
risk_flag: "liability_cap_above_36m_fees"
severity: "MEDIUM"
import operator
from dataclasses import dataclass
from typing import Any
OPS = {
"==": operator.eq,
"!=": operator.ne,
"<": operator.lt,
"<=": operator.le,
">": operator.gt,
">=": operator.ge,
}
@dataclass
class RiskFlag:
flag: str
severity: str
rule_id: str
def eval_condition(expr: str, ctx: dict[str, Any]) -> bool:
# very naive parser: "cap_multiple < 1.0"
tokens = expr.strip().split()
if len(tokens) != 3:
raise ValueError(f"Unsupported expression: {expr}")
left, op, right = tokens
left_val = ctx.get(left)
right_val = float(right) if "." in right or right.isdigit() else ctx.get(right)
return OPS[op](left_val, right_val)
def apply_policy(policy: dict, attributes: dict) -> list[RiskFlag]:
flags: list[RiskFlag] = []
for rule in policy["rules"]:
if eval_condition(rule["condition"], attributes):
flags.append(
RiskFlag(
flag=rule["risk_flag"],
severity=rule["severity"],
rule_id=rule["id"],
)
)
return flags
This type of pattern; policies as data plus deterministic evaluation is exactly how rule‑based contract risk platforms and compliance engines provide consistent clause‑level risk assessments and explainable results. Each RiskFlag can later be encoded as a binary feature.
For downstream ML, dashboards, and scoring, you want a fixed‑dimension binary feature vector per clause (and per contract). Many real systems expose features like “clause present/missing”, “clause deviates from standard”, “risk above threshold” as booleans and multi‑level risk labels.
CREATE TABLE clause_risk_features (
clause_id VARCHAR PRIMARY KEY REFERENCES clause(clause_id),
liability_cap_unlimited BOOLEAN,
liability_cap_below_12m_fees BOOLEAN,
liability_cap_within_standard BOOLEAN,
liability_cap_above_36m_fees BOOLEAN,
includes_indirect_damages BOOLEAN,
termination_for_convenience_present BOOLEAN,
termination_for_convenience_high_risk BOOLEAN,
dpa_missing BOOLEAN,
data_transfer_outside_region BOOLEAN,
-- additional features...
computed_at TIMESTAMP NOT NULL
);
A mapping function from RiskFlag instances and attributes to a feature vector:
from datetime import datetime
def features_from_flags(clause_id: str, flags: list[RiskFlag], attrs: dict) -> dict:
features = {
"clause_id": clause_id,
"liability_cap_unlimited": False,
"liability_cap_below_12m_fees": False,
"liability_cap_within_standard": False,
"liability_cap_above_36m_fees": False,
"includes_indirect_damages": attrs.get("includes_indirect_damages", False),
# ... other defaults ...
"computed_at": datetime.utcnow(),
}
for f in flags:
if f.flag in features:
features[f.flag] = True
return features
This mirrors how RealityCheck‑style tools and workflow engines compute multiple quantified risk metrics and boolean indicators from clause classifications and missing protection checks.
Clause‑level features must aggregate into contract‑level risk signals:
CREATE TABLE contract_risk_summary (
document_id VARCHAR PRIMARY KEY,
overall_risk_score INT,
high_severity_flag_count INT,
medium_severity_flag_count INT,
low_severity_flag_count INT,
missing_critical_clause_count INT,
computed_at TIMESTAMP NOT NULL
);
-- Example aggregation
INSERT INTO contract_risk_summary (document_id, overall_risk_score,
high_severity_flag_count, medium_severity_flag_count,
low_severity_flag_count, missing_critical_clause_count,
computed_at)
SELECT
c.document_id,
LEAST(100,
10 * SUM(CASE WHEN rf.severity = 'HIGH' THEN 1 ELSE 0 END) +
5 * SUM(CASE WHEN rf.severity = 'MEDIUM' THEN 1 ELSE 0 END)
) AS overall_risk_score,
SUM(CASE WHEN rf.severity = 'HIGH' THEN 1 ELSE 0 END) AS high_severity_flag_count,
SUM(CASE WHEN rf.severity = 'MEDIUM' THEN 1 ELSE 0 END) AS medium_severity_flag_count,
SUM(CASE WHEN rf.severity = 'LOW' THEN 1 ELSE 0 END) AS low_severity_flag_count,
SUM(CASE WHEN rf.risk_flag LIKE 'missing_%' THEN 1 ELSE 0 END) AS missing_critical_clause_count,
NOW() AS computed_at
FROM clause_risk_flag rf
JOIN clause c ON c.clause_id = rf.clause_id
GROUP BY c.document_id;
Patterns like these appear in risk scoring frameworks, multi‑agent contract analysis workflows, and commercial risk scoring platforms that provide overall scores plus detailed clause‑level evidence.
Here is a condensed Python function that ties the steps together for a single clause:
def score_clause(clause_record: dict, our_role: str, liab_policy: dict) -> dict:
# 1. Canonicalize
text = canonicalize_text(clause_record["raw_text"])
# 2. Perspective
perspective = derive_perspective(text, our_role)
# 3. Numeric normalization
amount_info = normalize_amount(text)
duration_months = normalize_duration_months(text)
# 4. Semantic attributes
attrs = extract_liability_attributes(text)
if amount_info:
attrs["cap_multiple"] = clause_record.get("cap_multiple", 1.0)
attrs["cap_reference_period_months"] = duration_months or 12
attrs.update(perspective)
# 5. Policy → flags
flags = apply_policy(liab_policy, attrs)
# 6. Flags → binary features
feature_row = features_from_flags(clause_record["clause_id"], flags, attrs)
return {
"clause_id": clause_record["clause_id"],
"attributes": attrs,
"flags": [f.__dict__ for f in flags],
"features": feature_row
}
In a production pipeline, you would run this as a batch job over all clauses in a document or as a streaming stage in your ingestion workflow. This end‑to‑end pattern normalized attributes → policy rules → flags → features; is at the heart of deterministic contract risk engines described in industry architectures and platforms.
Most modern systems combine deterministic risk engines with generative models:
Architectures like Ben Sweet’s AI solution for law firms and commercial tools such as Legiscope clearly separate “rule‑based risk detection” from LLM‑based explanation and drafting to keep decisions auditable. The binary risk features and flags you’ve engineered become inputs to LLM prompts, not outputs of the LLM itself.
For example:
text
System: You are a contract lawyer. You will receive a liability clause, normalized attributes, and binary risk flags.
User: Clause text: "...". Attributes: {...}. Flags: ["liability_cap_below_12m_fees", "includes_indirect_damages"].
Explain the risk to the customer in 3 bullet points and propose a safer alternative clause.
This design ensures that regulatory and internal audits can trace every high‑risk classification back to a specific rule, attribute, and clause text, a requirement increasingly emphasized in contract risk and compliance guidance.
By the end of this second part, the pipeline stops being an abstract “AI for contracts” idea and becomes a concrete set of operators: canonicalization, party normalization, numeric standardization, semantic attribute extraction, policy evaluation, and feature projection into a binary vector. Each step is deterministic, testable, and versionable, which is exactly what legal, risk, and compliance teams need when they are asked to justify why a contract was flagged as high risk.
Instead of treating risk as something an LLM intuits from free‑form prompts, you now have a normalized representation of each clause and a set of rule‑driven risk flags that can be computed on demand, replayed across new policy versions, and audited against the original text. This is the point where clause intelligence stops being “copilot‑style assistance” and becomes an input to serious risk governance: every HIGH, MEDIUM, or LOW signal is traceable back to a clause, an attribute, a rule, and a specific document version.
Taken together, Part 1 and Part 2 define a full contract‑to‑risk pipeline that looks and behaves like any other enterprise‑grade data product: a clear schema, stable identifiers, deterministic transformations, and explicit policies encoded as data instead of buried in prompts. Contracts flow in as PDFs and DOCX files and come out as clause graphs, feature rows, and risk scores that downstream systems dashboards, CLM workflows, case tools, and even LLM assistants can consume without having to “re‑interpret” the text each time.
For professional services organizations, this architecture does more than speed up review; it creates a defensible record of how risk was assessed at a given point in time, under a specific policy, over a specific clause set. That is what regulators, courts, and demanding clients increasingly expect: not just that AI helped, but that every automated decision is anchored in machine‑readable clause data and a transparent policy model that the firm controls.
