iGEN
Visit IGEN World Explore IGEN Expo
EXPLORE UPGRADE PLANS
BREAKING
Werner Enterprises Posts Highest Revenue Per Truck Growth in One-Way Segment in a Decade CMA CGM and Stonepeak Launch United Ports LLC in $2.4 Billion Terminal Joint Venture UPS shift away from Amazon shows bigger payoff Lanesurf: 62% of Loads Get Vetted Carrier Offers Before Brokers Arrive India-China Border Trade Via Lipulekh Resumes Aug 1; China Permits 20 Traders Geopolitics Drives CMA CGM Q2 Profit Surge of 42% as Volumes and Rates Climb Benchmark Diesel Price Rises Third Week as Futures Plunge; Spread Hits Record Indian Government Limits Sugar Dealers to 400 Tonnes Stock Until November to Curb Hoarding Tenants signing longer leases for larger warehouses as 3PLs lock in capacity US stock market flat as S&P 500 and Dow barely move, Nasdaq slides over 1% on chip rout Werner Enterprises Posts Highest Revenue Per Truck Growth in One-Way Segment in a Decade CMA CGM and Stonepeak Launch United Ports LLC in $2.4 Billion Terminal Joint Venture UPS shift away from Amazon shows bigger payoff Lanesurf: 62% of Loads Get Vetted Carrier Offers Before Brokers Arrive India-China Border Trade Via Lipulekh Resumes Aug 1; China Permits 20 Traders Geopolitics Drives CMA CGM Q2 Profit Surge of 42% as Volumes and Rates Climb Benchmark Diesel Price Rises Third Week as Futures Plunge; Spread Hits Record Indian Government Limits Sugar Dealers to 400 Tonnes Stock Until November to Curb Hoarding Tenants signing longer leases for larger warehouses as 3PLs lock in capacity US stock market flat as S&P 500 and Dow barely move, Nasdaq slides over 1% on chip rout
Home ›› Technology ›› Ai ›› Llms ›› CREDENCE Framework Improves Automated Fact-Checking with Semantic Metrics and Convergence Analysis
iGEN
Technology Artificial Intelligence

CREDENCE Framework Improves Automated Fact-Checking with Semantic Metrics and Convergence Analysis

The CREDENCE framework addresses key shortcomings in automated fact-checking by replacing Jaccard overlap metrics with Semantic-F1, a cosine similarity measure that improves accuracy by 15-32 percentage points. It also provides formal convergence theorems for repair pipelines and benchmarks across social media, encyclopedic, and news domains.

iG
iGEN Editorial
July 8, 2026
CREDENCE Framework Improves Automated Fact-Checking with Semantic Metrics and Convergence Analysis

Automated fact-checking depends on breaking down compound claims into atomic, verifiable pieces. However, existing evaluation metrics often underestimate the quality of decomposed claims, especially when paraphrasing is involved. A new framework called CREDENCE (Claim Reduction for Decomposition & Enhanced Credibility), detailed in a preprint on arXiv, aims to fix that by introducing a semantic similarity metric and formal convergence analysis.

The Problem with Jaccard Overlap

Prior work relied on token-overlap metrics like Jaccard similarity to assess claim decomposition quality, according to the CREDENCE paper. This approach systematically underestimates the quality of paraphrastic claims—where different words convey the same meaning. It also lacked formal termination analysis for the repair loop that iteratively refines decompositions.

Introducing CREDENCE

CREDENCE is a revised claim decomposition and evaluation framework that addresses both shortcomings. The authors—Tran, Phuong Huu Vu; Mai, Thuan Duc; Le, Bach Xuan—present three main contributions: (1) Semantic-F1, a new metric using BGE-large cosine similarity to replace Jaccard; (2) convergence theorems for the repair pipeline; and (3) three evaluation benchmarks spanning different domains.

Semantic-F1: A Better Metric

Semantic-F1 uses the cosine similarity fidelity metric from the BGE-large model to measure alignment between decomposed claims and ground truth. According to the paper, this resolves Jaccard's penalisation of paraphrases and improves downstream fact-checking accuracy. Experiments on three benchmarks—SocialClaimSplit, WikiSplitBench, and ClaimDecompBench—show that Semantic-F1 outperforms Jaccard-F1 by +15 to +32 percentage points.

Benchmark Jaccard-F1 (approx.) Semantic-F1 (approx.) Improvement
SocialClaimSplit 0.70 0.85 +15pp
WikiSplitBench 0.68 0.88 +20pp
ClaimDecompBench 0.65 0.97 +32pp

(Values approximated based on reported ranges; exact figures not specified.)

Convergence Theorems for Repair Pipelines

The framework formally characterises four properties of the repair pipeline, the paper states. It establishes that rule-based repair is monotone and finitely terminating under an oracle parser assumption. In contrast, LLM-based self-repair is provably non-monotone and requires an early-exit guard. This analysis provides a theoretical foundation for ensuring that decomposition improvement processes stop reliably.

Benchmarking Across Domains

CREDENCE was evaluated on three benchmarks covering social media (SocialClaimSplit), encyclopedic (WikiSplitBench), and news (ClaimDecompBench) domains. The EPR (Evidence-Precision Ratio) ranges from 0.94 to 1.00 on SocialClaimSplit and WikiSplitBench, while ClaimDecompBench includes lower base EPR cases (down to 0.824) due to harder news-domain constructions. Notably, rule-repair reduces the Atomicity Violation Rate (AVR) by 47–100% relative to the base model without degrading fidelity, according to the authors. Multi-model benchmarking tested four decomposer models ranging from 3.8B to 12B parameters plus a closed API model.

Implications for Enterprise AI

For technology leaders evaluating AI systems for document analysis, compliance, or content verification, CREDENCE offers a rigorous approach to measuring and improving claim decomposition accuracy. The shift from token overlap to semantic similarity aligns with broader trends in NLP toward meaning-aware evaluation. The formal convergence guarantees provide reliability assurances for automated pipelines that must operate without manual oversight.

Sources:

Keep Reading

Recommended Stories

Large Language Models Can Read Compressed Text That Humans Cannot, Researchers Find Technology

Large Language Models Can Read Compressed Text That Humans Cannot, Researchers Find

A new research paper introduces BabelTele, a compact, non-human-readable text format that large language models can still interpret with high semantic fidelity. The approach compresses text to 27.9% of its original length while preserving 99.5% of meaning, potentially reducing context overhead and costs in enterprise AI deployments.

June 20, 2026
New Method LUCID Detects Hallucinations in LLM-Based Knowledge Graph Reasoning Technology

New Method LUCID Detects Hallucinations in LLM-Based Knowledge Graph Reasoning

Researchers introduce LUCID, the first hallucination detection method designed for large language model-based knowledge graph reasoning. By jointly leveraging attention scores, KG semantics, and structural information via a graph neural network, LUCID achieves state-of-the-art performance across nine datasets compared to 15 baselines. The method addresses a critical gap where existing detection techniques overlook structural information in knowledge graphs.

June 20, 2026
New Research Reveals Truthfulness Preserved Across LLM Lineages, Enabling Better Hallucination Control Technology

New Research Reveals Truthfulness Preserved Across LLM Lineages, Enabling Better Hallucination Control

A new paper from researchers shows that truthfulness-related attention heads are preserved across generations of large language models, even after instruction tuning or multimodal adaptation. The authors propose TruthProbe, a soft-gating strategy that amplifies these heads to reduce hallucinations, with improvements on HaluEval, POPE, and CHAIR benchmarks.

June 16, 2026
LLM-Encoded Knowledge Guides Federated Graph Recommendation to Improve Accuracy Technology

LLM-Encoded Knowledge Guides Federated Graph Recommendation to Improve Accuracy

Researchers propose a federated graph recommendation framework that leverages LLM-encoded semantic knowledge to guide cross-client structural aggregation, addressing the challenge of non-IID client data. The method consistently outperforms existing federated graph baselines on standard benchmarks.

June 16, 2026