The Architecture of Financial Intelligence: A Comprehensive Analysis of Large Language Models in Finance

📄 Article ✨ New Page

---

1. Introduction: The Structural Paradigm Shift in Quantitative and Fundamental Finance

The integration of Large Language Models (LLMs) into the financial sector represents a structural paradigm shift, fundamentally transforming the mechanisms through which unstructured financial intelligence is processed, synthesized, and executed. Historically, quantitative finance and algorithmic trading have relied predominantly on structured data schemas, encompassing price-volume time series, tick data, macroeconomic indicators, and standardized accounting metrics. However, an estimated eighty percent of actionable market intelligence resides in highly unstructured text: regulatory filings, earnings call transcripts, analyst reports, geopolitical news streams, and real-time social media sentiment.

General-domain foundation models, while exhibiting remarkable natural language understanding and generative capabilities, frequently falter when applied to the specialized, highly technical, and strictly regulated domain of finance. These generalized systems struggle with domain-specific jargon, complex numerical reasoning over tabular data, and strict adherence to regulatory taxonomies, such as the eXtensive Business Reporting Language (XBRL) mandated in United States Securities and Exchange Commission (SEC) filings, which can contain thousands of highly specific accounting labels.

To bridge this operational deficit, the distinct discipline of Financial Large Language Models (FinLLMs) has emerged over the past several years. FinLLMs are specifically pre-trained, continuously adapted, or instruction-tuned on massive, highly curated corpora of financial literature, equipping them with the latent knowledge necessary to perform specialized tasks at an expert level. The development of these domain-specific models is not merely an exercise in scaling parameters or expanding context windows; it requires meticulous data curation pipelines, financial-specific instruction tuning paradigms, and the deployment of rigorous benchmarking frameworks designed to detect subtle hallucinations, prevent temporal leakage, and ensure logical consistency across multi-step reasoning tasks. The current landscape is categorized by a dichotomy between massive, proprietary models built by heavily capitalized institutions and lightweight, open-source models designed for democratized access, rapid iteration, and localized, secure deployment.

---

2. A Task-Centered Taxonomy for Financial Language Models

To systematically evaluate the utility, architecture, and deployment viability of FinLLMs, it is essential to map their theoretical capabilities to specific workflows within an investment, operational, or risk-management production pipeline. The existing literature and applied research propose a comprehensive taxonomy that categorizes financial LLM applications into several core functional domains, each supported by distinct algorithmic methodologies and specialized datasets.

2.1 Sentiment Analysis and Opinion as Signal Inputs

The extraction of polarity, stance, and emotional intensity from unstructured text is one of the most established applications of natural language processing in finance. Modern LLMs transform qualitative streams from financial news, social media platforms, earnings calls, and analyst notes into quantitative features utilized in event studies, return prediction engines, and risk monitoring systems. Fine-grained sentiment analysis relies on specialized instruction tuning datasets such as the Financial Phrase Bank, the Twitter-Financial-News-Sentiment dataset, and entity-level tracking datasets like FinEntity.

Unlike generic sentiment classifiers, financial sentiment extraction requires contextual awareness; for instance, understanding that an "upgrade to an MSCI ESG rating from BB to BBB" or "high double-digit retail sales growth" for a corporation like Xtep International constitutes a highly positive signal, whereas "fluctuations in raw material costs" or "demand weakening due to real estate regulation" represents a negative risk vector.

2.2 Information Extraction and Knowledge Graph Construction

Information Extraction (IE) involves converting unstructured prose into structured relational data, encompassing Named Entity Recognition (NER), relation extraction, and event detection. By populating proprietary knowledge graphs, LLMs act as intelligent controllers and generators that enable high-precision retrieval modules and point-in-time factor generation. Datasets such as FiNER, FinRED, and REFinD support supervised training for these exact tasks, enabling models to isolate the relationships between corporate subsidiaries, manufacturers, and global supply chains. Furthermore, causality detection tasks, supported by datasets like FinCausal20, train models to identify implicit cause-and-effect relationships within SEC filings and macroeconomic news, determining the underlying catalysts that influence market trends.

2.3 Numerical Question Answering and Economic Reasoning

A persistent vulnerability of early autoregressive language models is their inability to execute reliable arithmetic. In fundamental analysis, executing multi-step reasoning over tables, mathematical formulas, and unstructured text found in filings (e.g., 10-K, 10-Q) is critical for calculating Key Performance Indicators (KPIs) and validating investment theses. Specialized benchmarks and training datasets, including FinQA, ConvFinQA, TAT-QA, and DocMathEval, are utilized to probe and improve numerical correctness, drastically reducing miscalculation risks in filing-driven research. This requires models to not only comprehend text but to correctly identify table structures and cell boundaries, a capability enhanced by layout-aware training on datasets like FinTabNet and PACIFIC.

2.4 Summarization and Document Understanding

Institutional research requires the compression of voluminous documents — such as multi-hour earnings call transcripts or hundreds of pages of corporate prospectuses — into high-density executive briefs. This accelerates research and supports hypothesis generation while maintaining absolute materiality. The ECTSum dataset highlights the challenges inherent in automatic summarization, requiring high compression ratios and the ability to process documents that frequently exceed standard LLM token limits, all without discarding critical financial metrics or forward-looking guidance.

2.5 Multimodal Fusion and Audio-Visual Cues

The frontier of predictive modeling involves the fusion of text with non-textual inputs. Multimodal LLMs are trained to integrate the prosody (vocal nuances and emotional cues) of executives during earnings calls, visual data extracted from candlestick charts, and structured time-series data to inform trading signals. Datasets such as MAEC (Multimodal Aligned Earnings Conference Call) and MONOPOLY supply the necessary multimodal and policy-related cues, while general multimodal benchmarks like MMMU test the integration of financial charts, accounting tables, and geographic maps into the reasoning pipeline.

2.6 Agentic Workflows and Automated Trading Systems

Moving beyond passive query responses, the industry is transitioning toward agentic workflows that autonomously coordinate external tools for fundamental research, algorithmic backtesting, and trade execution. These sophisticated frameworks incorporate memory modules, role specialization, and "debate traces" between multiple AI agents to ensure logical consistency and auditability before a decision is finalized.

2.7 Governance, Compliance, and Security Risk Management

In heavily regulated environments, LLMs are tasked with ensuring adherence to legal standards, implementing policy checks, contradiction flags, and maintaining strict audit trails to shape allowable actions. Furthermore, financial language models must possess robust security knowledge to detect vulnerabilities, malware patterns, and cryptographic weaknesses within operational infrastructure, a capability rigorously tested in the latest iterations of Chinese benchmarks like FinEval.

---

3. Foundation Architectures and Pre-training Paradigms

The development of FinLLMs has bifurcated into two distinct methodological camps: massive, proprietary foundation models built via computationally exhaustive pre-training by heavily capitalized institutions, and lightweight, open-source models optimized through efficient adaptation techniques for democratized, secure local deployment.

3.1 The Proprietary Vanguard: BloombergGPT

BloombergGPT stands as a seminal milestone in the architectural development of domain-specific language models. Utilizing a decoder-only BLOOM-style architecture with 50 billion parameters, the model represents one of the largest specialized training efforts documented in the sector. The defining characteristic of BloombergGPT is its massive, mixed-dataset pre-training strategy, which aims to infuse deep, historical financial expertise without inducing catastrophic forgetting of general linguistic and cognitive capabilities. The model was pre-trained on a corpus exceeding 700 billion tokens, with 569 billion tokens utilized during the primary training run.

Data Category	Token Count (Billions)	Percentage of Total Training Data	Primary Sources
Financial Specific	363	51.27%	Web crawls (298B), Financial News (38B), Filings (14B), Press (9B), Internal Bloomberg (5B)
General Purpose	345	48.73%	The Pile (184B), C4 Web Corpus (138B), English Wikipedia (24B)

From a technical optimization standpoint, BloombergGPT utilized the AdamW optimizer with hyperparameters \( \beta_1 = 0.9 \), \( \beta_2 = 0.95 \), and a weight decay of 0.1. To maximize GPU utilization and throughput across the compute cluster, the training sequence length was strictly maintained at 2,048 tokens. However, the integration of ALiBi (Attention with Linear Biases) positional encoding theoretically allows the model to extrapolate to longer sequence lengths during inference without catastrophic degradation. The learning rate was governed by a cosine decay scheduler, peaking at \( 6 \times 10^{-5} \) following a linear warmup over the initial 1,800 steps, before decaying to a final rate of \( 6 \times 10^{-6} \). A critical operational outcome of this architecture is its few-shot proficiency; BloombergGPT demonstrated the capacity to translate natural language requests into valid Bloomberg Query Language (BQL) with as few as three in-context examples, bypassing the need for extensive task-specific instruction tuning for internal workflows.

3.2 The Open-Source Counter-Movement: FinGPT and LLM Pro Finance

In direct architectural opposition to the closed-API nature of institutional models, initiatives such as the AI4Finance Foundation's FinGPT provide open-source frameworks emphasizing data democratization, lightweight adaptation, and continuous retraining. Financial data is highly temporal and subject to rapid decay; therefore, FinGPT eschews static, multi-million-dollar pre-training runs in favor of automated data curation pipelines that source internet-scale financial data for rapid updates.

FinGPT achieves computational efficiency by leveraging Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA). By freezing the majority of the base model's weights and only updating a small, low-rank matrix, the framework allows for monthly or weekly retraining cycles at a compute cost of less than $300 per iteration. Furthermore, FinGPT heavily integrates Reinforcement Learning from Human Feedback (RLHF), enabling the model to align its outputs with specific individual investor preferences, risk-aversion levels, and trading habits — a personalization capability distinct from broad institutional models.

Complementing these efforts is the LLM Pro Finance Suite by DragonLLM, providing a tiered architecture of multilingual models specifically adapted for economics and business:

Model	Parameters	Specialization
Gemma Pro Finance	12B	Financial translation, batch processing, classification
Qwen Pro Finance R	32B	Financial mathematics, code generation, agentic systems
Llama Pro Finance	70B	Complex RAG, conversational chat, long-form content generation

These models demonstrate that targeted data curation pipelines can yield open-source variants that outperform larger, general-domain models on financial reasoning and translation tasks while maintaining rigorous risk controls.

3.3 Bilingual and Regional Architectures: CFGPT, PIXIU, and XuanYuan

Given the massive scale, unique regulatory frameworks, and linguistic nuances of the Chinese financial markets, significant resources have been devoted to developing bilingual (Chinese-English) and localized financial foundation models.

The CFGPT (Chinese Financial Generative Pre-trained Transformer) framework, built upon the InternLM-7B and InternLM2 base architectures, provides a highly localized solution encompassing large-scale pre-training, supervised fine-tuning, and a deployment framework (CFAPP). It integrates specialized modules for real-time compliance checking, risk monitoring, and fact verification, operating seamlessly within the Chinese regulatory context.

Addressing the limitations of monolingual models, the PIXIU project introduces a comprehensive bilingual framework featuring ICE-FIND (the first cross-lingual bilingual financial instruction dataset), the ICE-INTENT large language model, and the ICE-FLARE evaluation benchmark. The PIXIU model suite includes FinMA-7B-NLP (specialized strictly for NLP classification tasks), FinMA-7B-full (covering both NLP and predictive modeling), and FinMA-30B (fine-tuned atop the LLaMA-30B architecture). By simultaneously integrating translated and original English and Chinese datasets, PIXIU captures cross-border sentiment divergences and macroeconomic linkages often missed by regional models.

For institutions requiring massive context windows, the XuanYuan-70B model, based on LLaMA2-70B, extends the standard context length to 8k and 16k tokens during its pre-training on Chinese and English financial texts. Notably, XuanYuan offers 8-bit and 4-bit quantized versions, drastically reducing hardware constraints and enabling on-premises, secure deployment for firms barred by compliance from utilizing cloud-based APIs.

3.4 The Transition to Pure Reasoning: Fin-R1

While early financial language models prioritized information extraction and sentiment classification, the frontier of algorithmic research has decisively shifted toward complex logical reasoning. The Fin-R1 model serves as a prime example of this transition. Operating at a highly efficient parameter scale of just 7 billion, Fin-R1 employs a rigorous two-stage training framework: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL).

Rather than relying on brute-force parameter scaling, Fin-R1 achieves its performance through the curation of Fin-R1-Data, a dataset containing approximately 60,091 complete Chain of Thought (CoT) trajectories. By distilling the complex reasoning processes of larger models into high-quality training paths, Fin-R1 learns the underlying logic of financial problem-solving. Empirical evaluations demonstrate that this lightweight model achieved state-of-the-art scores of 85.0 on ConvFinQA and 76.0 on FinQA, outperforming significantly larger distillation models such as DeepSeek-R1-Distill-Llama-70B, and proving highly effective in real-world applications such as robotic advisory and financial compliance checking.

---

4. Data Engineering: Corpora, Instruction Tuning, and the SFT Ecosystem

The practical efficacy of a Financial Large Language Model is inextricably linked to the quality, density, and diversity of its underlying data. Supervised Fine-Tuning (SFT) is the critical mechanism that transforms a base foundational model — which merely predicts the next statistically probable token — into an interactive agent capable of following specific directives, extracting numerical tables, and adhering to professional compliance standards.

4.1 The CFData Corpus: Pre-training and Instruction Tuning at Scale

The Chinese CFGPT framework relies on an exceptionally detailed, multi-source financial dataset named CFData, divided into a massive pre-training set (CFData-pt) and a highly specialized fine-tuning set (CFData-sft).

The pre-training dataset comprises roughly 591 million documents and 193 billion tokens. The distribution of this data provides significant insight into the model's fundamental inductive biases:

Pre-training Sub-Dataset	Token Count (Billions)	Percentage	Content Description
CFData-SM	84	60.15%	Social media content; highly reflective of retail investor sentiment
CFData-FN	26	18.70%	Mainstream financial news and macroeconomic reporting
CFData-CA	17	12.28%	Standardized corporate announcements and regulatory filings
CFData-CP	13	6.24%	Lengthy, highly detailed corporate prospectuses for IPOs
CFData-RR	3	2.51%	Professional, high-density brokerage research reports
CFData-Wiki	0.137	0.09%	General-purpose Wikipedia content to maintain basic reasoning

The heavy reliance on social media (60.15%) indicates a model acutely attuned to market momentum and retail sentiment — a dominant force in the Chinese A-share market. However, such high-noise data requires rigorous subsequent fine-tuning to ensure the model produces professional, analytical outputs rather than echoing social media volatility.

This recalibration is achieved through the supervised fine-tuning dataset, CFData-sft, which consists of 1.6 million instruction pairs spanning 1.5 billion tokens. The task distribution dictates the model's operational utility:

• Report Summarization (CFData-RS): 50.60% (765 million tokens) — trains the model to condense lengthy research into actionable insights, identifying innovation points and strategic layouts
• Event Detection (CFData-ED): 22.69% (343 million tokens) — precise categorization of events across markets (e.g., distinguishing between the Derivative Market, Precious Metals, and Foreign Exchange)
• Topic Decomposition (CFData-TD): 12.37% (187 million tokens) — ensuring multi-faceted documents are broken down into discrete themes
• Stock Movement Prediction (CFData-SP): 8.27% (125 million tokens) — attempting to align qualitative textual sentiment with historical equity price trajectories
• Sentiment Analysis (CFData-SA): 5.69% (86 million tokens) — classifying market events as Positive, Negative, or Neutral

By utilizing prompt-based task reformulation, this SFT data is broken down further into hyper-specific operational functions, yielding 21K instances for product identification, 20K instances for risk generation (e.g., identifying real estate regulation risks in a lumber company report), and 18K instances for generating fully reasoned investment suggestions.

4.2 Open-Source Instruction Datasets and RLHF Formats

The broader open-source ecosystem, particularly repositories hosted on HuggingFace, provides a wealth of specialized datasets driving English-language financial instruction tuning. Datasets such as sujet-ai/Sujet-Finance-Instruct-177k (containing 178,000 instruction pairs) and AdaptLLM/finance-tasks offer broad coverage for financial querying.

To combat hallucinations during the SFT phase, novel methodologies are being employed in dataset generation. The Investopedia instruction tuning dataset utilizes a self-verification technique: unstructured scraped data is processed by an LLM to generate structured QA pairs (e.g., defining the differences between pro rata, excess, and no-liability insurance apportionment), followed by a secondary verification pass that mathematically reduces the probability of incorporating hallucinated facts into the final training corpus.

Furthermore, the alignment of LLMs with professional enterprise standards requires data formatted specifically for RLHF. The argilla/llama-2-banking-fine-tune dataset exemplifies this approach for the retail banking sector. Containing simulated interactions regarding failed transfers, unrecognised charges, card delivery timelines, and fraud disputes, the dataset provides a user request, two varying assistant outputs (response-1 and response-2), and a human preference ranking. This structure allows the model's reward function to optimize for the most helpful, polite, and professionally accurate response — a necessity for deploying autonomous customer-facing agents.

Fine-grained sentiment analysis also relies on entity-tracking datasets. While generic models assess the overall tone of a paragraph, datasets like yixuantt/FinEntity track sentiment at the specific entity level. By defining exact start and end character indices, the model learns to isolate sentiment directed at specific corporations (e.g., <JNJ.N> for Johnson & Johnson, <TSLA.O> for Tesla), financial institutions (Goldman Sachs, Morgan Stanley), market sectors, and commodities (Brent Crude <LCOc1>) co-occurring within a single text.

4.3 Data Augmentation and Domain Randomization

The creation of robust financial data also benefits from sophisticated preprocessing frameworks. Tools such as Cornucopia-LLM (an independent PyTorch-based augmentation framework) provide a generic framework for data augmentation, preprocessing, and domain randomization. By randomizing the structural layout of JSON, YAML, or Markdown tables during training, models are prevented from overfitting to specific positional formatting heuristics, ensuring they learn true semantic meaning and transferability across disparate financial reporting standards.

4.4 Key Financial LLM Datasets: Summary Reference

The following table consolidates the primary datasets discussed above, categorized by scale, task focus, and licensing status — a practical reference for practitioners selecting training data under compliance constraints:

Dataset Name	Size / Scale	Primary Task / Domain	License
CFData (CFData-pt & CFData-sft)	193B tokens (pre-training); 1.5B tokens (SFT)	Pre-training and SFT across multi-modal Chinese financial tasks	Open Source
BloombergGPT Financial Corpus	363B financial tokens	Large-scale mixed-dataset pre-training	Proprietary
FinEval	>26,000 questions (4,661 academic; 1,434 industry)	Comprehensive Chinese evaluation benchmark (Knowledge, Reasoning, Security)	CC BY-NC-SA 4.0
FinGPT Datasets	76.8K (Sentiment), 82.2K (Headline), 27.6K (Relation)	Internet-scale instruction tuning and RLHF	Open Source
twitter-financial-news-sentiment	11,932 documents	Multi-class sentiment analysis (Bearish, Bullish, Neutral)	MIT
FinEntity	979 rows	Entity-level sentiment classification and NER	Open Source
Sujet-Finance-Instruct-177k	~178,000 instruction pairs	Financial instruction tuning	Open Source
BBT-FinCorpus	~300GB raw text (105GB processed)	Large-scale financial pre-training	CC BY-NC-SA 4.0
Investopedia Instruction Tuning	Not explicitly specified	Fine-tuning embedding models and self-verification	CC BY-NC 4.0
llama-2-banking-fine-tune	100 rows	RLHF fine-tuning for retail banking interactions	Open Source

The licensing column is operationally significant: five of the ten datasets carry non-commercial or share-alike clauses (CC BY-NC-SA 4.0, CC BY-NC 4.0), which directly limits their use in production financial services deployments without additional legal review (see Section 7).

4.5 Alignment Data Formats: From SFT to Online RL

Selecting a dataset is only half the engineering problem. Each alignment algorithm in the training pipeline — from vanilla SFT to online RL — requires its data structured in a precise schema. Using the wrong format silently breaks training or produces misaligned models. The following breakdown covers the six canonical formats used across the HuggingFace TRL ecosystem, with financial-domain examples for each.

Format 1 — Language Modeling / Prompt-Completion (SFTTrainer, GKDTrainer)

Schema: A list of messages dictionaries with role (system / user / assistant) and content fields, or simply a flat prompt + completion string pair.

{
  "messages": [
    {"role": "system",    "content": "You are a financial analyst."},
    {"role": "user",      "content": "Summarize the Q3 10-Q filing for TSLA."},
    {"role": "assistant", "content": "Tesla reported revenue of $25.2B..."}
  ]
}

Usage: Standard Supervised Fine-Tuning — the model is trained purely to predict the next token to match the provided completion. Datasets like CFData-sft (1.6M instruction pairs) and Sujet-Finance-Instruct-177k are structured in this format. It is the entry point for every FinLLM pipeline and establishes the baseline instruction-following behavior before any preference alignment.

---

Format 2 — Preference Data (DPOTrainer, CPOTrainer, ORPOTrainer, RewardTrainer)

Schema: Each row contains a prompt, a chosen (preferred) response, and a rejected (dispreferred) response.

{
  "prompt":   "Classify the sentiment: 'Revenue growth decelerated sharply in Q4.'",
  "chosen":   "Negative",
  "rejected": "Positive"
}

Usage: Used for offline alignment methods — Direct Preference Optimization (DPO), Odds Ratio Preference Optimization (ORPO), and Contrastive Preference Optimization (CPO). In financial sentiment analysis, models fine-tuned with ORPO use datasets where correct financial categorizations are the chosen responses and inaccurate ones are rejected. The model learns to maximize the log-odds ratio of the chosen response over the rejected one, without needing a separate reward model. The argilla/llama-2-banking-fine-tune dataset (response-1 vs. response-2 with human preference ranking) is a direct example of preference-formatted data for the retail banking domain.

---

Format 3 — Unpaired Preference (KTOTrainer, BCOTrainer)

Schema: Each row contains a prompt, a single completion, and a boolean label — True if desirable, False if not.

{"prompt": "What is the YTM of a 5-year bond at par with a 4% coupon?", "completion": "4.0%",  "label": true}
{"prompt": "What is the YTM of a 5-year bond at par with a 4% coupon?", "completion": "6.2%",  "label": false}

Usage: Kahneman-Tversky Optimization (KTO) eliminates the need to collect paired comparisons — a significant practical advantage in finance. It is far cheaper to ask a financial analyst to give a single "thumbs-up" or "thumbs-down" on a generated research summary than to require them to rank two separate, lengthy outputs side-by-side. This dramatically reduces annotation cost when building preference-aligned FinLLMs on proprietary internal documents.

---

Format 4 — Stepwise Supervision (PRMTrainer)

Schema: Designed for Process Reward Models (PRMs), each row contains a prompt, a list of completions representing sequential reasoning steps, and a list of boolean labels evaluating the correctness of each individual step.

{
  "prompt": "Calculate the EV/EBITDA for a firm with EBITDA $120M, debt $400M, cash $50M, equity $800M.",
  "completions": [
    "Enterprise Value = Market Cap + Debt - Cash = 800 + 400 - 50 = $1,150M",
    "EV/EBITDA = 1,150 / 120 = 9.58x"
  ],
  "labels": [true, true]
}

Usage: Stepwise supervision is critical for financial math and multi-step economic reasoning. Models like Fin-PRM apply PRMs to evaluate the intermediate logic of financial calculations — ensuring the model doesn't just guess the final number correctly, but strictly follows correct financial rules and formulas at every step of the reasoning chain. This directly addresses the failure mode where a model arrives at a correct answer through an internally inconsistent path, which would be a compliance violation in auditable financial workflows. Fin-R1's 60,091 CoT trajectories are the raw material from which stepwise supervision labels can be derived.

---

Format 5 — Prompt-Only (GRPOTrainer, RLOOTrainer, OnlineDPOTrainer, NashMDTrainer, XPOTrainer)

Schema: The dataset simply consists of a list of raw prompts with no pre-defined answers or completions.

{"prompt": "Given the attached 10-K excerpt, identify three material risk factors and their potential EPS impact."}

Usage: Used for active, online Reinforcement Learning. The trainer feeds the prompt to the model, which generates multiple responses on the fly, and an external reward function — such as a Python script validating a calculated ratio, or an LLM-as-a-judge scoring report quality — scores the outputs dynamically. Group Relative Policy Optimization (GRPO) is the dominant technique here: rather than maintaining a separate critic model, GRPO samples a group of responses and uses their relative reward scores to compute the policy gradient. Fin-R1 and DianJin-R1 both use this prompt-only + GRPO paradigm to build complex financial reasoning chains through trial and error, which is why their training data is so compact (prompts only) while their learned behaviors are sophisticated.

---

Format 6 — Tokenized Language Modeling (PPOTrainer)

Schema: Requires pre-tokenized sequences formatted specifically for the model's tokenizer — typically a input_ids tensor alongside a separate value-model score.

Usage: Proximal Policy Optimization (PPO) is the classical RLHF method: an "actor" model generates tokens while a separate "critic" (value) model scores them at every step. It is more computationally expensive than offline methods like DPO because it requires loading multiple models simultaneously — the policy model, the reference model, the reward model, and the value model — to calculate rewards and KL-divergence penalties during training. FinGPT's RLHF pipeline uses a PPO-style approach to align outputs with individual investor risk preferences, making it one of the few FinLLMs where the cost overhead is justified by the need for fine-grained, personalized behavioral alignment rather than general task correctness.

---

Choosing the right format in practice:

Training Goal	Recommended Format	Key FinLLM Examples
Baseline instruction following	Prompt-Completion (SFT)	CFData-sft, Sujet-Finance-Instruct-177k
Offline preference alignment	Preference (DPO/ORPO)	argilla/llama-2-banking-fine-tune
Low-cost human feedback	Unpaired (KTO)	Proprietary analyst annotations
Verifiable step-by-step math	Stepwise (PRM)	Fin-PRM, Fin-R1 CoT trajectories
Online reasoning via RL	Prompt-Only (GRPO)	Fin-R1, DianJin-R1
Personalized RLHF alignment	Tokenized (PPO)	FinGPT RLHF pipeline

4.6 Dataset-to-Trainer Mapping: Practical Reference

The formats above are not abstract — every dataset in the FinLLM ecosystem has a concrete mapping to a specific TRL trainer. The following two tables make that mapping explicit.

Standard SFT and pre-training datasets:

Dataset Name	Underlying Data Structure	TRL Dataset Type	Typical Trainer
CFData-pt	Unstructured financial documents and text	Language modeling	SFTTrainer (continued pre-training)
CFData-sft	Instruction-response pairs	Prompt-completion	SFTTrainer
BloombergGPT Corpus	Unstructured raw text and web crawls	Language modeling	SFTTrainer (pre-training)
FinEval	Multiple-choice questions and answers	Prompt-completion (if fine-tuning)	SFTTrainer
FinGPT Datasets	Task-specific instruction-response dictionaries	Prompt-completion	SFTTrainer
twitter-financial-news-sentiment	Text mapped to multi-class labels	Prompt-completion	SFTTrainer
FinEntity	Text mapped to entity start/end indices	Prompt-completion	SFTTrainer
Sujet-Finance-Instruct-177k	General instruction-response pairs	Prompt-completion	SFTTrainer
BBT-FinCorpus	Raw text processed from corporate PDFs	Language modeling	SFTTrainer (continued pre-training)
Investopedia Instruction Tuning	Verified question-answer pairs	Prompt-completion	SFTTrainer
llama-2-banking-fine-tune	User request, two assistant responses, and a preference ranking	Preference	DPOTrainer, ORPOTrainer, RewardTrainer

The near-universal concentration on SFTTrainer reflects the current maturity of the field: almost all published FinLLM datasets were designed for supervised instruction tuning before the RL-alignment paradigm became widespread. Only the banking preference dataset breaks from this pattern.

Next-generation RL datasets:

The newest financial reasoning datasets are purpose-built for online RL trainers that were not available when earlier FinLLMs were designed:

Advanced Dataset	Underlying Data Structure	TRL Dataset Type	Target Trainer
Fin-R1-Data	Complex financial questions without pre-defined answers, used to generate live CoT reasoning paths	Prompt-only	GRPOTrainer, RLOOTrainer
Fin-PRM Dataset	Financial reasoning trajectories with boolean correctness labels for each individual step	Stepwise supervision	PRMTrainer
Vietnamese Finance KTO	Generated SQL/accounting completions tagged with a single True/False desirability label	Unpaired preference	KTOTrainer, BCOTrainer

The contrast between the two tables is structurally informative. The first generation of FinLLMs solved the knowledge access problem — getting financial domain vocabulary into the model's weights via massive SFT corpora. The second generation, represented by Fin-R1-Data and Fin-PRM, is solving the reasoning reliability problem — training models to execute multi-step financial logic correctly through outcome-based and process-based RL signals rather than imitation.

---

5. Benchmarking Frameworks and the Decoupling of Cognitive Capabilities

As financial language models scale in complexity, standard natural language benchmarks (such as GLUE, SuperGLUE, or MMLU) are insufficient for evaluating highly technical domain expertise. The industry requires specialized, multi-dimensional evaluation frameworks capable of measuring an LLM's ability to extract causal relationships, forecast market movements, execute precise numerical reasoning, and navigate compliance constraints.

5.1 English-Language Benchmarks: FLUE, FLARE, and FinBen

The initial effort to standardize financial NLP evaluation culminated in FLUE (Financial Language Understanding Evaluation) in 2022. FLUE establishes a baseline across five core tasks: Sentiment Classification (utilizing the Financial Phrase Bank and FiQA), News Headline Classification, Named Entity Recognition (assessed on loan agreement data), Structure Boundary Detection (FinSBD3), and Question Answering. A key technical innovation introduced alongside FLUE was the implementation of domain-specific pre-training objectives, including financial phrase masking and a Supervised Contrastive Learning loss ( \( L_{SCL} \) ), designed to capture latent similarities between examples of the same financial class.

FLARE (Financial Language Understanding and Prediction Evaluation) subsequently expanded upon the FLUE paradigm by bridging the gap between natural language understanding and predictive modeling. FLARE assesses an LLM's capacity to forecast actual stock price movements by synthesizing historical text sentiment with quantitative time-series data.

The most comprehensive recent advancement in English benchmarking is FinBen. Designed to replicate the complexities of real-world financial operations, this expansive framework encompasses 42 distinct datasets spanning 24 specific financial tasks across Information Extraction, Textual Analysis, Question Answering, Text Generation, Risk Management, Forecasting, and Decision-Making. FinBen is highly notable for introducing the first standardized evaluation of autonomous stock trading agents and for utilizing novel assessment methodologies that incorporate Retrieval-Augmented Generation (RAG) constraints, ensuring models are tested not just on static knowledge, but on their ability to ingest and synthesize external information on the fly.

5.2 Chinese-Language and Bilingual Benchmarks: FinEval and CFBenchmark

To address the unique regulatory environment and linguistic density of the Chinese financial markets, several rigorous regional frameworks have been developed.

FinEval is widely regarded as one of the most comprehensive Chinese financial benchmarks, utilizing over 26,000 diverse questions categorized to test both theoretical knowledge and practical application:

1. Financial Academic Knowledge: 4,661 multiple-choice questions derived from simulated professional exams, covering 34 highly technical subjects including Finance, Economy, Accounting, and Certificate examinations. It tests deep domain expertise, such as distinguishing complex theories in International Economics (Internalization Theory vs. Monopolistic Advantage Theory), identifying public interest entities in Auditing, and performing the practical, multi-step calculations required for the China Actuary certification.

2. Financial Industry Knowledge: 1,434 questions simulating real-world scenarios across 10 industry applications. Tasks include providing complex investment advisory (e.g., formulating strategies to adjust bond maturity structures in high-interest-rate environments) and extracting critical facts from operational corporate announcements, such as supply chain procurement contracts.

3. Safety Awareness / Security: Recognizing that financial LLMs deployed in production represent a massive systemic attack vector, FinEval rigorously evaluates model security across 11 domains, including Cryptographic protection, Malware analysis, Pentesting, Reverse engineering, and Vulnerability detection.

4. Financial Rigor and Agent Testing: FinEval assesses the LLM's capacity to function as an autonomous agent in a RAG environment. The model is fed retrieved data and instructed to execute precise financial calculations (e.g., calculating the annualized yield of a bond given specific principal and holding periods) and output solely the numerical result, testing the model's resistance to hallucination and strict adherence to formatting constraints.

Empirical results from FinEval indicate a performance divergence: while frontier general models (such as GPT-4o and Claude 3.5-Sonnet) often achieve the highest overall zero-shot scores due to massive parameter counts, specialized regional models (like Ant Group's Finix-CI-72B and XuanYuan-70B) frequently excel in domain-specific rigor and safety awareness metrics.

Other critical benchmarks include CFBenchmark, which evaluates 3,917 financial texts across recognition, classification, and generation tasks, and BBT-CFLEB, designed as the GLUE-equivalent for Chinese finance encompassing both understanding and generation tasks. Furthermore, FinanceIQ provides extensive testing with 7,173 single-choice questions across 36 subcategories relevant to economists and actuaries.

5.3 Cognitive Decoupling and Multimodal Evaluation

A fundamental flaw in traditional benchmarking is that single-task accuracy scores conflate a model's rote memorization of training data with its actual ability to reason and extrapolate. To address this, the FinEval-KR framework was introduced to decouple and independently quantify Knowledge and Reasoning abilities. Utilizing Bloom's taxonomy from cognitive science, FinEval-KR demonstrates that reasoning accuracy in complex financial tasks is bottlenecked primarily by a model's higher-order cognitive capabilities and its ability to apply logic, rather than sheer data recall.

The evaluation frontier is simultaneously expanding into non-textual domains. The MMMU (Massive Multi-discipline Multimodal Understanding) benchmark tests LLMs on college-level reasoning across highly heterogeneous image types. Consisting of 11.5K meticulously collected multimodal questions across 30 subjects, including Accounting, Public Health, Materials, and Architecture, MMMU integrates 30 image types, forcing the model to interpret financial charts, complex accounting tables, geographic maps, and chemical structures. Performance tracking highlights the ongoing difficulty LLMs face when processing visual structural data:

Model	MMMU Score
Human experts	76.2 – 88.6
GPT-4o	69.1
Claude 3 Opus	59.4

Further advancements in this area are supported by initiatives like Open-FinLLMs and benchmarks like MultiFinBen and DianJin-R1.

---

6. Operationalization, Hallucination Mitigation, and Governance

While rigorous benchmarking highlights theoretical capabilities, transitioning Financial Large Language Models from research assets into active production environments uncovers profound second and third-order operational challenges. Deploying an LLM as an actionable trading or advisory tool demands absolute precision, latency control, and strict regulatory governance.

6.1 Temporal Leakage and Time-Safe Evaluation

In fundamental analysis and quantitative backtesting, data must be strictly point-in-time. A critical and pervasive vulnerability in the deployment of FinLLMs is temporal leakage — the phenomenon where a model inadvertently incorporates intelligence generated after the targeted prediction date due to the chronological breadth of its pre-training corpus. If a model is pre-trained on a massive corpus extending through December 2023, utilizing that specific weight checkpoint to backtest stock predictions for mid-2023 will yield artificially inflated alpha, as the model "remembers" the future, rendering the backtest economically meaningless.

To mitigate this systemic flaw, robust deployment pipelines require time-safe document availability protocols. Advanced financial benchmarks and backtesting engines must enforce strict temporal boundaries, ensuring the model is evaluated solely on data that was publicly verified and available at the exact millisecond of the simulated decision. Furthermore, the industry is transitioning toward holistic evaluations that report not merely predictive accuracy, but also portfolio turnover metrics, exposure limits, execution latency, and capacity controls, embedding real-world market frictions directly into the AI assessment layer.

6.2 Hallucination Mitigation through RAG and Tool-Verified Numerics

In the financial sector, an LLM hallucination is not merely a statistical error; it represents a critical regulatory breach and a potential catalyst for massive capital loss. Autoregressive language models natively struggle with exact arithmetic precision and long-chain logical deductions, making them prone to fabricating revenue numbers or misinterpreting SEC filings.

To combat this, the architecture of production-grade FinLLMs is shifting decisively toward Retrieval-Augmented Generation (RAG) integrated with Tool-Verified Numerics. Instead of relying on internal parameter weights to recall the EBITDA of a specific corporation, a retrieval-first prompting pattern forces the LLM to halt generation, query an external, highly curated vector database or Knowledge Graph, ingest the exact text from the localized financial report, and output a response strictly bounded by that retrieved context. If mathematical calculations are required to answer the query, advanced agentic frameworks intercept the prompt, allowing the LLM to write and execute Python code in a secure, sandboxed environment rather than attempting to calculate the math directly via token prediction. This explicit separation of language generation from mathematical execution dramatically reduces numerical hallucination and ensures verifiable accuracy.

6.3 Agentic Workflows and Structural Compliance

As institutions move from passive conversational querying to active, autonomous execution, multi-agent systems are becoming the architectural standard. These frameworks coordinate multiple LLM agents, each assigned a specialized persona (e.g., Risk Manager, Sector Analyst, Quantitative Modeler). The agents engage in structured "debate traces," challenging each other's logic, hypotheses, and retrieved data before a final investment decision or portfolio rebalance is logged.

However, this autonomy introduces severe compliance and governance challenges. Production pipelines in heavily regulated jurisdictions must be heavily auditable. Every decision generated by a language-driven system must feature trace-links back to the specific evidentiary document that inspired it, enabling compliance officers to verify the algorithmic intent. Furthermore, models must seamlessly navigate complex structural boundaries, understanding the layout of financial tables and strictly adhering to taxonomies like XBRL to ensure that extracted numeric values are correctly associated with their underlying Generally Accepted Accounting Principles (GAAP).

---

7. The Complexities of Licensing and Open-Source Compliance

The rapid, decentralized proliferation of Financial Large Language Models has significantly outpaced the establishment of clear legal and commercial frameworks. The licensing of foundational model weights, scraped pre-training datasets, and instruction corpora creates a complex, often contradictory web of compliance constraints that heavily dictate how financial institutions can legally deploy these technologies.

7.1 The Phenomenon of Multi-Licensing and IP Contamination

A fundamental roadblock to the commercial deployment of open-source models is the phenomenon of multi-licensing and restrictive covenants. While an immense volume of leading financial AI research is built upon Meta's LLaMA architecture (powering models like Cornucopia-LLaMA-Fin-Chinese, XuanYuan-70B, and the PIXIU FinMA suite), the underlying LLaMA license inherently restricts usage to research and strictly non-commercial purposes. This effectively prohibits hedge funds, proprietary trading desks, and investment banks from utilizing these specific weights for live alpha generation or customer-facing advisory platforms without navigating bespoke, highly complex commercial licensing agreements.

Furthermore, the integration of diverse datasets during the Supervised Fine-Tuning phase introduces severe Intellectual Property (IP) contamination risks. When a permissively licensed open-source model (e.g., governed by Apache 2.0 or MIT, which allow commercial use) is fine-tuned using a dataset governed by a restrictive license (such as Creative Commons CC BY-NC 4.0, which strictly prohibits commercial use), the resulting neural network artifact exists in a highly contested legal gray area.

Research analyzing repository metadata highlights the severity of this issue. Across 43,455 model-dataset pairings analyzed on platforms like HuggingFace, there are 623 distinct model/dataset license combinations. Crucially, the license of the resulting model explicitly matches the license of at least one of its training datasets in only 41 of those 623 combinations. The most common structural conflict involves a model licensed under permissive terms (like Apache-2.0) trained on a dataset governed by a custom, "Other," or CC-BY-NC license, accounting for 11,731 conflicting pairs (roughly 27% of instances). Additionally, the widespread practice of multi-licensing — where a single model or dataset is released under overlapping open-source and Machine Learning specific licenses (like MIT and OpenRAIL) — creates novel legal complexities.

This disjointed licensing landscape forces compliance officers and risk managers at financial institutions to expend massive resources auditing the provenance of every data point utilized in the SFT pipeline. Utilizing models like FinBen, which explicitly shares all non-personal data under the MIT license, or tools governed strictly by CC-BY-SA 4.0, requires institutional tracking to ensure derivative works and quantitative strategies do not violate overarching intellectual property rights.

7.2 License Taxonomy: What You Can and Cannot Do

When constructing a training dataset for a FinLLM, understanding the precise legal implications of each license type is the difference between a deployable asset and a compliance liability.

Safest Choices: Public Domain and Permissive Licenses

For maximum freedom to use, modify, and commercialize a model, datasets governed by the following licenses impose the fewest restrictions:

• Public Domain (CC0 / PDDL): The creator has waived all rights. The Open Data Commons Public Domain Dedication and License (PDDL) and Creative Commons Zero (CC0) permit use without any restrictions whatsoever — no attribution required, no derivative-work conditions.
• MIT: Highly permissive and currently the most popular license for datasets on HuggingFace. Requires only attribution in software distributions, imposes no restrictions on commercial deployment of derived models.
• Apache 2.0: Equally permissive as MIT, with the additional advantage of an explicit patent non-aggression clause — important in finance, where patent litigation risk around proprietary trading algorithms is real.
• CDLA-Permissive-2.0 (Community Data License Agreement): Designed specifically for data sharing rather than software. Critically, it explicitly does not impose restrictions on the analytical results derived from the data (i.e., trained model weights), making it the cleanest license available for building commercially deployable FinLLMs.
• CC-BY 4.0: The standard for scientific and informational content. Allows commercial use and adaptation; requires attribution to the original creator. Widely used for academic financial datasets.

Licenses to Use with Caution: Restrictive and Copyleft

Including data with these licenses can contractually dictate how the resulting model may be used or distributed:

• Non-Commercial (CC BY-NC, CC BY-NC-SA): Explicitly prohibits use in any context where the LLM generates revenue or is deployed as part of a commercial enterprise. Five of the ten datasets in the Section 4.4 summary table carry this restriction. Fine-tuning a commercial model on CC BY-NC data is a direct license violation.
• ShareAlike / Copyleft (CC BY-SA, GPL, AGPL): These licenses require that any "derivative work" be distributed under the exact same license terms. There is active legal debate about whether an LLM constitutes a derivative work of its training data. If a court rules that it does, incorporating GPL or CC BY-SA data would legally require open-sourcing proprietary model weights — a commercially catastrophic outcome for institutional FinLLM deployments.

License	Commercial Use	Modification	Share-Alike Required	Patent Protection
CC0 / PDDL	Yes	Yes	No	No
MIT	Yes	Yes	No	No
Apache 2.0	Yes	Yes	No	Yes
CDLA-Permissive-2.0	Yes	Yes	No	No
CC-BY 4.0	Yes	Yes	No	No
CC BY-SA 4.0	Yes	Yes	Yes	No
CC BY-NC 4.0	No	Yes	No	No
CC BY-NC-SA 4.0	No	Yes	Yes	No
GPL / AGPL	Yes	Yes	Yes	No

7.3 The "Raw Facts" Exemption in Financial Data

Raw financial data occupies a distinctive legal position that practitioners frequently misunderstand. In jurisdictions like the United States, raw facts and single data points are not copyrightable — a historical stock price, a closing volume figure, or a macroeconomic indicator reading cannot be owned. This creates a significant surface area of freely usable data for FinLLM pre-training.

However, several adjacent protections remain:

• Database Rights and Terms of Service: While a single stock price is unprotected, the arranged database of a stock exchange may be protected under EU Database Directives as a structured collection. More practically, platforms distributing financial data (Bloomberg, Refinitiv, Seeking Alpha) use contractual Terms of Service to prohibit automated scraping and commercial redistribution, regardless of whether the underlying data is copyrightable. Violating ToS creates contract liability even where no copyright claim exists.
• Fair Use vs. EU Text and Data Mining (TDM) Exceptions: Relying on unlicensed copyrighted text (analyst reports, earnings call transcripts, news articles) for LLM training is legally contested. In the United States, developers typically argue such training constitutes "Fair Use" — a defense currently under intense judicial scrutiny, particularly when the resulting AI competes commercially with the original content creators. In the European Union, specific TDM exceptions permit training under certain conditions, but rights-holders may legally opt-out by machine-readable rights reservation, creating a shifting compliance surface.

7.4 Practical Recommendations for Dataset Construction

Industry best practices for building a legally defensible financial training corpus:

1. Filter strictly for MIT, Apache 2.0, CC0, CDLA-Permissive-2.0, and CC-BY 4.0 for any dataset that will feed a model with commercial application. Reject CC BY-NC and copyleft licenses at the ingestion stage.
2. Maintain meticulous provenance metadata. For every document ingested, record the source URL, retrieval timestamp, and exact SPDX license identifier (e.g., Apache-2.0, CC-BY-4.0). This audit trail allows targeted data removal if a license dispute arises post-training.
3. Treat ToS violations as equivalent to license violations. A model trained on Bloomberg Terminal data scraped in violation of Bloomberg's ToS carries contractual liability regardless of copyright status.
4. Monitor the EU AI Act and emerging TDM opt-out registries. The legal surface for training data is actively shifting; what was permissible in 2023 may require remediation by 2026 as case law and regulation crystallize.

---

8. Conclusion

The integration of Large Language Models into quantitative and fundamental finance is rapidly moving past the phase of theoretical research and entering the realm of hard operational deployment. The trajectory of this technology underscores a definitive shift away from the pursuit of brute-force parameter scaling and toward the rigorous application of domain-specific data engineering, cognitive reasoning distillation, and structural compliance.

The empirical evidence from the development of specialized architectures — ranging from the massive, mixed-dataset pre-training of BloombergGPT to the agile, RLHF-driven open-source pipelines of FinGPT and the highly localized, reasoning-optimized structures of CFGPT and Fin-R1 — demonstrates that contextual awareness and mathematical rigor dictate financial utility. The curation of hyper-specific datasets, such as the CFData corpus and entity-level sentiment tracking arrays, allows models to discern the granular nuances of market mechanics that generalized models overlook. Furthermore, the establishment of multi-dimensional evaluation frameworks like FinBen and FinEval ensures that as models transition into autonomous agentic workflows, their capabilities are rigorously decoupled, tested against hallucinations, and fortified against temporal leakage and systemic security vulnerabilities.

Ultimately, the successful capitalization of FinLLMs within the global financial infrastructure will depend entirely on the industry's ability to reconcile the inherently probabilistic nature of neural networks with the deterministic, highly regulated demands of capital markets. Through the deployment of retrieval-first architectures, tool-verified numerics, and strict adherence to verifiable data provenance and licensing frameworks, financial language models will continue to evolve into indispensable, highly auditable engines of modern quantitative intelligence.

---

---

Part II — Constructing Financial Hallucination-Suppression Alignment Datasets

Research Task: Propose a dataset construction method applicable to the vertical finance field, which generates high-quality, hallucination-suppression-specific alignment data based on existing large models (ChatGPT, GPT-4, DeepSeek).

This section synthesizes the research landscape across five dimensions — instruction tuning methodologies, synthetic generation techniques, hallucination typology, contrastive alignment data design, and end-to-end pipeline architecture — to arrive at a concrete, implementable proposal.

---

9. Recent Methodologies: Instruction Tuning and Alignment Datasets for Finance

9.1 The SFT-then-Align Paradigm

The dominant methodology for building vertical FinLLMs follows a two-stage pipeline:

1. Supervised Fine-Tuning (SFT): A general-purpose base model (LLaMA, Qwen, Mistral) is first fine-tuned on a large corpus of financial instruction-response pairs to acquire domain vocabulary and task format. Datasets like CFData-sft (1.6M pairs), Sujet-Finance-Instruct-177k, and FinGPT's task-specific corpora are the raw material here.
2. Alignment: The SFT model is then aligned via preference optimization (DPO, ORPO, KTO) or online RL (GRPO) to correct the residual failure modes introduced during SFT — most critically, hallucination.

The critical insight from recent work is that SFT alone does not suppress hallucination. SFT teaches the model to produce fluent, domain-appropriate text, but it also faithfully replicates any hallucinations present in the training data. A model trained on 1.6M CFData-sft pairs will confidently generate plausible-sounding but incorrect earnings figures or fabricated regulatory citations, because the training signal never distinguished factually grounded completions from plausible confabulations.

Alignment data specifically designed to teach the model when to refuse, when to hedge, and how to ground claims in retrieved sources is the gap that the current FinLLM literature has only partially filled.

9.2 Self-Instruct and Evol-Instruct Adaptations for Finance

Two general-purpose synthetic data techniques have been adapted for financial instruction generation:

• Self-Instruct (Wang et al., 2022): A seed set of human-written instruction-response pairs is used to prompt an LLM to generate new instruction-response pairs, then filtered for diversity and quality. Applied to finance, the seed set consists of expert-written financial QA pairs (CFA exam questions, analyst report templates), and the generator is GPT-4 or DeepSeek-V3.
• Evol-Instruct (Xu et al., 2023, WizardLM): A two-step process — first "evolve" existing instructions to be more complex (add constraints, require multi-step reasoning, introduce numerical tables), then generate responses. Evolving "What is Apple's P/E ratio?" into "Given the attached 10-K excerpt, calculate the trailing twelve-month P/E ratio, compare it to the sector median, and assess whether the current valuation is justified given projected revenue growth" produces much richer training signal.

The Investopedia instruction tuning dataset applies a variant of this: scraped financial text is processed by an LLM to generate structured QA pairs, followed by a secondary self-verification pass that filters hallucinated answers.

9.3 Constitutional AI and Process-Based Supervision for Finance

Two alignment paradigms from the general-purpose literature are directly applicable to financial hallucination suppression:

• Constitutional AI (CAI): A set of explicit principles ("do not fabricate financial figures", "always cite the source document", "express uncertainty when data is unavailable") guides both generation and critique. The model evaluates its own outputs against these principles and revises them — producing a self-critique-and-revision loop without requiring human preference labels.
• Process Reward Models (PRMs): Rather than rewarding only the final answer, PRMs assign correctness labels to each intermediate reasoning step. For financial QA, this means evaluating whether the revenue extraction step, the ratio calculation step, and the conclusion step are each independently correct. Fin-PRM operationalizes this for financial math.

---

10. Synthetic Generation with ChatGPT, GPT-4, and DeepSeek

10.1 Why Synthetic Generation is Necessary

Human annotation of financial preference data is prohibitively expensive. A single high-quality preference pair (prompt + chosen + rejected, both requiring domain expert verification) costs approximately $15–50 USD per item when sourced from credentialed financial analysts. At DPO-scale requirements (50K–500K pairs), this is economically infeasible for most research groups and mid-size institutions.

Synthetic generation using frontier models reduces this cost by 2–3 orders of magnitude while, under careful pipeline design, maintaining or exceeding the quality of purely human-labeled data.

10.2 Model Selection for the Generator Role

Generator Model	Strengths	Weaknesses for Finance
GPT-4o	Strong structured output, reliable JSON, multilingual	Proprietary; rate-limited
GPT-4-Turbo	Long context (128k), good at table-heavy 10-K analysis	Same as above
DeepSeek-V3	Open weights, competitive financial reasoning, cost-effective	Occasional hallucination on obscure tickers
DeepSeek-R1	Explicit CoT reasoning traces, strong on math	Slower inference; verbose outputs need post-processing
Claude 3.5 Sonnet	Reliable citation format, strong at regulatory text	Conservative refusals reduce negative sample diversity
Qwen2.5-72B	Strong on Chinese financial regulatory content	Weaker on IFRS vs. US GAAP distinctions

Recommended setup: Use DeepSeek-R1 (or GPT-4o) as the primary generator for CoT reasoning traces and preference pairs, with an independent GPT-4o (or Claude) acting as the verifier/judge — keeping generator and judge distinct to reduce confirmation bias.

10.3 Prompt Engineering for Financial Instruction Generation

The quality of synthetic data is entirely determined by the quality of the generation prompt. For financial hallucination suppression, prompts must:

1. Specify the source document — the generator should be forced to ground responses in a provided excerpt, not rely on parametric memory
2. Specify the output format — JSON with explicit fields for answer, calculation_steps, source_citations, confidence
3. Specify uncertainty conditions — the generator should produce hedged answers when the source document is insufficient, rather than confabulating

Example generation prompt for a grounded financial QA pair:

You are generating training data for a financial LLM.

SOURCE DOCUMENT:
{10-K excerpt: revenue $24.3B, COGS $15.1B, operating expenses $6.2B,
 interest expense $0.8B, tax rate 21%, shares outstanding 3.2B}

TASK: Generate a financial analysis question that requires multi-step calculation
over the above figures, and the correct, fully worked answer.

Requirements:
- The question must require at least 3 arithmetic steps
- The answer must cite specific line items from the source document
- The answer must express the confidence level (high/medium/low) for each step
- Do NOT introduce any figures not present in the source document

Output format:
{"question": "...", "answer": "...", "steps": [...], "citations": [...], "confidence": "..."}

10.4 Generating CoT Trajectories with DeepSeek-R1

DeepSeek-R1's explicit reasoning traces make it particularly valuable for generating stepwise financial reasoning data. The model's <think> tokens expose the full intermediate reasoning chain, which can be:

1. Extracted and formatted as PRMTrainer stepwise supervision data
2. Used as positive CoT examples for GRPO-based online RL (Fin-R1 style)
3. Compared against hallucinated reasoning chains to create DPO preference pairs

The key pipeline step is trace verification: after DeepSeek-R1 generates a reasoning chain, each intermediate step is checked against the source document and/or an external calculator before the trace is accepted into the training corpus.

---

11. Hallucination Typology in Financial LLMs

Financial hallucinations are not homogeneous. Effective suppression requires understanding the distinct failure modes and targeting each with appropriate training signal.

11.1 Financial Hallucination Taxonomy

Hallucination Type	Example	Detection Method
Numeric fabrication	"Apple reported revenue of $98.7B in Q3 2024" (actual: $85.8B)	Cross-reference financial data API (Yahoo Finance, Alpha Vantage)
Ticker/entity confusion	Using MSFT data when asked about MFST (typo)	Ticker validation against exchange symbol database
Temporal leakage	Citing a 2024 earnings figure when answering a 2022 query	Point-in-time filtering; date-aware retrieval index
Fabricated regulatory citation	"According to SEC Rule 17a-5(b)(3)..." (rule doesn't exist)	EDGAR/CFR lookup; legal citation validator
Ratio miscalculation	Calculating P/E as Price × Earnings instead of Price ÷ Earnings	Sandboxed Python execution with formula verification
XBRL tag hallucination	Using us-gaap:Revenue where us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax is required	XBRL taxonomy validator
Trend inversion	"Revenue grew 15% YoY" when the filing shows a 15% decline	Sign-check against source document
Spurious attribution	"Goldman Sachs analysts rate the stock a Buy" without a source	Citation grounding requirement; uncited claims flagged

11.2 Why Standard SFT Fails to Suppress These

SFT on prompt-completion pairs teaches the model what a good financial answer looks like, but does not teach it to prefer factually grounded answers over fluent confabulations. The cross-entropy training loss is indifferent to factual accuracy — a hallucinated earnings figure that matches the format of a correct one receives an identical training gradient.

Alignment training (DPO, KTO, GRPO with factual reward) creates a preference gap between grounded and fabricated responses that the SFT loss cannot create. This is the core motivation for building dedicated hallucination-suppression alignment data rather than simply scaling up SFT corpora.

---

12. Contrastive Alignment Dataset Design

The central data engineering challenge is constructing preference pairs where:

• The chosen response is factually grounded, calculation-correct, and appropriately hedged
• The rejected response is fluent and domain-appropriate but contains a specific, verifiable hallucination

The rejected sample must be plausible enough that the model cannot trivially distinguish it by stylistic cues alone — it must learn to distinguish on factual grounds.

12.1 Hallucination Injection Strategies

Strategy A — Numeric Perturbation:
Take a verified correct numerical answer and apply a systematic perturbation: scale by a random factor (×1.15, ×0.87), swap two digits, change the sign, or shift the decimal.

correct = 24.3  # billion
perturbations = [
    correct * random.uniform(0.85, 1.15),   # scale noise
    correct * -1,                            # sign flip
    float(str(correct)[::-1]),              # digit reversal (24.3 → 3.42)
    correct + random.randint(1, 5),         # additive noise
]

Strategy B — Temporal Displacement:
Replace a figure from the query period with the same metric from an adjacent period. "Q3 2024 revenue" is answered with "Q3 2023 revenue" — same ticker, same metric, wrong time. This trains the model to be sensitive to temporal context.

Strategy C — Cross-Entity Contamination:
Replace the correct company's figures with a competitor's figures from the same period. Apple's Q3 revenue replaced by Microsoft's Q3 revenue. This mirrors a realistic hallucination pattern (the model "knows" the figure, just associates it with the wrong entity).

Strategy D — Regulatory Citation Fabrication (via LLM):
Prompt a generator model to produce a plausible-sounding but non-existent regulatory citation. The generator is explicitly instructed to invent a rule number. The result becomes the rejected sample for queries requiring regulatory grounding.

12.2 DPO Preference Pair Schema

{
  "prompt": "Based on the attached 10-K excerpt, what was the company's gross margin in FY2023, and how does it compare to the prior year?",
  "source_doc": "FY2023: Revenue $24.3B, COGS $15.1B. FY2022: Revenue $21.8B, COGS $13.6B.",
  "chosen": {
    "response": "FY2023 gross margin = (24.3 - 15.1) / 24.3 = 37.9%. FY2022 gross margin = (21.8 - 13.6) / 21.8 = 37.6%. Gross margin expanded by approximately 0.3 percentage points year-over-year.",
    "hallucination_type": null,
    "verified": true
  },
  "rejected": {
    "response": "FY2023 gross margin = (24.3 - 15.1) / 15.1 = 60.9%. This represents a significant improvement from FY2022's margin of 37.6%.",
    "hallucination_type": "ratio_miscalculation",
    "injected_error_description": "denominator changed from revenue to COGS"
  }
}

12.3 KTO Unpaired Format (Lower Annotation Cost)

For institutions where pairwise annotation is still too expensive, KTO's unpaired format reduces the labeling burden to a single boolean per sample:

{"prompt": "What was Tesla's FY2023 automotive revenue?", "completion": "$78.5B", "label": true}
{"prompt": "What was Tesla's FY2023 automotive revenue?", "completion": "$91.2B", "label": false}

The false-labeled samples can be generated programmatically via the numeric perturbation strategies above, requiring no human annotation beyond the initial verification of the true samples.

---

13. End-to-End Pipeline Architecture

Stage 1 — Verified Source Data Ingestion

Build a clean, temporally indexed corpus of ground-truth financial facts.

Source	Data Type	License	Point-in-Time Safe
SEC EDGAR full-text search	10-K, 10-Q, 8-K filings	Public domain	Yes (filing date as timestamp)
World Bank Open Data	Macroeconomic indicators	CC-BY 4.0	Yes
Alpha Vantage API (free tier)	Historical price/volume, earnings	Permissive	Yes
Federal Register / CFR	US regulatory text	Public domain	Yes
XBRL inline viewer (SEC)	Structured GAAP line items	Public domain	Yes

Each ingested document is stored with metadata: {source_url, filing_date, ticker, fiscal_period, spdx_license, ingestion_timestamp}.

Stage 2 — Prompt Design and Instruction Seeding

Generate a diverse set of financial reasoning prompts spanning all 7 task classes from Section 2, starting from 500–1,000 human-written seed pairs (CFA exam questions, FINRA practice problems, auditor exam questions), then applying Evol-Instruct evolution passes to increase complexity.

Prompt diversity checklist:

• Numeric multi-step reasoning (gross margin, EV/EBITDA, YTM calculation)
• Entity and period disambiguation (correct ticker, correct fiscal quarter)
• Regulatory grounding (cite specific SEC/IFRS rule)
• Summarization under constraint (compress without losing KPIs)
• Uncertainty expression (hedge when source document is insufficient)

Stage 3 — Synthetic Generation with Verification

for each prompt p with source_doc d:
    1. Pass (p, d) to DeepSeek-R1 with grounding instruction
    2. Extract answer A and reasoning trace T from <think> block
    3. Verify each numeric claim in A against d via exact match or calculator
    4. Verify each regulatory citation via EDGAR/CFR API lookup
    5. If all verifications pass → accept (p, A, T) as positive sample
    6. If any verification fails → log failure type, discard or route to human review

Target acceptance rate: 75–85% for numeric QA; ~60% for regulatory citation tasks.

Stage 4 — Hallucination Injection for Negative Samples

Prompt Type	Primary Injection Strategy	Fallback
Numeric calculation	Numeric Perturbation (Strategy A)	Ratio Miscalculation
Time-series comparison	Temporal Displacement (Strategy B)	Numeric Perturbation
Multi-company comparison	Cross-Entity Contamination (Strategy C)	Temporal Displacement
Regulatory citation	Citation Fabrication via LLM (Strategy D)	Entity Confusion

After injection, a plausibility filter (LLM judge, 1–5 scale) screens out rejected samples that are obviously wrong. Samples scoring below 3 are regenerated.

Stage 5 — Automated Quality Verification (LLM-as-a-Judge + Tool Verification)

Track A — Tool-Verified Numerics: All numerical answers in chosen samples are re-verified by executing the claimed calculation in a sandboxed Python environment. Any sample where the Python result diverges from the stated answer beyond a configurable epsilon is flagged for removal.

Track B — LLM-as-a-Judge:

Judge prompt:
"You are evaluating a financial QA preference pair.
PROMPT: {p}
SOURCE DOC: {d}
CHOSEN RESPONSE: {A_chosen}
REJECTED RESPONSE: {A_rejected}

Score on 1-5:
1. Is CHOSEN factually correct and grounded in SOURCE DOC?
2. Is REJECTED plausible but factually wrong?
3. Is the factual distinction clear to a domain expert?

Output: {"chosen_correctness": X, "rejected_plausibility": X,
         "distinction_clarity": X, "overall_accept": true/false}"

Acceptance criteria: chosen_correctness >= 4, rejected_plausibility >= 3, overall_accept = true.

Estimated pipeline yield:

Stage	Items In	Acceptance Rate	Items Out
Stage 1 (ingestion)	—	—	~100K source documents
Stage 2 (prompt seeding)	1,000 seeds	Evol ×10	~10K prompts
Stage 3 (generation + verify)	10K prompts	75%	~7,500 positive samples
Stage 4 (hallucination injection)	7,500	80% plausibility filter	~6,000 preference pairs
Stage 5 (LLM judge)	6,000 pairs	85%	~5,100 final pairs

A single pipeline run produces ~5,000 high-quality DPO preference pairs — sufficient for meaningful hallucination-suppression alignment at 7B–13B scale. Multiple runs with different source document batches can scale to 50K+ pairs.

---

14. Proposed Dataset Schema

The final dataset is stored in a format compatible with DPOTrainer, KTOTrainer, and PRMTrainer:

{
  "id": "fin-halluc-00001",
  "prompt": "...",
  "source_doc": "...",
  "source_metadata": {
    "url": "https://www.sec.gov/...",
    "filing_date": "2024-02-02",
    "ticker": "AAPL",
    "fiscal_period": "Q1FY2024",
    "spdx_license": "public-domain"
  },
  "chosen": {
    "response": "...",
    "reasoning_trace": "...",
    "verification_method": "calculator",
    "verified": true
  },
  "rejected": {
    "response": "...",
    "hallucination_type": "numeric_perturbation",
    "injected_error_description": "...",
    "plausibility_score": 4
  },
  "judge_scores": {
    "chosen_correctness": 5,
    "rejected_plausibility": 4,
    "distinction_clarity": 5,
    "overall_accept": true
  }
}

---

15. Open Questions and Future Directions

1. Hallucination rate measurement: No standardized financial hallucination benchmark equivalent to HaluEval exists for the general domain. FinBen and FinEval measure task accuracy but not hallucination rate specifically. A dedicated hallucination evaluation suite covering all eight types in Section 11.1 is a prerequisite for measuring pipeline effectiveness.

2. Reward model calibration: For GRPO-based online RL (Fin-R1 style), the reward function determines everything. A binary correct/incorrect reward is insufficient for the nuanced spectrum of financial correctness (exactly right vs. close enough vs. directionally correct vs. completely wrong). Designing a continuous, calibrated financial reward function is an open problem.

3. Distribution shift: A model aligned on SEC EDGAR filings will hallucinate differently when deployed on earnings call transcripts or analyst reports. Domain-specific alignment data needs to be constructed for each sub-domain of deployment.

4. Adversarial robustness: The pipeline above assumes a cooperative generation setting. In real deployment, users may craft prompts specifically designed to elicit hallucinated financial figures (prompt injection, jailbreak-style queries). Building adversarial financial prompts into the training corpus is a necessary extension.

---

Works Cited

1. The New Quant: A Survey of Large Language Models in Financial Prediction and Trading — arXiv 2510.05533
2. A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges — IDEAS/RePEC arXiv 2406.11903
3. Parameter Efficient Instruction Tuning of LLMs for Financial Applications — IJCAI 2024
4. A Comparative Analysis of Instruction-Tuning LLMs for Financial Text Classification — arXiv 2411.02476
5. BloombergGPT: A Large Language Model for Finance — arXiv 2303.17564
6. arXiv 2402.02315
7. GitHub: AI4Finance-Foundation/FinGPT
8. GitHub: adlnlp/finllms — FinLLMs benchmarks and datasets
9. LLM + Datasets: Finance — HuggingFace Collection
10. GitHub: TongjiFinLab/CFGPT
11. GitHub: SUFE-AIFLM-Lab/FinEval
12. Announcing LLM Open Finance Models — DragonLLM on HuggingFace
13. CFGPT: Chinese Financial Assistant with Large Language Model — arXiv 2309.10654
14. GitHub: YY0649/ICE-PIXIU
15. TheFinAI/finma-7b-nlp — HuggingFace
16. Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning — arXiv 2503.16252
17. FinLang/investopedia-instruction-tuning-dataset — HuggingFace
18. Large Language Models in Finance: A Survey — arXiv 2311.10723
19. BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark — arXiv 2302.09432
20. CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models — arXiv 2407.02301
21. WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain — arXiv 2211.00083
22. [FinBen: A Holistic Financial Benchmark for Large Language Models](https://proceedings.neurips.cc/paper_files/paper/2024/file/adb1d9fa8be4576d2870 3b396b82ba1b-Paper-Datasets_and_Benchmarks_Track.pdf) — NeurIPS 2024
23. FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models — ACL Anthology
24. CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model — arXiv 2311.05812
25. FinEval-KR: A Financial Domain Evaluation Framework for LLMs' Knowledge and Reasoning — arXiv 2506.21591
26. An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face — arXiv 2502.04484