How Athyna Intelligence Scaled RLHF and RLVR to Hundreds of Expert Finance Hours per Week

TL;DR: Athyna Intelligence staffed a bench of US-GAAP finance domain experts for RLHF and RLVR, handling roughly 1,500 paired comparisons and 300 to 400 senior expert hours per week. Across RLHF preference labeling and RLVR environment review, up to 20 experts evaluated model responses and helped turn professional financial judgment into an executable reward signal, with five-business-day turnaround per batch.

Training models on corporate finance tasks requires finance domain experts who can evaluate more than numerical accuracy. In reinforcement learning from human feedback (RLHF) and reinforcement learning with verifiable rewards (RLVR), a response can produce the right calculation while relying on unsupported data, applying the wrong financial treatment, or giving an explanation that would not hold up with the finance professional reviewing it.

Athyna Intelligence built its expert bench to apply that financial judgment at scale. Running RLHF preference labeling and RLVR environment review in parallel required experts who could evaluate model responses and determine what should count as materially correct in a finance context.

What Does RLHF Preference Labeling Look Like for Finance Tasks?

RLHF preference labeling for finance tasks requires domain experts to compare two agent transcripts and determine which response demonstrates stronger financial accuracy, reasoning, and judgment. In Athyna Intelligence's workflow, RLHF finance domain experts read both transcripts end to end, including every tool call, across multi-turn scenarios involving personas such as a CFO, external auditor, or operations coordinator.

Experts evaluate more than the final answer. They look at:

• Numerical accuracy: The correctness of calculations and figures in the response.
• Use of available data: How well the agent works within the available information and pushes back when a request cannot be supported.
• Response to misdirection: How the agent handles deliberate attempts to steer its reasoning incorrectly.
• Audience fit: How well the explanation matches the persona’s level of financial expertise.

Each comparison produces a ranking and a two-to-four-sentence rationale citing specific numbers or tool calls from the transcripts. Generic preference statements are rejected. Experts must identify exactly why one response demonstrates stronger financial reasoning.

How Does RLVR Environment Review Work for Finance Tasks?

RLVR environment review checks whether the reward structure used to evaluate a model is financially accurate, solvable, and specific enough to reward the right outcome. In reinforcement learning with verifiable rewards (RLVR), model outputs are evaluated against machine-checkable criteria. Athyna Intelligence's finance experts review those criteria before the environments reach a model.

Each environment contains a series of gates. Every gate includes a description of what is being evaluated, an expected value, and a tolerance type that defines how the result is checked.

Who Reviews Reward Model Gates for Financial Correctness?

Senior finance domain experts review reward gates to determine whether they are financially accurate and whether a well-reasoned response can realistically satisfy them. For Athyna Intelligence's senior RLVR workstream, CPA or CFA credentials were preferred, alongside applied experience in roles such as Controller, FP&A Lead, or Finance Manager.

Experts also write tolerances in plain language, which the platform resolves into machine-checkable gates. They can suggest rewrites and review version history as the environment is refined. The expert’s judgment about what counts as materially correct becomes part of the executable reward signal the model trains against.

All scenario data used in this work is synthetic. The environments, personas, and financial information are created for training purposes and do not use real company financials.

What Qualifications Do RLHF and RLVR Finance Experts Need?

RLHF and RLVR finance experts need applied US-GAAP experience, professional experience at U.S. firms, and the financial judgment to evaluate model reasoning in practice. These requirements reflect the level of applied expertise needed from finance subject matter experts for LLM training.

Athyna Intelligence’s qualifications varied by workstream:

• Applied US-GAAP experience: Experts needed experience using US GAAP in day-to-day finance work and evaluating financial decisions in practice.
• U.S. corporate finance experience: Professional experience at U.S. firms provided familiarity with the accounting and finance standards reflected in the tasks.
• Big 4 or Fortune 500 background preferred: These backgrounds provided relevant experience for scenarios requiring senior-level financial judgment.
• CPA or CFA preferred for senior RLVR review: Senior reviewers assessed gates and tolerances that could become part of the model's executable reward signal.
• Controller, FP&A Lead, or Finance Manager experience: These were among the senior profiles targeted for RLVR environment review, where experts needed to determine what should count as materially correct.

For a broader breakdown of expertise by AI training task, see Athyna's guide to best practices for hiring AI model trainers.

How Do You Source Finance Domain Experts for RLHF and RLVR?

Finance domain experts for RLHF and RLVR are sourced by defining the financial decisions they will evaluate, then matching those tasks to the required experience, seniority, and credentials. For Athyna Intelligence’s finance workstreams, hiring a finance AI trainer meant identifying professionals with applied US-GAAP experience and matching their seniority and credentials to the type of evaluation they would perform.

Athyna Intelligence sourced and vetted these profiles before they entered calibration and live production. This gives teams access to the level of finance expertise required for reward modeling without having to build and manage a specialist bench internally.

How Do You Maintain Quality Across a Bench of Finance Domain Experts?

Quality across a bench of finance domain experts is maintained through calibration, gold-labeled tasks, and double-rating throughout production. Athyna Intelligence used three quality controls as the bench scaled:

• Mandatory calibration: Every expert completed calibration before starting live work to establish a consistent evaluation standard.
• Gold-labeled tasks: Pre-evaluated tasks were seeded invisibly into live batches to check whether experts continued applying that standard.
• Double-rating: Experts independently evaluated the same work to measure inter-rater agreement and identify differences in judgment.

This process supported 300 to 400 senior expert hours per week while applying the same quality controls across the bench.

What Breaks When Generalists Review Financial Reasoning?

Generalist reviewers are able to verify whether a calculation is numerically correct without recognizing a problem with the financial reasoning behind it. A result can satisfy an automated check while relying on a treatment that an experienced finance professional would reject. For financial reasoning training data, that creates a gap between getting the number right and applying the right financial standard.

Research on specialized annotation shows why domain expertise matters. The 2025 COLM study ⁠Evaluating Large Language Models as Expert Annotators evaluated expert annotation across finance, biomedicine, and law and found a persistent gap between model and human expert annotations across the domains studied. The researchers also found that reasoning models did not produce statistically significant improvements over non-reasoning models.

The same expertise gap matters when selecting human reviewers. RLHF finance experts need to identify weak financial reasoning even when the final number looks plausible. RLVR reviewers need to identify reward gates that are technically executable but encode the wrong financial standard. Athyna covers the broader risks of mismatching expertise to AI training work in its guide to common mistakes when hiring AI model trainers.

Where Does Human Finance Expertise Become the Bottleneck?

Human finance expertise becomes the bottleneck when automated verification can confirm a result but cannot determine whether the financial reasoning behind it is professionally sound.

Consider a model that produces a plausible ARR figure based on an unsupported assumption. The calculation may be correct and the output may pass a numerical check, but the underlying financial reasoning is still wrong. The same problem appears when a financial treatment computes correctly but follows an approach an experienced controller would reject.

Automated verification can check the criteria it has been given. A finance expert still has to determine what should count as materially correct in the first place. In RLHF, that judgment determines which response provides the stronger training signal. In RLVR, it helps define the reward gates used to evaluate model outputs. As finance tasks become more specialized, this is where human expertise becomes the bottleneck.

Athyna Intelligence builds expert benches around the domain experience, credentials, and quality standards each AI training workstream requires. Whether you’re scaling RLHF preference labeling, RLVR environment review, or both, talk to Athyna about the finance expertise your training pipeline needs.