LLMs in Finance: Use Cases, Risks, Architecture & Implementation Guide

Banks and fintechs can use LLM in finance to find answers in internal policies, review financial documents, summarize research, and turn client meetings into useful notes. But the model shouldn’t make things up or make critical calls on its own. It needs access to approved company information, and a responsible person must review outputs that could affect customers, lending decisions, or regulatory filings.

Getting access to an LLM in finance is the easy part. Major banks have already demonstrated how these models can work at scale. The real challenge for a mid-sized lender, payments company, insurer, or wealth platform is figuring out where they make sense and how to use them safely. Which workflows are worth the risk? What will hold up under an audit? And what do the rules require as of October 2026? Here, we answer these questions and take a closer look at generative models within the broader field of AI in financial services.

In brief

• Don’t start with customer-facing AI. Internal workflows are a more practical first step, as deployments at Morgan Stanley, JPMorgan, and Bank of America demonstrate.

• An LLM is only as good as the information it retrieves. In a 2023 benchmark, GPT-4-Turbo with a shared retrieval index got 81% of questions about public-company filings wrong or left them unanswered.

• Use the right tool for the job. Traditional ML remains the better fit for fraud scoring and credit decisions. LLMs are better suited to summarizing, explaining, and routing information.

• Keep an eye on the rules. The EU’s high-risk credit-scoring obligations now start on December 2, 2027, while new US model risk guidance does not cover generative AI.

How an LLM Differs from the Models Finance Already Runs

Machine learning is nothing new to banks and fintechs. Traditional credit and fraud models work with structured data and return scores that teams can check against historical results. They follow a defined logic, and risk teams have established ways to test their performance.

An LLM works on unstructured language. It reads contracts, emails, filings, and call transcripts, and it produces open-ended text. It is often a third-party foundation model accessed through an API. Its output can vary for the same input, and it can state false information with confidence. NIST’s Generative AI Profile lists confabulation among twelve risks that are unique to or made worse by generative AI.

That changes how banks need to manage risk around large language models in finance. You can’t rely on backtesting alone when the model generates a paragraph instead of a score. Teams need:

• evaluation sets of real questions;

• checks that every answer is grounded in a cited source;

• logging of prompts and outputs;

• clear rules on who reviews what.

The CFA Institute’s practical guide to LLMs in the financial industry describes adoption of large language models in finance as still early. It notes interest in analyzing financial reports, automating customer service, detecting fraud, and market sentiment analysis. The term financial LLM usually describes one of two things: a general frontier model connected to a firm’s financial data, or a model pre-trained on financial text. Purpose-built financial LLM models exist, and BloombergGPT is a widely cited example. For a mid-sized firm, the first option is the practical starting point, for reasons covered in the architecture section below.

Which LLM Use Cases in Finance Deliver Value Today

The most credible LLM use cases in finance share the following:

• the input is mostly text;

• a qualified person reviews the output;

• an error costs time before it costs money.

FINRA’s 2026 Annual Regulatory Oversight Report names summarization and information extraction as the top generative AI use cases it observed among member firms. Both fit that profile.

Where an LLM in Banking and Finance Adds Value First

Internal knowledge assistants are one of the clearest examples of how banks can put LLMs to work. Morgan Stanley Wealth Management rolled out its AI @ Morgan Stanley Assistant in September 2023. By June 2024, 98% of its Financial Advisor teams had adopted the tool.

There’s also an important engineering lesson in OpenAI’s case study. Morgan Stanley worked with OpenAI to improve retrieval through repeated evaluations, increasing document access from 20% to 80%. That figure comes from OpenAI, but the takeaway applies more broadly: testing and refining retrieval can make internal knowledge tools much more useful. The same principle matters for AI in wealth management.

Bank of America uses a more conservative approach to AI in customer interactions than in employee-facing applications. In an August 2025 interview, the bank’s head of digital platforms said Erica still relies primarily on rules-based technology rather than generative AI.

By July 2026, the bank had brought generative AI into EricaAssist, an internal tool used by more than 18,000 customer service representatives. It gives employees a quick summary of why a customer is calling, surfaces relevant information, and recommends what to do next. Bank of America says the tool reduces average call time by nearly a minute. It’s a practical example of how LLMs can speed up banking operations while leaving customer conversations in human hands. Copilots of this kind are among the most defensible LLM use cases in banking because the reviewer is already in the loop. Among the options for an LLM in banking and finance, an internal assistant is also the easiest to take through risk and compliance review, because errors stay inside the firm.

When employees need answers about legal policies or compliance rules, they shouldn’t have to ask the legal team the same questions over and over. Lumitech built an AI-powered legal knowledge assistant for a fintech institution operating across the MENA region to make that information easier to find.

Employees no longer need to dig through internal policies or ask the same routine questions repeatedly. The assistant provides answers based on approved documents and links back to the sources, with access limited according to each person’s role. Company data remains within the organization’s environment. It can explain policies and processes, but questions that require legal judgment are passed to the legal team. During the pilot, the team noticed fewer repeat questions and said answers were available within minutes. These were qualitative observations, not measured results.

LLMs for Document Review and Compliance

Financial documents aren’t always easy to work with. Onboarding packs, loan files, and regulatory updates can include dozens of pages, which makes employees hunt for important details in dense tables. An LLM for financial data extraction can sort through these files, pull out the information teams need, compare existing policies with new rules, and prepare summaries for analysts to check. Add intelligent document processing, and teams can spend less time digging through documents and more time putting the information to use.

How LLMs Help Analyze Financial Information

An LLM for financial analysis can reduce the time analysts spend reviewing earnings reports, comparing disclosures from different periods, and searching large volumes of financial research. The key issue? Even the strongest model can generate incorrect answers when given unreliable information. In the FinanceBench study, researchers evaluated 16 model configurations on questions about public-company filings. GPT-4-Turbo answered only 19% of questions correctly when all filings were stored in a single shared index. The remaining 81% were answered incorrectly or left unanswered. When researchers created a separate index for each filing, the correct-answer rate rose to 50%. The models used in these tests date back to 2023, so the findings say more about the importance of retrieval design than the performance of today’s finance LLMs. The same principle holds for deal teams adopting AI in investment banking.

LLMs in Customer Service

Putting an LLM in front of customers comes with a real trade-off: lower service costs don’t mean much if the experience gets worse. Klarna saw this firsthand. In February 2024, the company announced that its OpenAI-powered assistant had handled 2.3 million conversations in just one month, which is two-thirds of all customer service chats.

Cost reductions mean little when customers can no longer access adequate support. In May 2025, Klarna’s CEO acknowledged that the company had placed too much emphasis on savings rather than quality. Klarna then began testing new human agents to make human support easier to access. Banks and fintechs must also account for transparency obligations under Article 50 of the EU AI Act, applicable from August 2, 2026.

Where Traditional ML Still Works Best

Using an LLM for every financial decision can be risky. Fraud scoring and credit decisioning need models that deliver consistent results quickly and make their decisions explainable. Traditional models built through AI and ML development services, with proper model risk oversight, are still better suited to this role.

LLMs can help investigators interpret cases, understand reason codes, and direct alerts. Final decisions should still rely on a model with logic that validators can test and scrutinize. Our guide to AI credit scoring explains why this separation matters.

What the Public Evidence on Results Shows

Adoption is far easier to document than outcomes. Evident’s 2026 AI Index for Banks, published in October 2026 and covering 50 of the world’s largest banks, found that only 12% of reported AI use cases show impact against operational KPIs and barely 1% disclose a concrete financial return. Twelve of those banks now report a realized or projected return across their AI activities, up from eight a year earlier.

JPMorgan’s estimated $2 billion in AI value should not be attributed to generative AI alone. In September 2024, President Daniel Pinto linked that value to the bank’s AI use cases, including fraud prevention. JPMorgan was also rolling out its LLM Suite assistant to 140,000 employees. But these were broader AI initiatives, and the $2 billion figure did not measure the impact of GenAI separately. Public evidence on large language models in financial services at mid-sized firms is thinner still, and independently verified, quantified results from mid-market deployments are scarce in public sources.

Before launching a pilot, define baseline metrics such as handling time, time spent searching, and review effort. LLM financial services deployments can then be judged on measured change. The same gap between demos and documented results is a strong reason to rank LLM use cases in finance by how easily their impact can be measured. Start with workflows where time saved is visible within weeks.

Not sure where to start?

Lumitech’s fintech and finance software development team can identify the strongest use case, assess data readiness and risk, and define a prototype around it.

Get in Touch

Choosing the Architecture: Retrieval, Fine-Tuning, or a Domain Model

Teams working with large language models for finance can choose among three architectural paths.

Retrieval-augmented generation (RAG) connects a general model to an indexed library of your documents and requires answers to cite them. The CFA Institute notes that many organizations pair frontier LLMs with RAG. This keeps strong general language ability while adding domain-specific data through retrieval. It is a sensible default for teams evaluating an LLM for finance because it keeps knowledge current and auditable.

Fine-tuning helps a model learn how you want it to respond, whether that means using a specific tone or format, or applying the right classification labels. But it won’t keep facts current, so it usually works best alongside retrieval.

Domain pre-training builds a model from a large financial corpus. Bloomberg trained BloombergGPT, a 50-billion-parameter model, on 363 billion tokens of financial text and 345 billion general tokens. Few firms have that data or budget. As a planning rule of thumb, a custom finance LLM makes sense only after retrieval and fine-tuning have demonstrably failed on a high-value task. The comparison below uses planning estimates drawn from these trade-offs.

For any LLM in finance that touches client data, the deployment model matters as much as the technique. Private model endpoints, zero data retention terms, or open-weight models running inside your own cloud account keep client data out of public training pipelines. In Morgan Stanley’s case, OpenAI’s zero data retention policy addressed the bank’s concern that proprietary data could be used to train public models.

Lumitech has applied a retrieval-first design in a RAG-based portfolio analytics platform for a Panama-based fintech startup. The platform combines multi-broker data aggregation with AI-generated explanations of market events.

The 2026 Regulatory Picture

The rules that apply to an LLM for financial services depend mainly on what its output decides. The summary below reflects the status as of October 2026. It is general information and does not constitute legal advice.

European Union

• Digital Omnibus on AI. Regulation (EU) 2026/1744 entered into force on July 27, 2026. High-risk rules for stand-alone Annex III systems now apply from December 2, 2027.

• What counts as high-risk. Annex III covers creditworthiness evaluation and credit scoring of individuals, except for fraud detection. It also covers risk assessment and pricing in life and health insurance.

• What was not delayed. Article 50 transparency requirements, including informing people when they’re interacting with an AI system, have applied since August 2, 2026. The deadline for machine-readable marking of generative AI systems already on the market was extended to December 2, 2026.

• DORA. In effect since January 17, 2025, it may cover LLM API and cloud providers as ICT third-party service providers, with implications for contracts, third-party registers, and exit planning.

United States

• SR 26-2. On April 17, 2026, the Federal Reserve, OCC, and FDIC replaced SR 11-7 with revised model risk guidance, issued by the Fed as SR 26-2. The new guidance states that generative and agentic AI models are novel and rapidly evolving and therefore outside its scope. It is expected to be most relevant to banking organizations with over $30 billion in total assets.

• Treasury framework. On February 19, 2026, Treasury released the Financial Services AI Risk Management Framework. It is non-binding guidance that adapts the NIST AI RMF to financial services.

• CFPB. In May 2025, the CFPB withdrew its 2022 and 2023 circulars on adverse action notices, including the one on credit decisions based on complex algorithms. The underlying ECOA and Regulation B duty to give specific reasons for credit denials still stands.

• FINRA. FINRA gives generative AI its own section in the 2026 report, making clear that broker-dealers cannot treat these tools as an exception to existing supervision and recordkeeping rules.

• Colorado. SB 26-189, signed in May 2026, replaces the state’s 2024 AI Act with an automated decision-making technology law that takes effect January 1, 2027, and covers lending among other areas. It requires a post-adverse-outcome disclosure within 30 days.

Financial institutions should make accurate, evidence-based claims about their use of AI. In March 2024, the SEC settled charges against Delphia and Global Predictions over misleading AI claims, resulting in $400,000 in combined penalties.

Key Challenges and Practical Safeguards

Five common risks can affect LLM applications in finance, but practical controls can help keep them in check.

• Confabulation. Make the model cite its sources and avoid answering when retrieved information doesn’t provide enough support. Test it against a golden set of real questions whenever the model or prompt changes.

• Data privacy. Classify documents before indexing, enforce access controls at query time, and remove unnecessary personal data. Contractually require providers to retain no data.

• Prompt injection. Treat uploaded files and web content as untrusted input, and limit the tools the model can access. Require human approval for any action involving money transfers or changes to records. Prompt injection ranks first in the OWASP Top 10 for LLM Applications.

• Provider lock-in. Document your model and vendor dependencies, and plan for switching if a service becomes unsuitable. Keep prompts and data portable across APIs to avoid vendor lock-in.

• AI bias and misleading claims. Test AI outputs across different customer segments wherever they may influence decisions or service outcomes. Back claims about the system’s capabilities with measurable results rather than assumptions. Our guide to ethical AI in finance covers the broader principles.

A financial LLM that passes these controls in testing still needs monitoring in production. Model updates, new documents, and new user behavior all shift accuracy over time.

How to Implement an LLM in Finance: From Pilot to Production

A phased rollout helps firms control the starting scope of LLM finance projects, evaluate performance, and expand based on measurable outcomes.

1. Choose a focused use case and measure the baseline. Identify a text-heavy internal workflow with a designated owner, then track current handling and search times to compare against pilot results.

2. Classify the risk tier. Map the workflow against the use-case table above. Obligations for an LLM for financial services vary by market, so confirm which rules apply in each one you serve.

3. Get the data and permissions right. Choose which documents to include, set access rules, and decide how to handle personal data. This step often reveals data quality problems that need to be fixed early.

4. Design retrieval and evaluation together. Build an evaluation set using realistic user questions and verified answers. Check source grounding and citation accuracy, and test how the system handles requests it should not answer.

5. Set up review and audit logs. Define approval responsibilities and record prompts, retrieved sources, and responses. FINRA includes prompt and output logging among the practices firms may want to consider.

6. Treat the system as a production service. Track prompt and model versions, monitor changes in performance and costs, and retest the system after every update. Our guide to LLMOps for enterprises covers additional operational practices.

Expand only after the first workflow outperforms its baseline. Apply the same risk assessment and evaluation steps to each new use case to build a repeatable LLM finance roadmap. Documenting every stage also makes it easier to show regulators and auditors how an LLM in finance program is governed.

The work doesn’t end once you choose an LLM for finance. Model versions can change over time; providers adjust their pricing; new technical or usage restrictions may come up. A financial LLM architecture built around retrieval, evaluation, and logging makes it easier to respond without starting over. For LLM financial services programs, it also gives teams a consistent approach to reporting and oversight and enables them to scale without repeating governance work for each new use case.

Take a governed LLM from pilot to production.

Lumitech designs and builds generative AI solutions and RAG assistants for regulated teams, including the retrieval, evaluation, access-control, and monitoring layers these workflows depend on.

Let’s Talk