Transformers are not just about language

Transformers are not just about language.

They are about structure.

We often think of LLMs as text models because ChatGPT made language the most visible use case. But the deeper lesson is broader: transformer models can learn hidden relationships inside large sequences of real-world events.

That is why transaction foundation models in finance are interesting.

Financial transactions are not just rows in a database. They are records of real events: who interacted with whom, when, through which channel, under what context, and with what outcome.

The old way

The old way was often semi-manual: humans selected features, engineered signals, and trained specialist models for fraud, credit, authorization, or recommendations.

The problem is simple: if humans do not notice a latent structure, it may never be exposed to the model.

The new direction is different: feed much richer transaction sequences into transformer-style models and let them learn the structure of the financial space.

Examples from the NVIDIA article

A few examples from NVIDIA’s article:

• Revolut and NVIDIA built PRAGMA, trained on 24 billion events across 26 million user records in more than 100 countries.
• Mastercard is developing a large tabular foundation model for payments.
• Adyen is applying transaction foundation models to payment authorization at massive scale.
• Stripe is using transaction context for fraud reduction and the emerging world of agentic commerce.

My intuition

1. Time should be part of the representation.

Time should not be treated as just another metadata field. It should be embedded into the transaction sequence, similar to how position embeddings work in LLMs.

Events that happen together, or close in time, may become close in representation space. That gives the model a way to see structures humans did not explicitly encode.

2. Proprietary transaction data is the real opportunity.

Public foundation models cannot easily learn the private structure of a bank, payment network, or fintech platform: customer behavior, merchant relationships, authorization outcomes, fraud feedback, and temporal patterns. That private event space is the real training ground.

3. The model does not necessarily need to be frontier-scale.

The important question is not “how large is the model?” but “does the model have enough capacity to represent the complexity of the structure it is trying to learn?” A transaction foundation model needs to be large enough for the financial structure, not large for the sake of scale.

This is not “ChatGPT for banking.”

It is more interesting than that: a foundation model for the structure of financial behavior.

Original article: Why Financial Institutions Are Converging on Transaction Foundation Models to Build Their Own Intelligence