How we use AI to learn a broker's file formats from headers, filenames, and stale samples, then compile that understanding into deterministic, documented Python that runs on SOC 2 infrastructure. No client record ever reaches a model.

Every firm putting AI to work inside a regulated business runs into the same question eventually: where does the model's access to data actually stop.

Most answers are either too cautious or too loose. The cautious answer keeps AI away from anything sensitive and pays for it in engineering time, because the tasks AI is best at, reading unfamiliar structure, mapping one system's vocabulary onto another's, are exactly the tasks that show up wherever confidential data lives. The loose answer lets a model see whatever is convenient and hopes the confidentiality agreement covers it.

There is a third option. Split the work into two kinds. Discovery is figuring out what an unfamiliar system is: what a file contains, what a column means, how one counterparty's report codes correspond to another's. Production is doing that same work correctly, every day, forever, on data nobody outside the firm should ever see. AI is very good at the first kind of work. It should never touch the second, because production should run on deterministic code regardless of who or what figured out how to write it.

The discipline is finding the smallest set of facts discovery actually needs, proving it never needs more, and compiling whatever it learns into code that runs with no model in the loop. Here is what that looks like in the most literal integration problem investment operations has: reading a new counterparty's raw data feed for the first time.

A new prime broker drop arrives as a zip: hundreds of files, nested three folders deep, no documentation. Some are .xls with a three-line banner above the real column header. Some are .csv with a broker-internal report code in the filename and nothing else to identify them. One is encrypted. Nobody at the broker will send you a schema, because internally the schema is obvious to them and has been for fifteen years.

Somebody now has to work out what each of those files is. Which one carries trades, which carries positions, which carries the cash ledger. Whether the quantity column is signed. Whether BL / SL / SS / BC means buy-long, sell-long, sell-short, buy-cover, or something else at this particular broker. Whether the file dated Tuesday contains Monday's activity.

This is the actual bottleneck in investment operations integration. Not the reconciliation logic, which is well understood. Not the reporting, which is a solved problem. The bottleneck is that every counterparty speaks a private dialect, and learning a dialect has historically cost weeks of engineering time per counterparty.

Large language models are extremely good at this specific task. Give one an unlabelled tabular file and it will tell you what it is with high accuracy. That capability is real and it is available today.

There is also an obvious problem. Those files contain positions, counterparty names, account numbers, and trade-level detail for a client who signed a confidentiality agreement. Most firms resolve the tension by declining to use AI on the data at all, and continue to pay the integration cost in engineering weeks.

We split the work instead. The rest of this piece is about exactly what crosses that line, and what does not.

What Actually Crosses the Boundary, a Working Example

To identify a broker file, you need almost none of its content. This is the observation the whole architecture rests on. Four signals are sufficient in nearly every case:

• The filename. SRPB_104283_3300214657_Custody_Position_20260616.xls tells you the broker, the report family, the account, the domain, and the data date before you open it. Broker filenames are structured because broker file generation is structured.

• The column header row. The first non-empty row is the schema. Account ID | Client Trade Ref | PB-Id | Traded Quantity | Cash Consideration | Buy/Sell | SEDOL | ISIN is a complete description of a trade blotter. Discovery only needs to see that row. It has no reason to open anything below it.

• A stale or specimen sample. A file from four months ago describes today's schema exactly as well as today's file does, and describes today's positions not at all.

• The delivery mechanics. Where the files land and how they behave. One prime broker delivers into /download and rotates a file into /download/.previous the moment you retrieve it. Another delivers into /in, not the login root. A third delivers into /outgoing and deletes on retrieval, so the acquisition job gets exactly one attempt per day. An OMS feed hides its files in a client-named subfolder that a listing of the login root will never show you.

None of that is client data. A broker's file layout is identical for every client that broker serves. The delivery path is an operational fact about the counterparty. The header row is a schema.

The economically valuable signal in a broker feed is its shape. Shape is not confidential. Values are confidential, and discovery does not need them.

Discovery Produces a Specification, Not an Answer

The second half of the design matters as much as the first. The model's output never flows into a client deliverable.

What discovery produces is a specification: a mapping from header tokens to canonical fields, a filename glob, a folder hint, a delivery path, a decode format, a set of value translations. It then gets compiled into ordinary, documented Python that runs inside the platform.

After that, the model is out of the loop permanently. The daily production run contains no model call at all.

That is the property that makes the whole thing defensible in a regulated setting. When an auditor asks why a particular trade matched, or why a break was raised, the answer is a persisted execution record. Not a prompt, not a temperature setting, not "the model determined." Same input, same output, every day, reproducible on re-execution.

It also means the production path inherits the platform's controls rather than needing new ones. Credentials never appear in code. SFTP passwords, SSH private keys, and PGP private keys resolve at runtime from the platform Secret store by path; the code references a secret id and a key name, and the build script never sees a value. Outbound connections leave from a static IP the counterparty has whitelisted. PGP decryption happens inside the worker, in memory, on infrastructure covered by SOC 2 and ISO 27001.

The model that read a column header never had access to any of that, and could not have.

The Library

Once a broker's format is understood and verified, it does not need to be learned again. The filename pattern, the header tokens, and the parsing code go into a library: a lookup table that every new onboarding checks first, before discovery ever runs.

The next client who clears through that same prime broker is recognised from the filename alone. No discovery pass, no model call, no engineering week. The format is already known, and it stays known.

In a recent onboarding, a 338-file drop, most of it broker report extracts that the routing correctly set aside, matched every one of fourteen known families from filenames alone, with full coverage of the account master. The parsers reproduced the reference dataset exactly: every trade, position, cash line item, and fee, before anyone had to look at a value.

That is what the library actually buys a client. Not just speed, attention. The question stops being how to get a counterparty's data in cleanly, every day, and becomes what the data is actually saying. The library is what lets an operations team stop managing feeds and start managing risk.

Reconciliation, Trade Marking, and Compliance

A clean, current feed from every counterparty is not the end goal. It is what reconciliation, trade marking, and compliance reporting need in order to run at all, and each one breaks quietly the moment a feed arrives late, mistyped, or misread.

Reconciliation compares two independent views of the same book, usually the OMS against the prime broker or custodian, and flags what does not match. It only works if both feeds are parsed correctly and consistently, which is exactly what the library exists to guarantee. With standardised formats already in place, reconciliation runs as a daily check that puts breaks in front of an analyst, instead of a parsing problem in front of an engineer.

Trade marking, deciding how a fill books, at what price, against which account, depends on fields that live in whatever format the broker happens to use that day. When the library already knows where those fields sit, marking runs against structured data instead of a manual hunt for which column holds the settlement price this time.

Compliance reporting has to show a regulator or an investor that a portfolio stayed within its mandate on every day it was live, not just the days someone remembered to check. That requires a complete, dated feed history. Everysk's compliance tools run against the same data the library keeps current, so the audit trail is a byproduct of the daily run, not a separate project.

None of these three is a new problem. What changes is where the effort goes. Instead of every counterparty relationship costing an integration project before any of that work can start, the integration is already done, and the effort goes straight into the reconciliation, the marking, and the compliance check.

Your Feeds Have Dialects Too. We Already Speak Most of Them.

To learn more about how Everysk handles counterparty data feeds, reconciliation, and compliance, or to request a demo, get in touch with our team.

About Everysk

Everysk Technologies is a New York-based fintech company providing intelligent automation and risk analytics solutions for capital markets. Our no-code platform enables asset managers, hedge funds, and wealth platforms to design, automate, and scale investment workflows across risk management, analytics, reporting, and compliance. Founded by investment professionals, Everysk helps firms modernize operations while maintaining transparency, control, and institutional rigor.

Follow Everysk on LinkedIn for the latest updates.