What a model evaluation package must include

A model evaluation package is a self-contained bundle that lets a lab run a model on a specific business task and receive a score. It arrives with the execution code, the task definition, the grading logic and the data the model will see. Think of it as a plug-in you drop into a testing pipeline; the surrounding environment is the world the model interacts with.

Core components of a package

A complete package contains four tightly coupled pieces. First, the environment implementation reproduces the workflow you care about, perhaps a support-ticket system, a sales-funnel sequence, or a project-tracking board. Second, the task specification spells out the input format, the actions the model must take, and the criteria for success. Third, the grader or metric module inspects the model's output, compares it to the ground-truth, and returns a numeric score or a pass/fail verdict. Fourth, the data payload supplies de-identified operating records, support tickets with resolutions, CRM activity logs, or project histories, that the environment reads.

Each piece is version-controlled and documented so that the same run can be reproduced months later. The grader is deterministic: running the same model twice on the same data yields the same result, which is essential for a fair evaluation.

What an environment must ship with

When a seller licenses an environment, the package must contain more than raw files. The following items are required for a lab to evaluate the data safely and fairly:

• Executable code. A container image, a virtual environment, or a set of scripts that install all dependencies and start the environment without extra configuration.
• Interface definition. A clear API (for example a REST endpoint or a function signature) that tells the lab how to feed inputs and receive outputs.
• Complete documentation. A concise README that explains the data schema, the call flow, any required environment variables, and how to interpret the grader's output.
• Rights and provenance memo. Proof that the seller owns the data, that customer contracts allow de-identified use, and that the data has been cleared for export. This memo travels with every package we broker.
• De-identification guarantee. Before any sample reaches a lab, personally identifiable information and any confidential fields are removed or replaced with stable placeholders such as "employee 632". A compliance test checks that no personal data remains based on the residence of the people in the records; the company's counsel signs off on the result.

Because the value of an environment rises with the richness of its workflow history, sellers often include several years of continuous logs on the same tools. Those logs let a lab build reinforcement-learning (RL) trajectories that reflect real decision making, not just isolated snapshots.

Need licensed data for a model? Tell us what you need

Rights, de-identification and compliance

All licensed environments are covered by a data-license agreement. The seller retains ownership of the data and can continue using it internally. The lab receives a perpetual, non-exclusive right to use the de-identified data for model training and evaluation. If a buyer asks for exclusivity, the term is limited, typically 24 months, and the price adjusts accordingly.

We never move raw files until the seller approves a quote. Once the deal closes, the data is exported to a secure, logged workspace. De-identification can be performed by the buyer's lab, by the seller, or by us as a separate service; who does the cleaning and who pays for it are settled during contract negotiations. We do not claim any independent certification, and we always advise the seller to let counsel confirm which laws apply. The process is a matter of legal clearance, not technical magic.

How we help you get the right package

You already know the workflow you want to evaluate, maybe support tickets with resolutions, or a sales-order lifecycle. Tell us what data you need, and we match you with sellers who hold the appropriate operating records. We handle the questionnaire, collect the one-page summary, and bring back the best quote from a large AI lab. No data moves until you give the final go-ahead, and the broker fee is paid by the buyer only after the deal closes.

If you need a custom environment that includes a specific grader or a unique API, we work with the seller to define those requirements before any data leaves their premises. The brief overview of the buyer request process is available on our buyer request page, and you can see the kinds of data we can source for AI labs on the buy page.

Use the button below to tell us what data you need, and we will start the matching process.