Can a company that builds AI products sell its data to AI labs?
Possibly, but these companies face more questions than most. They hold prompts, model answers, human ratings and evaluation sets, and every one of those has a source: a user, a model provider or the company's own staff. Who agreed to what decides which parts can go further.
What an AI product company has that is licensable
| Evaluation sets and test cases | Questions with expected answers and scoring rules, written by the team. Purpose-built, and often the cleanest asset in the company. |
| Human feedback and rating records | Reviewer judgments on which answer was better and why, from contractors or staff under clear terms. |
| Prompt and response logs | What users asked and what the system returned. Valuable, and the most restricted record in the building. |
| Support and feedback data | Tickets, bug reports and feature requests about the product, which show how people describe problems with AI behaviour. |
| Failure and red-team notes | Cases where the product gave a poor or unsafe answer, with the analysis and the fix. |
Where the records live
Logs are usually in an observability or analytics store, evaluation sets in a repository or spreadsheet, and ratings in an annotation tool. Support sits in a help desk like any other company.
We ask what each store holds, how long it has been kept and whether the terms shown to users at the time of collection are on file. Without those terms a log cannot be assessed.
What users agreed to
A user who typed a prompt into a product agreed to some terms. Those may allow the company to use the content to run and improve the service, to train models, or neither. They may promise that business customers' content is never used for training. Whatever the text says, it limits what the company can pass on, and a change to the terms usually applies only going forward.
Prompts also contain personal data and confidential business material that users did not mean to share broadly. Privacy law applies to that content, and a de-identification step is likely to be needed even where the terms are permissive. This is general background for orientation, and legal advice comes from your own counsel.
Model-output rules
If the product runs on a model from another provider, that provider's terms may limit what can be done with its outputs, including using them to train competing models. Those terms vary, so each provider agreement is read before any answer text is included. Evaluation sets written by the company's own staff avoid this question, which is why they come first.
What stays out for an AI product company
| Business customers' prompts and files | Where the contract promises no reuse, or is silent, those logs stay with the customer. |
| Logs with personal or sensitive content | Names, health details, credentials and private messages found in prompts are removed, or the record is dropped. |
| Model outputs restricted by a provider | Answers from a third-party model whose terms bar this use are left out. |
| Model weights and proprietary training data | The core product assets are not part of a data package unless the company decides to offer them separately. |
More on this: how personal details are removed, who carries the risk and how you get paid.
A worked case
A company with seventy staff builds an AI writing assistant. Its first package is the evaluation set its team wrote, with the scoring rules and the reviewer notes explaining each judgment. User logs are not included, because the old terms did not mention training and many users were business accounts. The company's counsel reads the sample before anything proceeds.
Questions owners ask about this
Can we update our terms now and then use past logs?
Generally, new terms apply to content collected after users accept them. Past content stays under the terms in force when it was collected, so counsel should confirm.
Are our own evaluation sets safe to offer?
Usually yes, if staff wrote them and no third-party text or restricted model output is inside. Both are checked.
What if reviewers were contractors?
Their agreements need to assign the work or allow this use. Annotation platforms can add their own terms, so those are read as well.
Do enterprise customers have to approve?
For logs that contain their content, the contract controls, and written consent is the clear route. Without it, those logs are left out.
Didn't find your question? Ask us directly.
Keep reading
Records to look at first
- Support tickets: The whole thread, wrong guesses included, with customers replaced by stand-ins.
- Code and pull request history: The commit, the review and the failed fix before the working one, with secrets and borrowed code removed.
- GitHub repository export: How to take code, pull requests and review comments out of GitHub, and the keys to remove first.
- Zendesk data export: How to get tickets with every reply and internal note out of Zendesk, and what customers’ details need first.
- Labeled tables and datasets: Rows, a result column and a data dictionary: when a table can be licensed.
- Intercom data export: Which Intercom routes return the actual chat text, which return only reporting columns, and what customers typed that must go.
- Evaluation datasets: Exam questions, graded work and checked test cases from real jobs, kept private so they can measure a model.
Other industries
- Software companies: Tickets, postmortems and renewal threads; customers appear everywhere and code is a separate deal.