How do you export a GitHub repository with its history for an AI data license?
A repository is stored with its whole history, which makes exporting the code straightforward. The pull requests, the review comments and the issues are not part of the code itself. They live on the hosting service, and they need a separate route. Know both before you promise what a set will hold.
What a GitHub organization usually holds
| The code and every commit | The full history of changes with their messages, branches and authors. |
| Pull requests and review comments | The proposals, the discussions and the approvals. Kept by the service, not inside the code. |
| Issues | Bug reports and requests with the conversations under them. |
| Workflow and build configuration | The automated checks that run on each change and how they evolved. |
| Wikis and project boards | Design notes and planning kept beside the code. |
How code and reviews leave GitHub, route by route
| Tool | What an export gives you | What is missing or needs an admin |
|---|---|---|
| A complete clone of the repository | All commits, branches and tags, so the code and its history are complete. | Pull requests, review comments and issues, which live on the service. |
| The service’s programming interface | Pull requests, reviews, issues and comments, pulled by a developer. | A developer has to write and run the pull, and access limits shape how fast it goes. |
| A migration or export archive | A bundle that includes issues and pull requests, where the plan allows it. | It is built for moving to another service and needs conversion. |
| Organization-wide mirroring | A repeatable copy of many repositories at once, run by an owner. | It needs owner access and a place to store a large amount of data. |
Tool names belong to their owners and are used only to say where records usually live. Export features change by plan and version, so check the current rules for your account.
The history and the discussion are two exports
A clone carries the code, the commit messages and the authors, and that already holds the sequence of changes. The review discussion is a second layer that arrives only through the interface or an archive. Teams that review carefully keep the more valuable one there.
A repository with a long review history, written by a team, with tests that run, is the version of this data that matters. A single author or a short life makes the set thin.
Secrets, ownership and open source
Old commits can hold keys and passwords that were later deleted from the current files but not from the history. Scanning the whole history is part of any export, and the keys should be changed regardless.
Employees usually assign their work to the company, but contractors are not automatic. Client work, forks of open-source projects and third-party code are not yours to license.
What a repository set leaves out
| Keys, tokens and passwords in the history | Found across every commit and removed, with the keys changed. |
| Forks, client code and third-party code | Excluded because the company does not own it. |
| Personal data in tests and examples | Customer records that crept into fixtures are removed. |
An example, invented for illustration
A software company keeps its product in private repositories on GitHub, with pull requests reviewed by at least one other engineer. An owner mirrors the repositories, and a developer pulls the pull requests and review comments through the interface. A set scans every commit for keys, replaces author names with stand-ins and leaves out a fork of an open-source project. The owner reviews a sample of the cleaned pull requests before anything is shared.
Questions owners ask about this
Will cloning the repository be enough?
For the code and its history, yes. For the review discussion and the issues, no. Those need a second route, and they are usually worth having.
We use another hosting service. Is this different?
The idea is the same: a clone gives the code and history, and the service’s interface gives the discussion. The details differ by service.
Who can run this?
An organization owner or a developer with the right access. It is not something to hand to one person without the team knowing.
Didn't find your question? Ask us directly.
Keep reading
Related kinds of records
- Code and pull request history: The commit, the review and the failed fix before the working one, with secrets and borrowed code removed.
- Jira data export: How to get issues with their comments and links out of Jira, and the secrets to remove first.
Industries that keep these records
- Software companies: Tickets, postmortems and renewal threads; customers appear everywhere and code is a separate deal.