Delivery Robot RL Environment in Unreal Engine
The first try
The first try
This delivery robot has one job: carry a pizza across the street.
First try. It hits a tree, then a parked car.
So how does it get good at this?
The video
Full script
This delivery robot has one job: carry a pizza across the street.
First try. It hits a tree, then a parked car.
So how does it get good at this?
A delivery robot learns a job like this in an RL environment: a practice world where it learns by trying.
Quick note: this is a demo world, built in Unreal Engine for this video.
A real one for a company takes more training and the company's own data.
RL is short for reinforcement learning: software learns a job by trial and error.
The software that learns is called an agent, and this robot is one.
An RL environment is a simulation where the agent practises, and when it copies a real place, people call it a digital twin.
It has 3 parts: a task, a lot of tries, and a score that grades every try.
So why are AI labs training agents like this?
They want AI that does real jobs: delivery, trucking yards, warehouses, farms, ports and call centres.
An agent only gets good by trying a job thousands of times, so it needs a practice world.
And the best practice worlds are built from real work data, the kind most businesses already have.
It is not only AI labs. A business can build its own RL environment.
Mostly to automate work and save time and money.
It can be private: self-hosted on your own servers, so your data stays in-house.
It is safe, because the agent crashes in the simulation, not on a real street.
And it is measured: you see a score before you trust it with real work.
Here is our world: one city street, with a pizza shop, parked cars and people on the sidewalk.
The customer waits at the yellow door across the road.
Now the score: every step closer to the door earns points, and the delivery adds a bonus.
Every bump costs points, and so does every wasted second.
So who gives the grade? A set of rules written once in code, called a reward function.
The computer scores every try by itself, and the business decides what good means: on time, no damage, less fuel.
Nobody tells the robot the route, because the score teaches it what good looks like.
Keep that in mind, because the next part looks messy.
Next it practises, sped up.
It tries a route, gets a score, and tries again.
After thousands of tries, the score starts to climb.
Here is the run after practice.
It leaves the shop and stays on the sidewalk.
It crosses at the crosswalk, straight across.
On the other side it rolls past the people on the sidewalk.
No bumps, and the score stays green.
That is the goal for a business: the same careful job, every time.
Pizza delivered. The customer waves, and the robot gets full points.
To be fair, this demo has a limit.
This street is made up, and these runs were set up for the video.
A real environment is built from real places and real data.
When the agent scores well, it is copied onto the real robot and tested carefully, step by step, before it does real work.
A question you might have: where does a real environment get real behavior from?
From data like yours: delivery times, routes, trip logs and incident reports.
AI labs look for exactly this kind of data.
You do not need an AI team to start.
Pick one job that repeats, gather the data on how it is done today, and decide what a good score looks like.
So what happens to your data? There are 2 common paths.
You can keep it in-house and build a private, self-hosted environment that trains your own agents.
Or you can license it to an AI lab, which builds practice worlds and training data from it.
In that case, personal details are de-identified first, and the contract sets what the lab may use it for.
You keep your own copy, and anything that cannot be licensed is left out.
I'm Jimmy. I have worked in data at Fortune 500 companies, as a data analyst, a data engineer and a software engineer, and I am trained in data science. I am also a Canadian military veteran, specialized in data analysis and reporting.
If you want to sell your data, I am your broker and technical advisor. I bring your data to several AI companies, and I guide your team through the technical side, including compliance. AI companies pay my fee.
Already shopping your data around? Contact us anyway. We can still work through the terms with you and help you negotiate a better price.
If you are buying data for an AI lab, I find the right data and see it through to delivery, so you can focus on AI research.
Your data already shows how the work gets done, and that is what makes a practice world real.
More on how licensing works is at AITrainingDataBroker.com.
Which job in your business would you want software to practise first? Tell me in the comments.
Thank you very much for watching, and I will see you next time.
More slides
AITrainingDataBroker.com
Can a company build
its own RL environment?
RL = reinforcement learning. Will it save time and money?
3 parts: the payoff · the risk · how to build oneJimmy Lin
AITrainingDataBroker.com
How much money did
companies get for
selling data to AI?
What the filings say Reddit, Wiley, Informa and HarperCollins were paid, and what that means for your data
Jimmy Lin
AITrainingDataBroker.com
Can You Sell Private
RL Environments
to AI Labs?
What an RL environment is, and what decides whether an AI lab may license one you built on your own
Jimmy Lin
