By Ahti Heinla, Co-Founder and CTO of Starship Technologies
I see robots every day. I see them sliding down sidewalks at pedestrian speed, stopping to make sure it’s safe to cross the road. Sometimes I find them talking to pedestrians. It’s a glimpse into the fantasies of tech-savvy people – the wonderland of artificial intelligence. But this is not a hallucination, nor a dream, it is a reality built by our team of dedicated visionaries over the past five years; We brought the future into the present.
As recently as a couple of years ago, these robots needed a little human support and were accompanied on their journeys, just like the coordination followed by autonomous car makers, who test their cars in public with ‘safety drivers’.
Starship became the first robotics team to begin operating regularly in public about 18 months ago, without using safety drivers; We allow our robots to explore the world on their own. We now operate our network of robots every day in several cities around the world, serving people dinner, parcels, and groceries.
Shared knowledge is acquired knowledge
It’s exciting to be the first.
When I was a founding engineer at Skype, we were the first to make voice over IP access practical; We are now working on doing the same with robots in public places. For four years, our engineering teams have worked behind closed doors on what has been a major achievement and an amazing experience.
I’d like to share some details from our tech journey with you. Over the coming weeks and months, other members of the Starship engineering team will be sharing aspects of their journey as well.
During this trip, we worked with computer vision, path planning, and obstacle detection – well-researched topics in academic robotics. In fact, Starship started as a research project, but quickly moved into a hands-on and practical delivery.
This means that in addition to tuning the Levenberg-Marquardt algorithm for nonlinear optimization, we had to develop a program for:
- Calibrate most sensors automatically – after all, we don’t want to spend hours calibrating them manually; We have built hundreds of robots and are currently preparing for a larger operation.
- Predict how much power each flight will draw from the bot’s battery – so we can coordinate which bot will send, based on the state of the battery.
- Predict how many minutes a restaurant will take to prepare food – until a robot shows up at the right time!
Most of the autonomous robots in the world today are very expensive, they are built as demonstrations of technology or research vehicles and are not used in commercial operations. The sensor package alone for a standalone device can cost upwards of $10,000. This simply won’t work in the delivery space, it’s not a luxury industry where you can charge a premium.
Autonomous research vehicles often have 3 kilowatts of computing power in the trunk; Impractical for a small robot, safe for childbirth. So, part of our engineering journey has been about designing for low unit economics. Here are some of the topics we had to consider:
- Advanced image processing on a low-end computational platform.
- Work to solve hardware problems in software.
- Keep track of how often bots need maintenance and why.
- Develop advanced road planning systems, to ensure we use our network of robots efficiently.
It’s been a huge journey in visual design as well, involving hundreds of sketches, drawings, and surveys before we made our first plastic robot body.
Back in the early days when we were still in stealth, we didn’t want to reveal what our robots looked like. Regular public testing required the creative use of rubbish bags attached to the robot’s body as camouflage!
Building practical robots is a combination of science, systematic engineering, and hacking. This combination of different disciplines is the main feature of Starship. There is absolutely nothing simple about robotics. All your knowledge of the situation is probabilistic; All sensors have failure and error modes, even a seemingly simple task like Make the robot stop at obstacles It could become its own small research project.
Starship is a fast-moving startup and it’s important that it doesn’t become just a big research project. Engineers who are excited about Starship are often not pure scientists, not pure hackers, not pure engineers; They have many of these attributes and can use them as appropriate for the task at hand. We need complex technology solutions to be implemented quickly and within the resource constraints of low-cost hardware.
Dexterity and resourcefulness are valuable skills.
A week is a long time in Starship
At the beginning of the week, our team will implement a new algorithm to detect constraints from point clouds and retest it against a full test case database overnight, and it will be tested live on our test ground by the end of the week.
He’ll be on the streets next Monday, with the team already reporting on their progress during our engineering meeting on Monday. On most Mondays, some members of the engineering team report a 300%+ increase on at least one of the metrics achieved, just in the previous week.
Data as a result and facilitator of the scale
Metrics and data have become a huge part of Starship architecture.
You see, when we were just getting started, we had no data – we hadn’t driven much yet. Every day we modify our robot (yes, just the one that was at the time), take it to the sidewalks and see how it performed. We now have a lot more, walking around independently every day – too many for engineers to directly observe.
Thanks to the data, we can now see the performance of our bots, hundreds of them. We can organize weekly “dive into the data” seminars, where engineers share results and watch random deliveries to stay connected to their work in action.
When we work to make our bots run more smoothly, we analyze the data in the Acceleration Events table in our data warehouse; There are at least a billion rows in this table. Other tables include Road Crossing Events, our maps, every command each bot received from our servers, and obviously the data collected from every delivery it makes.
Four years ago, we had none of this. When we were just getting started – and we weren’t running commercial deliveries yet – I often had to convince people that automated delivery really worked. People found it hard to believe and were quick to point out a variety of reasons.
Do doubt and fear always accompany new technology?
Several years ago, I landed at JFK Airport in New York with a robot in my bag. Apparently the customs man asked, “What is this thing?” I explained that it was a dock delivery robot, to which he replied, “Dude, this is New York! You’ll be robbed in a matter of minutes!”
In fact, at the time, just about everyone thought these robots would be stolen – I’m sure they probably would (postal delivery trucks get stolen, even if rarely). So far, our robots have traveled more than 200,000 kilometers (130 thousand miles) and we have not yet seen this problem.
There are of course security features in place. The robot has a siren and 10 cameras, is constantly connected to the internet and knows its location with an accuracy of 2cm (thanks to the aforementioned Levenberg-Marquardt algorithm, and 66,000 lines of automatically generated C++ code that enables our bots to use).
People also think that pedestrians may be afraid of robots on the sidewalk or may not accept their presence. Will people call the police? Honestly, we weren’t sure about this either! However, as soon as we put one of the robots there on the sidewalk, we were very surprised.
What happened next surprised us: People just ignored him. The vast majority of the public did not pay attention at all to the robots, even those who saw them for the first time, and people were certainly not afraid. Others might pull out their phones and post on Instagram about how they saw the future.
And that’s what we wanted.
We want people to care as much about robots as they do their dishwashers. This pattern of silently accepting robots as if they were always with us has repeated itself in every city around the world in which we have worked.
He is getting better. Once people learn that these robots are doing a useful service to the neighborhood, they develop an affinity with them. Kids even write thank-you letters to the robots, we’ve got a “wall of thank-you letters” to prove it!
Automating last-mile delivery was never easy, and we knew it would be a bold project. We’ve also known all along that there will be more than one basic roadblock that needs solving – it turns out there are hundreds of roadblocks! But we have long realized that all these problems are solvable – it just required ingenuity and perseverance.
Some startups like running a sprint start a minimum viable product in 3 months. For Starship, it’s more like a marathon – great and consistent effort is required, but the end result brings great benefits to the world.
Last mile delivery is one of the industries in the world that has experienced little technological disruption since the adoption of the car. The team at Starship is looking to change that, and with more than 20,000 deliveries under our belt, we’re well on our way.
If you are interested in learning more, check out our second engineering blog on neural networks and how our robots work here – https://medium.com/starshiptechnologies/how-neural-networks-power-robots-at-starship-3262cd317ec0