April 7, 2020

Enabling A Data Infrastructure for Autonomous Guided Vehicle (AGV)

Written by Tyler Moore

As the Autonomous Guided Vehicle market continues to grow so will its storage and data transfer needs.

Advanced infrastructure and the processing of large data sets will play a critical role in the growth and advancement of the AGV market

The next three years will see an explosive growth in connected cars; and this year, 2020, it is expected that 98% of all new cars being sold will be connected. This growth is driven by ubiquitous mobile networks plus government-required connectivity. 


As the car becomes a big part of our digital edge, the data from cars will be owned by different constituents. Car manufacturers will likely own and process data related to critical car systems while any personally identifiable or safety-critical data, roughly 70%–90% of today’s car data, will be processed in real time on the vehicles themselves, to minimize latency and comply with data privacy requirements. Deep learning and non-critical data processing, such as route optimization, can occur where data processing power is delivered via cloud, instead of in a centralized or remote location. This arrangement enables manufacturers to reduce risk and solve compliance challenges.


A manufacturers choice of business model, coupled with regulatory and geopolitical constraints, are key drivers for its data strategy and its infrastructure requirements. Key items for consideration are what data is collected, its criticality, how to handle large data volumes, how frequently data changes, and how that data is protected, analyzed, and stored. If a manufacturer chooses to monetize data, the infrastructure must be able to efficiently and cost-effectively filter data, move it where it is needed, as needed. When data resides in the cloud, moving data out of one cloud and sending it to another can add expenses as most cloud providers bill for every byte downloaded from the cloud. In order for systems to be viable, cloud storage business models need to change and offer flat fees rather than variable costs. Some multi-cloud providers already do so.

Data aggregation and networking have their own infrastructure requirements and so must compute and storage. It is not realistic or cost-effective for manufacturers to build private data centers all over the globe, given the uncertainties around business models and revenue opportunities. A distributed digital infrastructure, incorporating hybrid and multicloud, is a critical enabler.

Data networks today and the Internet are designed to distribute data from the core out to consumers who access data via smart devices. The rise and rapid growth of the Internet of Things (IoT) requires collection and processing of massive quantities of data from connected mobile devices at the digital edge. This creates challenges for traditional IT architectures, which are not distributed to the edge and struggle to manage large data streams. Autonomous vehicles will require a dramatic evolution of the stack to support new functionality with massive CPU and data requirements. For example, manufacturers will have to design ways to off-load critical data and analytics through API gateways to third parties, eliminate vendor lock-in, ensure data integrity and enable real-time insights while ensuring data sovereignty compliance.

Given the sheer volume, variety, velocity and veracity of data in the autonomous test environment, this approach will likely reach its limits. Data management will become more complex at the edge, and it will need to be optimized to create separate data sets for monetization. Cost, scale and performance constraints will soon surface as test fleet data consume conventional storage infrastructure. Therefore, companies testing autonomous vehicles will need to look for alternative solutions such as edge infrastructure to improve efficiencies and contain costs.


In order to allow the cloud to be transformative in the AGV market, five things need to take place: 

  • Fixed cost data storage – eliminate variable ingress, egress, and API fees
  • High network speed – move large data sets rapidly across great geographic distances
  • Geo-dispersion – replicate and locate data in multiple locations where it is needed
  • Low single tier pricing – hot storage at archival rates
  • Ability to work with mobile IoT (fast packaging and transit of large volumes of small data packets)

Even as data at rest becomes an infrastructure issue, data in motion presents additional, more complex infrastructure issues. Autonomous test fleets ideally should be colocated with infrastructure where they can be locally trained and aligned with conditions and regulations in those geographies, however, this is not practical.

Instead, huge data volumes must be moved to geographically dispersed workstations where engineers can develop, test, create and train new models. This data interchange consumes network, storage and compute resources at an unprecedented rate. It requires companies to rethink their infrastructure investments as a variable cost cloud model can add significant and unpredictable cost to vehicle testing and validation. 

No matter how decentralized the data and infrastructure, one must centralize governance, security, and privacy across the data life cycle in order to allow adequate data control plus accurate logging, monitoring and reporting. Governance policies and management of privileges, traceability and audit trails must also be centralized. This is much harder to do when data is spread across multiple clouds, on-premises, and other siloed storage. There are also cases where features such as automatic braking or lane changing are developed by different teams located across multiple geographies. These applications need to access the same data; therefore, data quality, integrity, synchronization and availability are critical. 

Perhaps it is time to think of a one-stop multi-cloud shop with infrastructure optimized for the IoT.  Companies such as RStor have anticipated such changes and have developed both the advanced infrastructure and flat rate low cost business models to accommodate the needs of the AGV market and similar data intensive edge businesses.

You may also like…