Creating Virtual Scenes & Scenarios with MathWorks (RoadRunner & Unreal Engine)
1
Table of Contents
• Introduction
• Context
• Component Overview
• Design and Implementation
• Results
• Common Pitfalls & Troubleshooting
• Summary & Next Steps
• References
2
Introduction
This article explains how virtual scenes and driving scenarios can be created and used within a Model-Based Design workflow using MathWorks tools. It focuses on how MATLAB® and Simulink® integrate with RoadRunner and Unreal Engine to enable realistic, repeatable, and scalable simulation environments for developing and validating advanced automotive systems. The article is aligned with the NXP Model-Based Design Toolbox (MBDT) workflow and targets users working on control, perception, and system-level validation.
In our demo setup, the same workflow presented in this article was applied to build a Driver-in-the-Loop simulation scenario. By leveraging MATLAB®, Simulink®, RoadRunner, and Unreal Engine, we created a realistic virtual environment that allowed direct interaction with the system running on NXP hardware. This approach highlights the practical value of these simulations, not only for early validation and testing, but also for closing the loop between model-based design and real-time execution on target hardware, enabling faster iteration, safer validation, and improved system reliability.
3
Context
As automotive systems become more complex, early validation is increasingly important. Engineers must assess advanced functionality under tight development timelines, often before hardware is available. Model-Based Design supports this need by enabling system logic and behavior to be verified early using executable models. Virtual scenes extend this approach by embedding those models in realistic, controlled environments that reflect real-world operating conditions.
Within an NXP-based development workflow, virtual scenes enable teams to explore a wide range of driving situations quickly, safely, and repeatably. Complete applications can be evaluated at Model-in-the-Loop (MIL), Software-in-the-Loop (SIL), and Processor-in-the-Loop (PIL) stages, helping uncover issues early and reducing risk before hardware integration. This structured use of virtual validation supports smoother transitions from simulation to deployment on automotive microcontrollers.
4
Component Overview
Creating and using virtual scenes with MathWorks relies on several tightly integrated components:
MATLAB and Simulink – used for algorithm development, control logic, and system modeling.
RoadRunner – a dedicated environment for building detailed road networks, traffic infrastructure, and driving scenarios.
Unreal Engine – responsible for high-fidelity 3D visualization and sensor realism.
Simulation interfaces – enabling data exchange between Simulink, RoadRunner, and Unreal Engine during runtime.
5
Design and Implementation
This section describes the design principles and implementation flow used to create virtual scenes and scenarios. The process emphasizes modularity, repeatability, and tight integration with control and system models.
5.1 System Requirements
The following prerequisites must be satisfied to build and execute virtual scenes with MATLAB and Simulink and follow our path:
MATLAB and Simulink with Automated Driving Toolbox and Simulink 3D Animation Toolbox installed.
RoadRunner.
Adequate GPU resources for real-time rendering and sensor simulation.
These requirements ensure smooth interaction between simulation models and the visualization environment.
5.2 Architecture & Model Description
At a higher level, the architecture consists of a Simulink model acting as the system under test, connected to a virtual world generated by RoadRunner and Unreal Engine. The Simulink model publishes vehicle states and receives environmental feedback, such as lane boundaries, traffic participants, or sensor detections.
Clear interface definition between the model and the virtual environment is essential. Signals representing vehicle position, velocity, and actuator commands are exchanged at each simulation step, enabling closed-loop execution.
vladmitroi_0-1782905875750.png
5.3 MATLAB/Simulink Implementation
Connecting to RoadRunner and Loading a Scenario
MATLAB connects directly to RoadRunner to open projects and load driving scenarios:
% Launch RoadRunner and open a project
rrApp = roadrunner('C:\RoadRunnerProjects\VirtualScenes');
openProject(rrApp, 'HelloWorld_Project');
% Open a RoadRunner scenario and start simulation
scenarioName = 'Intersection_CrossTraffic';
openScenario(rrApp, scenarioName);
rrSim = createSimulation(rrApp);
start(rrSim);
Integrating RoadRunner with Simulink
Once configured, Simulink and RoadRunner run synchronously. RoadRunner updates the virtual environment, while Simulink computes vehicle behavior and control actions.
sim('ConfiguredVirtualVehicleModel');
close(rrApp);
5.4 Integration (RoadRunner ↔ Unreal Engine)
RoadRunner is used to design road geometry, traffic signs, intersections, and actor paths. These assets are exported to Unreal Engine, which provides photorealistic rendering and sensor simulation.
% Open the Simulink model
open_system('ConfiguredVirtualVehicleModel');
% Path to the RoadRunner project containing the scene
scenarioPathFull = 'C:\RoadRunnerProjects\VirtualScenes\HelloWorld_Project';
% Configure the Simulation 3D Scene Configuration block
set_param('ConfiguredVirtualVehicleModel/Visualization/3D Engine/3D Engine/Simulation 3D Scene Configuration', ...
'RoadRunnerProjectPath', scenarioPathFull); vladmitroi_0-1782905795216.png
5.5 Creating a Custom Scene from Real Map Data
Custom scenes can be created by importing real-world map data into RoadRunner. Geographic information such as road layouts and elevation profiles can be converted into editable road networks.
This is an example of how to create a custom scene for recreating the Silverstone Racing Circuit in RoadRunner, using OpenStreetMap and Driving Scenario Designer.
(function() { var wrapper = document.getElementById('lia-vid-6399904467112w960h540r872'); var videoEl = wrapper ? wrapper.querySelector('video-js') : null; if (videoEl) { if (window.videojs) { window.videojs(videoEl).ready(function() { this.on('loadedmetadata', function() { this.el().querySelectorAll('.vjs-load-progress div[data-start]').forEach(function(bar) { bar.setAttribute('role', 'presentation'); bar.setAttribute('aria-hidden', 'true'); }); }); }); } }})(); (view in My Videos)
And the result using Simulink 3D with Unreal Engine.
vladmitroi_1-1782906964231.png vladmitroi_2-1782906974722.png
5.6 Testing & Validation
Once scenarios are defined, automated simulation runs can be executed to validate system behavior across multiple variants. Key metrics such as trajectory tracking, sensor coverage, and control stability can be evaluated offline.
This systematic testing approach increases confidence before integrating software with NXP hardware targets.
(function() { var wrapper = document.getElementById('lia-vid-6399905825112w960h540r143'); var videoEl = wrapper ? wrapper.querySelector('video-js') : null; if (videoEl) { if (window.videojs) { window.videojs(videoEl).ready(function() { this.on('loadedmetadata', function() { this.el().querySelectorAll('.vjs-load-progress div[data-start]').forEach(function(bar) { bar.setAttribute('role', 'presentation'); bar.setAttribute('aria-hidden', 'true'); }); }); }); } }})(); (view in My Videos)
6
Results
Using virtual scenes significantly reduces development time. Engineers can identify functional issues early, explore edge cases, and refine algorithms without hardware constraints.
In practice, this results in higher software quality at the time of hardware deployment and a smoother transition to real-world testing.
7
Common Pitfalls & Troubleshooting
Simulation time overhead can become noticeable when RoadRunner scenes are used directly in the Simulation 3D Scene Configuration block, as Unreal Engine re-imports RoadRunner assets at the start of each simulation. While this is well suited for iterative development and scene refinement, it can slow down repeated runs.
Note: For final validation or deployment-oriented testing, improved performance can be achieved by using a precompiled Unreal Engine project, which avoids repeated asset import and significantly reduces startup time.
In addition, repeatedly launching RoadRunner for each simulation introduces unnecessary overhead. A recommended practice is to keep RoadRunner running across multiple simulations and reuse the existing connection. This can be achieved using the RoadRunner roadrunner.connect API, allowing MATLAB and Simulink to reconnect to an active RoadRunner instance instead of restarting it for every run, thereby improving iteration speed and overall workflow efficiency.
8
Summary & Next Steps
Virtual scenes turn simulation into experience. Combined with an NXP Model-Based Design workflow, MathWorks tools enable engineers to innovate faster, validating complex behavior early while reducing risk, cost, and development effort.
Next, these environments can be expanded with high-fidelity sensor models, automated regression testing, and hardware-in-the-loop execution, closing the gap between virtual validation and real-world deployment.
9
References
Import OpenStreetMap Data into Driving Scenario — MathWorks Help
Driving Scenario Designer App — MathWorks Help
RoadRunner — MathWorks Product Page
Simulink 3D Animation Toolbox — MathWorks Help
OpenStreetMap
Automated Driving Toolbox — MathWorks Product Page
Visualize 3D Scenes with Unreal Engine — MathWorks Help
roadrunner.connect API — MathWorks Help
NXP Model-Based Design Toolbox — Community
View full article
