UK software specialist rFpro has integrated Solectrix’s SXIVE image-processing technology into its AV elevate simulation platform, creating a virtual environment in which engineers can design, optimise and test complete automotive camera pipelines before physical hardware is available.
The collaboration is aimed at addressing growing demand from OEMs for accurate digital twins of camera systems from Tier 1 suppliers. By bringing image signal processing (ISP) into simulation, the companies say engineers can move sensor optimisation, perception development and training-data generation earlier in the vehicle development cycle, potentially reducing development time and cost.
The integration incorporates Solectrix’s software-based ISP into rFpro’s camera model, allowing engineers to optimise processing stages including debayering, colour correction, tone mapping, denoising, lens distortion and lens shading correction. Camera parameters can be assessed against difficult driving scenarios, while SXIVE-processed synthetic data can be used to train perception models.
“With camera-based perception being central to most ADAS and autonomous vehicle performance, ISP optimisation is a key competitive differentiator,” said Matt Daley, technical director at rFpro. “Advantages can be gained by enhancing the images for machine rather than human vision and ultimately improving the perception model performance and overall safety of the system. Our combined simulation environment and ISP model create a far faster and safer way for engineers to experience tuning set options in dynamic driving conditions, rather than heading out onto public roads and finding these edge cases conditions.”
The approach enables large numbers of scenarios to be generated without requiring corresponding physical vehicle tests. Weather, location and driving conditions can be varied, while image-processing modules and sensor configurations can be modified within the simulation.
“Providing sufficient amounts of high-quality video data is a real challenge for the development of any modern AD or ADAS system,” said Dr. Roman Tzschoppe, R&D manager at Solectrix. “Our collaboration with rFpro enables engineers to design and execute hundreds of thousands of test scenarios and generate a wide range of highly-specific training data for camera-based perception models. It enables critical image quality ISP parameters to be varied through the dataset production, this simply isn’t feasible when relying solely on physical hardware image collection.”
The technology also has applications beyond automated driving. Digital mirrors, surround-view systems and other camera-based human-vision applications are increasingly being deployed in passenger cars, buses and heavy goods vehicles. Such systems can encounter challenging conditions including low sun, reflective road surfaces, bright headlights and underground or covered parking.
AV elevate provides an integrated simulation environment for sensor tuning, perception and control-system training, and testing of the wider automated-vehicle technology stack. SXIVE combines image-processing software with hardware accelerators, applications and plugins, supporting rapid prototyping and real-time processing.
The combined platform also allows engineers to examine the impact of image-processing decisions on perception performance. For example, colour-balancing strategies can be evaluated for their effect on pedestrian detection in different lighting and weather conditions.
“OEMs now increasingly expect Tier 1 suppliers to deliver a highly accurate digital twin of their sensor system alongside a new product, except they want this long before the hardware is available,” said Matt Daley, technical director at rFpro. “Developing a high-quality camera model isn’t just a benefit for development but it is now becoming a commercial necessity. By integrating SXIVE into AV elevate we are giving the industry a complete environment to tune, train and test the entire camera system.”
