Complete solution for neural network dataset generation, automatic and random camera and object movement, segmented images, and python scripts for export to COCO dataset format.
We have developed a solution for the UE4 marketplace. Check out the video on our YouTube channel.
This project contains several blueprints and python scripts designed to streamline capturing of images in UE4 with instance segmented images, exporting data from the scene, and processing it into COCO dataset format in JSON file, to be ready for artificial neural network training.
Everything is built in Blueprints, no C++ is necessary. There are several blueprints that allow you to spawn desired static meshes, add tags to them and have the camera and objects move randomly and automatically while capturing the image of the scene and the segmented image pair. Included python scripts use OpenCV 2 python library to detect polygons (and bounding boxes) in the masked segmented images which are then exported to COCO dataset format (JSON file).
Features:
- Export of images and instance segmented images of the scene
- Automatic camera and actor movement
- Automatic static mesh segmentation by adding an actor tag to them
- Export of scene data, colors of the objects in the masked segmented images linked with their actor tag in a CSV file format
- Python scripts to extract the polygons, bounding boxes, and actor tags to export them as a COCO dataset format JSON file, ready for Neural Network training
See the solution on Unreal Engine Marketplace.
