openni_cam: openni_cam
rgbd + thermo camera sensor
openni_cam is ...
A multi-modal NIFTI sensing package. It collects thermal measurements, color RGB measurements and depth measuremets (from Asus Xtion and laser scans) into a single reference frame. The detector run in the reference frame has acces to RGB, temperature and depth in every pixel (with some of the measurements possibly missing).
List of modules:
- openni_test1 standalone test/viewer for Thermo-RGB-D inputs, frame capturing for calibration
- openni_node ROS publisher for RGB-D sensor (Asus Xtion Pro), using OpenNI library.
- openni_view ROS viewer for RGB-D sensor.
- openni_thermo_node ROS synchronous capturing driver for thermo camera (OPTRIS PI-IMAGER M140) and RGB-D sensor
- laser_render Semi-dense depth reconstruction from rotating laser scans for a given viewpoint using OpenGL
- victim1_node victim detector using multimodal Temperature-RGB-D input
ROS APIs
openni_node
Publishes RGB image, depth image and camera intrinsic parameters at
Usage
ROS topics
Publishes to:
- "openni_camera/rgb": [sensor_msgs::image_encodings::RGB8] RGB image
- "openni_camera/depth": [sensor_msgs::image_encodings::MONO16] Depth image, in mm from camera center
- "openni_camera/camera_info": [sensor_msgs::CameraInfo] projection internal parameters
ROS parameters
openni_view
openni_view displays (Qt) combined view of image and disparity (1/depth)
Usage
ROS topics
Subscribes to:
- "openni_camera/rgb": [sensor_msgs::image_encodings::RGB8] RGB image
- "openni_camera/depth": [sensor_msgs::image_encodings::MONO16] Depth image, in mm from camera center
ROS parameters
openni_thermo_node
ROS synchronous capturing driver for thermo camera (OPTRIS PI-IMAGER M140) and RGB-D sensor
Usage
ROS topics
Publishes to:
- "openni_camera/rgb": [sensor_msgs::image_encodings::RGB8] RGB image
- "openni_camera/depth": [sensor_msgs::image_encodings::MONO16] Depth image, in mm from camera center
- "openni_camera/camera_info": [sensor_msgs::CameraInfo] projection internal parameters
- "thermo_camera/ftemp": [sensor_msgs::image_encodings::TYPE_32FC1] temperature image, degrees Celcium
- "thermo_camera/camera_info": [sensor_msgs::CameraInfo] thermo camera projection internal parameters
- "thermo_camera/info": [openni_cam::thermo_info] extra thermo camera state information (chip, box, flag temperature)
ROS parameters
Reads the following parameters from the parameter server
- "~loop_rate" : [int] desired capturing (also publishing) frame rate
laser_render
laser_render: Semi-dense depth reconstruction from rotating laser scans for a given viewpoint using OpenGL
width=0.1
Usage
$ launch/laser_render.launch [display frame node_name]
- Examples
$ roslaunch openni_cam laser_render.launch display:=true fram:=openni_camera node_name:=render_to_openni_frame
$ roslaunch openni_cam test_laser_image.launch
ROS topics
Subscribes to:
- "camera_topic/camera_info": [sensor_msgs::CameraInfo] camera to which semi-dense depth is to be rendered
- "scan_point_cloud": [sensor_msgs::PointCloud2] Point cloud from a laser scan (single line scan is assumed)
Publishes to:
- "camera_topic/laser_image": [sensor_msgs::image_encodings::MONO16] depth from the camera center in mm
ROS parameters
Reads the following parameters from the parameter server
- "~fixed_frame" : [string] name of the frame in which interpolation of measurements in time is performed
- "~point_cloud_queue_size" : [int] how many 2D scans to remember
- "~wait_for_transform" : [double] timeout to wait for transform of the "frame"
- "~camera_topic" : [string] name of the camera topic to read "/camera_info" (external and internal projection parameters). Being set by launch arg "frame".
- "~display" : [bool] whether to display the rendered image in a window (debug purposes)
- "~scan_point_cloud" : [string] name of the scan point cloud topic [sensor_msgs::PointCloud2]
victim1_node
Fusion of different modalities (temperature-RGB-D) into one frame (RGB frame). Victim detector using multimodal input
Usage
Keys:
- "Return / Backspace / mouse left" : capture images into a matlab file (requires #define MATLAB)
- "Tab" : cicle through viewing modes: outline, detector responce 1, detector responce 2, overlay depth, overlay temperature
- "Esc" : exit
ROS topics
Subscribes to:
- "openni_camera/rgb": [sensor_msgs::image_encodings::RGB8] RGB image
- "openni_camera/depth": [sensor_msgs::image_encodings::MONO16] Depth image, in mm from camera center
- "openni_camera/camera_info": [sensor_msgs::CameraInfo] projection internal parameters
- "thermo_camera/ftemp": [sensor_msgs::image_encodings::TYPE_32FC1] temperature image, degrees Celcium
- "thermo_camera/info": [openni_cam::thermo_info] extra thermo camera state information (chip, box, flag temperature)
- "camera_topic/laser_image": [sensor_msgs::image_encodings::MONO16] depth from the camera center in mm
Publishes to:
- "openni_camera/detections": [sensor_msgs::image_encodings::MONO16] depth from the camera center in mm
- "openni_camera/points": [vision_msgs::PoseEstimate] detection clusters
ROS parameters
Reads the following configuration files
- "config/camera_serial_number_geom_cali_P.txt" : [int] Transform from Thermo frame to RGB frame (estimated by matlab/califrames)
- "config/camera_serial_number_geom_cali_Pinv.txt" : [int] Inverse transform from RGB frame to Thermo frame
- "config/temp_model.txt" : [int] Human temperature likelihood model
openni_test1
Displays (Qt) combined view of RGB temperature and disparity (inverse depth)
Usage
Keys:
- "Return / mouse left" : capture images into a matlab file for calibration (requires #define MATLAB)
- "Esc" : exit
Calibration
Load frames.mat with captured image pairs. Use matlab/califrames.m utility to clic corresponding points and estimate reprojection. Only click the points for which depth estimate is available (overlaid in color). 3-4 point pairs in 5-10 frames are needed for a good estimate.
width=0.1
