Kinect Stream Saver Application


The Kinect Stream Application allows users to display and store the Kinect streams. This application is developed based on a sample called "Kinect Explorer - D2D C " developed by Microsoft corporation. (Compatible with SDK 1.7)

Noncommercial — You may not use this application for commercial purposes.  
>> © The AI & Robotics Research Team (AIRR), Toronto Rehabilitation Institute.
Prepared by Elham Dolatabadi,

The Kinect Stream Saver application is meant to aid research in the general area of mobility monitoring. The AI & Robotics Research Team (AIRR) within the Toronto Rehabilitation Institute (TRI) holds copyright of the dataset.


 

Implementation:

The Kinect Stream Saver application is based on the KinectExplorer-D2D (for C++) sample code.
Modifications to the sample code are explained below.

- Two new classes were added.

  • StreamSaver:

This class has methods that store stream data (color, depth, skeleton) to a dynamic FIFO
buffer as the data is received from the Kinect device, and separately write the data from
the FIFO buffer to output files as described above. The disk writes are done in a separate
thread to avoid degradation of frame processing speed during data collection.

  • SaverViewer:

This class implements a fourth GUI tab in the KinectExplorer application. The tab is labeled
"Recording" and it allows users to specify which data streams should be saved to disk.

- A few classes in the KinectExplorer sample code were modified. The changes were kept to minimum, but   were necessary to invoke the functionality of the new classes.

  • KinectWindow
  • NuiColorStream
  • NuiDepthStream
  • NuiSkeletonStream

- A new GUI "Recording" tab was designed and included in the application resources. This included modifications to the following files.

  • KinectExplorer.rc
  • resource.h

 

Instructions for use:

To collect data with the Kinect Stream Saver:

Run KinectExplorer-D2D.exe:

  • In the lower right corner of the window navigate to the 4th tab, labeled Recording.
  • Enter the Local Disk name and Folder Name if desired. Otherwise, data will be collected at C:\KinectData
  • Choose which data streams to collect.
  • Press Capture to begin data collection.
  • Press Stop to end data collection.

Recording.jpg

 The collected data will appear in Local Disk:\KinectData\Folder Name.

  • LiTimeStamp.csv :

This file has fields: counter#, frame#, timestamp

    • The counter starts at 1 when the Capture button is pressed and data collection begins.
    • The frame number is from the Kinect API; it starts when KinectExplorer-D2D.exe is launched.
    • The timestamp is in milliseconds, and is useful for recording frame rate. (At present data is collected at approximately 30 frames/second, so frames are approx 33 msecs apart.)

 

  • Color images are saved as either
        . as .bmp files on each frame.
        . as "binary" data stored in a .binary file.

  • Depth images* are saves either

. as .bmp files on each frame.
. as "raw" data stored in a .csv file.

  • Skeleton data:

The program can track up to six skeletons at once. For the first four, all joints are tracked.
  For the last two, only the center of mass is tracked.
 
  Saving The Joint Positions: one data files is created, named  Joint_Position.binary.

  Saving The Joint Orientations: one data files are created, named  Joint_Orintations.binary.

  The 20 skeleton joints are written in the following order.

CoM Position
HIP_CENTER
SPINE
SHOULDER_CENTER
HEAD
SHOULDER_LEFT
ELBOW_LEFT
WRIST_LEFT
HAND_LEFT
SHOULDER_RIGHT
ELBOW_RIGHT
WRIST_RIGHT
HAND_RIGHT
HIP_LEFT
KNEE_LEFT
ANKLE_LEFT
FOOT_LEFT
HIP_RIGHT
KNEE_RIGHT
ANKLE_RIGHT
FOOT_RIGHT

--- "JointPosition....csv" File ---

Each row contains following fields:
- position (x, y, z)
- frame number (in CoM row); joint tracking state (in joint rows)*

* Tracking states: 0=joint is not tracked; 1=joint is inferred; 2=joint is tracked


--- "JointOrientation....csv" File ---

Each row contains following fields:
- hierarchical joint angle as a quaternion (w, x, y, z),
- angle status*

* Angle status:
- in joint rows: 0=successful, any other value implies angle computation failed
and the angle has been set to the Identity quaternion (1,0,0,0).
- in CoM rows: Not applicable. Angle status is set to S_FALSE=1.


Caveats:

  • If fewer than 6 skeletons are tracked, the data may appear in any of the 6 files. Check the file size to determine which files store skeleton data. The unused files will contain only frame numbers and zeros for all positions in all frames.
  • If a user that is being tracked leaves the field of view and later returns, the Kinect does not know it is the same person, so they will be assigned a new skeleton id. This can mean that the person's data appears in more than one data file.

 

Acknowledgement:

Special thanks to Marge Coahran for all the help with putting together the README file ! 


 

Terms of Use:

If your research benefits from the Kinect Stream Saver application provided by AIRR, please consider the following Terms of Use:

- Please cite the following paper in any publications about your work:

  •     Elham Dolatabadi, Babak Taati, Gemma S. Parra-Dominguez, Alex Mihailidis, “A markerless motion tracking approach to understand changes in gait and balance: A Case Study”, the Rehabilitation Engineering and Assistive Technology Society of North America (RESNA 2013 Annual Conference). Student Scientific Paper Competition Winner

- please acknowledge this application in any publications about your work. Thank you for your support.

    Acknowledgement Text Examples:

    We would like to acknowledge AI & Robotics Research Team (AIRR) for the application that facilitated this research.

- Without prior approval from the Toronto Rehabilitation Institute AIRR Research Team, the application, may not be used for commercial purposes.

Last edited Jan 17 at 9:38 PM by edolataabadi, version 9