Because of server limitation, we need to save HDD space strictly.

For this reason, we archived data while skipping no motion frames.

This yields a hidden intrinsic frame number: “running frame num,” that is frame number while capturing images by camera.
Because we drops no motion frames, this number does not corresponds to the frame number of archived video.

How to refer the running frame number.

In timestamp.csv files for some kinds of camera data,  running frame num. is printed with timestamps.

If running frame num at nth column is m, it means the real frame number of nth frame in the video is m. Please be careful about the treatment of frame number in Optical / Kinect Cameras.

The detail of motion detection algorithm (for skipping no motion frames) is shown in following paper.

A. Hashimoto et al, “KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity,”  CEA2014

Kyoto University Dataset(s) Release Agreement

Introduction

Introduction
The goal of the Kyoto University Smart Kitchen (KUSK) dataset(s) is to develop new techniques, technology, and algorithms for cooking image processing such as Human Activity Recognition, Ingredient Recognition, and Food Tracking. Kyoto University has copyright on the data and is the principal distributor of the KUSK dataset(s). The dataset is meant to aid research efforts in the general area of developing, testing and evaluating algorithms for cooking image processing.
Release of the dataset
To advance the state-of-the-art in various cooking image processing technology, this dataset is distributed to the researcher community. All further uses of the dataset will be considered on the case- by-case bases. To obtain the right to access the full dataset, the requestor must sign this document and agree to observe the restrictions listed below.
Consent: The researcher(s) agrees to the following restrictions on the KUSK dataset(s):

1. Redistribution
   Without prior written approval from Kyoto University, the KUSK dataset(s), in whole or in part, will not be further distributed, published, copied or disseminated in any way or form whatsoever, whether for profit or not. This includes further distributing, copying or disseminating to a different facility or organizational unit in the requesting university, organization, or company.
2. Modification and Commercial Use
   Without prior approval from Kyoto University, the KUSK dataset(s), in whole or in part, may not be changed or used for commercial purposes.
3. Requests for the KUSK dataset
   All requests for the KUSK dataset(s) will be forwarded to Kyoto University Principal Investigator(s).
4. Publication Requirements
   In no case should the still frames are used in any way that could cause the original subject embarrassment or mental anguish. Data of human subjects is provided in coded form (without personal identifying information). Subject consent permits publication (paper or web-based) of the data (including image data) for scientific purposes only.
5. Citation/Reference
   All documents and papers that report on research that uses the KUSK dataset(s) will acknowledge the use of the dataset by including an appropriate citation to the following.
   • KUSK Food: A. Hashimoto et al, “Recognizing ingredients at cutting process by integrating multimodal features,” ACM Multimedia 2012 workshop on Multimedia for Cooking and Eating Activities
   • KUSK 2013IJ: A. Hashimoto et al, “How Does User’s Access to Object Make HCI Smooth in Recipe Guidance?” 6th International Conf. on Cross-Cultural Design, Held as Part of HCI International 2014
   • KUSK 2014RC: A. Hashimoto et al, “KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity,” Workshop on Smart Technology for Cooking and Eating Activities,” 2014
6. Publications to Kyoto University
   A copy of all reports and papers that are for public or general release that use the KUSK dataset(s) are appreciated to forward upon release or publication to Kyoto University Principal Investigator(s).
7. No Warranty
   THE PROVIDER OF THE DATA MAKES NO REPRESENTATIONS AND EXTENDS NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED.
   THERE ARE NO EXPRESSED OR IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, OR THAT THE USE OF THE MATERIAL WILL NOT INFRINGE ANY PATENT, COPYRIGHT, TRADEMARK, OR OTHER PROPRIETARY RIGHTS.

KUSK dataset(s) Principal Investigator(s) Large-Scale Text Archive Laboratory Academic Center for Computing and Media Studies, Kyoto University. Yoshida Nihonmatsu-cho, Sakyo-ku, Kyoto, 606-8501, JAPAN

To obtain the right to access all locked data (with ), you need to read the Terms and Conditions, and submit the application form below.
After five business days for a confirmation and user creating process, we would provide you an account for data access.

• Due to the compatibility problem between contact form 7 and its (unofficial) signature add-on, you might temporally need to click the submit button twice (in that case, you will see empty yellow box after the first click.) We are sorry for the inconvinience.

About CHIFFON

Check Here

Log Format

Example

[2014-05-13 20:27:31 UTC] [INFO] [Chiffon::Web::Navigator 97] CHECK(guest-2014.05.13_19.56.10.308258): substep03

[timepstamp][log level][Internal Info] Event Name (Session ID) Event Parameters etc.

Events

Events self-annotated by Subjects

NAVI_MENU,CHANNEL,CHECK,PLAY_CONTROL,END

Events from wizard or any external recognizer

EXTERNAL_INPUT
* event parameter is a command thrown by the wizard or the external recognizer.
* basic commands are shown here.

Other Events

START

Note

* all the events logged in KUSK2014RC are annotated by subjects.
* KUSK2013IJ include both annotation by subjects and wizard.

 Sensor

HBM PW6D

Sensor Location

load_table.csv : now preparing

load_stove.csv :  now preparing

* see also this paper to estimate center of gravity.

(A. Schmidt et al, “Context Acquisition Based on Load Sensing,” Ubicomp 2002 )

Data Logger

File Format

.csv (comma separated values)

(compressed by 7zip)

Data Header (table, stove)

1st column: unit ID for each data. (400 => Kg.)

2nd column: offset values.

3rd column: Start Time

Data Alignment of a Column (table, stove)

1. Elapsed Time (in seconds)
2. Left Top (Kg.)
3. Right Top (Kg.)
4. Left Bottom (Kg.)
5. Right Bottom (Kg.)

Sensor

Keyence
Supply: FD-MZ5AT (Japanese Page)
Discharge: FD-MZ10AY(Japanese Page)
(Similar Sensor’s English Page for 10AY, but no 5AT)

Data Logger

File Format

.csv (comma separated values)

Data Header

1st column: unit ID for each data. 2801 => little / min.
2nd column: offset values.
3rd column: Start Time

Data Alignment of a Column

1. Elapsed Time (in seconds)
2. Supply (little / min.)
3. Discharge (little / min.)

Sensor

File Format

.csv (comma separated values)

Data Alignment of a Column

1. time stamp
2. Microwave
3. IH Cooking Stove
4. Other Electronic Appliance (1)

* Unit: kW

Sensor

File Format

Raw Data

32bit float

.7z (7zip)
* The png files are saved as 4channel 8bit images. Hence, you need to load the png files by following code, for example.

cv::Mat read 	= cv::imread("temp.png", CV_LOAD_IMAGE_UNCHANGED);
cv::Mat readImage	= cv::Mat(read.rows, read.cols, CV_32FC1, read.data);

Visualized Data

Extension .mp4

Codec:  H265 (not H264!)

Audio: none

This data image visualized temperature by assigning min. and max. values to blue and red dynamically during the observation. If you need a time sequence of absolute temperature values, use Raw Data provided by 7z.

Decoding Command Sample

ffmpeg -i downloaded_movie.mp4 -c:v png ./output_dir/%7d.png

Encoding Command at compression

ffmpeg  -i  input_%07d.png -c:v hevc -pix_fmt yuv420p -strict experimental -preset veryslow -qp 0 output.mp4

*Currently, not many video player support H265 codec. You should convert H265 video to any other codec you prefer by ffmpeg.

Sensor

Kinect for Xbox 360

File Format

RGB

Extension .mp4

Codec:  H265 (not H264!)

Audio: none

Caution

In some datasets, no motion video frames are dropped for saving the server storage.
If you intend to process a time sensitive process, please check running frame number treatment. 2014RC has also a problem of jitter in frame rate. (This problem is limited to the data with 10fps videos.)

Decode/Encode commands

Decoding Command Sample

ffmpeg -i downloaded_movie.mp4 -c:v png ./output_dir/%7d.png

Encoding Command at compression

ffmpeg  -i  input_color_%07d.png -c:v hevc -pix_fmt yuv420p -strict experimental -preset veryslow -qp 0 output.mp4

*Currently, not many video player support H265 codec. You should convert H265 video to any other codec you prefer by ffmpeg.

Depth

Extension .mov

Codec:  ffv1 (lossless)

Audio: none

Decoding Command Sample

ffmpeg -i downloaded_movie.mp4 -c:v png ./output_dir/%7d.png

* Caution: because the depth data are 16bit grayscale, jpg, bmp or any other video codes will loose the higher bits.

Encoding Command at compression

ffmpeg -i  input_depth_%07d.png\" -c:v ffv1 -pix_fmt gray16le output.mov

Time Stamp & Bone Track

See also Microsoft Document for Skelton Frame.

1. Frame Running Number
2. time stamp
3. skeleton.dwFrameNumber
4. skeleton.dwFlags
5. skeleton.vFloorClipPlane (x:y:z) (separator ‘:’)
6. skeleton.vNormalToGravity(x:y:z)
7. SKELTON_DATA 1 (if detected)
8. SKELTON_DATA 2 (if detected)
9. …

SKELTON_DATA (sepalator ‘|’)

See also Microsoft Document for Skelton Data.

1. data.eTrackingState
2. data.dwTrackingID
3. data.dwEnrollmentIndex
4. data.dwUserIndex
5. data.Position (x:y:z)
6. data.SkeletonPositions[0] (x:y:z)

…
26.  data.SkeletonPositions[20] (x:y:z)

Sensor

Point Grey Research FL3-U3-32S2C-CS

File Format

Extension .mp4

Codec:  H265 (not H264!)

Audio: none

Caution

In some datasets, no motion video frames are dropped for saving the server storage.
If you intend to process a time sensitive process, please check running frame number treatment. 2014RC has also a problem of jitter in frame rate. (This problem is limited to the data with 10fps videos.)

Decode/Encode commands

Decoding Command Sample

ffmpeg -i downloaded_movie.mp4 -c:v png ./output_dir/%7d.png

Encoding Command at compression

ffmpeg  -i  input_%07d.png -c:v hevc -pix_fmt yuv420p -strict experimental -preset veryslow -qp 0 output.mp4

*Currently, not many video player support H265 codec. You should decode the H265 video to any other codec (or png files as the above decoding command sample).

Time Stamp

1. Frame Running Number
2. time stamp

Sensor

Point Grey Research FL3-U3-32S2C-CS

File Format

Extension .mp4

Codec:  H265 (not H264!)
Audio: none

Command:
ffmpeg  -i  input_%07d.png -c:v hevc -pix_fmt yuv420p -strict experimental -preset veryslow -qp 0 output.mp4