News

What is EPIC-KITCHENS-100?

The extended largest dataset in first-person (egocentric) vision; multi-faceted, audio-visual, non-scripted recordings in native environments - i.e. the wearers' homes, capturing all daily activities in the kitchen over multiple days. Annotations are collected using a novel 'Pause-and-Talk' narration interface.

Characteristics

  • 45 kitchens - 4 cities
  • Head-mounted camera
  • 100 hours of recording - Full HD
  • 20M frames
  • Multi-language narrations
  • 90K action segments
  • 20K unique narrations
  • 97 verb classes, 300 noun classes
  • 6 challenges

Previous versions...

EPIC-KITCHENS-100 Stats and Figures

Some graphical representations of our dataset and annotations

Annotation Pipeline

Automatic Annotations

Download

Dataset publicly available for research purposes

Data and Download Script

Extended Sequences (+RGB Frames, Flow Frames, Gyroscope + accelerometer data): Available at Data.Bris servers (740GB zipped) or via Academic Torrents

Original Sequences (+RGB and Flow Frames): Available at Data.Bris servers (1.1TB zipped) or via Academic Torrents

Automatic annotations (masks, hands and objects): Available for download at Data.Bris server (10 GB). We also have two Repos that will allow you to visualise and utilise these automatic annotations for hand-objects as well as masks.

We also offer a python script to download various parts of the dataset

Annotations and Pipeline

All annotations (Train/Val/Test) for all challenges are available at EPIC-KITCHENS-100-annotations repo

Code to visualise and utilise automatic annotations is available for both object masks and hand-object detections.

The EPIC Narrator, used to collect narrations for EPIC-KITCHENS-100 is open-sourced at EPIC-Narrator repo

Publication(s)

Cite the extension's ArXiv paper (available now on Arxiv):

@ARTICLE{Damen2020RESCALING,
   title={Rescaling Egocentric Vision},
   author={Damen, Dima and Doughty, Hazel and Farinella, Giovanni Maria  and and Furnari, Antonino 
           and Ma, Jian and Kazakos, Evangelos and Moltisanti, Davide and Munro, Jonathan 
           and Perrett, Toby and Price, Will and Wray, Michael},
           journal   = {CoRR},
           volume    = {abs/2006.13256},
           year      = {2020},
           ee        = {http://arxiv.org/abs/2006.13256},
} 

Additionally, cite the original paper (available now on Arxiv and the CVF):

@INPROCEEDINGS{Damen2018EPICKITCHENS,
   title={Scaling Egocentric Vision: The EPIC-KITCHENS Dataset},
   author={Damen, Dima and Doughty, Hazel and Farinella, Giovanni Maria  and Fidler, Sanja and 
           Furnari, Antonino and Kazakos, Evangelos and Moltisanti, Davide and Munro, Jonathan 
           and Perrett, Toby and Price, Will and Wray, Michael},
   booktitle={European Conference on Computer Vision (ECCV)},
   year={2018}
} 

A journal version of the ECCV 2018 paper is (available now on Arxiv and IEEE Early Access):

@ARTICLE{Damen2020Collection,
   title={The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines},
   author={Damen, Dima and Doughty, Hazel and Farinella, Giovanni Maria  and Fidler, Sanja and 
           Furnari, Antonino and Kazakos, Evangelos and Moltisanti, Davide and Munro, Jonathan 
           and Perrett, Toby and Price, Will and Wray, Michael},
   journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
   year={2020}
} 

Copyright Creative Commons License

All datasets and benchmarks on this page are copyright by us and published under the Creative Commons Attribution-NonCommercial 4.0 International License. This means that you must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. You may not use the material for commercial purposes.

EPIC-KITCHENS-100 2021 Challenges

Challenge and Leaderboard Details COMING SOON

For Challenge Results and winners on EPIC-KITCHENS-55, go to: Challenge 2020 Details. Note that these are NEW leaderboards, and results are not directly comparable to last year's results.

EPIC-Kitchens 2021 Challenges - Dates

Sep 1, 2020
EPIC-Kitchens Challenges 2021 Launched!
May 28, 2021
Server Submission Deadline at 23:59:59 GMT
Jun 4, 2021
Deadline for Submission of Technical Reports
TBC
Results announcement dates will be confirmed later

Challenges Guidelines

The five challenges below and their test sets and evaluation servers are available via CodaLab. The leaderboards will decide the winners for each individual challenge.

For each challenge, the CodaLab server page details submission format and evaluation metrics. To enter any of the five competitions, you need to register an account for that challenge using a valid institute (university/company) email address. A single registration per research team is allowed. We perform a manual check for each submission, and expect to accept registrations within 1 working day.

For all challenges the maximum submissions per day is limited to 1, and the overall maximum number of submissions per team is limited to 50 overall, submitted once a day. This includes any failed submissions due to formats - please do not contact us to ask for increasing this limit.

To submit your results, follow the JSON submission format, upload your results and give time for the evaluation to complete (in the order of several minutes). Note our new rules on declaring the supervision level, given our proposed scale, for each submission. After the evaluation is complete, the results automatically appear on the public leaderboards but you are allowed to withdraw these at any point in time.

To participate in the challenge, you need to have your results on the public leaderboard, along with an informative team name (that represents your institute or the collection of institutes participating in the work), as well as brief information on your method. You are also required to submit a report (details TBC).

Make the most of the starter packs available with the challenges, and should you have any questions, please use our info email uob-epic-kitchens@bristol.ac.uk

Frequently Asked Questions

  • Q. Who is allowed to participate?
    A. Any researcher, whether in academia or industry, is invited to participate in the EPIC-KITCHENS-100 Challenges. We only request a valid official email address, associated with an institution, for registraton. This ensures we limit the number of submissions per team. Do not use your personal email for registration. You registration request will be declined without further explanation.
  • Q. Can I participate in more than one challenge? Do I need to register separately for each challenge?
    A. Yes, and yes. You can participate in all challenges but you need to register separately for each. Winners for each challenge will be announced and there will be no 'overall' winner across challenges.
  • Q. Can I get additional submission limits to debug my file format?
    A. No. Please check your format in advance. We do not offer additional allowance for submission failures.
  • Q. Can I participate in the challenge but not submit a report describing my method?
    A. No. Entries to the challenge will only be considered if a technical report is submitted on time. This should not affect later publications of your method if you restrict your report to 4 pages including references.
  • Q. Are there any prizes given?
    A. There are no monetary prizes. Certificates will be awarded.
  • Q. What is the submission limit?
    A. Once a day, with a maximum of 50 submissions.
  • Q. Can you give me access to pre-trained models of baselines?
    A. Yes, check the details of this per challenge.

The Team

We are a group of researchers working in computer vision from the University of Bristol and University of Catania. The original dataset was collected in collaboration with Sanja Fidler, University of Toronto

Dima Damen

Principal Investigator
University of Bristol, United Kingom

Giovanni Maria Farinella

Co-I
University of Catania, Italy

Davide Moltisanti

(Apr 2017 - )
(prev.) University of Bristol
(curr.) Nanyang Tech University

Michael Wray

(Apr 2017 - )
University of Bristol

Hazel Doughty

(Apr 2017 - )
University of Bristol

Toby Perrett

(Apr 2017 - )
University of Bristol

Antonino Furnari

(Jul 2017 - )
University of Catania

Jonathan Munro

(Sep 2017 - )
University of Bristol

Evangelos Kazakos

(Sep 2017 - )
University of Bristol

Will Price

(Oct 2017 - )
University of Bristol

Jian Ma

(Sep 2019 - )
University of Bristol

Research Funding

The work on extending EPIC-KITCHENS was supported by the following research grants