1 year, 5 months ago

FedCSIS 2023 Challenge: Cybersecurity Threat Detection in the Behavior of IoT Devices

FedCSIS 2023 Challenge: Cybersecurity Threat Detection in the Behavior of IoT Devices is the 9th data science competition organized in association with Conference on Computer Science and Intelligence Systems (https://fedcsis.org/). This year, the task is related to cybersecurity - participants will construct scoring models to detect anomalous operating system behavior of IoT devices under attack. The challenge is sponsored by Łukasiewicz Research Network - Institute of Innovative Technologies EMAG and EFIGO sp. z o.o. companies.

Overview

The topic of this year's data science competition is the detection of cybersecurity breaches in log data from IoT devices. The data sets contain 1-minute logs of all related system calls. The task for the competition participants is to develop a model that assesses the chances that a cyber attack was ongoing during the monitored period. Such a model could play a vital role in improving the safety of IoT systems.

Special session at FedCSIS 2023: As in previous years, a special session devoted to the competition will be held at the conference. We will invite authors of selected challenge reports to extend them for publication in the conference proceedings (after reviews by Organizing Committee members) and presentation at the conference. The papers will be indexed by the IEEE Digital Library and Web of Science. The invited teams will be chosen based on their final rank, the innovativeness of their approach, and the quality of the submitted report.

Terms & Conditions
 
 
Please log in to the system!

The task in this challenge is to design an accurate method for predicting if logs from an IoT system indicate whether a cyberattack was performed on the system or not. The available training data contains 15027 log files in a .csv table format (to be found in the directory train_data/) a small fraction of which indicates that a cyberattack was performed (the list of all files indicating a cyberattack can be found in file train_files_containing_attacks.txt). All files have uuid4 random names. All timestamps have been replaced with one single timestamp relating to the datetime 2023-04-12-00:00:00.

Solution format: the test data, containing 5017 files, is to be found in the directory test_data/. The test files have the same format and naming scheme as the training files.

Solutions in this competition should be submitted to the online evaluation system as text files with predictions for test instances. The solution file should consist of exactly 5017 lines, each with a single number from the [0, 1] interval that indicates a chance for a cyberattack. The ordering of predictions should correspond to the lexicographic ordering of log files from the test set.

Evaluation: the quality of submissions will be evaluated using the ROC AOC measure. Solutions will be evaluated online, and the preliminary results will be published on the public leaderboard. The preliminary score will be computed on a small subset of the test records, fixed for all participants. The final evaluation will be performed after the completion of the competition using the remaining part of the test records. Those results will also be published online. It is important to note that only teams which submit a report describing their approach before the end of the challenge will qualify for the final evaluation.

Rank Team Name Is Report   Preliminary Score Final Score Submissions
1
MathLogic
True True 1.0000 0.999900 76
2
dymitr
True True 1.0000 0.999700 59
3
The Fellowship of the Cybersecurity
True True 0.9997 0.999500 5
4
DML
True True 0.9999 0.999300 176
5
Y-Team
True True 1.0000 0.998600 8
6
Cyan
True True 0.9940 0.996600 69
7
PisaTeam
True True 1.0000 0.995700 10
8
hieuvq
True True 0.9772 0.971800 101
9
Stan
True True 0.9190 0.929300 14
10
baseline
True True 0.9633 0.925700 1
11
kubapok
False True 0.9962 No report file found or report rejected. 25
12
NxGTR
False True 0.9591 No report file found or report rejected. 26
13
halo
False True 0.9042 No report file found or report rejected. 9
14
Yicli
False True 0.8927 No report file found or report rejected. 13
15
Venator Invictus
False True 0.6938 No report file found or report rejected. 2
16
ChatGPT
False True 0.5406 No report file found or report rejected. 1
17
Lavandula
False True 0.5250 No report file found or report rejected. 2
  • June 2, 2023 (23:59 GMT): deadline for submitting the predictions
  • June 4, 2023 (23:59 GMT): deadline for sending the reports, end of the competition
  • June 09, 2023: online publication of the final results, sending invitations for submitting short papers for the special session at FedCSIS'23
  • July 09, July 12, 2023: deadline for submitting invited papers
  • July 16, 2023: notification of paper acceptance
  • July 28, 2023: camera-ready of accepted papers, and registration for the conference are due

Authors of the top-ranked solutions (based on the final evaluation scores) will be awarded prizes funded by the Sponsors:

  •     1000 USD for the winning solution + 600 EUR for one FedCSIS 2023 registration
  •     500 USD for the 2nd place solution + 600 EUR for one FedCSIS 2023 registration
  •     250 USD for the 3rd place solution + 600 EUR for one FedCSIS 2023 registration
  • Andrzej Janusz, QED Force / University of Warsaw
  • Artur Kozłowski, Łukasiewicz Research Network – Institute of Innovative Technologies EMAG
  • Błażej Adamczyk, EFIGO sp. z o.o.
  • Daniel Iwanicki, QED Software
  • Maksym Brzęczek, EFIGO sp. z o.o.
  • Marcin Michalak, Łukasiewicz Research Network – Institute of Innovative Technologies EMAG
  • Marcin Tynda, EFIGO sp. z o.o.
  • Michał Czerwiński, QED Software
  • Piotr Biczyk, QED Software

The challenge is organized within the SPINET project (``Development of a tool for protecting users, systems and devices of the Internet of Things, based on machine learning and behavioral analysis’’) - a project financed by the Polish National Centre for Research and Development as part of the fourth CyberSecIdent–Cybersecurity and e–Identity competition (agreement number: CYBERSECIDENT/489240/IV/NCBR/2021). The project is being carried out by Łukasiewicz Research Network - Institute of Innovative Technologies EMAG, EFIGO sp. z o.o., and QED Software sp. z o.o. companies.

   

This forum is for all users to discuss matters related to the competition. Good manners apply!
  Discussion Author Replies Last post
Test Labels Yichao 3 by M
Monday, June 19, 2023, 22:05:50
a short delay in the final results Andrzej 0 by Andrzej
Sunday, June 11, 2023, 09:27:08
the evaluation system is temporarily unavailable Andrzej 6 by M
Sunday, May 21, 2023, 19:55:06
data files Utkarsh 1 by Andrzej
Sunday, April 23, 2023, 14:50:49
Explicit Submission Order. Carlos 1 by Andrzej
Thursday, April 20, 2023, 08:28:20