2 weeks, 3 days ago

IEEE BigData 2019 Cup: Suspicious Network Event Recognition

Suspicious Network Event Recognition is a data mining challenge organized in association with IEEE BigData 2019 conference. The task is to decide which alerts should be regarded as suspicious based on information extracted from network traffic logs. The competition is kindly sponsored by Security On-Demand (https://www.securityondemand.com/) and QED Software (http://qed.pl/).

Cyber threat detection and analytics play a pivotal role in providing security to organizations that provide web services, and to their users. Importance of this field is continuously growing due to the increasing abundance of Internet services, wireless networks, smart devices, etc.   Since the cybersecurity domain is hugely complex, it is also one of the major challenges of the contemporary world.

In this challenge, the task is to detect truly suspicious events and false alarms within the set of so-called network traffic alerts, that the Security Operations Center (SOC) Team members @ SOD have to analyze on an everyday basis. An efficient classification model could help the SOC Team to optimize their operations significantly. It is worth adding that although the competition sponsor is entirely commercial, the knowledge and experience that can be gathered by the competition participants may be highly beneficial to improve the intelligent cybersecurity modules in many organizations.

More details regarding the task and a description of the challenge data can be found in the Task description section.

Special track at IEEE BigData 2019: A special session devoted to the challenge will be held at the IEEE BigData 2019 conference. We will invite authors of selected challenge reports to extend them for publication in the conference proceedings (after reviews by Organizing Committee members) and presentation at the conference. The publications will be indexed in the same way as regular conference papers. The invited teams will be chosen based on their final rank, innovativeness of their approach and quality of the submitted report. 

References:

  • D. Ślęzak, A. Chądzyńska-Krasowska, J. Holland, P. Synak, R. Glick, M. Perkowski: Scalable Cyber-security Analytics with a New Summary-based Approximate Query Engine. IEEE BigData 2017: 1840-1849
Terms & Conditions
 
 

IEEE BigData 2019 Cup: Suspicious Network Event Recognition has finished. We are happy to announce that the competition attracted a total of 249 teams which, in total, submitted over 2400 solutions. Thanks!

The official Winners:

  1.     Team hieuvq:

    Quang Hieu Vu, ZALORA
    Dymitr Ruta, EBTIC, Khalifa University
    Ling Cen, EBTIC, Khalifa University

  1.     Team test_123:

    Cong Dong, Institute of Information Engineering, Chinese Academy of Sciences
    Yu Fan Chen, College of Management and Economics, Tianjin University
    Dong Xu Han, Institute of Information Engineering, Chinese Academy of Sciences

    the team would also like to honorably mention their mentors:

    Song LiuBo Jiang, and Jun Rong Liu from Institute of Information Engineering, Chinese Academy of Sciences

  1.     Team HSOC

    Tian Wang, Institute of Information Engineering, Chinese Academy of Sciences
     Chen Zhang, Institute of Information Engineering, Chinese Academy of Sciences
     Zhigang Lu, Institute of Information Engineering, Chinese Academy of Sciences

We would like to thank all participants for their effort and invaluable contribution. In particular, we want to express our gratitude to all participants who decided to send us descriptions of their solutions in the form of brief reports. We have sent invitations to selected teams to extend their reports and submit conference papers for a special session at IEEE Big Data Conference. 

 

Rank Team Name Score Submission Date
1
hieuvq
0.931743 2019-09-28 11:16:09
2
test_123
0.930295 2019-09-29 13:35:58
3
HSOC
0.926885 2019-09-29 19:51:33
4
FightForPALI
0.918421 2019-09-29 13:50:22
5
bingogogogogo
0.917840 2019-09-29 17:07:18
6
DeepIf
0.914965 2019-09-27 17:26:00
7
VerifiedXiaoPAI
0.910551 2019-09-26 13:29:09
8
Pit
0.910134 2019-09-29 22:29:00
9
security
0.910003 2019-09-29 19:37:03
10
NSEC SJTU
0.906779 2019-09-27 04:22:40
11
extended baseline
0.906375 2019-07-13 02:52:55
12
AGW
0.905450 2019-09-29 11:57:16
13
maper1
0.903533 2019-08-19 09:10:30
14
Pisa
0.902961 2019-09-27 13:53:11
15
UMF Canada
0.896850 2019-08-23 00:20:18
16
IF
0.889315 2019-09-29 22:21:37
17
Chain
0.888660 2019-07-11 17:30:36
18
sink
0.874392 2019-07-30 09:59:56
19
baseline solution
0.870464 2019-07-2 18:09:18
20
CREDIT
0.854441 2019-09-21 23:51:04
21
TTUiDVL
0.839764 2019-09-25 23:03:07
22
M
0.839644 2019-09-3 17:47:10
23
hello_there
0.838661 2019-07-3 18:40:53
24
ISO_Project
0.836666 2019-09-29 22:22:24
25
InTensorty
0.781533 2019-09-28 19:49:49
26
melanzana
0.668400 2019-09-5 10:52:11
27
Lzxc
0.535459 2019-09-29 20:12:40
28
LiuZ
0.530611 2019-09-29 17:11:27
29
tc
No report file found or report rejected. 2019-09-29 08:13:15
30
wt
No report file found or report rejected. 2019-09-29 14:00:31
31
6e
No report file found or report rejected. 2019-09-29 17:36:32
32
jinmeizhi
No report file found or report rejected. 2019-09-29 05:03:13
33
perfect
No report file found or report rejected. 2019-09-19 11:39:00
34
dinesh
No report file found or report rejected. 2019-09-7 01:51:34
35
323_touch_fish
No report file found or report rejected. 2019-07-13 05:51:20
36
test
No report file found or report rejected. 2019-08-27 10:02:48
37
12213
No report file found or report rejected. 2019-08-28 11:56:01
38
iddqd
No report file found or report rejected. 2019-07-13 15:59:58
39
r2
No report file found or report rejected. 2019-08-8 23:37:56
40
DennisShaw
No report file found or report rejected. 2019-07-31 04:07:49
41
michalm
No report file found or report rejected. 2019-07-23 08:52:01
42
GoGoPowerRangers
No report file found or report rejected. 2019-07-8 18:04:04
43
shire
No report file found or report rejected. 2019-07-5 13:05:06
44
xyz
No report file found or report rejected. 2019-07-18 09:11:33
45
piero
No report file found or report rejected. 2019-07-15 15:47:08
46
hopium
No report file found or report rejected. 2019-07-26 02:01:45
47
Sinister Three
No report file found or report rejected. 2019-07-2 16:42:48
48
Josh
No report file found or report rejected. 2019-09-18 10:37:33
49
LIDIA_UDC
No report file found or report rejected. 2019-09-12 17:06:59
50
maa
No report file found or report rejected. 2019-07-14 13:55:20
51
marcb
No report file found or report rejected. 2019-07-3 16:59:43
52
Dan
No report file found or report rejected. 2019-08-5 19:04:38
53
MAGIX.AI
No report file found or report rejected. 2019-08-7 09:05:44
54
Marshalls
No report file found or report rejected. 2019-07-5 11:24:05
55
nightking
No report file found or report rejected. 2019-09-28 14:05:23
56
Rookie
No report file found or report rejected. 2019-07-13 02:06:17
57
safak
No report file found or report rejected. 2019-07-18 20:45:30
58
roger
No report file found or report rejected. 2019-08-6 14:18:41
59
Surrey
No report file found or report rejected. 2019-07-10 15:35:11
60
zhaohui
No report file found or report rejected. 2019-08-6 13:18:18
61
阿贵去哪了
No report file found or report rejected. 2019-08-6 14:48:16
62
agui
No report file found or report rejected. 2019-08-6 07:17:02
63
WASP
No report file found or report rejected. 2019-09-6 02:10:39
64
Mars
No report file found or report rejected. 2019-07-22 07:31:57
65
The artic lab
No report file found or report rejected. 2019-07-6 18:53:18
66
j
No report file found or report rejected. 2019-07-30 16:58:33
67
Sof
No report file found or report rejected. 2019-08-5 15:46:07
68
Toy
No report file found or report rejected. 2019-07-12 13:24:48
69
钢棍谢师傅
No report file found or report rejected. 2019-08-17 22:20:22
70
CBRL-TOPCODERS
No report file found or report rejected. 2019-07-22 08:03:14
71
Cloud
No report file found or report rejected. 2019-07-18 08:44:12
72
lol
No report file found or report rejected. 2019-07-3 14:50:36
73
Radosne Kurki
No report file found or report rejected. 2019-07-8 14:30:55
74
---
No report file found or report rejected. 2019-07-8 21:14:42
75
FuzzyMelon
No report file found or report rejected. 2019-07-9 00:03:30
76
Test Team
No report file found or report rejected. 2019-08-1 14:49:35
77
Kefi
No report file found or report rejected. 2019-08-3 09:57:42
78
WinTeam
No report file found or report rejected. 2019-08-27 20:27:37
79
LiuShuang-team
No report file found or report rejected. 2019-09-26 06:10:07
80
mm_nitro
No report file found or report rejected. 2019-09-22 21:45:27
81
J_theboss
No report file found or report rejected. 2019-07-30 19:15:56
82
panand
No report file found or report rejected. 2019-07-11 11:46:24
Please logIn to the system!

The data set available in the challenge consist of alerts investigated by a SOC team at SoD (called ‘investigated alerts’). Each record is described by various statistics selected based on experts’ knowledge, and a hierarchy of associated IP addresses (anonymized), called assets. For each alert in the ‘investigated alerts’ data tables, there is a history of related log events (a detailed set of network operations acquired by SoD, anonymized to ensure the safety of SoD clients).

In total, training and test data sets cover a period between October 1, 2018, and March 31, 2019. A description of columns from the ‘investigated alerts’ data is provided in a separate file column_descriptions.txt. The main data was divided into a training set and a test set based on alert timestamps. Approximately four months are used as the training set (the file cybersecurity_training.csv) and the remaining part is used as a test set (the file cybersecurity_test.csv). The format of those two files is the same - columns are separated by the '|' sign, however, the target column called 'notified' is missing in the test data.

There will also be a second data set available to participants of the challenge. Due to its large size, it will be hosted on an external platform. We will provide access to this data on request, to participants who exceed the baseline score on the public leaderboard. This data contains information about individual event logs associated with each of the alerts from the main data (both training and test parts). A more detailed description of this set will be provided at a later stage of the competition.

The task and the format of submissions: the task for participants of this challenge is to predict which of the investigated alerts were considered truly suspicious by the SOC team and led to issuing a notification to SoD’s clients. In the training data, this information is indicated by the column 'notified'. A submission should have a form of scores assigned to every record from the test data - each score in a separate line of a text file. An example of a correctly formatted submission file is provided in the Data files section.

Evaluation: the quality of submissions will be evaluated using the AUC measure. Solutions will be evaluated online and the preliminary results will be published on the public leaderboard. The preliminary score will be computed on a small subset of the test records, fixed for all participants. The final evaluation will be performed after completion of the competition using the remaining part of the test records. Those results will also be published online. It is important to note that only teams which submit a report describing their approach before the end of the challenge will qualify for the final evaluation. 

In order to download competition files you need to be enrolled.
Rank Team Name Score Submission Date
1
FightForPALI
0.9521 2019-09-29 13:50:22
2
hieuvq
0.9514 2019-09-28 11:16:09
3
bingogogogogo
0.9492 2019-09-29 17:07:18
4
test_123
0.9453 2019-09-29 13:35:58
5
VerifiedXiaoPAI
0.9438 2019-09-29 10:13:31
6
jinmeizhi
0.9430 2019-09-29 05:03:13
7
HSOC
0.9429 2019-09-29 19:51:33
8
wt
0.9426 2019-09-29 14:00:31
9
NSEC SJTU
0.9420 2019-09-27 04:13:43
10
tc
0.9410 2019-09-29 08:13:15
11
security
0.9403 2019-09-29 19:37:03
12
DeepIf
0.9402 2019-09-23 18:56:05
13
6e
0.9391 2019-09-29 17:36:32
14
Pisa
0.9313 2019-09-29 20:18:39
15
AGW
0.9305 2019-09-29 11:57:16
16
Pit
0.9305 2019-09-27 23:34:46
17
perfect
0.9287 2019-09-19 11:39:00
18
UMF Canada
0.9253 2019-08-23 00:20:18
19
maper1
0.9239 2019-09-17 17:46:24
20
dinesh
0.9238 2019-09-7 01:51:34
21
extended baseline
0.9224 2019-07-13 02:52:55
22
sink
0.9180 2019-07-30 09:59:56
23
iddqd
0.9151 2019-07-13 15:59:58
24
r2
0.9126 2019-08-8 23:37:56
25
IF
0.9119 2019-09-29 22:21:37
26
test
0.9106 2019-08-27 02:41:18
27
GoGoPowerRangers
0.9095 2019-07-8 18:04:04
28
piero
0.9082 2019-07-15 15:47:08
29
xyz
0.9066 2019-07-18 09:11:33
30
DennisShaw
0.9048 2019-07-31 04:07:49
31
Sinister Three
0.9045 2019-07-2 16:42:48
32
12213
0.8982 2019-08-28 11:56:01
33
shire
0.8978 2019-07-5 13:05:06
34
hopium
0.8966 2019-07-26 02:01:45
35
Chain
0.8943 2019-07-11 17:30:36
36
maa
0.8935 2019-07-13 16:39:04
37
323_touch_fish
0.8934 2019-07-13 05:51:20
38
michalm
0.8901 2019-07-23 08:52:01
39
baseline solution
0.8899 2019-07-2 18:09:18
40
Marshalls
0.8861 2019-07-5 11:24:05
41
Dan
0.8852 2019-08-5 19:04:38
42
TTUiDVL
0.8841 2019-09-25 23:03:07
43
MAGIX.AI
0.8836 2019-08-7 09:05:44
44
M
0.8769 2019-09-3 17:47:10
45
marcb
0.8760 2019-07-3 16:59:43
46
nightking
0.8753 2019-09-28 14:05:23
47
ISO_Project
0.8753 2019-09-29 22:22:24
48
LIDIA_UDC
0.8632 2019-09-12 17:06:59
49
InTensorty
0.8612 2019-09-28 19:49:49
50
Rookie
0.8598 2019-07-13 02:06:17
51
Josh
0.8581 2019-09-18 10:37:33
52
CREDIT
0.8581 2019-09-21 23:51:04
53
hello_there
0.8557 2019-07-3 18:40:53
54
agui
0.8350 2019-08-6 07:17:02
55
zhaohui
0.8345 2019-08-6 13:18:18
56
阿贵去哪了
0.8293 2019-08-6 14:48:16
57
roger
0.8246 2019-08-6 14:18:41
58
safak
0.8143 2019-07-18 20:45:30
59
Surrey
0.8095 2019-07-10 15:35:11
60
WASP
0.7990 2019-09-6 02:10:39
61
j
0.7925 2019-07-30 16:58:33
62
Mars
0.7774 2019-07-22 07:31:57
63
Sof
0.7679 2019-08-5 15:46:07
64
The artic lab
0.7216 2019-07-6 18:53:18
65
Toy
0.6388 2019-07-12 13:24:48
66
melanzana
0.6311 2019-09-5 10:52:11
67
钢棍谢师傅
0.5542 2019-08-17 22:20:22
68
LiuZ
0.5419 2019-09-29 17:11:27
69
lol
0.5356 2019-07-3 14:50:36
70
Radosne Kurki
0.5356 2019-07-8 14:30:55
71
---
0.5356 2019-07-8 21:14:42
72
FuzzyMelon
0.5356 2019-07-9 00:03:30
73
Test Team
0.5356 2019-08-1 14:49:35
74
Kefi
0.5356 2019-08-3 09:57:42
75
WinTeam
0.5356 2019-08-27 20:27:37
76
LiuShuang-team
0.5356 2019-09-26 06:10:07
77
Lzxc
0.5327 2019-09-29 20:12:40
78
CBRL-TOPCODERS
0.5281 2019-07-22 08:03:14
79
mm_nitro
0.5239 2019-09-22 21:26:09
80
Cloud
0.5175 2019-07-18 08:44:12
81
J_theboss
0.5025 2019-07-30 19:15:56
82
panand
0.5000 2019-07-11 11:46:24
  • May 27, 2019: web site of the challenge opens, the task is revealed,
  • June 15, 2019 July 1, 2019: start of the competition, data become available,
  • September 29, 2019 (23:59 GMT): deadline for submitting the solutions,
  • October 2, 2019 (23:59 GMT): deadline for sending the reports, end of the competition,
  • October 7, 2019: online publication of the final results, sending invitations for submitting papers for the special track at the IEEE BigData 2019 conference,
  • October 28, 2019: deadline for submiting invited papers,
  • November 4, 2019: notification of paper acceptance,
  • November 15, 2019: camera-ready of accepted papers due,
  • December 9-12, 2019: the IEEE BigData 2019 conference (special track date TBA).

Authors of the top-ranked solutions (based on the final evaluation scores) will be awarded prizes funded by our sponsors:

  • First Prize: 1500 USD + one free IEEE BigData 2019 conference registration,
  • Second Prize: 1000 USD + one free IEEE BigData 2019 conference registration,
  • Third Prize: 500 USD + one free IEEE BigData 2019 conference registration.

The award ceremony will take place during the special track at IEEE BigData 2019 conference.

  • Dominik Ślęzak, QED Software & Security On-Demand & University of Warsaw
  • Agnieszka Chądzyńska-Krasowska, Security On-Demand & Polish-Japanese Academy of Information Technology
  • Joel Holland, Security On-Demand
  • Andrzej Janusz, QED Software & University of Warsaw
  • Daniel Kałuża, QED Software
  • Bartek Konarski, Security On-Demand
  • Agnieszka Sochal, QED Software

In case of any questions please post on the competition forum or write an email at contact {at} knowledgepit.ml 

             

This forum is for all users to discuss matters related to the competition. Good manners apply!
  Discussion Author Replies Last post
Data File Download Speed Fernando 0 by Fernando
Tuesday, October 01, 2019, 01:20:04
Can not change the final submissions 5 by cong
Monday, September 30, 2019, 11:41:15
issues related to team merging Andrzej 0 by Andrzej
Thursday, September 26, 2019, 12:16:46
the end of competition approaches Andrzej 4 by Andrzej
Thursday, September 26, 2019, 00:35:15
Page not found J 1 by Andrzej
Monday, September 23, 2019, 13:06:28
Baseline submission code release? Dan 1 by Andrzej
Tuesday, September 10, 2019, 16:29:12
maintenance of the evaluation system Andrzej 0 by Andrzej
Wednesday, August 28, 2019, 11:31:50
the first of additional data sets released Andrzej 2 by Daniel
Monday, August 19, 2019, 12:00:09
alert_time Tian 1 by Daniel
Monday, August 19, 2019, 11:31:36
Training set order Scott 5 by Scott
Wednesday, August 14, 2019, 12:42:43
Role of localized_alerts_data and submission score A 3 by Andrzej
Tuesday, August 06, 2019, 19:47:06
Additional data - event logs Andrzej 1 by Daniel
Friday, August 02, 2019, 18:45:25
why I keep getting this error jayesh 1 by jayesh
Monday, July 29, 2019, 23:22:23
The data sets were released! Andrzej 2 by Andrzej
Thursday, July 04, 2019, 00:32:06
The submission system is online! Andrzej 0 by Andrzej
Tuesday, July 02, 2019, 18:18:22
the submission system opens soon Andrzej 0 by Andrzej
Tuesday, July 02, 2019, 01:59:22
a delay in disclosure of the competition data Andrzej 2 by Andrzej
Monday, July 01, 2019, 21:42:55
No option to add team memeber after creating the group AMIT 1 by Andrzej
Thursday, May 30, 2019, 14:42:57