SCML 2021 League
One of the ANAC 2021 Competition Leagues (@ IJCAI 2021)
AGENT SUBMISSION IS CLOSED
REPORT SUBMISSION IS CLOSED

SCML 2021 Final Results

Oneshot Track

# AgentTeam InstituteScore Winnings
1 Agent 112 Assaf Tirangel, Yossi Weizman, Inbal Avraham Bar Ilan University 1024 (1016) 500€
2 Gentle Takumu Shimizu Tokyo University of Agriculture and Technology 1024 (908) 300€
3 Agent 74 Sagi Nachum Bar Ilan University 993 200€
3 UCOneShot Agent Yuchen Liu, Rafik Hadfi and Takayuki Ito Kyoto University 947 200€

Standard Track

# AgentTeam InstituteScore Winnings
1 M4 Kazuki Komori Tokyo University of Agriculture and Technology 167 500€
2 Charlies Agent Mehmet Onur Keskin, Umit Cakan, Gevher Yesevi, Reyhan Aydogan, Amy Greenwald Ozyegin University and Brown University 003 300€
3 Artisan Kangaroo Koki Katagiri Nagoya Institute of Technology -016 200€

Collusion Track

# AgentTeam InstituteScore Winnings
1 M4 Kazuki Komori Tokyo University of Agriculture and Technology 383 500€
2 Charlies Agent Mehmet Onur Keskin, Umit Cakan, Gevher Yesevi, Reyhan Aydogan, Amy Greenwald Ozyegin University and Brown University 87 300€

BIRD Innovation Award

# AgentTeam InstituteWinnings
1 God Father Gold Fish Jackson De Campos, Benjamin Fiske, Chris Mascioli, and Amy Greenwald Brown University 400€

NEWS

  • 2021.8.18 The ANAC session is scheduled at 8:00 to 13:00 on August 25th GMT. We are looking forward to meeting your all there. The finalists have already been notified by email on 2021.8.15.
  • 2021.8.15 The qualification round is concluded and finalist teams have been informed by email. We thank all our participants and hope that we will see you all at IJCAI soon. The winners will be announced at ANAC 2021.
  • 2021.8.10 Based on requests from participants, an extended academic report of more than 2-pages can be submitted as the official agent description to be released with the agent code after the official competition at IJCAI. The page limit of this optional extended report is 5 pages. Even if you are going to submit an extended report, you still have to submit a 2-pages report to be used primarily for evaluation of agents for the BIRD Innovation Award (the extended version may also be consulted if provided). You can submit one of the two reports up to 15 hours after the deadline given that the other was submitted in time (i.e. before August 10 23:59:59 AoE) and the 2-pages report must be available to be considered for the BIRD Innovation Award. Alternatively, you can submit a single report of more than 2-pages as long as everything after page 2 is an optional appendix
  • 2021.8.1 We updated the leader boards below to show the agents running in the official competition. Participants still have 10 days to upload their final reports and updated the description so it is OK to submit a placeholder as the report/description for now.
  • 2021.7.28 We published the sixth SCML Newsletter today. If you did not receive it by email, you can find it here.
  • 2021.7.28 Due to an issue with time-zone processing, the count-down timer in this page was different in different time-zones. To avoid any sudden reduction in the perceived remaining time to the deadline, we extended the final submission deadline by twelve hours. Please note that the count-down timer is an unofficial estimate and the official deadline is indicated under "Important Dates" in this page (i.e. August 1st 23:59:59 UTC).
  • 2021.7.16 The fourth data release is available here.
  • 2021.7.15 We published the fifth SCML Newsletter today. If you did not receive it by email, you can find it here.
  • 2021.7.4 Deadline extension (see Important Dates).
  • 2021.6.30 The third data release is available here.
  • 2021.6.30 We published the fourth SCML Newsletter today. If you did not receive it by email, you can find it here. To receive future newsletters, please register here
  • 2021.6.19 As per the rules of the competitions, agents must have a non-trivial strategy to be considered for the official competition. As one of the mechanisms to enforce this rule, agents with no contracts in any simulations (most likely submitted to test the submission process) will not be ranked in the online competition and cannot become finalists in the official competition.
  • 2021.6.15 The second data release is now available.
  • 2021.6.14 We published the third SCML Newsletter today. If you did not receive it by email, you can find it here. To receive future newsletters, please register here
  • 2021.6.1 The first data release is now available.
  • 2021.6.1 We published the second SCML Newsletter today. If you did not receive it by email, you can find it here. To receive future newsletters, please register here
  • 2021.5.18 A new version of the SCML visualizer (0.2.4) was just published with bugfixes and a new tournament-only visualization of score distribution.
  • 2021.5.17 We published a blog post about SCML standard here.
  • 2021.5.10 We published the first SCML Newsletter today. If you did not receive it by email, you can find it here. To receive future newsletters, please register here
  • 2021.5.10 A new tutorial for the visualizer is now YouTube.
  • 2021.5.10 An updated version of scml and NegMAS are now on PyPi. Please upgrade your development environment.
  • 2021.5.2 The visualizer was improved substantially. You can now visualize contracts, negotiations, and offers. Moreover we added free data exploration to it. Stay tuned for the tutorial.
  • 2021.4.14 A visualizer for SCML was created. You can see a quick introduction about it here

This is the home of the Supply Chain Management League one of ANAC 2021 leagues @ IJCAI 2021.

The main motivation behind the SCM league is to increase the relevance of autonomous negotiation research by focusing on real-world scenarios that are characterized by situated negotiations, and complex dynamic utility functions.

The agent needs not only decide how to negotiate in a predefined single negotiation session, but when to join negotiations, and how to coordinate the behavior of its negotiators across multiple concurrent negotiations.

It is highly recommended to register your interest in the league by registering to this website by May 30th (No need to upload an agent by that time). It is mandatory to upload a preliminary version of the agent by July 10th.

Please note that the score on the live-competition will not impact in any way your final score in the official competition to be announced at IJCAI 2021.

The live competition is run using fewer configurations than the official competition to facilitate faster feedback when submitting new agents. This may lead to different results in the official competition.

Changes from SCML 2020

  • Agents will have access to more information about the market to help them develop their strategies. These are specifically trading prices and exogenous contract summary (See Section 5 of the game description).
  • We introduce a simpler form of the game running in the same environment as a third track called SCML-OneShot. An overview of of this simpler form is found here. The full description of this simpler form is found here. The main differences between SCML-OneShot and the other two tracks can be found here.
  • New in May 10th, 2021 The truncated mean will be used instead of the median for comparing agents.
  • The evaluation criteria for the collusion track will be slightly modified to take into account a “the consolidated financial state” of all factories run by the same participant. More specifically, the total balance (and inventory) of all factories controlled by an agent type will be aggregated before calculating the profit giving a single score for the agent type in every simulation instead of three as was the case in 2020. This means that median-pumping by getting one factory to lose while pushing the profits of the other two up will not be effective in 2021.
  • Finalists for the “collusion” track MUST have a dedicated “collusion strategy” section in their reports with a non-trivial strategy for colluding among their factory managers.
  • We created discussion rooms for the three tracks here to help participants get together and contact organizers. You can also report bugs, feature requests and open new discussion topics on the same page.

Getting started and getting help

  1. The first thing you need to do is to read the game overview for the one shot track and/or the standard/collusion tracks.
  2. If you are not registered here, do so to received updates and be able to submit your agents later here.
  3. After deciding which track(s) to participate in, check there detailed description( OneShot Standard/Collusion).
  4. Next, Check the tutorials. You can find video tutorials and consult the documentation and we are happy to see you joining the discussions.
  5. Next, Download the appropriate skeleton (OneShot or standard/collusion) and test it. You will find a full description on how to do that in the README as well as the docstring on top of myagent.py in the skeleton .
  6. Happy hacking :-) If you have any questions, bug reports, feature requests, etc, you can open an issue on the SCML GitHub page

Sponsors

Links

Important Dates

Registration (Optional)May 30th
Preliminary SubmissionJuly 10th
Last Online TournamentJuly 28th
Final SubmissionAugust 1st
Report SubmissionAugust 10th
Finalist AnnouncementAugust 15th
Winner AnnouncementAugust

Statistics

Past Competitions

Leader Board (OneShot)

Based on the tournament run on 2021-07-29 22:47:22 UTC.

# Agent Score
1 Agent74 14,194
2 Gentle 14,062
3 Agent112 13,716
4 TheSopranos78 13,190
5 StagHunter 12,030
6 Zilberan 11,111
7 UCOne 10,483
8 UcOneshotAgent 10,188
9 BondAgent 8,741
10 QlAgent 8,431
11 PDPSAgent 6,894
12 Godfather 6,411
13 Agent97 5,237
14 agentelfaroltest 4,367

Running in the official competition (12 Competitors):
StagHunter, Agent 74, QlAgent, TheSopranos 78, Zilberan, BondAgent, Agent 97, Gentle, UcOneshotAgent, Agent 112, PDPS Agent, GodfatherGoldfish,

Missing report (1 Competitors):
BondAgent,

Leader Board (Standard)

Based on the tournament run on 2021-07-29 22:24:36 UTC.

# Agent Score
1 CharliesAgent 930
2 M4 604
3 ArtisanKangaroo 478
4 SteadyMgr (2020 winner) 240
5 PhonyTales 89
7 Agent30 (2020 2nd place) -145
8 Mediocre -251
9 PAIBIU -368
10 BlueWolf -627
11 IyibiAgent -770
12 Esnaf007 -813
13 WTAgent -839
14 SorceryAgent -989
15 PrudentAgent -1,054
16 AgentPerry -1,443
17 Agent137 -1,764
18 E3BIU -2,030
19 PA -2,442
20 Perseverance -2,477
21 YIYAgent -6,171
DursunAgent 0

Running in the official competition (20 Competitors):
SteadyMgr, Agent 30, M4, Sorcery Agent, ArtisanKangaroo, WTAgent, PrudentAgent, SolidAgent, E3BIU, PAIBIU, PA, Mediocre, Perseverance, Esnaf-007, YIYAgent, Agent 137, Agent Perry, IyibiAgent, BlueWolf, CharliesAgent,

Missing Reports (3 Competitors):
Perseverance, Esnaf-007, Agent 137,

Leader Board (Collusion)

Based on the tournament run on 2021-07-29 23:55:46 UTC.

# Team/Agent Score
1 CharliesAgent 495
2 M4 (2020 winner updated) 418
3 Merchant (2020 2nd place) 225
4 ArtisanKangaroo 108
6 PAIBIU -288
7 IyibiAgent -564
8 WTAgent -667
9 Esnaf007 -771
10 Mediocre -849
11 PrudentAgent -973
12 AgentPerry -1,009
13 BlueWolf -1,243
14 E3BIU -1,447
15 Agent137 -1,484
16 Perseverance -1,717
17 PA -2,015
18 SorceryAgent -2,211
19 YIYAgent -2,737
DursunAgent 0

Running in the official competition (9 Competitors):
M4, Merchant, ArtisanKangaroo, E3BIU, Mediocre, Perseverance, Esnaf-007, Agent 137, CharliesAgent,

Missing report (3 Competitors):
Perseverance, Esnaf-007, Agent 137,

Sponsored by NEC-AIST AI Collaborative Research Laboratory and BIRD Initiative