2023/01/23 | < back

Multiplayer in Dark Engine games

Cooperative 4-player multiplayer was added into System Shock 2 in version 2.23 and is still present in NewDark v2.48. It is notoriously unstable, especially with more than 2 players, and it is almost impossible to get it to work with 4, as trying to get the 4th player into the game usually just kicks out somebody else.

NewDark Thief 2 v1.27 adds this 4-player multiplayer capability as well, with the same thing happening with the 4th player.

Increasing the player cap in Thief 2

Before v1.27 there was an old mod that would add multiplayer to Thief 2, and that did not have the 4-player cap that v1.27 does. After obtaining the leaked partial Dark Engine source I set out to correct this horrible injustice with zero knowledge of Dark Engine internals or modding.

It seems networking in Dark Engine is mostly run by the cNetManager class. It has a field called m_MaxPlayers which determines the session player cap:

src/FRAMEWRK/NETMAN.CPP:1756
    // The limit to the number of players allowed in a game:
    int m_MaxPlayers;
#define DEFAULT_MAX_PLAYERS 4
    

The field is initialized in the constructor:

src/FRAMEWRK/NETMAN.CPP:3517
  cNetManager(IUnknown* pOuter)
    : m_Parsers(),
      ...       
      m_MaxPlayers(DEFAULT_MAX_PLAYERS)
  {
      ...
      MI_INIT_AGGREGATION_1(pOuter, INetManager, kPriorityNormal, NetManConstraints); 
      ...
    

From a cursory glance nothing much seemed to depend on this number being specifically 4, so naturally it's possible to patch thief2mp.exe to increase it.

cNetManager has been changed quite a bit in Thief 2 v1.27 compared to the code above. Its constructor can still be identified by looking for references to the string "IID_INetManager" because the class is inherited from some COM interface bullshit and the MI_INIT_AGGREGATION macros generate that string, among other things. It starts at offset 0x2F440 and the interesting part is at 0x2F4F9:

thief2mp.exe
.text:004300D9     mov     [edi+11Ah], ebp
.text:004300DF     mov     [edi+11Eh], ebp
.text:004300E5     lea     eax, [edi+122h]
.text:004300EB     call    sub_42DDF0
.text:004300F0     mov     eax, [esp+4Ch+var_38]
.text:004300F4     mov     byte ptr [esp+4Ch+var_4], 6
.text:004300F9     mov     dword ptr [edi+136h], 4       ; <--
.text:00430103     mov     [edi+13Ah], ebp
.text:00430109     mov     [edi+13Eh], ebp
    

That 4 is at offset 0x2F4FF. After replacing it with a 5 I discovered that now 4 players can join the game normally, but when the 5th joins, somebody gets kicked out like before again. The genius solution to this problem then is to just set the number to whatever you want + 1. Testing with up to 5 players revealed that the multiplayer was just as jank as with 3 people, but the game didn't seem to mind the extra players, other than perhaps being a bit more unstable.

Increasing the player cap in System Shock 2

System Shock 2 is somewhat older than Thief 2 and its cNetManager is much closer to the leaked source, even in v2.48. Its constructor can be identified the same way. It starts at offset 0x2E080 into ss2.exe and m_MaxPlayers is set slightly differently:

ss2.exe
.text:0042ED20     mov     eax, 4                        ; --+
.text:0042ED25     mov     byte ptr [esp+50h+var_4], al  ;   |
...                                                      ;   |
.text:0042ED5D     mov     byte ptr [esp+50h+var_4], 7   ;   |
.text:0042ED62     mov     [edi+215h], eax               ; <-+
    

This time the 4 is at offset 0x2E121. After changing it to a 10 and doing some testing I discovered that now 4 or more people could join the lobby just fine, but usually at least one of them desynced or got dropped at the end of the character creation/training segment. On the rare occasion that all clients actually got through, any players over the old max count of 4 would not actually spawn in the game, in contrast to Thief 2.

This happens because of the way player spawning works in SS2. Each player avatar is cloned from a net avatar (player character) archetype and there are 4 of them, one for each player:

missing

This allows the game to pick a different player model and minimap icon for each player. By default one of those archetypes is picked by the net node based on its player index in cShockNetServices::MyAvatarArchetype():

src/SHOCK/SHKNETAP.CPP:57
    if (!config_get_raw("net_avatar", 
                        abstractName, 
                        sizeof abstractName))
    {
      // Use the default player (DEFAULT_AVATAR_NAME is "Default Avatar")
      sprintf(abstractName, "%s %d", DEFAULT_AVATAR_NAME, myPlayerNum);
      abstractPlayer = pObjSys->GetObjectNamed(abstractName);
      AssertMsg1(abstractPlayer,
                 "Default avatar %s not in gamesys!",
                 abstractName);
    } else {
      abstractPlayer = pObjSys->GetObjectNamed(abstractName);
    ...
    

As seen above, it's possible to pick your own avatar by setting net_avatar to some other archetype in cam.cfg. The archetype name is received by other net nodes in cNetManager::HandleCreatePlayerMsg(). If a node doesn't wish to deal with other players' custom models, it can set net_simple_avatars 1 in cam.cfg, which will use the default avatar for all players:

src/FRAMEWRK/NETMAN.CPP:1978
    ObjID arch;
    if (config_is_defined("net_simple_avatars")) {
      // This player wants to just use the default avatar for all
      // other players. Less cool, but much cheaper in terms of
      // video memory.
      // Use the default player
      arch = gm_ObjSys->GetObjectNamed(DEFAULT_AVATAR_NAME);
      AssertMsg1(arch,
                 "Default avatar %s not in gamesys!",
                 DEFAULT_AVATAR_NAME);

    } else {
    ...
    

The stock object hierarchy only contains Default Avatrs 1 through 4, so any player index above 4 just doesn't get spawned in. This can be fixed either by forcing everyone to use net_simple_avatars or by adding more default archetypes into the hierarchy, for example, by using a mod with a custom gamesys.dml that would look something like this:

gamesys.dml
DML1

// note that you can also provide custom map icons and player models
CreateArch "Default Avatar" "Default Avatar 5"
{
    +ObjProp "Obj" { "Map Icon" "plrpip" }
    +ObjProp "Shape" { "Model Name" "player" }
}
CreateArch "Default Avatar" "Default Avatar 6"
{
    +ObjProp "Obj" { "Map Icon" "plrpip" }
    +ObjProp "Shape" { "Model Name" "player" }
}
// etc
    

Thief 2 v1.27 uses a similar system, except it defaults to "Garrett" regardless of player index, thus having more than 4 players more or less just works.

Attempting to make System Shock 2 more stable

As mentioned above, it's a very rare occasion that all players escape the SS2 character creation intro without dropping. At least one of the reasons for it appears to be the way the game handles that sequence in multiplayer. After establishing the initial connection and loading into the intro level (earth.mis), the networking gets "paused" by calling cNetManager::NonNetworkLevel(). In this state no packets are sent out and any incoming packets are not processed and instead put into a queue. When the game hits the "Synchronizing" screen at the end of the intro sequence, networking is "unpaused" and all packets in that queue are processed at once. There are at least two problems with this:

  1. Processing the queued messages during sync can cause more messages to be sent/received. There is an attempt to handle this by running the processing loop until there are no more messages left in the queue, or up to 8 times.
  2. The game sets the DirectPlay message timeout to 60 seconds for non-blocking (overridable via config var net_player_timeout) and 5 seconds for blocking messages. This probably doesn't matter, since messages still get received, they're just not processed, unless a message hits you during a loading screen.

One or both of these problems result in the fact that sitting at the sync screen for a long time seems to decrease your chances of successful connection. Thus one possible way to increase those chances is to reduce the time it takes everyone to go through character creation.

To this end I wrote a small script (.zip.png) that optionally replaces the character creation sequence with a simple dialog window that pops up at the start of earth.mis. It also loads you directly into medsci1.mis after you're done, skipping one loading screen. This can also be used in single player. Chuck the .zip into your DMM Archives folder and install it in the mod manager. It can also theoretically be applied to any custom intro map.

missing

Unfortunately it is not easily possible to increase the amount of rounds the sync process runs for, as the data for all rounds is stored in a static sized array, but we can at least do something about the timeouts. Thief 2 v1.27 disables them altogether, but that is a bit too drastic. The blocking message timeout in Shock 2 is set in cNetManager::SendToDPID(), which can be identified by looking for references to the string "Added player #%d", which is used in cNetManager::PollNetwork() right after a call to SendToDPID():

src/FRAMEWRK/NETMAN.CPP:2796
    ...
                m_bNonNetworkLevel = FALSE;
                SendToDPID(createMsg->dpId, &msg, sizeof(msg), TRUE); // <--
                m_bNonNetworkLevel = saveNonNetworkLevel;
                char temp[40];
                sprintf(temp, "Added player #%d", m_NumPlayers);      // <--
    ...
    

And the part of SendToDPID() we're looking for:

src/FRAMEWRK/NETMAN.CPP:2188
    ...
    if (blocking) {
      // Don't wait *too* long on a blocking message. Use blocking
      // messages *very* sparingly, only when absolutely necessary:
      timeout = BLOCK_DELAY; // = 5000
    } else {
    ...
    

That 5000 is at offset 0x2C7B4 in ss2.exe. Replace it with a 60000. While we're at it, might as well change the default m_playerTimeout from 60000 to 600000. It is set in cNetManager::Init(), which can be identified by references to the string "net_player_timeout":

src/FRAMEWRK/NETMAN.CPP:2524
    ...
    if (!config_get_int("net_player_timeout", &m_playerTimeout))
      m_playerTimeout = DEFAULT_PLAYER_TIMEOUT; // = 60000
    ...
    

The 60000 is at offset 0x2CAE3. Replace it with a 600000.

The patched executables for T2MP and SS2 can be found in a .zip.png here.

Results

After all this suffering the netcode is still unstable as shit, but at least it's possible to actually get in game with more than 3 players in SS2. After testing with some real people, it appears it is indeed possible to get into the game and play it with 4+ people. The save system makes it difficult to play with a large amount of people since you have to continue the game with the same amount of players as you started.

missing
missing

I've compiled the Engineering cutscene skip (since that cutscene also affects netgame stability), the aforementioned gamesys.dml and a new multiplayer respawn system into a single mod, called ss2_netfixes. It can be downloaded here (.zip.png). Should be compatible with SCP, but probably not with most other mods.

< back