Polymorphic Mutex Names


Perhaps there was never any legitimate reason to use Polymorphic Mutex Names, so it’s understandable many developers never provided a solution.

It could be argued, poly mutexes serve only as a way for malicious applications to evade detection. On the other hand, they could potentially be used to defend applications from Denial of Service (DoS) attacks or termination by a malicious application.

Many developers use named mutexes to avoid multiple instances of their application running. The problem is that applications using this technique are also vulnerable to Denial of Service attacks.

First, some defintions…

Named Mutex

A mutual exclusion (mutex) is a program object that allows multiple program threads to share the same resource, but not simultaneously. Global Named Mutexes can be created to avoid multiple instances of the same application running on a system.

How to limit 32-bit applications to one instance in Visual C++ explains how to do this.

The following snippet of code is very common.

int WINAPI WinMain(...)
   const char szUniqueNamedMutex[] = "com_mycompany_apps_appname";
   HANDLE hHandle = CreateMutex( NULL, TRUE, szUniqueNamedMutex );
   if( ERROR_ALREADY_EXISTS == GetLastError() )
      // Program already running somewhere
      return(1); // Exit program

   // Program runs...

   // Upon app closing:
   ReleaseMutex( hHandle ); // Explicitly release mutex
   CloseHandle( hHandle ); // close handle before terminating
   return( 1 );

The problem with this example is when an entirely different application either intentionally or unintentionally creates a mutex of the same name. In this case: com_mycompany_apps_appname

In the remarks section for the CreateMutex API, it states the following.

If you are using a named mutex to limit your application to a single instance, a malicious user can create this mutex before you do and prevent your application from starting. To prevent this situation, create a randomly named mutex and store the name so that it can only be obtained by an authorized user. Alternatively, you can use a file for this purpose. To limit your application to one instance per user, create a locked file in the user’s profile directory.

Raymond Chen covers the topic in A single-instance program is its own denial of service

A discussion on stack exchange and another in Multiple program instance prevention in C suggest possible ways to avoid Denial of Service conditions.

Polymorphic Code

Uses a Disassembly Engine (DE) to rewrite instructions while retaining the original algorithm. The code changes itself each time it runs, but functionality of the code will remain the same.

Imagine setting the EAX register of an x86 CPU to zero. Each of the following instructions accomplish this, but they certainly appear different, don’t they?

"\x31\xc0"                 /* xor  eax, eax             */

"\x29\xc0"                 /* sub  eax, eax             */

"\xb8\x00\x00\x00\x00"     /* mov  eax, 0               */

"\x8d\x05\x00\x00\x00\x00" /* lea  eax, [0]             */

"\x83\xe0\x00"             /* and  eax, 0               */

"\x6a\x00"                 /* push 0                    */
"\x58"                     /* pop  eax                  */

"\x6b\xc0\x00"             /* imul eax, eax, 0          */

Malware Evasion Techniques

It was a blog post by the Talos Group titled “Cyber Conflict” Decoy Document Used In Real Cyber Conflict which reminded me of how malware continues to use named mutexes to prevent multiple instances of the same code running simultaneously.

This malware in particular decodes a mutex name at runtime in order to evade detection by string. Here you see the mutex name in its decoded state. That is, after it’s been deobfuscated using an XOR operation.

Talos go on to say: the actor changed the XOR key and the MUTEX name. We assume that these modifications were performed in order to avoid detection based on public IOCs.

Indicators of Compromise (IOC) is something that can be used to identify an attack, such as a Mutex name.

@osxreverser joked about the concept of polymorphic mutex names back in July. Since a polymorphic mutex name would always be different, the assumption is that it would be ineffective as a way to prevent multiple instances running.

Are polymorphic mutex names possible? Yes, they are.
Do they have a legitimate purpose? Potentially, yes.

Some searching online lead me to the following question at stack overflow which was posted in 2012.

Can a polymorphic/metamorphic worm use (the same) mutex? This would solve the problem of grinding the network into the ground and consuming all resources with multiple instances of the worm….

The answer which was accepted:

Short answer: no.

The point of polymorphic malware is to have no identifiable pattern. …. To know the name of the mutex, there must be a pattern to it. If there’s a pattern to it, the AV can use that pattern to detect the malware.

However, this answer is not entirely true…

The general consensus is that poly mutex names aren’t possible.

A proposal to generate Mutex Revocation List (MRL) is discussed in What if we had mutex revocation lists?

As the malware authors use these markers to check if they have already infected a specific machine, the mutex names canโ€™t be truly random.

No, they can’t be truly random, but they can be more difficult to detect.

Using encryption

To the best of my knowledge, the first documented example of malware using encryption algorithms to derive mutex names was demonstrated in the TreasureHunter malware.

How Malware Generates Mutex Names to Evade Detection discuses how the TreasureHunter malware derives an MD5 hash of the Windows Product ID before using this as the global mutex name. This prevents more than one instance of the application running while simultaneously evading detection via string matching.

Loki-Bot does something similar using the MachineGUID from the Windows registry.

For more information, see Looking at Mutex Objects for Malware Discovery and Indicators of Compromise

Although the above methods don’t generate polymorphic names, only entropy and additional code to enumerate mutexes are missing.

So what kinds of crypto can be used to generate poly mutex names?

  • Key derivation function (KDF)
  • Use a password hashing algorithm along with nonce to create unique output.

  • Symmetric Encryption
  • Use a stream or block cipher with unique key to encrypt a message, can later be decrypted using same key.

  • Message Authentication Code (MAC)
  • Use a cryptographic primitive and unique key to derive hash of string. Hash can be verified using same message and key.

  • Pseudo Random Number Generator (PRNG)
  • Seed RNG using hash of unique information obtained from operating system.
    Use byte stream to obfuscate a message.

  • Asymmetric Encryption
  • Use a digital signature algorithm such as ElGamal or DSA.

Although all of the above can be used or misused to generate polymorphic mutex names, using a KDF seemed sufficient.

If we use entropy, this must be included in the name, otherwise we cannot verify ownership of the named object later when performing detection.

You can think of the mutex name as being a password if we use a KDF, and a nonce/salt being the entropy.

Mutex Name Generation

For illustration purposes, I’m using the permutation function of SipHash for a 64-bit hash function.

void permute(w128_t *state, int cnt) {
  uint32_t *s=(uint32_t*)&state->w[0];
  int      i;
  for (i=0; i<cnt; i++) {
    s[0] += s[1]; 
    s[1]=ROTL32(s[1], 5); 
    s[1] ^= s[0]; 
    s[2] += s[3]; 
    s[3]=ROTL32(s[3], 8); 
    s[3] ^= s[2]; 
    s[0] += s[3]; 
    s[3] ^= s[0]; 
    s[2] += s[1]; 
    s[1]=ROTL32(s[1], 7); 
    s[1] ^= s[2]; 
uint64_t Hexe(void *in, size_t len, uint64_t nonce)
    w128_t s;
    int    idx, i;
    uint8_t *p=(uint8_t*)in;
    w64_t   *seed=(w64_t*)&nonce;
    // zero initialize
    memset(s.b, 0, sizeof(s));

    // set 64-bit seed
    s.w[1] = seed->w[0];
    s.w[3] = seed->w[1];
    // absorb data
    while (len) {
      idx = MIN(len, HEXE_BLOCK_LEN);
      for (i=0; i<idx; i++) {
        s.b[i] ^= p[i];
      p += idx;
      len -= idx;
      permute(&s, 2);
    // add padding
    s.b[idx] ^= 0x1F;
    s.b[3]   ^= 0x80;
    // permute last time
    permute(&s, 4);
    // return 64-bit hash
    return s.q[0];

The 1st 32-bits highlighted in red are the nonce/seed value provided to Hexe with mutex name. The remaining 64-bits are returned by Hexe.

Validation is simply using the same mutex name and nonce/seed value to derive a hash which is then compared with the mutex name enumerated from system.

Additional steps

Since a lot of unique mutex names are derived from a GUID function such as CoCreateGUID() we can take things a step further and increase the length of nonce to 64-bits. This along with 64-bit hash can be converted to a GUID string.

int main(int argc, char *argv[])
    w128_t  s, r;
    OLECHAR *guid_str;
    if (argc!=2) {
      printf ("usage: hexe_hash <string>\n");
      return 0;
    // doesn't matter about source of seed value
    // just multiply using golden ratio value
    s.w[0] = rand() * 0x9e3779b9;
    s.w[1] = rand() * 0x9e3779b9;
    // derive hash of input string
    s.q[1] = Hexe(argv[1], strlen(argv[1]), &s);
    wprintf(L"Hexe Hash : %016llX%016llX\n", s.q[0], s.q[1]);
    // convert to string
    StringFromCLSID((GUID*)&s, &guid_str);
    wprintf(L"GUID      : %s\n", guid_str);
    // convert to binary
    CLSIDFromString(guid_str, (GUID*)&r);
    wprintf(L"Hexe Hash : %016llX%016llX\n", r.q[0], r.q[1]);
    return 0;  

As you can see, it looks exactly like a regular GUID now except it can also be verified using the same algorithm that generated it.

All that remains now is how to find and validate poly mutex names.

Enumerating Mutexes

We enumerate handles (file descriptors on linux) for each process of the local computer. Filter by type Mutant, filter those with no name, then attempt verification of each name.

BOOL IsRunning(PWCHAR mutex_name)
    pNtQuerySystemInformation  NtQuerySystemInformation;
    pNtQueryObject             NtQueryObject;

    w128_t                     r;
    PWCHAR                     p;
    HRESULT                    hr;
    ULONGLONG                  hash;
    ULONG                      len=0, total=0;
    NTSTATUS                   status;
    LPVOID                     list=NULL;
    DWORD                      i;
    HANDLE                     hProcess, hObject;

    BOOL                       bRunning=FALSE;
    size_t                     mutex_len;

    NtQuerySystemInformation =
        GetModuleHandle(L"ntdll"), "NtQuerySystemInformation");

    NtQueryObject =
        GetModuleHandle(L"ntdll"), "NtQueryObject");

    if (!NtQuerySystemInformation ||
        !NtQueryObject) {
      // we couldn't resolve API address
      return FALSE;

    SetPrivilege(SE_DEBUG_NAME, TRUE);

    list = xmalloc(2048);

    do {
      len += 2048;
      list = xrealloc (list, len);

      if (list==NULL) {
        // we couldn't reallocate memory
      status = NtQuerySystemInformation(SystemHandleInformation,
          list, len, &total);

    } while (status == STATUS_INFO_LEN_MISMATCH);

    if (!NT_SUCCESS(status)) {
      // we were unable to obtain list of process
      return FALSE;

    mutex_len = lstrlen(mutex_name);

    // for each handle
    for (i=0; i<h->HandleCount && !bRunning; i++)
      // skip system
      if (h->Handles[i].ProcessId == 4) continue;

      // open the process
      hProcess = OpenProcess(PROCESS_DUP_HANDLE,
         FALSE, h->Handles[i].ProcessId);

      if (hProcess != NULL)
        // try duplicate handle
        status = DuplicateHandle(hProcess,
            (HANDLE)h->Handles[i].Handle, GetCurrentProcess(),
            &hObject, 0, FALSE, DUPLICATE_SAME_ACCESS);

        if (status)
          // query basic info
          status = NtQueryObject(hObject,
              ObjectBasicInformation, &obi, sizeof(obi), &len);

          if (NT_SUCCESS(status))
            // skip object if there's no name
            if (obi.NameInformationLength !=0)
              // query the type
              len = obi.TypeInformationLength + 2;
              t = (POBJECT_TYPE_INFORMATION)xmalloc(len);

              if (t != NULL) {
                status = NtQueryObject(hObject,
                    ObjectTypeInformation, t, len, &len);

                if (NT_SUCCESS(status)) {
                  // skip object if it isn't a mutant
                  if (lstrcmpi(t->Name.Buffer, L"Mutant")!=0) {

              // query the name
              len = obi.NameInformationLength + 2;
              n = (POBJECT_NAME_INFORMATION)xmalloc(len);

              if (n != NULL) {
                status = NtQueryObject(hObject,
                    ObjectNameInformation, n, len, &len);

                if (NT_SUCCESS(status)) {
                  // obtain the absolute name
                  p = wcsrchr(n->Name.Buffer, L'\\');
                  if (p != NULL) {
                    p += 1;
                    // attempt conversion to binary
                    hr = CLSIDFromString((OLECHAR*)p, 
                    if (hr == NOERROR) {
                      // generate hash using seed
                   hash = Hexe(mutex_name, 
                       mutex_len*2, r.q[0]);
                      // do we have a match?
                      if (hash == r.q[1]) {
                        // we found poly mutex
          // close object
        // close process
    return bRunning;

An example of output from handle.c


    PWCHAR  mutex_name = L"com_mycompany_apps_appname";
    BOOL    bRunning; 
    w128_t  s, r;
    OLECHAR *guid_str;
    HANDLE  hMutex;
    // already running?
    bRunning = IsRunning(mutex_name); 
    wprintf (L"Checking: %s : %s.\n", mutex_name,
      bRunning ? L"Found instance" : L"Instance not found");
    // if not running, create it
    // doesn't matter about source of seed value
    // just multiply using golden ratio value
    s.w[0] = rand() * 0x9e3779b9;
    s.w[1] = rand() * 0x9e3779b9;
    // derive hash of input string
    s.q[1] = Hexe(mutex_name, lstrlen(mutex_name)*2, s.q[0]);
    // convert to string
    StringFromCLSID((GUID*)&s, &guid_str);
    wprintf(L"Creating: %s for %s\n", guid_str, mutex_name);
    // create
    hMutex = CreateMutex(NULL, TRUE, guid_str);
    if (ERROR_ALREADY_EXISTS == GetLastError()) {
      wprintf (L"already exists!\n");
      return 0;
    // already running?
    bRunning = IsRunning(mutex_name); 
    wprintf (L"Checking: %s : %s.\n", mutex_name,
      bRunning ? L"Found instance" : L"Instance not found");
    // close mutex 
    wprintf (L"Closing : %s\n", mutex_name);

    // already running?
    bRunning = IsRunning(mutex_name); 
    wprintf (L"Checking: %s : %s.\n", mutex_name,
      bRunning ? L"Found instance" : L"Instance not found");

Although the above example uses a hardcoded string, the MachineGUID value could also be used.


While something like this can be used for malicious purposes, defensive applications need to evade detection too.

Ideally, generation of mutex names should be created by the kernel using digital signatures, but since there is no demand for such a feature, we’re unlikely to see it implemented.

@yassine_lemmou pointed out to me that if 2 or more applications run at the same time, this would result in multiple instances running.


This entry was posted in cryptography, programming, windows and tagged , , , , . Bookmark the permalink.

3 Responses to Polymorphic Mutex Names

  1. Jackson5 says:

    Awesome post! Any chance you have a copy of handle.c? I want to use it in our next CTF reversing challenge ๐Ÿ™‚

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s