Perspectives

The x86 Jmp Call Additive Encoder, Explained

Written by Nick Hoffman, Jeremy Humble, Toby Taylor

A technical overview

This blog is the sixth installment of our Shellcode Signature Series, where we disassemble shellcode and obfuscation techniques and write signatures to detect them. In this post, we will be looking at the x86_jmp_call_additive payload encoder that in found within Metasploit. The encoder itself is one of the older encoders in Metasploit, but it uses a novel approach to rotate an XOR key that can be helpful to understand from a defender's perspective.

Understanding the x86 Jmp Call Additive Encoder

The x86 Jmp Call Additive encoder contains a statically generated decoding stub, with a substitution for the key and payload. This is a great encoder for a beginning reverse engineer to study since it is not as complex as an encoder like Zutto Dekiru.

The primary logic of the module is contained within the init function as shown in the ruby source:

"\xfc" + # cld
"\xbbXORK" + # mov ebx, key
"\xeb\x0c" + # jmp short 0x14
"\x5e" + # pop esi
"\x56" + # push esi
"\x31\x1e" + # xor [esi], ebx
"\xad" + # lodsd
"\x01\xc3" + # add ebx, eax
"\x85\xc0" + # test eax, eax
"\x75\xf7" + # jnz 0xa
"\xc3" + # ret
"\xe8\xef\xff\xff\xff", # call 0x8

Let’s walk through each step of the encoder to understand its instructions. Viewing the init function in a disassembler breaks the chunk of assembly into two functions, making it easier to read. The first function at the entry point is:

The shellcode will preload the XOR key into EBX and perform a non-conditional jump to offset 0x14, which is a call to offset 0x8 (not shown in the window below). When the call is performed, EIP is pushed onto the stack. Following the call instruction is junk code which is decoded in the next function.

Seeking to offset 0x8, which was the destination of our previous call, will show the following function:

If we break down each step, the control flow becomes a bit easier to understand, as shown below.

pop esi ; pop the value of EIP into ESI
push esi ; PUSH ESI back onto the stack (used in the ret at the end)

This is our method of obtaining the value of EIP. Since the call instruction happened right before the encoded payload bytes, the value at ESI will now point to the beginning of the encoded buffer. The value is pushed back on to the stack so when the RET instruction executes, it will result in an effective jump to the beginning of our newly decoded buffer, as shown here:

xor dword [esi], ebx ; XOR the bytes at buffer start with key
lodsd eax, dword [esi] ; Store result of XOR into EAX, inc ESI
add ebx, eax ; Add result from XOR to key
test eax, eax ; Is the loop finished?

The core logic of the encoder happens within these instructions. Let’s walk through an iteration of this code:

xor dword [esi], ebx ; [esi] ^ ebx = 479603F9 ^ 4714EB05 == 0082E8FC
lodsd eax, dword [esi] ; EAX == 0082E8FC
add ebx, eax ; eax + ebx 0082E8FC + 4714EB05 == 4797D401
test eax, eax ;
jne <offset> ; Jump is taken while EAX != 0

EBX changes with each round which results in a rotating key. From an attacker’s perspective, this is advantageous because a defender cannot write detection signatures that look for XOR’d variants of known shellcode structures.

Yara Signature

The Yara signature for this encoder is easy to create, given that most of the decoder’s code is static. The following signature will trigger on payloads that use this key rotation method:

rule x86_jmp_call_additive
{
meta:
author = "Nick Hoffman / Jeremy Humble"
description = "X86 JMP Call Additive Encoder Stub"
strings:
$encoder_op = {fc bb [4] eb 0c 5e 56 31 1e ad 01 c3 85 c0 75 f7 c3 e8 ef ff ff ff}
$encoder_ascii = /fcbb.{8}eb0c5e56311ead01c385c075f7c3e8efffffff/ nocase
condition:
any of them
}

When running the signature against Virustotal and other online sandboxes, it’s not uncommon to find payloads that pentesters submitted to test the encoder’s impact on antivirus detection rates.

Conclusions and Suggestions

This encoder contains a novel method of rotating the XOR key with each pass during the decoding process. Performing a jump ahead and then a call backwards is a common method in shellcode, and we will go over the process in greater detail in later posts.

There are several ways to improve this encoder. Slightly changing the order of operations or inserting NOP instructions will often break signatures that are written to look for the exact encoder. If the authors want the encoder or decoder to be more robust, they could re-write it to utilize different registers which would change up the assembly, making it more difficult for signatures to detect.

Read more insights from our Managed Threat Services team and learn how our accredited analysts can deliver 24/7 threat detection, investigation, and response for your organization.

Article