Reversing an Undocumented Protocol

In this post, we will step through reverse engineering a proprietary D-Link protocol called “DCP”.  This protocol is used to provision the D-Link DCS camera line for first time use, and is used for some management features when connected to the MyDLink cloud web site.


The device we will be testing is the D-Link DCS-930L camera.  It comes with a Windows and Mac installer for first time provisioning.  You can obtain a copy of the EXE installer from their support site here.

Wireshark was used to capture all the protocol messages sent when the install wizard runs.  For my testing, I mirrored a port on the switch connected to the camera to see all the messages used to provision the device on the MyDLink cloud as well.  There is a lot of interesting chatter to, but that will have to wait until another day.  In this post we will focus on the protocol messages sent via UDP/5978.

DCP protocol messages have an interesting format.  From the Wireshark captures, it was immediately clear that the messages start with a binary header, including a two byte big-endian size field, and two six byte fields with the destination and source MAC address.  The size field does not include the size of this header, only the following payload.  The payload is text-based field, which appears to be base64-ish.

What does base64-ish mean?  Here’s an example:


It’s like base64 with a different character set.  The trailing dollar signs are a dead give-away.

Still, in order to understand this encoding and the protocol messages contained within, we will need to take apart the firmware and disassemble the MIPS binaries.  Or we could take the easy route and disassemble the x86 binaries in the installer.  There is an even easier way though.

These DCP messages are also produced when logged into the MyDLink cloud site.  They are sourced from a Java applet, which has the capability to change settings on the local device from the cloud web site when layer-2 adjacent to the device.  Since it uses the same protocol, we’ll hit the easy button and decompile the applet JAR instead.

I actually used two apps to decompile the JAR.  I found that JD-GUI was great for browsing the code and looking for high-level relationships, but in order to get accurate decompilation I used Jode.  But before we write an encoder, it would be nice to decode the existing PCAPs to get a feel for the attack surface.

Instead of re-writing the code at this point, we can just use it as is.  JRuby or Jython are great for this task — we can just open the JAR file and start using objects and methods with ease.  My preference is Ruby, so we’ll use JRuby for this task.

First, let’s use a native Ruby script to do the PCAP decode.  JRuby no longer supports C extensions, so we’ll need to use a separate script to parse the PCAPs.  I’ll admit that this is super-hacky, but it works.  The native Ruby script is shown below:

#!/usr/bin/env ruby
# This is pcap_reader.rb

require 'pcaprub'

pcap = Pcap.open_offline(ARGV[0])
packets = []
pcap.each do |pkt|
  packets << pkt[(40+14+2)..-1]

puts packets.join("|")

Easy stuff, we open the packets, print the obfuscated payload separated with pipe characters.  Now for the magic:

#!/usr/bin/env jruby
# This is pcap_decode.rb

require 'java'
require 'tsa.u45.jar'

a ="qazwersdfxcvbgtyhnmjklpoiu5647382910+/POIKLMJUYTGHNBVFREWSDCXZAQ", '$'.ord)

pkts = `ruby pcap_reader.rb #{ARGV[0]}`.chomp.split("|")
pkts.each do |pkt|
  puts a.b(pkt)

Yep, that’s it.  I’m just using the Java that D-Link was kind enough to provide me.  The tsa.u45.jar file is automatically downloaded when you manage a camera device on  You can get it here if you’d like to take a look.

Now that I’ve decoded the PCAPs, the attack surface becomes apparent.  Here’s an example of a command that is encoded within the obfuscated portion of the DCP message:


Seems simple enough — 102 is the length of the message after the first comma, ‘S’ is some kind of message type indicator, the M field is obviously the target MAC, D is the device type, C appears to be some actual base64, and X is a mystery (for now).  First, let’s decode the C field:

"ls /tmp/provision.conf"

Um… okay.  So, wild guess — ‘S’ means Shell.  That’s very kind of D-Link to provide us with a remote shell.  What’s better, the response messages actually return the result of the shell command in the R field.  It appears to be well worth our time to write our own encoder/decoder.

Before we do that though, the X field is still a mystery.  This field is the length of an MD5 hash, and seems to be the same only when the message content is the same, so I can guess that it’s a signature.  However, a straight-up MD5 of the message does not yield the same value.  Back to the Java code:

try {
  String string_12_ = string_9_.substring(string_9_.indexOf(",") + 1);
  MessageDigest messagedigest;
  (messagedigest = MessageDigest.getInstance("MD5")).reset();
  byte[] is = messagedigest.digest();
  StringBuffer stringbuffer = new StringBuffer();
  for (int i_13_ = 0; i_13_ < is.length; i_13_++)
        (new Object[] { Integer.valueOf(is[i_13_] & 0xff) })));
  string_12_ = new StringBuilder()
          (new Object[] { stringbuffer.toString() })))
  string_9_ = String.format("%d,%s",
      (new Object[] { Integer.valueOf(string_12_.length()),
          string_12_ }));
} catch (Exception exception) {
  c.a(1, "Generate DCP Message w/ Signature Failed", exception);
  return 0;

Seems pretty obvious what’s going on here.  The message portion after the first comma is MD5 hashed along with another string value.  After a little guess and test, it turns out that the other value is indeed the password.  It would seem that we need the password in order to get a shell, which is no fun.

Before we give up, let’s consider what happens when a message is sent with an invalid hash.  As it turns out, the DCP service is nice enough to answer with an R=0, to let us know that the command did not run.  Interestingly, the response is signed too.  Why is this important?  Because cracking salted MD5 is dead simple.

In order to crack the password, we can simply make a text file with the hash value, a space, and the “salt”.  In this case, the salt is the actual message, without the length and first comma, and without the last semi-colon and X parameter.  We can then use hashcat (or a GPU-assisted cracker) to crack the password:

hashcat -m 20 -a 0 -p " " hashfile.txt /usr/share/wordlists/rockyou.txt

If we take everything we’ve learned so far, add Ruby and stir with vigor, we get a shiny new Metasploit module.  I’ve chosen to implement as an auxiliary module since there’s no shellcode involved.  I’ve also used Ruby sockets instead of Rex sockets, since I didn’t know how to receive a broadcast from a Rex socket.  Note that if you would like a persistent shell to play with, you can use the module to enable telnet via:

set CMD "/usr/sbin/telnetd -l/bin/sh"

Protocol flaws are fun, but where to go from here?  There’s some very sketchy looking VPN-like tunnel code in that Java applet, and there’s a MIPS implementation of the same thing in the /mydlink/tsa binary in the firmware.  You can pull apart the firmware quite easily with binwalk (two layers of LZMA) and cpio, if you’d like to check.  Pretty sure we’ve just scratched the surface.

No doubt you’ve noticed we haven’t talked about more traditional buffer-handling exploits… those MIPS binaries are chock full of strcpy too.  Maybe in another post…

1 thought on “Reversing an Undocumented Protocol”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s