HID Handshake

Before the Host and the Controller can exchange any meaningful data, they have to carry out a handshake. PioneerDJ implemented a simple Challenge-Response-Protocol to authenticate the DJ software and the controller (presumably to be able to charge licensing fees from 3rd party DJing software for being compatible with the CDJ). This is accomplished via a DLL called libpcon.dll which are included "Native Instruments TraktorDJ 2" and "SeratoDJ Pro" installations. The sole purpose of that DLL is to implement the logic required to carry out a successful handshake. The handshake is initiated via the CDJ by using the Link menu: LINK  CONTROL MODE (CDJ)  USB MIDI. This will cause the CDJ to start sending USB-MIDI packets and the first Packet of the handshake procedure:

USB-Devices are by definition unable to send data to the host on their own. The host instead has to constantly pull for changes. So make sure the Software running on the host is doing that before connecting initiating the connection on the CDJ.
This also mostly applies to other Pioneer devices than the CDJ, notably pioneer devices that only have MIDI+Sysex controls instead of HID, but are still locked without performing this handshake first (via Sysex).

Serialization Specifics

in the following documentation, buffers and integer values are used interchangeably. All integers larger than one byte are always (de-)serialized in big-endian order.

The handshake packets exchanged over HID always have the following structure:

0123456789abcdef00f0ILTSLSNLNArgN Payload0010More Arguments20i+00

Since the payload is of variable length and can exceed the length limit, the packet begins with an extended header. This is followed by the first TLV (which will be referred to as "super-tlv" for the rest of this doc) which contains a list of arguments. The "T" of the super-tlv indicates the current stage in the handshake procedure. The "L" of the super-tlv indicates the length of the entire payload including the TLV header (so for a payload with 9 Bytes length, the length of the TLV will be 9+2=11).

if messages are exchanged over sysex messages, the packet header is instead of the Sysex Extended Header form.
Table 1. super-tlv stages
TLV Type Description Direction

0x11

CDJ announces itself to the host

CDJ→Host only

0x12

Host start the handshake by sending the CDJ the challenge

Host→CDJ only

0x13

CDJ responds with response for the hosts challenge and its own challenge

CDJ→Host only

0x14

Host responds with response to CDJs challenge and a vendor-specific device ID

Host→CDJ only

0x15

CDJ confirms that a successful handshake was established

CDJ→Host only

Each argument is another TLV and follows the same rules as the super TLV. However, its type value indicates the purpose of the data transmitted, but the meaning of the payload changes based on the direction of transfer. It seems like the CDJ accepts the arguments in arbitrary order, but the following figures will reflect the ordering in which they were observed in the wild.

Table 2. argument types
Argument Type Description Notes

0x01

Manufacturer-name

ASCII encoded

0x02

Product-name

ASCII encoded

0x03

CDJ authentication

Buffer spread out

0x04

Host authentication

Buffer spread out

0x05

Vendor-specific device ID

Buffer spread out
For compatibility reasons with sysex, the buffers in the arguments 0x03, 0x04 and 0x05 are spread out. Meaning each byte is split in half and serialized into two bytes. So the buffer [0x12,0x34] will actually be transmitted as [0x01, 0x02, 0x03, 0x04]. This is because you can only send 7-bit bytes via sysex, which is fine for transmitting Ascii-text, but not for binary data containing 8-bit data. When describing how the arguments will be formed, the compact version will always be assumed while the bytefield figures will contain the sizes of the spread out buffer as they are transmitted.

Handshake procedure

0123456789abcdef00f00001000111040104MajMin0000000000100000000000000000000000000000000020000000000000000000000000000000003000000000000000000000000000000000

The CDJ initiates the connection by sending a super tlv of type 0x11 and a single nested tlv of type 0x01 which contain the major and minor version of the CDJs firmware. At the time of writing, the newest version is 1.85.

At this point the host has to respond with the following message:

0123456789abcdef00f00001000112L01LMSoftware-Manufacturer001002LPSoftware-Product0312SeedA2000000000000000003000000000000000000000000000000000

After the host has sent its challenge packet, the CDJ will respond with the following:

0123456789abcdef00f00001000113L01LMManufacturer02LP0010Product040aHashA030d20SeedE00000000003000000000000000000000000000000000

Which is then again followed up by a packet from the host.

0123456789abcdef00f00001000214L01LMManufacturer02LP0010Product040aHashE051620Vendor-specific device ID30000000000000000000000000

The Handshake ends with an acknowledgment from the CDJ:

0123456789abcdef00f0000200021502000000000000000000100000000000000000000000000000000020000000000000000000000000000000003000000000000000000000000000000000

In case anything goes wrong during the authentication steps, the CDJ will send a reset packet before starting the entire handshake from the beginning again. A reset packet contains no data and instead just all 0:

0123456789abcdef0000000000000000000000000000000000100000000000000000000000000000000020000000000000000000000000000000003000000000000000000000000000000000

"Cryptography"

This section will describe in detail how the different transmitted parameters are generated. The authentication follows the simplest pattern of a mutual challenge-response authentication scheme: One party provides a challenge which the other party combines with a shared secret. This combined challenge is then hashed to form the actual response. The shared secret depends on the manufacturer of the DJ software trying to communicate with the device in question. The secret is formed by XORing SeedE with the manufacturer-dependent value (which are hardcoded into the CDJs firmware):

Table 3. Manufacturer-dependent secret input
Manufacturer Value

PioneerDJ

0x680131FB

Serato

0x0D6F55AB

NativeInstruments

0x8C5B3F5D

Atomix

0x97779123

Algoriddim

0x384B522B

Mixvibes

0x9EA1A8B6

Numark

0x889127C7

Dolby

0x7A2B59AB

Generating SeedA

The exact way seedA is generated is currently unknown. However, I was able to identify a pattern which is present when posing as seratoDJ or Traktor Pro 2. It seems like the algorithm generates two ushorts (16bit unsigned integers) and then concats them after serializing them as uints (32bit). So the resulting buffer could look like this (before being spaced out for actual transmission):

012345670000ffff0000ffff

Hashing

The CDJ uses a Fowler-Noll-Vo hash function (FNV-1a). The value 0x811c9dc5 is used as the algorithms 32-bit FNV offset basis while 0x1000193 serves as its FNV Prime. You can read more about FNV hashes on wikipedia Here is a complete reference implementation in python3.8:

def FNV(buff: bytes) -> int:
  hash = 0x811c9dc5
  for val in buff:
    hash = ((val^hash) * 0x1000193) & 0xFFFFFFFF
  return hash

Validating the CDJs response

The CDJs response can be validated by simply calculating appending the common secret and SeedA to each other and hashing that: hashAd = hash(seedA+secret)

If hashAd matches hashA, the CDJs response is valid.

Building the response for the CDJ (HashE)

Building HashE is just as simple as validating the CDJs response: hashE = hash(seedE+secret)

Building the vendor-specific device ID

The vendor-specific is static and can be taken from the following table:

Contributions Wanted: If you happen to have access to software from any of these manufacturers or hardware that is not yet registered, feel free to contact me by opening an issue on this projects github repository
Table 4. vendor-device ID index
NativeInstruments Serato PioneerDJ Atomix Algoriddim Mixvibes Numark Dolby

CDJ2000NXS2

0x02F2F4510ED30A2C1042

0XC0BD940FB700BB8FC75B

0x02F2F4510ED30A2C1042

 — 

 — 

 — 

 — 

 — 

DJM900NXS2

0x43F4CDEE2C07C9590742

0xD557CE8F5EF07BC27E98

0xC03251EE5DF69245025B

 — 

 — 

 — 

 — 

 — 

CDJTOUR1

0x768DCEDA0DD7F010434E

 — 

 — 

 — 

 — 

 — 

 — 

 — 

DJMTOUR1

0x502876738F3DC7CF1DA7

 — 

 — 

 — 

 — 

 — 

 — 

 — 

XDJ1000MK2

0xABA778D3B506E22EBBAE

 — 

 — 

 — 

 — 

 — 

 — 

 — 

DJM450

0x08EF3F2F1E7A9017F6AF

 — 

 — 

 — 

 — 

 — 

 — 

 — 

DJM250MK2

0xD2F58B612C6206819139

 — 

 — 

 — 

 — 

 — 

 — 

 — 

DJM750MK2

0x866E33BD04852E71ED21

 — 

0x490a44496383029BF371

 — 

 — 

 — 

 — 

 — 

DJMV10

0x3C83DB259C76FE8DB2AB

0x1D3EAC55C613D297623D

 — 

 — 

 — 

 — 

 — 

 — 

XDJXZ

Not supported

0x2D99858D0A5A913D4EBD

Captures lead me to believe that there actually is no handshake taking place in this configuration

 — 

 — 

 — 

 — 

 — 
From my experiments, it looks like Atomix VirtualDJ actually poses as Rekordbox instead of using its own identifiers.

Keepalive

Once a successful handshake occurred, the host has to keep the connection alive by sending packets at least once per second. However, it is recommended to send a packet at least every 400ms since that has been observed in tests with commercial DJing software. In order to keep the connection alive in cases where the host has no new information to update the CDJ with, the CDJ also accepts empty packets:

0123456789abcdef0000000000000000000000000000000000100000000000000000000000000000000020000000000000000000000000000000003000000000000000000000000000000000