Example
Measurement Data Acquisition/Network Communication
The objective is to control a fictional measurement device via a binary message protocol. A controlling client software could be connected over TCP/IP, UDP or any other suitable transport layer to the device. The client software and the device are communicating with each other by using the specified binary message protocol.
A proper communication solution would consist of a serializer to encode/decode the binary message protocol, a meaningful protocol documentation and analysis tools to debug/inspect the protocol in the transport layer. Additionally the binary message protocol should be optimized for size and speed. Every byte counts and saves bandwidth, memory and computation power.
In this example the client software will be implemented in C++. Hence the generated serializer will be in C++ too. The analysis tooling consists of the extern network analyzer Wireshark® in conjunction with a generated protocol dissector. A structured documentation will be generated with HTML.
At first the message protocol is described. Then the serializer, Wireshark® Dissector and the documention generation is explained.
Message Protocol Description
The fictional measurement device supports the following set of messages. Each message will be specified with Protlr Language (PGL).
Establish connection. Establishes a connection to the measurement device. A client which connects to the device must provide a proper version number.
/**
* Message the client must send to the
* device to establish a connection.
* */
@use MsgType 1 alias MSG_CLIENTCONNECT
struct ClientConnect {
1 : uint8 version //!< Client version
}
Close connection. Closes an open connection to the measurement device. No additional data needed.
/**
* Message the client must send to the
* device to disconnect.
* */
@use MsgType 2 alias MSG_CLIENTDISCONNECT
struct ClientDisconnect {
}
Start measurement. Configures the device for measuring and starts the measurement process. The device supports two data measurement modes: low and high. Furthermore up to four measurement channels can be activated.
/**
* The device supports two measurement
* modes.
* */
enum Mode {
ModeLow = 1 //!< Sets sample rate to 50 Hz
ModeHigh = 2 //!< Sets sample rate to 100 Hz
}
//! Starts the measurement process.
@use MsgType 3 alias MSG_CLIENTSTART
struct ClientStart {
1 : bool channel[4] //!< Control array to activate measurement channels
1 : Mode mode //!< Measurement mode
}
Abort measurement. Aborts a running measurement process. Every started measurement process must be aborted by the client.
/**
* Each measurement process must be
* aborted by the client.
* */
@use MsgType 4 alias MSG_CLIENTABORT
struct ClientAbort {
1 : int8 reason //!< The reason why the measurement was cancled
}
Measurement data reception. Acquired measurement data is sent from the device to the client. The data consists of sampled measurement values and various state flags.
/**
* Message which contains acquired
* measurement data.
* */
@use MsgType 5 alias MSG_DEVICEDATA
struct DeviceData {
1 : uint8 channel //!< Channel the data belongs to (0..3)
8 : uint64 timestamp //!< Timestamp of data acquisition
1 : bitfield {
1 : bool overheating //!< Flag which indicates if device is overheating
3 : int8 battery //!< Value which indicates battery charge condition
4 : int8 pendingFrames //!< Value which indicates how many frames are pending
}
4 : int32 sampleCount //!< Measurement sample count
4 : float32 samples[sampleCount] //!< Array of measurement sample points
}
Message frame. Every protocol message is wrapped in a message frame. The message frame acts as entry point/starting point into the protocol.
/**
* This is a message protocol to communicate
* with a fictional measurement device.
* */
endianness-be protocol MeasurementDevice
/**
* Frame structure which acts as container
* for messages. The containing message
* will be specified by a message id.
* */
@entry
struct MessageFrame {
1 : int8 msgId //!< Id of the protocol message
* : MsgType<msgId> msg //!< Dynamic protocol message
}
Message Protocol Generation
The 66 lines of message protocol description will now be fed into Protlr. The generation process will roughly create:
- 900 lines of high quality C++ source code to serialize,
- 300 lines Wireshark® LUA Dissector source code to inspect/debug,
- 4 pages of meaningful documentation.
A bug free human implementation and documentation would take days, weeks or even months! Changes in the message protocol description would lead to cumbersome adaption of source code and documentation. Every developer would cry in pain :-).
ANSI C89
Here you can see an excerpt of an ANSI C89 compatible serializer for the defined protocol.
/*******************************************************************************
* MessageFrame
*/
typedef struct
{
pbyte *_begin;
pbyte *_end;
pint8 msg_type_id;
union {
MeasurementDeviceClientConnectDesc clientconnect;
MeasurementDeviceClientDisconnectDesc clientdisconnect;
MeasurementDeviceClientStartDesc clientstart;
MeasurementDeviceClientAbortDesc clientabort;
MeasurementDeviceDeviceDataDesc devicedata;
} msg;
} MeasurementDeviceMessageFrameDesc;
PNewResult measurementdevice_messageframe_new(MeasurementDeviceMessageFrameDesc *, PMemoryIterator *);
pint8 measurementdevice_messageframe_msgid(const MeasurementDeviceMessageFrameDesc *);
void measurementdevice_messageframe_set_msgid(MeasurementDeviceMessageFrameDesc *, pint8);
const pbyte *measurementdevice_messageframe_ptr__begin(const MeasurementDeviceMessageFrameDesc *);
const pbyte *measurementdevice_messageframe_ptr__end(const MeasurementDeviceMessageFrameDesc *);
const pbyte *measurementdevice_messageframe_ptr_msgid(const MeasurementDeviceMessageFrameDesc *);
void measurementdevice_messageframe__clear(MeasurementDeviceMessageFrameDesc *);
PParseResult measurementdevice_messageframe__parse(PParseInput *);
C++ Serializer
The generated serializer consists of various source and header files and can be embeded into the C++ client software project. The software must provide a suitable access to a transport layer to send/receive the messages. For example a network socket. Protocol messages now can be encoded/decoded into/from byte streams and send/received over the transport layer. Changes in message protocol can be easily integrated into the client software.
Excerpt of the generated C++ source code:
/** Message which contains acquired measurement data. */
class DeviceData
: public MsgType
{
DeviceData(const DeviceData &);
DeviceData& operator=(const DeviceData &);
public:
DeviceData();
~DeviceData();
/* Getters for public fields. Protocol-defined types are returned as reference. */
uint8 channel() const;
uint64 timestamp() const;
bool overheating() const;
int8 battery() const;
int8 pendingFrames() const;
int32 sampleCount() const;
CppTarget::Runtime::Array<float> &samples();
const CppTarget::Runtime::Array<float> &samples() const;
/* Setters for public non-const fields.*/
void setChannel(uint8 value);
void setTimestamp(uint64 value);
void setOverheating(bool value);
void setBattery(int8 value);
void setPendingFrames(int8 value);
void setSampleCount(int32 value);
/* IO-Operations */
void read(std::istream &stream);
void write(std::ostream &stream);
void reset();
private:
bool hasAliasIntern(DynamicType dynamicTypeId, int64 numericAlias) const;
int64 aliasIntern(DynamicType dynamicTypeId) const;
private:
struct Bitfield0Data;
MsgType::Alias m_aliasMsgType;
uint8 m_channel; /**< Channel the data belongs to (0..3) */
uint64 m_timestamp; /**< Timestamp of data acquisition */
std::auto_ptr<Bitfield0Data> m_Bitfield0Data;
int32 m_sampleCount; /**< Measurement sample count */
CppTarget::Runtime::Array<float> m_samples; /**< Array of measurement sample points */
};
Wireshark® Dissector
The generated Wireshark® Dissector is based on LUA and can be directly loaded into Wireshark® to analyse whats happening in the transport layer. Knowing whats happening in the transport layer is the key to discovering implementation errors. Changes in the message protocol will be reflected immediately in the dissector.
Excerpt of the generated Wireshark® Dissector LUA source code:
_G.MessageFrame = Proto("MessageFrame", "MessageFrame")
require("Runtime.RevertRange")
require("Runtime.BitfieldValue")
local M = {}
-- static dependencies can be computed only once. Structure Proto objects for dynamic array types have to
-- be initialized before dissection because fields cannot be registered at runtime
require("MessageFrame_modules.MessageType")
-- data fields
local messageId = ProtoField.int8("messageId", "MessageFrame.messageId")
table.insert(MessageFrame.fields, messageId)
-- dissector
function _G.MessageFrame.dissector(buf, pkt, root)
if buf:len() == 0 then return end
_ENV = _G
__byteOffset = 0
local __startByteOffset = __byteOffset
root = root:add(MessageFrame, buf(__byteOffset), buf)
pkt.cols.protocol = _G.MessageFrame.name
local __subtree = {}
local __typeId = 0
local __streamSize = 0
-- messageId --
root:add(messageId, buf(__byteOffset, 1))
__byteOffset = __byteOffset + 1
__typeId = M.
-- msg --
M.msg = assert(loadfile("MessageFrame_modules/MessageType_instance.lua"))(__typeId)
__oldByteOffset = __byteOffset
__subtree = root:add(buf(__byteOffset), "MessageFrame.msg")
M.msg.dissector(buf, pkt, __subtree)
__streamSize = __byteOffset - __oldByteOffset
__subtree:set_len(__streamSize)
__streamSize = __byteOffset - __startByteOffset
root:set_len(__streamSize)
end
-- registration
local udp_dissector_table = DissectorTable.get("udp.port")
udp_dissector_table:add(1234, _G.MessageFrame)
Documentation
The generated documentation explains the message protocol in a meaningful structure. It can be shared and always stays up to date.