pdserv  3.4
Process data server
Loading...
Searching...
No Matches
MSR language syntax

Introduction

The MSR data server is one means for a client to interact and communicate with a process encapsulated by the pdserv library, possibly running in real time.

The client attaches to the pdserv server via a TCP network socket, by default on port 2345. Communication is bi-directional.

The communication syntax between client and server follows an XML-like format.

Upon connection, the server sends an initial greeting and waits for commands from the client.

Command stream from client to server

Commands are sent by the client to the server in a single XML element. The XML-like command format in BNF is:

command := '<' tag attribute* '/>'
tag := name
attribute := ' ' name '="' value '"'
name := any character except the obvious (space, angled brackets, qoutes)

Example of setting a parameter value:

<write_parameter index="55" value="123457"/>
Note
The command parser is very relaxed (attributes can have single ('), double (") or even no quotes if the value does not contain spaces, the closing can be '>' or '/>', boolean attributes can use the attribute name stand alone without '="1"'), but do not rely on this feature! This is only useful when a human interacts on the network socket directly. Programs must always use the strict definition as shown above.

Unrecognized commands return a warning:

<silly/>
<warn num="1000" text="unknown command" command="silly"/>

Unrecognized attributes are silently ignored.

There are state changing and query commands, and they may or may not generate a reply. Commands are processed in order of reception. It is not required to synchronize commands to replies, although the client will do that for its own purpose.

To determine whether a command was processed by the server, the id attribute is used within the command element. The reply (if one is generated) will have an id attribute with the same value. Additionally an <ack> element with the same id attribute value is sent immediately after command is processed.

For example, here are two commands sent in one line:

<rk index="99" id="rk99"/><rk index="3" id="rk3"/>
<ack id="rk99"/>
<channel index="3" name="/osc/amplitude/0/0" datasize="8" typ="TDBL"
bufsize="1000" task="0" HZ="100" time="1565346298.661905"
value="1.358077306222715e-312" id="rk3"/>
<ack id="rk3"/>

The first line contains two <rk> (query a signal) commands with an id attribute. The first is unsuccessful and produces no reply, however the <ack id="rk99"/> is replied before successfully processing the second query. Note the id attribute in <channel>.

This mechanism can be used to synchronize communication between client and server, especially when the commands do not produce a reply.

Reply stream from server to client

The reply stream is well formed XML that can be processed with a streaming XML parser like SAX or expat. A message is a single XML element that may contain child elements.

Note
Although the server sends a greeting in the form of a <connected> element, it is a complete element with a closing and is thus a finished document! Normal parsers will stop there and need to be reset for parsing to continue. Thus parsers must first be primed with any well formed opening element (like <msr>) which is never closed. Content generated by the server is ASCII which is valid UTF-8, although variable path names are encoded in native program format. To avoid problems, use UTF-8 throughout.

The communication from server to client is interleaved with solicited and unsolicited messages:

  • Solicited messages are replies generated by the server due to a command. An example is a parameter value poll to which the server replies.
  • Unsolicted messages are sent ad-hoc by the server to the client as required. Examples are:
    • The client configured the server to monitor a signal or parameter for value change. The value is sent as soon as it changes without the client explicitely requesting it.
    • Error, warning, info, etc events generated by the process are sent when they occur.
    • All clients are immediately notified of a parameter value changes.

The client must carefully separate the return stream to distinguish between these message types.

The problem with unsolicited messages is that they occur at any point in time without any cause from the client. Thus the client has to monitor this stream continuously or run the risk of its input stream overflowing and thus being cut off by the server. For most clients (GUI's, etc) this is no problem, but for clients that want to excercise synchronous single threaded communication could choke. To alleviate this problem, unsolicited messages can be disabled using the <remote_host/> command.

Data types

When querying the server for its variables (signals or parameters), its data type and binary size (sizeof operator) is returned in the typ and datasize attributes respectively.

An example of quering a parameter:

<rp index="4"/>
<parameter index="4" name="/osc/amplitude/Limit" datasize="8" typ="TDBL"
flags="7" mtime="0.000000" value="20"/>

The following table maps these types to the corresponding C data type.

typ datasize c-type
TUCHAR 1 uint8_t
TCHAR 1 int8_t
TUSHORT 2 uint16_t
TSHORT 2 int16_t
TUINT 4 uint32_t
TINT 4 int32_t
TULINT 8 uint64_t
TLINT 8 int64_t
TFLT 4 float
TDBL 8 double
COMPOUND n struct {}

Dimension

The dimensionality of a variable is reflected in attributes when querying the variables.

Generally a variable can be scalar or non-scalar. Scalar variables contain only one of the data types as shown above (a compound can also be a scalar). If a variable contains more than one element, it is said to be non-scalar.

For non-scalar variables, the reply has the following additional attributes:

  • typ has either _VECTOR or _MATRIX is appended to the data type
  • orientation contains either
    • VECTOR
    • MATRIX_ROW_MAJOR
  • anz: element count (relevant for matrices and vectors) (German: anzahl)
  • cnum: column count (only for matrices)
  • rnum: row count (only for matrices)

The data values are ordered in a C-like fasion. Adjacent values are the last dimension of a multi-dimensional array: for the

double array[4][5];

with 4 rows and 5 columns, the 5's will be adjacent (row major).

An example of a scalar:

<channel index="32" name="/SawTooth" datasize="1" typ="TUCHAR"
bufsize="100" task="0" HZ="10"/>

An example of a vector:

<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
anz="5" cnum="5" rnum="1" orientation="VECTOR" dir="1" flags="7"
mtime="0.000000" value="0,0,0,0,0"/>

An example of a matrix:

<channel index="2" name="/osc/amplitude" datasize="8" typ="TDBL_MATRIX"
anz="25" cnum="5" rnum="5" orientation="MATRIX_ROW_MAJOR"
dir="1" bufsize="1000" task="0" HZ="100"/>

Incedentally, the dir flag in the last two examples show that there are more variables listed under its name (see the list command).

Binary value representation

Binary values are transmitted in native machine format. Check the endian flag in the greeting.

MSR uses two methods of binary encoding:

hexdec-format

This format simply walks though the value's entire memory space byte by byte and prints it as a 2-digit hex value.

Example: pi = 182D4454FB210940 on a little endian platform.

This representation is used between client and server interaction.

To query the value of a variable, set the hex boolean attribute in the command and the value is returned in the hexvalue reply attribute.

To set the value of a parameter, use the hexvalue instead of the value** attribute in the command.

Base64 encoding

The value is encoded as base64 string which is more efficient than hexdec (four base64 vs six hexdec characters per three bytes binary data).

Example: pi = GC1EVPshCUA= on a little endian platform.

This representation is used for streaming subscribed signals when the coding attribute is set to Base64

Atomized variables

To support old clients that can only handle scalar variables, the server can atomize non-scalar variables in a path beneath it. This behaviour is set in the configuration file.

Atomized variables can cause an exponential explosion (think of a 10x10 matrix yielding 100 separate variables)

When requesting a directory list, use the attribute noderived to suppress listing of directories containing atomized variables only.

Commands

ping

Ping the server. The server will reply with its current time. This function is used to detect network presence of the server.

Command attributes: none

Reply: <ping/> with attributes:

  • time (after v3.0): Server reception time

Example:

<ping/>
<ping time="1565257713.113707"/>

write_parameter

Alias: wp

Set the value of a parameter. The connection must be authorized, see access in remote_host

Command attributes:

  • index [uint] (m*): Parameter index (see read_parameter)
  • name [string] (m*): Parameter path
  • hexvalue [string] (m**): Hexadecimal representation of the value
  • value [value list] (m**): Comma separated list of doubles
  • startindex [uint] (o): Linear starting index when writing a non-scalar parameter. Out of range indices are ignored.
  • aic [bool] (o): If true, suppress parameter update notification. This is used when parameters need to be written at high frequency.

*) One of index (dominant) or name is mandatory, otherwise nothing happens.

**) One of hexvalue (dominant) or value is mandatory, otherwise nothing happens.

Values are written until either the parameter's end is reached or input value is exhaused.

Reply: none (possibly a parameter update notification follows)

See also
pdserv_parameter()

Example:

<wp index="6" hexvalue="DEADBEEF"/>

read_parameter

Alias: rp

Read a parameter's value value and attributes.

Command attributes:

  • name [string] (m*): Parameter path
  • index [uint] (m*): Parameter index (name dominates)
  • short [bool] (o): Only show index, name, mtime and value/_hexvalue_ in reply. This reduces communication overhead when the parameter is known.
  • hex [bool] (o): Return value in hexadecimal representation using the hexvalue attribute.

*) If neither name (dominant) or index is specified, a complete parameter listing of all parameters is performed. The listing is enclosed in a <parameters></parameters> element with individual child <parameter/> elements.

Reply: <parameter/> with attributes:

  • index [uint]: Parameter index
  • name [string]: Parameter path
  • comment* [string]: Comment string
  • datasize* [uint]: see data types
  • typ* [enum]: see data types
  • anz* [uint]: see dimension
  • cnum* [uint]: see dimension
  • rnum* [uint]: see dimension
  • orientation* [enum]: see dimension
  • dir* [bool]: see atomized variables
  • flags* [uint]: Bit coded value
    • 0x01: readable
    • 0x02: writeable
    • 0x100: Parameter is part of another parameter (atomized)
  • persistent* [bool]: Parameter is persistent
  • mtime (time): Modification time. A modifiation time of 0.0 indicates that the parameter is unmodified.
  • value** [value list]: List of (comma separated) values
  • hexvalue** [binary]: Hexadecimal value representation

*) These attributes are not transmitted when short attribute is set.

**) hexvalue is returned when hex attribute is set.

See also
pdserv_parameter()

Example:

<rp index="5"/>
<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
anz="5" cnum="5" rnum="1" orientation="VECTOR" dir="1" flags="7"
mtime="1565276030.213719" value="56746,1,0,0,0"/>

xsad

Subscribe a signals to the data stream sent by the server.

Command attributes:

  • coding [enum] (o): If set to Base64, value is base64 coded
  • event [bool] (o): Transmit value only when it changes
  • sync [bool] (o): If true, data transmission is synchronized by resetting all decimation counters.
  • channels [uint list] (o): List of channels to transmit
  • reduction [uint] (o): Decimation counter; Value is sent every n'th count
  • blocksize [uint] (o): Number of values to group before transmission
  • precision [uint] (o): Number of decimal points to use (when not using base64)
  • group [uint] (o): Data group to send channel in. A data group can only contain a channel once, although a channel can exist in more than one data group. This is used to group similar task/decimation rates together. Introduced in v3. Feature test: group
Note
When channels is not specified, only sync takes affect. If a channel exists in a group, it is overwritten with the new attribute values.

Reply: none (but see the unsolicited `<data>` message)

Example:

<xsad channels="2,4" group="3" reduction="5" blocksize="10"/>

xsod

Removes signal subscription from a data transmission group.

Command attributes:

  • channels [uint list] (o): List of channels to remove. If this attribute is not specified, all signal subscriptions are cancelled.
  • group [uint] (o): Group id to remove channels from. Introduced in version 3. Feature test: group

Reply: none

Example:

<xsod channels="4" group="3"/>

xsap

Version: after v3.0

Monitor parameters, transmitting the value on change.

Command attributes:

  • hex [bool] (o): Values are transmitted in hexadecimal represenation
  • monitor [bool] (o): If true, transmit all parameter changes. This state can only be reset using <xsop monitor="0"/>.
  • parameters [uint list] (o): List of parameters to monitor

Reply: none (but see the unsolicited `<data>` message)

Example:

<xsap parameters="5,6"/>

xsop

Version: after v3.0

Cancel monitoring of parameter changes.

Command attributes:

  • monitor [bool] (o): If false, stop monitoring all parameters. This does not clear the list of parameters to monitor!
  • parameters [uint list] (o): List of parameters to remove from monitor

Example:

<xsop parameters="6"/>

echo

Unused

read_kanaele

Alias: rk rc

Read the value and all attributes of a signal.

Command attributes:

  • name [string] (m*): Signal path
  • index [uint] (m*): Signal index (name dominates)
  • short [bool] (o): Only show index, name, time and value/_hexvalue_ in reply. This reduces communication overhead when the signal is known.
  • hex [bool] (o, since v3): Return value in hexadecimal representation using the hexvalue attribute.

*) If neither name (dominant) or index is specified, a complete signal listing of all signals is performed. The listing is enclosed in a <channels></channels> element with individual child <channel/> elements. Polling signal values is expensive, thus the value is not transmitted when a complete listing is performed,

Reply: <channel/> with attributes:

  • index [uint]: Parameter index
  • name [string]: Parameter path
  • comment* [string]: Comment string
  • datasize* [uint]: see data types
  • typ* [enum]: see data types
  • anz* [uint]: see dimension
  • cnum* [uint]: see dimension
  • rnum* [uint]: see dimension
  • orientation* [enum]: see dimension
  • dir* [bool]: see atomized variables
  • bufsize [uint]: Parameter is persistent
  • task [uint]: Task id signal belongs to
  • HZ [double]: Signal calculation value (may be slower than task!)
  • time [time]: Sample time
  • value** [value list]: List of (comma separated) values
  • hexvalue** [binary]: Hexadecimal value representation

*) These attributes are not transmitted when short attribute is set.

**) hexvalue is returned when hex attribute is set. Supported since v3.

Example:

<rk index="29"/>
<channel index="29" name="/osc/derivative" comment="Derivative of [cos,sin]"
datasize="8" typ="TDBL_LIST" anz="2" cnum="2" rnum="1"
orientation="VECTOR" text="Derivative of [cos,sin]" dir="1"
bufsize="1000" task="0" HZ="100"
time="1565280630.398467" value="11.04316657549318,-5.031003686345138"/>

read_param_values

Alias: rpv

Return a list of all parameter values, sequencially sorted by index.

Reply: <param_values/> with attributes:

  • value [value list]: Semicolon separated list of parameter values.
See also
pdserv_signal()

Example:

<read_param_values/>
<param_values value="1.2;10;1;0;20;56746,1,0,0,0"/>

auth

Version: since v3.0

Feature test: login

Perform SASL authentication dialog.

Command attributes:

  • clientdata [base64 string]: Data from client
  • mech [string]: Chosen mechanism
  • logout [bool]: Close session

Reply: <saslauth/> with attributes:

  • mechlist [string]: List of supported mechanisms.
  • serverdata [base64 string]: Server SASL reply.
  • success [bool]: If supplied, this is the final step with login indicating success or failure.

1) The first step is to obtain a list of supported mechanisms.

<auth/>
<saslauth mechlist="DIGEST-MD5 CRAM-MD5 NTLM PLAIN LOGIN ANONYMOUS"/>
Note
When login is mandatory, the <saslauth mechlist="..."/> element is sent automatically upon connection without prior request.

The list of mechanisms does not change. This first step is required only once upon initial connection by the client.

This step is not required if the mechanism is known apriori.

2) The client chooses a mechanism. Depending on the mechanism, the client library will start the conversation with some data or not. Supplying a mech** attribute initiates the server side of the conversation (client-send-first). Depending on the mechanism, the server may require more data or even signal success or failure.

<auth mech="PLAIN" clientdata="lkjsdlfjkl"/>
<saslauth serverdata="lkjasdflkj" success="1"/>

The mech attribute may only be sent on initiation. Certain mechanisms may require more than one interaction. Subsequent commands will only have clientdata, until the server replies with a success attribute.

list

Version: since v2.0

Feature test: list

Return a list of parameters, signals and directories.

Command attributes:

  • path [string] (m): Directory path to list
  • noderived [bool] (o): Omit the dir attribute from <parameter/> and <channel/> elements when only atomized versions of the parent variable exist beneath it.

Reply: <listing></listing> element with children:

Example:

<list path="/"/>
<listing>
<dir path="/Event"/>
<channel index="32" name="/SawTooth" datasize="1" typ="TUCHAR"
bufsize="100" task="0" HZ="10"/>
<dir path="/Taskinfo"/>
<dir path="/osc"/>
</listing>

remote_host

Version: since v2.0

Feature test: polite

Command to set miscellaneous states.

Command attributes:

  • name [string] (o): Remote host name
  • applicationname [string]: Name of application logging in
  • access [bool] (o): Request permission to change parameters. This is another ugly feature of MSR, as the client decides whether it is allowed to write paramters or not.
  • polite [bool] (o): Do not produce any unsolicited message, such as <pu>, messages, etc. Only commands that produce an immediate reply will send data from server to client.

Example:

<remote_host name="myhostname" applicationname="GUI" access="1"/>

read_statistics

Alias: read_statics (no, this is not a spelling mistake) rs

Get information about other MSR clients connected to the server.

Reply: <clients></clients> element with <client/> child elements with attributes:

  • name [string]: Text as specified using the name attribute in `<remote_host/>`.
  • appname [string]: Text as specified using the applicationname attribute in `<remote_host/>`.
  • countin: Received bytes
  • countout: Transmitted bytes
  • connectedtime: Time when client connected

Example:

<read_statistics/>
<clients>
<client name="lansim (192.168.22.56:52276)"
apname="Persistent Manager, Version: 0.3.1"
countin="19908501" countout="27337577"
connectedtime="1282151176.659208"/>
<client .../>
</clients>

message_history

Feature test: history

Arguments:

  • seq (uint) (o): request a message with a specific sequence number

Request the server to send past messages. Without attributes, the latest message including all active messages are sent enclosed in a <message_history></message_history> element in increasing order of sequence numbers.

With seq argument, the specific message with that sequence number is returned if it exists.

Reply:

  • No attributes: a list of all active messages as well as the latest message. The list is enclosed in <message_history></message_history>
  • When seq is specified and invalid: Nothing
  • When seq is specified and valid: The requested message

Notes:

Typically this command is used to discover all past messages. To achieve this, first send

<message_history/>

and process all messages returned, taking note of the sequence numbers in the replies.

Then use

<message_history seq="XX" id="nn"/>

to retrieve individual messages, counting down from the highest sequence number. If a message does not exist, nothing is returned. Therefor the id tag should be used in this context to detect an empty reply.

Note
The sequence number is a uint32_t data type and can wrap (though very unlikely, but possible). Counting down from 0 will yield sequence number 4294967295 (2^32-1).
See also
pdserv_event()

Example of retrieving active messages:

<message_history/>
<message_history>
<crit_error name="/Limit" index="3" seq="150" prio="0"
time="1565386296.209379" text="Event message 3"/>
<crit_error name="/Limit" index="1" seq="4196" prio="0"
time="1565389793.155808" text="Event message 2"/>
<reset name="/Limit" index="0" seq="4220" time="1565389796.195809"/>
</message_history>

The last message has sequence number 4220 (/Limit[0]) but it is inactive. Messages /Limit[1] and /Limit[3] still persist.

To regenerate the list, retrieve messages with sequence numbers counting down from the highest sequence number (4220): 4219, 4218... (possibly, but not necessarily skipping 4196):

<message_history seq="4219" id="mh"/>
<crit_error name="/Limit" index="0" seq="4219" prio="0"
time="1565389795.655814" text="Event message 1" id="mh"/>
<ack id="mh"/>

Example of retrieving a specific message with sequence number 3220:

<message_history seq="3220" id="mh"/>
<ack id="mh"/>

Message 3220 does not exist any more becuase it was purged by the server (in this example it keeps 1000 messages - see configuration file). The list is now complete as far as possible.

However, even though the history has been exhausted, message /Limit[3] with sequence number 150 is still known to be active.

Unsolicited stream

ack

This element is sent just after the reply to any command with an id attribute is processed. This feature can be used to synchronize communication with the server.

Element attributes:

  • id [string]: Same as id attribute of command
  • time [time] (after v3.0): System time

Example:

<echo id="echo id"/>
<ack id="echo id" time="1565618806.761301"/>

connected

This element is sent as a greeting upon connection to the server. It is used to greet the client with a information about the server, running process and a list of capabilities.

Element attributes:

  • name (string): Always MSR
  • host (string): Host name
  • features (enum):
    • pushparameters:
    • binparameters:
    • eventchannels:
    • statistics:
    • pmtime:
    • aic:
    • messages:
    • polite: (since v2) See remote_host
    • list: (since v2) See list
    • login: (after v3.0) See auth
    • history: (after v3.0) See message_history
    • xsap: (after v3.0) See xsap/xsop
    • group: (since v3) See xsad/data
  • recievebufsize (uint): Size of input buffer. An non-terminated command exceeding this limit will cause network disconnection. (yes, recievebufsize is a spelling mistake!)
  • app (string): (since v2) Application name
  • appversion (string): (since v2) Application version
  • endian (enum): (after v3.0)
    • big
    • little
  • login (string): (after v3.0) If mandatory, client must log in

Example:

<connected name="MSR" host="lappi" app="PdServ Test" appversion="1.234"
version="393226"
features="pushparameters,binparameters,eventchannels,statistics,pmtime,
aic,messages,polite,list,login,history,xsap"
endian="little" recievebufsize="8192"/>

data

A <data></data> element is a container enclosing child elements of values for signals that were subscribed using xsad.

Element attributes:

  • time [time]: Time when <data> element is transmitted.
  • group [int]: Value of xsad:group, if not zero. Introduced in version 3. Feature test: group
  • level: unused

Child elements:

  • <F/> signal data of fixed decimation rate subscription as specified in xsad:reduction.

    Attributes:

    • c [uint]: Signal index
    • d [data]: Array of signal values, grouped together as specified in xsad:blocksize
  • <E>: Same as <F/>, but for event based subscribtion (xsad:event) instead of stream based.
  • <time/>

    Attributes:

    • d [uint64 array]: uint64_t array of size xasd:blocksize of time values when the sample was taken. Time is in nano-seconds since epoch. Coding is base64

      This time is more accurate than data:time, since it contains the actual timestamp when a sample was taken and not one calculated backwards from data:time.

Example of event-based subscription (with group supported):

<xsad channels="29" event="1" group="9" id="1234"/>
<ack id="1234" time="1565613895.117706"/>
<data group="9" level="0" time="1565613895.122557">
<time d="XWcUZ5ItuhU="/>
<E c="29" d="-9.844291201868986,-5.137457097023935"/>
</data>

pu

Notification to all clients of a parameter change. Clients interested in its value must then request it.

See also: xsap

Element attributes:

  • index [uint]: Parameter index

Example:

<pu index="5"/>
<pu index="7"/>

parameter

A client can subscribe to monitor parameter value changes using xsap. Upon change, the value is sent automatically by the server using a <parameter/> element upon value change.

Example:

<xsap parameters="5" id="pmon"/>
<ack id="pmon" time="1565627421.396449"/>
<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
anz="5" cnum="5" rnum="1" orientation="VECTOR" flags="7"
mtime="1565627429.852331"
hexvalue="0100000000000000000000000000000001000000" pm="1"/>
<parameter index="5" name="/Event/State" mtime="1565627438.417570"
hexvalue="0000000000000000000000000000000001000000" pm="1"/>
Note
This this is very similar to read_parameter reply, except for:
  • the pm attribute, indicating that this is a monitored parameter
  • no atomized versions of the parameter is sent
  • only a hexdec value is sent
  • parameters that are known to the client (it was sent to the client in a long version already) produce a short reply (see sort attribute in read_parameter.

Event: crit_error, error, warn, info, reset

A message is generated by the process when some event occurs. They have the following categories as tag name:

  • crit_error
  • error
  • warn
  • info
  • reset

<reset/> is a special event element signalling that the event has been cancelled.

Attributes are:

  • name [path]: (since v3) Path that identifies the event
  • path [path]: (until v3) Path that identifies the event
  • state [bool]: (until v3) Event is set/unset
  • index [int]: Event element:
    • -1: Scalar event
    • 0..N-1: element index when event path is a vector
  • seq [uint32]: (since v3) Event sequence number, linearly increasing, rolling over at 2^32-1
  • prio [enum]: (since v3) Priority
    prio Priority Tag name
    0 Emergency crit_error
    1 Alert crit_error
    2 Critical crit_error
    3 Error error
    4 Warning warn
    5 Notice info
    6 Info info
    7 Debug info
  • time [time]: Time when event occurred
  • text [string]: Descriptive text message
See also
pdserv_event()

Example (since v3):

<warn name="/Limit" index="1" seq="7" prio="4" time="1565689720.991359"
text="Event message 2"/>
<reset name="/Limit" index="1" seq="8" time="1565689724.251307"/>

Example (until v3):

<crit_error path="/Event2" index="0" state="1" time="75683.785450"
text="Single Event"/>