payload format indicator

[MartinL0815]

Dear devs,

I just care for a problem ticket of a customer in our simulation embedded client.
The customer complained, that JSON payload published is identified to be binary in our client.

Checking the MQTT properties of the published package I see, that the payload format identificator is missing and the publish packet contains unspecified or application specifc data (handled as binary).

This is identical to providing the MQTT property "payload format indicator" with the value 0.

A payload format indicator of 1 tells the publish packet would contain an UTF-8 string as payload. An additional exiting MQTT property for the content-type may than detail the payload content format.

Did I miss something or is there a bug to be filed?

Here a cite from the specs I followed while implementing our embedded client:

3.1.3.2.3 Payload Format Indicator
1 (0x01) Byte, Identifier of the Payload Format Indicator.
Followed by the value of the Payload Format Indicator, either of:

• 0 (0x00) Byte Indicates that the Will Message is unspecified bytes, which is equivalent to not sending a Payload Format Indicator.
• 1 (0x01) Byte Indicates that the Will Message is UTF-8 Encoded Character Data. The UTF-8 data in the Payload MUST be well-formed UTF-8 as defined by the Unicode specification [Unicode] and restated in RFC 3629 [RFC3629].

It is a Protocol Error to include the Payload Format Indicator more than once. The Server MAY validate that the Will Message is of the format indicated, and if it is not send a CONNACK with the Reason Code of 0x99 (Payload format invalid) as described in section 4.13.

https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901063

If it is a bug, I would file it here in github.

Could we please connect and discuss this before I have to react to the customer?

Thanks!

Best regards

Martin

[MartinL0815]

Ok - 😓 - should have cited from https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901111 which is about publishing not about connecting.

From your experience, because the specs do not really say, that the payload format indicator must be present, what to do if the payload format indicator is not present?

Thanks in advance!

[ysfscream]

@MartinL0815 Hi! Welcome to the MQTT X community!

As you say, the Payload Format Indicator is a property in MQTT 5.0 that, if not set, will be the same as if it were set to 0, which means that the payload is an unspecified format of bytes, which is equivalent to not sending a payload format indicator. It just means that the recipient can use it to do some payload formatting checks if specified, and we don't deal with that. However, in the transport layer, it is always transmitted in binary format.

[MartinL0815]

Dear @ysfscream,

What confuses me is, that MQTTX allows the user to specify the format and MQTTX doesn't use the MQTT v5 capabilities in a V5 connection in the case, the user specifies e.g. JSON as payload data format.

Why not dealing with this user decision and supporting listening clients to identify the payload data format by using the two MQTT properties, the payload format indicator and the content-type?

[ysfscream]

OK, sorry for the misunderstanding, but specifying JSON in MQTT X is just to make the Payload editor in the client more friendly to support input and format conversion. MQTT X does not yet support setting the payloadFormatIndicator value in MQTT 5.0, this is an issue that we will consider adding, thanks.

Why not dealing with this user decision and supporting listening clients to identify the payload data format by using the two MQTT properties, the payload format indicator and the content-type?

This is a good question, but we also don't yet know if we can support it, and I'll explore it later in development.