Complete protocol description in ethermess.7

ethermess.7

@@ -7,7 +7,11 @@
 .Sh DESCRIPTION
 .Nm
 is a protocol for direct messaging over raw Ethernet frames.
-.Pp
+.Ss Byte order
+.Nm
+uses network byte order when representing values larger than 8 bits.
+That is, 0xAABB will be encoded as 0xAA 0xBB.
+.Ss Packet format
 The structure of an Ethernet frame containing an EtherMess packet is as
 follows:
 .TS
@@ -26,19 +30,21 @@ Ethernet header	Destination	6	Recipient's MAC
 	EtherType	2	0xDA7A
 EtherMess header	Version	1	0x00
 	Packet type	1	(defined below)
-Payload		0 - 1494	(depends on packet type)
-Padding		0 - 1494	0x00
+Payload		0 - 1498	(depends on packet type)
+Padding		0 - 1498	0x00
 .TE
 .Pp
 Padding may always be present, and it can be any length that fits into the
 Ethernet frame after the headers and the payload.
 Only requirement for padding is that it has to be all zero bytes.
+If this requirement is violated, the packet must be ignored even if it
+otherwise parses fine.
 .Pp
 NOTE: Frame check sequence is not present in the table above, as at least
-on Linux if you read and write raw Ethernet frames using
+on Linux if raw Ethernet frames are read and written using
 .Xr packet 7
 it gets stripped off and added by the lower level drivers.
-.Ss Packet type
+.Ss Packet types
 .Bl -tag -width Ds
 .It Speak version (0x00)
 .TS
@@ -50,8 +56,8 @@ Version	1	EtherMess protocol version
 .TE
 .Pp
 Tells the recipient to speak the given version of the protocol.
-Should be generated whenever you receive an EtherMess packet with a version
-you don't speak.
+Should be generated whenever an EtherMess packet with a version one doesn't
+speak is received.
 .It Status request (0x01)
 Requests the recipient to tell its status and nick.
 Upon receiving this packet, a status packet should be sent to the sender.
@@ -79,24 +85,204 @@ Offline
 .El
 .Pp
 The nick field must contain valid utf-8.
-That is, it may not contain overlong encodings, codepoints over U+10FFFF,
+That is, it must not contain overlong encoding, codepoints over U+10FFFF,
 or surrogate pairs.
 .Pp
-Additionally, the nick may not contain any non-characters (U+xxFFFE,
+Additionally, the nick must not contain any non-characters (U+xxFFFE,
 U+xxFFFF, U+FDD0 - U+FDED), C0 control characters (U+00 - U+1F), C1 control
 characters (U+80 - U+9F), or the codepoints U+2028 LINE SEPARATOR and
 U+2029 PARAGRAPH SEPARATOR.
 .Pp
-If the message has a status that is not recognized or the nick is not valid
-utf-8 or contains disallowed codepopoints, the message must be ignored.
-.It Msgid request (0x03)
-<>
-.It Msgid (0x04)
-<>
-.It Message (0x05)
-<>
-.It ACK (0x06)
-<>
+The packet must be ignored if
+.Bl -bullet
+.It
+Status is not one of the recognized values.
+.It
+Nick length is more than the remaining size of the packet
+.It
+Nick is not valid utf-8.
+.It
+Nick contains disallowed codepoints.
 .El
+.It Msgid request (0x03)
+Requests the recipient to tell the message ID of the next message it is
+expecting from the sender.
+Upon receiving this packet, a msgid packet should be sent to the sender.
+.It Msgid (0x04)
+.TS
+allbox;
+lb	lb	lb
+l	l	l.
+Field	Length (B)	Contents
+Msgid	2	Expected ID of next message
+.TE
+.Pp
+Informs the recipient of the expected ID of next message the recipient will
+send to the sender.
+Generally speaking, this should be one greater than the message ID of the
+message sender last received from the recipient.
+.Pp
+For special cases, see the Msgid section.
+.It Message (0x05)
+.TS
+allbox;
+lb	lb	lb
+l	l	l.
+Field	Length (B)	Contents
+Msgid	2	ID of the message
+Length	2	Length of the message field in bytes
+Message	0 - 1494	Message encoded as utf-8
+.TE
+.Pp
+Is used to transmit messages.
+Upon receiving this packet, an ACK packet should be sent to the sender.
+.Pp
+If the message ID is considered to be in the future (for details see the
+Msgid section), the message should be displayed to the user and the next
+message ID to expect should be made msgid + 1.
+If instead the ID is considered to be in the past, the message should be
+skipped as a repeat. Note however than even in this case an ACK should be
+generated.
+.Pp
+The message field has similar requirements to the nick field in its
+encoding and contents, except that the message field may additionally
+contain the codepoints U+0A LINE FEED and U+09 CHARACTER TABULATION.
+.Pp
+The packet must be fully ignored (no ACK) if
+.Bl -bullet
+.It
+Message length is more than 1494 or the remaining size of the packet.
+.It
+Message is not valid utf-8.
+.It
+Message contains disallowed codepoints.
+.El
+.It ACK (0x06)
+.TS
+allbox;
+lb	lb	lb
+l	l	l.
+Field	Length (B)	Contents
+Msgid	2	ID of the received message
+.TE
+.Pp
+Confirms that the sender has received a message with the given message ID
+from the recipient.
+.Pp
+If the message ID matches a message sent by the recipient, the recipient
+should consider that message to be received and not transmit it again in
+the future.
+On the other hand if the recipient doesn't recognize that message ID, the
+packet should be ignored.
+.El
+.Ss Msgid
+Message IDs in
+.Nm
+are unsigned 16 bit integers.
+.Pp
+They are unique to each sender and recipient pair.
+This means that the message IDs of messages A sends to B are completely
+unrelated to the IDs of messages A sends to C or messages B sends to A.
+.Pp
+Because they are 16 bit, there is a possibility of a rollover.
+For that reason, a simple comparison whether the ID of the message received
+is equal or greater than the ID the recipient was expecting next doesn't
+work.
+The correct way to test if a message ID is considered to be in the future
+is to do
+.Li (received_msgid - next_expected_msgid) modulus 2^16
+and see if the result is in the range of [0, 0x8000].
+.Pp
+There are special cases for if the recipient doesn't know what the ID of
+the next message they receive should be.
+If they have received a message, it should be accepted and the next ID to
+expect should be set to the ID of the message plus one, just like if the
+message ID was considered to be in the future.
+(Though it may be marked as a possible repeat.)
+If they have received an msgid request packet, they should set the next ID
+to expect to a random value and send that value in the msgid packet.
+.Pp
+Implementations should not store the next message IDs to use in any
+persistent storage.
+This is because if those values are out of date, messages sent by the
+implementation may be silently dropped.
+Storing next message ID to expect is safer, though a value more than 0x8000
+out of date will lead to valid messages being dropped by the implementation
+itself.
+.Ss Sending a message
+To send a message, the sender should first figure out the correct message
+ID to use.
+If they have cached the next message ID to use for the recipient, they
+should use it.
+Otherwise they should send an msgid request packet to the recipient.
+.Pp
+If the recipient doesn't answer the msgid request, the sender should retry
+the transmission a few times.
+The reference implementation,
+.Xr ethermess 1 ,
+will attempt to transmit at most 5 times, and uses a randomized timeout
+around 1 to 1.5 seconds.
+Implementations should randomize the retry timeout to reduce chances of
+packet collisions with other
+.Nm
+implementations.
+If there is still no msgid packet, the message should be considered to have
+failed to send.
+.Pp
+Once the message ID to use is known, the sender should send the message to
+the recipient and wait for an ACK with the same message ID.
+(Note that the sender might receive ACKs for different messages.)
+The transmission should have a similar timeout and retry mechanism as with
+the msgid request.
+If no matching ACK is received, the message should be considered to have
+failed to send, but the implementation may keep it around in case an ACK
+with its message ID is received later.
+.Pp
+After sending the message, regardless of if an ACK was received for it or
+not, the sender should update it's cached value of the next message ID to
+use when sending to the target.
+.Ss Changing one's own nick or status
+When a user changes their nick or status, they should broadcast (send to
+the MAC FF:FF:FF:FF:FF:FF) the status and nick with the status packet.
+.Ss Offline timeout
+An implementation should keep track of when it has last received a valid
+packet from its peers.
+If it has been more than 5 minutes, the implementation should consider the
+peer to have quit.
+The quit notification displayed to the user should mark that this was
+because of the timeout and not due to a status packet with an offline
+status.
+.Pp
+To avoid being considered to have quit, an implementation should
+periodically broadcast a status packet containing its current status and
+nick.
+The period used by
+.Xr ethermess 1
+is in the range of 60 to 64 seconds.
+The period should be randomized in order to avoid packet collisions with
+other
+.Nm
+implementations.
+.Ss Finding peers
+To find peers, an implementation should broadcast a status request package.
+Alternatively it may wait until it receives a keep-alive status broadcast
+from all of its peers.
+.Ss Getting the status of a peer
+An implementation should keep a cache of statuses and nicks that is updated
+whenever a status packet is received.
+.Pp
+If the cache doesn't have a given peer, the implementation may request the
+status using the status request packet targeted at that peer.
+The timeout and retry mechanism should be designed to avoid spammyness and
+minimize the possibility of packet collisions.
+.Pp
+An implementation may also implement a user-initiated status check by first
+looking at the cache and then regardless of whether it was in the cache or
+not sending a status request packet.
+This way the user can both receive an answer fast and in case the cached
+value was outdated get the correct result once it arrives.
+.Ss Shutting down
+When an implementation exits, it should broadcast a status packet with the
+status set to 0x02 (offline) and nick set to the current nick.
 .Sh SEE ALSO
 .Xr ethermess 1