Realtime updates with SSE

Server Sent Events is a browser/javascript component that allows a server to push events to a client freeing the client from needing to poll to check for updates. SSE has been around for several years and is widely available in many browsers. It is similiar to websockets, but is single direction only, unlike websockets which are bi directional.

More information about SSE. W3C specification

Notes about SSE

• You cannot add additional headers to the EventSource class in Javascript. This means the url specifying the connection request must include additional details such as authorisation
• SSE can be easily Polyfilled with XHR if necessary
• It is possible to use fetch() in streaming mode as an alternative to the EventSource class, which has the advantage of permitting additional headers. Fieldpine do not have any examples of doing this available, but there are several examples available in the internet
• SSE in the browser will handle reconnection after network failure automatically, however it will not restart after certain errors from the webserver (such as 502).
• Prefer to view examples and cut/paste? browse to /report/pos/tests/eventsource.htm when logged into your Fieldpine Account to see a simple example in action.

Using SSE with Fieldpine

To use an event source within a browser, simply create the EventSource object and register event listeners. Here is the code used in our eventsource test page

var Token = null;
var TestEventSource = null;

function StartSSE() {
    // Check if we are using Fieldpine Online, and get a token.
    if (Token === null) {
        // GetOpenApi() is a simple XHR/fetch() function in elink.js to query OpenApi
        GetOpenApi("/ActiveSession/AboutMe", function (oo) {
            Token = eLink_ExtractStr(oo, "data.UrlToken", "");
            StartSSE();
        });
        return;
    }

    // Ready to open
    let url = "/eventsource";

    // Do we have a Token, which implies RmSystem is required
    if (Token.length > 0) {
        url += sessionStorage.getItem("LginRmSuffix");
    }
    url += "?want=products,levels";

    // If restarting from a fixed point, add RVE to the query string.
    // Commented out for this examplw
    // url += "&rve=" + LastRveSeen;

    if (Token.length > 0) url += "&token=" + Token;

    if (TestEventSource !== null) TestEventSource.close();
    TestEventSourceRecv = new Array;
    TestEventSourceCount = 0;
    TestEventSource = new EventSource(url);
    TestEventSource.withCredentials = true;

    // Hook several handlers as the test source sends different message types

    // onmessage is called for each event that does not have an explicit name
    TestEventSource.onmessage = function (oEventData) {
        // Handle any messages that arrive without an event name
    };

    // Register a seperate listener for each event we want
    // If we dont register, the message is dropped
    TestEventSource.addEventListener('products', function (oEventData) {
        // Handle a product event
    });

    TestEventSource.addEventListener('customers', function (oEventData) {
        // Handle a customer event
    });
}

If you are consuming SSE feeds in a non browser program, such as an eCommerce web server, then several open source examples exist for various languages

To use an EventSource directly from sockets, use the following pseudo-code. When working outside the browser, you generally can add additional headers such as x-api-key safely.

sock = new socket();
sock.url = "/eventsource_1_2_3_4?want=products"  // Where 1,2,3,4 is your RmSystem
sock.HttpHeader.Last-Event-Id = "???"   // Add this header if you are restarting from a known point
sock.HttpHeader.X-Api-Key = Your-Api-Key
sock.HttpHeader.Accept = "text/event-stream"    // Set this to indicate we want event stream mode

sock.connect()
while (sock.read()) {
    // Process the data stream.
    // You will need assemble packets and decode the feed, see the W3C spec above
    // This is relatively simple, but beyond this brief example
}

Implementation Details

• The URL for eventsources is /eventsource_RmSystem?want=request&token=Auth Token&session=sessionid...
  • If you are using inhouse servers, not Fieldpine online, then you can omit the RmSystem component, but we recommend always using it as added protection.
  • Token= or sessionid= is required if you are connecting to Fieldpine Online. These both provide evidence of your authorisation to access the data.

Available "want" items

Specify which data you wish to receive with the want= query paramter. You must specify each item required, there is no "all" parameter. An "all" function would cause unncessary network traffic, and you need to register eventListeners for each data type anyway.

Keyword	Description
debug	A testing interface that returns a series of test packets. These packets should match normal data packets.
products	Sends a message whenever a product is created or editted.
customers	Sends a record whenever the customer definition is created or editted
accounts	Reserved
sales	Under dev/review. Will provide a realtime stream of sales being completed
levels	Sends a message when the stocklevel for an item changes, for any reason. more

Most events fire only when "direct attributes" are changed. You will receive updates if any direct attribute of the product is altered. This may mean the product appears the same to you, but isn't internally. Direct product attributes are things like unit price, description, color etc, but not things like orders being raised, or stock level changes

Packet Format

Each event packet will have the following standard format when using JSON. The response consists of an array of objects, however the array will almost always have one element, but uses an array for future expansion. Your use should always check for multiple elements in the array.

{
 "type": [
   ... type specific data ...
]
}

For example, a products packet might appear as follows

{ "Products": [
    {"Pid": 6017, "Physkey": "KEPJ28BER9WEXXY45J", "Publish": 0, "Description": "**ASCANI 7139 SHY/GUN CPG", "RVE": 20210727.091105994}
]}