SOFTWARE ARCHITECTURES
Chapter 4 – Push Notification Services: Google and Apple
Zachary Cleaver
Summary The goal of this paper is to define the structure of push notification systems (PNS), and specifically to analyze the architecture of Android’s PNS’s versus Apple’s as well as to analyze the advantages and limitations of each PNS. I first discuss the basics of a push notification system, looking at how it typically works and highlighting some key features. The next three sections cover Google’s Cloud to Device Messaging system, Google’s Cloud Messaging system, and Apple’s push system. Each section will give an overview of the systems prerequisites, discuss some advantages and disadvantages about its method for sending data, and finally will cover its architectural structure.
Push Notification Systems
General Structure Push notification systems are similar in design to client-server models. The basic structure of a push notification system is broken down into three parts: an application that will receive data/instructions, a third party service that will provide instructions or information for the app, and a service (Google Cloud Servers, Apple Push Notification Service, etc.) that handles the exchange of information between the two (see Figure 1).
Figure 1. High level architectural view. (Provided by Basavraj)
History Middleware is software above the level of the operating system but that provides reusable infrastructure services that many applications can share. Notifications systems are a kind of middleware. One of the earliest Notification system specifications was specified by Object Management Group in the early 1990s.
1
SOFTWARE ARCHITECTURES
Advantage The server that sends information to the application from the third party server acts as a middleman to communicate and control the rate at which data is exchanged. This allows for information to be “pushed” to a device without having to stress the device by keeping multiple applications running at once. Applications can remain off or idle on a device while the user receives a notification that new information or updates are available for a specific application. This greatly increases run time efficiency on a device by allowing the user to be in control of what apps are running while still being able to receive information from the third party servers.
Google’s Cloud to Device Messaging
Prerequisites In order for an Android device to receive messages from an application server, there are some basic requirements that must first be met when using the C2DM service (Basavraj) The device must be version 2.2 or higher A registered Google account must be present on the device The server that wishes to send data must be able to “talk” HTTP and connect to the C2DM server The application should have the proper permissions the receive messages from the C2DM service and be able to make an internet connection The developer of the application must register through the C2DM site to use the service
Data Sent The amount of data that could be sent by Google’s first notification system was limited to 1024 bytes. Google also restricted the number of continuous messages that could be sent, as well as the number of messages a server can send to a specific device. The maximum number of messages a service could send was 200,000 per day. A drawback of C2DM is that it does not guarantee the delivery of a message to the application and it does not guarantee that the order of multiple messages will be maintained (Basavraj). Bourdeaux elaborates on this and believes in is due to the “file-and-forget” nature of Google’s service. Because of this, developers would be wary to send a notification that their app has a new message rather than sending the message itself. However, Bourdeaux pointed out an improvement at the time versus Apple’s push notification—C2DM allowed the application to be woken up rather than taking Apple’s method of simply sending a notification. This may seem like a minor feature compared to today’s services, but this development helped improve the capabilities and standards for future push notification systems.
Main Sequence of Events There are five basic steps that C2DM follows according to Google’s documentation: 1. The third party service sends a message to the C2DM servers that it wishes to pass on to its application 2. The Google servers enqueues and stores the message in its database until it can be delivered if the device is currently offline 2
SOFTWARE ARCHITECTURES
3. The Google servers then pass along the third party service’s message once the recipient device comes online 4. The Android device’s system then broadcasts the message sent from the C2DM servers to the application via Intent broadcast1, checking for the proper permissions so that only the intended target application receives the message. This step “wakes up” the application, meaning the application does not need to be running in order to receive a message. 5. Finally, the application processes the message. C2DM does not specify how the message should be processed; rather, this decision is left to the third party service. This gives the application full freedom as to how the message should be processed, whether it is posting a notification, updating/synchronizing data in the background, or even displaying a custom user interface.
Architecture This section looks in detail at the structure of how the C2DM service works with the application and the third party service. There are three primary steps in this structure: setting up a connection with and enabling C2DM, sending a message, and receiving a message. Figure 2 illustrates a more in-depth look at C2DM’s architecture that expands on Figure 1.
1 Intent broadcasting is Android’s method for sending and receiving information between application states within a single app or between multiple applications. Intent broadcast uses key-value pairs to pass along information, essentially acting as a local communication system on the device. See http://developer.android.com/guide/ for more information. 3
SOFTWARE ARCHITECTURES
Figure 2. Lower level architectural view. (Provided by Basavraj)
C2DM Registration A third party service that wishes to use C2DM servers must first signup using a unique package name or app ID to define their application, as well as an email address that is responsible for this application. This is a relatively quick process for an application, and is only required once per application. The next step is registration. A registration intent is fired off from the application to the C2DM server upon its first use. This intent (com.google.android.c2dm.intent.REGISTER) contains information like the aforementioned app ID and email that the C2DM server will use to register the application. Upon a successful registration, the C2DM server broadcasts a registration intent that supplies that application with its own registration ID. The application must then send its registration ID to the third party service, which is then stored in the service’s database. This ID will be used to facilitate communication between the application and its service; the ID lasts until the application unregisters itself if it wishes to stop receiving messages from the C2DM server (in which case a notification will be returned to the application to alert the user of this event) or if Google refreshes the ID registration for the application.
Sending a Message For a third party service to send a message, it must first have a ClientLogin authorization token. This token “authorizes the application server to send messages to a particular Android application” (see Google documentation). While an application may have multiple registration IDs for each device/user that connects to the servers, only one authorization token is required for the app. The third party server then sends an HTTP Post request to Google’s servers, passing along the registration ID and the authorization token. There are a few more credentials that are passed along in the request: the payload data, a collapse key, and an optional parameter called delay_while_idle. The payload is a key-value pair that contains the message(s) being passed to the application. There is no limit to the number of key-value pairs that are allowed; however, there is a limit to message size (1024 bytes). The collapse key is a string that aggregates or collapses similar messages into a group that collect in the server if the target device is offline at the time the message was sent. This prevents too many messages from being sent to the device once it comes online by only sending the last message—it should be noted that the order of messages is not guaranteed, so the last message sent to the application may not be the last one that was stored in the queue. The delay_while_idle does exactly what its namesake implies—this tells the service whether to delay sending a message to the target device if the device is idle at the time. Once the device becomes active, the last message from each collapse key will then be sent.
4
SOFTWARE ARCHITECTURES
The C2DM service then verifies the credentials of the request through its authentication service, queues the message for delivery, sends the message to the target device, and finally removes the message from its queue after a successful delivery.
Receiving a Message Once the message has been received, the device’s system then extracts the key-value pairs from the payload and passes this data on to the application by broadcasting a receive intent. Finally, the data from each key is extracted by the application and processed. Since C2DM merely facilitates the exchange of information between an application and its server and does not worry about how to display the messages, personalization of display is left up to the discretion of the application.
Response Description
200 Includes body containing: id=[ID of sent message] Error=[error code] o QuotaExceeded — Too many messages sent by the sender. Retry after a while. o DeviceQuotaExceeded — Too many messages sent by the sender to a specific device. Retry after a while. o InvalidRegistration — Missing or bad registration_id. Sender should stop sending messages to this device. o NotRegistered — The registration_id is no longer valid, for example user has uninstalled the application or turned off notifications. Sender should stop sending messages to this device. o MessageTooBig — The payload of the message is too big, see the limitations. Reduce the size of the message. o MissingCollapseKey — Collapse key is required. Include collapse key in the request.
503 Indicates that the server is temporarily unavailable (i.e., because of timeouts, etc ). Sender must retry later, honoring any Retry-After header included in the response. Application servers must implement exponential back off. Senders that create problems risk being blacklisted.
401 Indicates that the ClientLogin AUTH_TOKEN used to validate the sender is invalid.
Figure 3. Listed above is a chart of the possible response codes a service may receive due to message failure. (provided by Google documentation)
Google Cloud Messaging
Architecture GCM’s architecture is very similar in design to C2DM’s. The basic three step formula of enabling the GCM service, sending a message to a device, and then processing the message that was received is still followed. Minor improvements are made to increase efficiency through authentication services, and to allow for a smoother delivery of messages between the 5
SOFTWARE ARCHITECTURES application services and the devices receiving data. Figure 4 shows a more in-depth and detailed description of GCM’s workflow. GCM is an improvement on Google’s C2DM servers that came about in June 2012. The C2DM service was deprecated at this time, and applications have been moving towards this newer, more efficient service. Improvements in data capacity and message delivery are a couple key improvements on Google’s recent service. The overall architecture of Google’s service is the same (enable the Google service on the target device, send a message, and receive a message), but advancements have been made that make GCM a notable improvement over C2DM.
Figure 4. GCM architectural overview. (Provided by Basavraj)
Notable Differences Basavraj highlights several key differences that GCM implements versus C2DM: Simple API Key – The API key is used in the header that communicates with the GCM server. This unique key is used during authentication checks Sender ID – C2DM relied on an email address register the app and communicate with the third party service. GCM uses the project number acquired from the API console JSON format –This format is now supported, as well as allowing support for the previous plain text style when sending messages 6
SOFTWARE ARCHITECTURES
Multiple senders – Using a single registration ID, and application can receive a message from multiple parties Time-to-live messages (Expiry) – The time of expiration for a message to be removed from the Google server after not sending can be set between 0 and 4 weeks.
Advancements From C2DM Compared the C2DM’s payload limit of 1024, GCM has quadrupled the amount up to four kilobytes of data that can be pushed. Another advancement GCM has come up with since C2DM is the accessibility provided for users across multiple devices. If a user has, say, a smart phone and a tablet that have installed the same app and are synced through their Google accounts, the user will receive a notification on both devices. For example, if a user receives a reminder via their calendar application, the reminder will be sent to both devices. As is expected, checking the notification and clearing it on one device clears it on all. Also, if the message has not been sent to one of the devices but has already been dismissed or handled on the other, the duplicate message that is still queued in the Google server will be removed. This is achieved by way of Google’s notification_key parameter. This key is the “token that GCM uses to fan out notifications to all devices whose registration IDs are associated with the key” (Bourdeaux).The key is mapped to all registration IDs for a user across their devices; so instead of sending out messages one at a time, the server can send the message to the notification_key which will then forward the message to all of the user’s registration IDs. However, only 10 different keys can be included in the notification_key (though who would have over 10 devices they need to sync?).
Connection Servers GCM provides the option of using one of two connection servers: HTTP and CCS (XMPP). These two servers can either be used alone or together depending on the application and its goals. Basavraj lists three major ways in which the servers differ from one another according to GCM’s documentation: Upstream/Downstream messages o HTTP: Can only send messages downstream (from the cloud to a device) o CCS: Upstream (sending messages from a device to the cloud) and downstream Asynchronous messaging o HTTP: The 3rd-party server sends messages as HTTP POST requests and waits for a response. This mechanism is synchronous o CCS: The 3rd-party server connects to the Google infrastructure using a persistent XMPP connection and sends/receives messages to/from all their devices. CCS sends acknowledgment /failure notifications (in the form of special ACK and NACK JSON-encoded XMPP messages) asynchronously. JSON o GCM: Messages using JSON are sent using an HTTP POST 7
SOFTWARE ARCHITECTURES
o CCS: Messages using JSON are encapsulated in XMPP messages
Cloud Connection Server GCM’s Cloud Connection Server (CCS) is an “XMPP endpoint that provides a persistent, asynchronous, bidirectional connection to Google servers” (see Google documentation). This leaves two streams open (one from device-to-cloud, the other cloud-to-device) that allows for data to be exchanged in near real time between a third party server and its GCM connected applications. HTTP can be used in conjunction with CCS to continue sending message to the GCM server.
Benefits XMPP is asynchronous in design, allowing for more messages to be sent while using fewer resources. Communication is bidirectional—not only can your server send messages to the device, but the device can send messages back to your server. The device can send messages back using the same connection used for receiving, thereby improving battery life.
Flow Cycle Each message sent to the CCS receives one of two responses: acknowledged or not acknowledged (or ACK/NACK). If a message has not received either option, it is considered pending. Once the pending message count reaches 100, messages should no longer be sent by the third party server until the CCS acknowledges some of the previously sent messages. Figure5 illustrates an example situation of a CCS connection and a case in which the third party server must wait before sending any more new messages.
8
SOFTWARE ARCHITECTURES
Figure 5. Message/ack flow. (see Google documentation)
ACKing Messages Just as the third party server will stop sending messages if too many are pending, the CCS will stop sending messages if there are too many messages that have not been acknowledged in order to avoid overloading the third party server with responses. To mitigate this problem, the third party server should send the ACK (acknowledgement) responses as soon as possible to keep a consistent flow of messages coming in. However, there is no limitation on the number of pending ACK’s that may be present at any given time. This is why it is imperative that ACK responses be handled quickly to avoid blocking the delivery of new upstream messages that are sent. ACKs are only valid during the time of a single connection. Because of this, the third party server needs to be able wait for the CCS to resend the upstream message again if it has not been “ACKed” before the connection is closed. Then, all ACK/NACK responses that were not received from the CCS for each pending message before the connection was closed should be sent again (see Google documentation).
Apple Push Notification System
Basic Architecture Apple’s Push Notification System follows the template of a push notification service. The application must register with the iOS for push notifications. Confirmation of this request returns a “device token” or an identifier that signifies that the target device will be receiving information. This token can be thought of as an address that tells the service where notifications are to be delivered to. The device token is passed on to the third party server. Whenever a notification or message needs to be passed along to the application, the server sends this data to the APNS along with the device token that it stored earlier for the target device. The APNS can then use this token to pass along the message/notification from the third party service to the device’s application.
9
SOFTWARE ARCHITECTURES
Figure 6. Basic workflow of APNS. (Table provided by Hafizji)
Payload Apple’s payload comes in some distinct ways compared to Google’s notification system. The structure is the same in that the payload is represented by a key-value pair. However, Apple’s maximum size allowed for a payload is only 256 bytes—a small amount compared to Google’s four kilobytes. Similarly to Google, Apple also uses JSON objects to exchange information between applications and servers. However, Apple allows for three different styles of notifications to be delivered: 1) A standard alert message 2) A badge icon over the app icon with a number indicating the number of new notifications 3) A sound to play Below is an example of a JSON object. The “alert” and “sound” tags are the key-value pairs of the “aps” object, which is referred to as a dictionary type (this type is similar to the map 10
SOFTWARE ARCHITECTURES function seen in C++). Once this object is received by the application, the text “Hello, world!” will be displayed as an alert view and the corresponding sound, bing, will be played. { "aps": { "alert": "Hello, world!", "sound": "bing" } }
JSON object: aps is a dictionary type with fields “alert” and “sound”. The specified alert is played/shown if a notification is delivered to an application that is not running at the time of arrival. Apple’s documentation warns that the delivery of notification is a “best effort”, meaning the delivery of the notification is not guaranteed to carry through. The server sending the payload can specify a payload value that differentiates from the Apple-reserved aps namespace, but the custom data must be sent using JSON. Figure 7 lists a more detailed explanation of what is expected for each type of key-value pair.
11
SOFTWARE ARCHITECTURES
Key Value type Comment
alert string or If this property is included, the system displays a standard alert. You may dictionary specify a string as the value of alert or a dictionary as its value. If you specify a string, it becomes the message text of an alert with two buttons: Close and View. If the user taps View, the application is launched.
badge number The number to display as the badge of the application icon. If this property is absent, the badge is not changed.
sound string The name of a sound file in the application bundle. The sound in this file is played as an alert. If the sound file doesn’t exist or default is specified as the value, the default alert sound is played. The audio must be in one of the audio data formats that are compatible with system sounds.
content- number Provide this key with a value of 1 to indicate that new content is available. This available is used to support Newsstand apps and background content downloads. Newsstand apps are guaranteed to be able to receive at least one push with this key per 24-hour window.
Figure 7. Different key-value pairs. (Table provided by Hafizji)
Quality of Service This component of APNS is a store-and-forward function that helps the efficiency of message delivery. It is like Google’s GCM service in that it will store a notification if the target device is currently offline and send it once the device is back online; however, APNS only stores one notification for an application for each device. If another message is sent to the same app on the same device before the device has come back online, the older message will be discarded. Furthermore, if the device remains offline for too long, APNS will discard any notifications that were being stored (Ashok).
Binary Representation of Data Framework APNS uses a TCP socket connection to stream notifications. The framework for a notification is broken down into five distinct parts: the device token, the payload, a notification reminder, an expiration date, and a byte for priority. The chart below, provided by Apple’s documentation, gives a look into these attributes.
12
SOFTWARE ARCHITECTURES
Item Item Name Length Data ID
1 Device token 32 bytes The device token in binary form, as was registered by the device.
2 Payload less than or equal to The JSON-formatted payload that contains the 256 bytes messages/notifications in key-value pairs.
3 Notification 4 bytes An arbitrary, opaque value that identifies this notification. identifier This identifier is used for reporting errors to your server.
4 Expiration 4 bytes A UNIX epoch date expressed in seconds (UTC) that date identifies when the notification is no longer valid and can be discarded. Specify zero to indicate that the notification expires immediately and that APNs should not store the notification at all.
5 Priority 1 byte . 10 The push message is sent immediately. . 5 The push message is sent at a time that conserves power on the device receiving it.
Figure 8. Binary framework piece-by-piece
Conclusion This final table displays a comparison of a few popular push notification services. This chart is a little outdated since it does not include the parameters of GCM, but it is still relevant in displaying the different types of services available. There are numerous third party services that are also available for use. Common features in each service are highlighted, such as their payload limit, expiration time, and quota of messages that are allowed to be sent. I would encourage future developers who wish to create applications and the service that accompanies them to use this paper as reference to the Google and Apple systems. There is much more information that was not covered in this paper, and individual research will help provide the reader with the information they need to make an informed decision.
13
SOFTWARE ARCHITECTURES
Platform → Android iOS Blackberry Windows Phone 7 Parameters ↓ Name C2DM APN BPS MPNS Registration Yes No Yes No (For push services) OS Version 2.2+ (Froyo) iOS 3.0 + 4.2+ 7.x Mac X v7.0 Payload Size 1 KB 256 bytes 8 KB 3 KB (+ 1 KB Header) Payload Format String JSON XML XML Expiry Time No Controllable Controllable up to No 30 days Quota (per day) 200,000 NA 100,000 Unlimited (500 for unauthenticated web services) Queue/Channels support Yes Yes Configurable Channels (4 per app) (1 per app) 1 per app, 30 per device SSL/TLS Support Yes Yes Yes Yes Notification UI App has to In-Built In-Built Toast, Tile or Raw provide Free/Paid Free Free Free Free Delivery guarantee No No No No Delivery status No No Yes No
Figure 9. Comparison of different push notification services, provided by Basavraj.
References "Android Cloud to Device Messaging Framework." Google Developers: Android. Google, 26 Sep 2012. Web. 29 Apr 2014.
14
SOFTWARE ARCHITECTURES
15