Microfeed

Specification

Henrik Hedberg
Innologies
<henrik.hedberg (@) innologies.fi>

Version 0.1 (20090628)

Note! This specification is out-dated. See the reference implementation for the changed details.

The permanent URL of the latest version of this document is http://microfeed.org/specification/.

Contents

1. Introduction
2. Architecture
3. Data Model
4. Predefined Properties
5. Predefined Item Types
6. Predefined Feeds
7. File System Usage
8. D-Bus Messaging

1. Introduction

The presumption is that there is a web site or sites providing feed-type information and an application fetching that information with Microfeed.

2. Architecture

The microfeed architecture is built on the following concepts:

• A provider provides access to a specific web site. There are usually one provider for each web site or for each type of an information provider.
• A publisher publishes the feed-type information. A publisher is an instance of a provider configured with user-specific data, such as credientals. There are at least one publisher for each provider that is used by an application. However, there can be multiple publishers, if a provider supports multiple web sites or an application wants to login into a single web site with multiple credientals.
• A feed represents a data source in a web site. It consists of messages, contacts, or similar web site specific items. Also publisher configuration is a feed.
• An item is one message, contact or similar in a feed. It consists of unique identifier, timestamp, Microfeed-specific status, and varying number of properties embodying the content of an item.
• A subscriber is an application that subscribes one ore more feeds from one or more publishers (and thus from one or more providers).

These elements are depicted in the Figure 1. After a subscriber has subscribed a feed from a publisher (which is an instance of a provider), the publisher publishes items, i.e. sends item data to the subscriber.

Figure 1. Microfeed architecture.

3. Data Model

The following identifiers are used to distinguish various elements in Microfeed:

• Provider identifier is an unique string identifying a provider in a system. Only letters a-z and A-Z, numbers 0-9 as well as dot (.) and underscore (_) characters are allowed in a provider identifier. A domain name owned by a developer or developing organization should be used as a prefix in provider identifier to make sure that no two identifiers are the same. Example provider identifier is org.microfeed.provider.TestProvider.
• Publisher identifier is an unique string identifying a publisher in a system. It consists of two parts separated with hyphen (-). The first part is publisher part identifying a publisher within a provider. Only letters a-z and A-Z, numbers 0-9 as well as dot (.) and underscore (_) characters are allowed in publisher part. The other part is provider part identifying a provider in a system. Provider part must be the provider identifier of an existing provider. Example publisher identifier is TestPublisher-org.microfeed.provider.TestProvider.
• Feed URI is an unique string identifying a feed within a publisher. The format of the string conforms to universal resource identifier (URI) except that the unreserved characters must not be percent-encoded, i.e. two feed URIs must be comparable without any additional decoding procedures. A subscriber must handle a feed URI only as an unique identifier, because the actual resource may not exist. However, a provider may use a feed URI that directly identifies an existing resource of an external web site and utilize that information internally. An example feed URI is http://microfeed.org/Feed/Overview.
• Item UID is an unique string identifying an item within a feed. It may contain any valid character, but it is suggested to use only the same letters than in provider identifiers.

The item consists of the following data elements in addition to the unique identifier:

• Timestamp represents the time an item has been created (or modified if an external web site uses that semantic). It is an integer containing Unix/POSIX time, i.e. number of seconds elapsed since midnight of January 1, 1970 (UTC).
• Status is Microfeed specific information containing the following flags:

  Value	Name	Description
  1	Active	An item is recently fetched from an external web site and thus up-to-date. If the flag is unset an item is archived meaning that it is still in a provider's internal cache but not necessarily in an external web site's feed.
  2	Unread	An end-user has not read an item yet. A subscriber should unset this flag with the ReadItems method after an end-user has read an item.
  128	-	Reserved for implementation purposes.

  The status value is the resulting integer of bitwise OR operation of the flags. For forward compatibility, a subscriber must ignore all flags it does not support.
• Properties contain the actual data of an item. An item may have zero or more properties. A property consists of the following two parts:

  Part	Description
  Name	An unique string identifier of a property within an item. Only letters a-z and A-Z, numbers 0-9 as well as dot (.) and underscore (_) characters are allowed in a property name.
  Value	A string containing the actual data. Also other types, such as integers, are represented in string-form.

  There are predefined properties and a provider may also introduce its own properties (see Section 4). Example property is content.image (name) with value http://microfeed.org/microfeed_architecture.png.

In addition, an item may contain binary data. That feature is usually used only in the http://microfeed.org/Feed/Images feed (see Section 6).

4. Predefined Properties

The predefined properties provides a common semantic for known item properties. Those are divided into various categories to ease reading.

• Content represents the actual information of the item and has the following predefined properties:

  Name	Description
  content.image	The item UID of an image in the http://microfeed.org/Feed/Images feed. The image is either the actual content of the message or an icon representing the message. For example, images.abcdef.
  content.text	A string embodying the actual content of the message. For example, Microfeed architecture really works!
  content.title	A string embodying the title of the message. A title should be less than 140 characters. For example, Microfeed architecture.
  content.url	An URL specifying the location of the item in an external web site. A subscriber may use this URL to open the item in a web browser. For example, http://microfeed.org/specification/.

• User represents typically the sender of an item or a contact and has the following predefined properties:

  Name	Description
  user.image	The item UID of an image in the http://microfeed.org/Feed/Images feed. The image represents the user. For example, avatars.hhedberg.
  user.nick	For example, hhedberg.
  user.feed	for example, http://www.external.site/user/hhedberg.
  user.url	An URL specifying an external web site of an user. A subscriber may use this URL to open the site in a web browser. For example, http:/www.henrikhedberg.net/.

• Location represents the geographical location where an item has been sent or an user currently is. It has the following predefined properties:

  Name	Description
  location.text	For example, Somewhere, Earth.
  location.latitude	A decimal number (as a string) containing the latitude of the location of an user. For example, 65.05.
  location.longitude	A decimal number (as a string) containing the longitude of the location of an user. For example, 25.47.
  location.altitude	A decimal number (as a string) containing the altitude of the location of an user in metres. For example, 15.

• Comments are used by the providers that support commenting (if the actual site supports that). It has the following predefined properties:

  Name	Description
  comments.count	A non-negative integer embodying the number of comments refererring to the item. If an item (type) does not support comments this property must be omitted. For example, 3.
  comments.feed	A feed URI identifying a feed that contains the item and comments referring to it. If an item (type) does not support comments this property must be omitted. For example, http://www.external.site/feed/123456.

• Referred is representing the opposite direction of comments. It is used when the item is a comment of another item, and thus referring to it. The following predefined properties are available:

  Name	Description
  referred.item	The UID of the item this item is referring.
  referred.feed	The URI of the feed of which the referred item is part.

• Group represent somekind of grouping of items, such as channels in some micro-blogging services. It has the following predefined properties:

  Name	Description
  group.name	A string representing the name of a group where the item is originally published. This property must be used only in conjunction with the group.feed property. For example, #Microfeed.
  group.feed	A feed URI identifying a group where the item is originally published if it is different than the feed URI of a sender (user.feed). If the current feed is the original feed of the item, i.e. the group.feed and the URI of the feed containing the item are equal, this property must be omitted. If an external web site supports the special type of feeds called channels this property is typically used in the overview feed to identify a channel where the item has been published. For example, http://www.external.site/channel/Microfeed.

• A settings value has the following predefined properties:

  Name	Description
  setting.length	A maximum length of the value of the setting in characters. For example, 10.
  setting.text	An user-readable string describing the setting. For example, Update period.
  setting.type	A type of the value of the setting. A type must be one of the following alternatives: string, integer, non-negative integer, or positive integer. For example, non-negative integer.
  setting.unit	An user-readable string describing the unit of the setting. For example, seconds.
  setting.value	The actual value of the setting as a string. For example, 300.

• A feed metadata has the following predefined properties:

  Name	Description
  feed.name	The name of a feed that can be displayed to an end-user.
  feed.permissions	The concatenated string of the following values: :add (it is possible to add new items into the feed).

• A item metadata has the following predefined properties:

  Name	Description
  item.permissions	The concatenated string of the following values: :modify (it is possible to modify the properties of the item), :remove (it is possible to remove the item), and :reply (it is possible to reply to the item).

5. Predefined Item Types

The generic item types are message, image, contact information, and settings value. Each of these types has its own set of predefined properties and semantics for the UID value.

• A message is a single piece of information received from the external service. The UID of a message item is only unique identifier for a subscriber, but it may embody internal meaning for a publisher. A message consists of at least the following predefined properties: content.*, user.*, location.* (if supported by the external site), comments.* (if supported by the external site), referred.* (if supported by the external site), group.* (if supported by the external site), and item.*.
• An image contains the binary data of an image. The UID of an image is an unique identifier that is referred by the user.image and content.image properties. The UID may be the real URL of the image in an external site, but that must not be expected. An image does not have any predefined properties.
• A contact represents an user in an external site. The UID of a contact is the nick of the user. A contact typically has the following predefined properties: user.* (except user.nick, since it is already available as the UID of an item), location.* (if supported by the external site), and item.*.
• A setting stores the configuration of a publisher. The UID of a setting is an internal name of it. If the UID starts with a dot (.), it should not be displayed to an end-user. The following predefined properties must be available in a setting item: setting.* and item.*.
• A feed metadata contains the metainformation of an feed. In addition, it may containt the default metainformation of the items belonging in the feed. The UID of a feed metadata item is always http://microfeed.org/Metadata/Feed. The following predefined properties may be available in a feed metadata item: feed.* and item.*.

6. Predefined Feeds

A publisher should support the following feeds:

• http://microfeed.org/Feed/Overview must contain the summary of the most recent activity in the external site. Typically this contains the status messages sent by the end-user's contacts. The feed consists of message items.
• http://microfeed.org/Feed/Images must contain all the images that are referred in some other feed by using the content.image or user.image property. The feed consists of image items.
• http://microfeed.org/Feed/Contacts must contain the contacts of an user, if the external web site supports that. A single item should have the predefined properties of a contact.
• http://microfeed.org/Feed/Sent must contain the items an user has sent, if the external web site supports that. The feed consists of message items.
• http://microfeed.org/Feed/Settings must contain the configuration of a publisher. The feed consists of setting items.

If a provider does not support a predefined feed it must immediately respond to subscribe method with NoSuchFeed error reply (see Section 8.3).

In addition to predefined feeds, a publisher may support other feeds. For example, while the summary of messages of an external web site is published in the http://microfeed.org/Feed/Overview feed the messages of a single user called hhedberg may reside in the http://www.external.site/user/hhedberg feed.

7. File System Usage

A provider must create several files into a file system. Thus, a subscriber is able to find providers and publishers by iterating the well-known directories in a file system.

7.1 Provider File

A provider must create a provider file into one of the following locations:

• A directory specified by the environment variable MICROFEED_PROVIDERS_DIRECTORY if the variable is set.
• A directory .microfeed/providers under the user's home directory (specified by the environment variable HOME).
• The directory /usr/local/lib/microfeed/providers.
• The directory /usr/lib/microfeed/providers.

Directories must be iterated in that order. The provider file should be created into /usr/lib/microfeed/providers (or /usr/local/lib/microfeed/providers) when a new provider is installed into a system.

The provider file must contain the following lines separated with line-feed (ASCII value 0x10):

Line number	Description
1	An user readable name of a provider.
2-n	The predefined and special feeds that are supported by the provider. The line is divided into two parts that are separated with a space ( ): the feed URI and the user-friendly name of the feed.

7.2 Publisher File or Directory

A provider must create one publisher file or directory for each of its publishers into one of the following locations:

• A directory specified by the environment variable MICROFEED_PUBLISHERS_DIRECTORY if the variable is set.
• A directory .microfeed/publishers under the user's home directory (specified by the environment variable HOME).
• The directory /usr/local/lib/microfeed/publishers.
• The directory /usr/lib/microfeed/publishers.

Directories must be iterated in that order.

The contents of the publisher file or directory is not specified. A publisher may store its internal cache into the file or directory.

8. D-Bus Messaging

The messaging between a subscriber and a publisher (actually a provider) is based on D-Bus.

The D-Bus service name of a publisher is the provider part of the publisher identifier. The D-Bus object path of a publisher is /org/microfeed/publisher/name, where the name is the publisher part of the publisher identifier. For example, the publisher that has identifier ExamplePublisher-org.microfeed.Provider.ExampleProvider, registers a service URI org.microfeed.Provider.ExampleProvider, and is using object path /org/microfeed/publisher/ExamplePublisher.

8.1 Publisher Methods

A Microfeed publisher must implement an D-Bus interface org.microfeed.Publisher with the following methods:

• AddItem method is used to add a new item into a feed. A subscriber must give a temporary UID for the item, which is used in an error signal if the operation fails. At first, publisher must only check the validity of the request and send back a reply. A publisher must silently ignore properties it does not support. After that actual network activity may start. If any errors occur an error signal must be emitted. Thus, if an addition succeeds no specific notification is sent. However, the new item should appear in a feed of an external web site and eventually in a subscribed feed causing emission of the ItemAdded signal. The AddItem method has the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Temporary item UID
  0 - n times the following pairs:
  STRING	Property name
  STRING	Property value

• CreatePublisher method is used to create a new publisher, i.e. a new instance of a provider with specific settings, such as credientials. Altough a subscriber sends the method call to a non-existing publisher, thus specifiying an identifier of the publisher to be created, it is handled by a provider. A provider must instantiate a new publisher and initialize it with default values. No network activity should occur. The CreatePublisher method does not have any parameters.
• DestroyPublisher method is used to destroy an existing publisher. A publisher must emit the NoSuchPublisher signal to all subscribers before removing them from internal book keeping. The DestroyPublisher method does not have any parameters.
• ModifyItem method is used to modify values of properties of an existing item in a feed. At first, a publisher must only check the validity of the request and send back a reply. A publisher must silently ignore properties it does not support. After that actual network activity may start. If any errors occur an error signal must be emitted. Thus, if a modification succeeds no specific notification is sent. However, the modified item should appear in a feed of an external we site and eventually in a subscribed feed causing emission of the ItemChanged signal. The ModifyItem method has the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Item UID
  0 - n times the following pairs:
  STRING	Property name
  STRING	Property value

• Ping is used to ensure that a publisher is still alive. A publisher must respond to the Ping method as soon as possible. If a subscriber does not receive reply from a publisher quick enough, the subscriber should try to resubscribe all feeds it has subscribed. That action should restart the publisher if it is, for example, crashed. The Ping method does not have any parameters.
• ReadItems method is used to change the unread status of existing items. A publisher must unset the unread flag of items in its internal book keeping and emit the ItemStatusChanged signals with updated statuses. The ReadItems method has the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Start item UID (inclusive) or empty string if the first item in the feed (inclusive)
  STRING	End item UID (inclusive) or empty string if the last item in the feed (inclusive)

• SendItemData method is used to request optional binary data of an existing item. At first, a publisher must only check the validity of the request and send back a reply. After that actual network activity may start. When the item data has been received, a publisher must send the ItemData signal. If any errors occur an error signal must be emitted. A publisher should cache item data to minimize network trafic. The SendItemData method must have the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Item UID

• RepublishItems method is used to request a republising of existing items. Usually a subsciber wants to do this when presenting a new feed for an end-user. A publisher should not limit the maximum number of items to republish. Thus, a subscriber must give a reasonable value for the parameter, because a publisher may have large item cache and republishing that may waste system resources. The RepublishItems method has the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Start item UID (exlusive) or empty string if the first item in the feed (inclusive)
  STRING	End item UID (exlusive) or empty string if the last item in the feed (inclusive)
  UINT16	Maximum number of items to republish

• SubscribeFeed method is used to subscribe a feed. At first, a publisher must only check the validity of the request and send back a reply. After that the publisher must send a feed metadata item. It may be possible to subscribe a non-existing feed, because the network activity begins later when the feed is updated by the publisher.

  Type	Description
  STRING	Feed URI

• UnsubscribeFeed method is used to unsubscribe a subscribed feed.

  Type	Description
  STRING	Feed URI

• UpdateFeed method is used to force update of an feed. Typically publisher (provider) is updating subscribed feeds periodically, so this method is rarely needed.

  Type	Description
  STRING	Feed URI

• MarkItem method is used to mark an item. A publisher (provider) must store marked items in its local cache, and if the external site supports similar functionality, also mark the item that way.

  Type	Description
  STRING	Feed URI
  STRING	Item UID

• UnmarkItem method is used to remove existing marking from an item.

  Type	Description
  STRING	Feed URI
  STRING	Item UID

8.2 Publisher Signals

A Microfeed publisher must send a signal with a D-Bus interface org.microfeed.Publisher when broadcasting information to all subscribers and with a D-Bus interface org.microfeed.Publisher.ToDestination when sending information to a specific subscriber.

• FeedUpdateStarted signal must be emitted when a publisher starts to update a specific feed. An application may indicate the situation somehow to an user, for example, by disabling an update button. Usually varying number of ItemAdded, ItemUpdated, and ItemRemoved signals are emitted after this signal. However, those signals may be emitted even if a publisher has not emitted this signal. The FeedUpdateStarted signal must have the following parameters:

  Type	Description
  STRING	Feed URI

• FeedUpdateEnded signal must be emitted when a publisher ends updating a specific feed. An application may recall the actions made after receiving a FeedUpdateStarted signal. The FeedUpdateStated signal must have the following paramters:

  Type	Description
  STRING	Feed URI

• FeedRepublishingStarted signal must be emitted after a subscriber has requested republishing of some items in a feed. A publisher must emit the signal using the D-Bus interface org.microfeed.Publisher.ToDestination with the destination parameter set in the D-Bus message, since other subscribers are not interested in that information. The FeedRepublishingStarted signal must have the following parameters:

  Type	Description
  STRING	Feed URI

• FeedRepublishingEnded signal must be emitted when republishing has ended. A publisher must emit the signal using the D-Bus interface org.microfeed.Publisher.ToDestination with the destination parameter set in the D-Bus message, since other subscribers are not interested in that information. The FeedRepublishingEnded signal must have the following parameters:

  Type	Description
  STRING	Feed URI

• ItemAdded signal must be emitted when a new item has been added into a feed. The ItemAdded signal has the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Item UID
  UINT64	Timestamp
  BYTE	Item status
  0 - n times the following pairs:
  STRING	Property name
  STRING	Property value

• ItemChanged signal must be emitted when a property or timestamp of an existing item has been changed in a feed. An existing item is an item that has been earlier added into a feed using the ItemAdded signal by a publisher whether a subscriber has received that signal or not. Thus, the parameters of the signal must contain all properties, not only changed ones. A provider should avoid changing the timestamp after an item has been added, because typically subscribers order items by timestamps and thus changing it reorders items. The ItemChanged signal has the same parameters as the ItemAdded signal:

  Type	Description
  STRING	Feed URI
  STRING	Item UID
  UINT64	Timestamp
  BYTE	Item status
  0 - n times the following pairs:
  STRING	Property name
  STRING	Property value

• ItemData signal must be emitted after a subscriber has requested it with the SendItemData method. A publisher must emit the signal using the D-Bus interface org.microfeed.Publisher.ToDestination with the destination parameter set in the D-Bus message, since other subscribers are not interested in the data. The ItemData signal must have the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Item UID
  ARRAY of BYTEs	The actual binary data

• ItemRemoved signal must be emitted when an existing item has been removed from a feed. It is not error to emit the ItemRemove signal for items that a subscriber has not known beforehand. An application may silently ignore those signals. The ItemRemoved signal has the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Item UID

• ItemRepublished signal must be emitted for each existing item that a subscriber has been requested. These signals must be emitted between FeedRepublishingStarted and FeedRepublishingEnded signals. A provider may cache recent items and republish only those without contacting to an external web site. The ItemRepublished signal has the same parameters as the ItemAdded signal:

  Type	Description
  STRING	Feed URI
  STRING	Item UID
  UINT64	Timestamp
  BYTE	Item status
  0 - n times the following pairs:
  STRING	Property name
  STRING	Property value

• ItemStatusChanged signal must be emitted when a status of an item has been changed. Note, that this status is specific to Microfeed and does not correspond to a potential status of an item in an external web site. The ItemStatusChanged signal must have the following parameters:

  Type	Description
  STRING	Feed URI
  STRING	Item UID
  BYTE	Item status

8.3 Publisher Errors

A Microfeed publisher must send a signal with a D-Bus interface org.microfeed.Publisher.Error when broadcasting an error message to all subscribers and with a D-Bus interface org.microfeed.Publisher.Error.ToDestination when sending an error message to a specific subscriber. In addition, A Microfeed publisher must use the same error message names with the D-Bus interface org.microfeed.Publisher.Error when returning an error as a reply to a method call.

A publisher should use the following error names if appropriate:

• UnknownError
• DBusMessageFailed
• ProviderClosedConnection
• InvalidPublisherIdentifier
• InvalidFeedUri
• InvalidArguments
• MissingProperty
• FeedNotSubscribed
• FeedAlreadySubscribed
• PublisherAlreadyExists
• ItemAlreadyExists
• NoSuchProvider
• NoSuchPublisher
• NoSuchFeed
• NoSuchItem
• NoSuchProperty
• NoItemData
• CannotModifyItem
• ConnectionFailed
• LoginFailed
• CommunicationFailed
• FromService

8.4 Subscriber Methods

A Microfeed subscriber must implement a D-Bus interface org.microfeed.Subscriber with the following method:

• Ping is used to ensure that a subscriber is still alive. A subscriber must respond to the Ping method as soon as possible. If a publisher does not receive reply from a subscriber quick enough, the publisher should unsubscribe all feed the subscriber has subscribed and remove the subscriber from its internal book keeping. The Ping method does not have any parameters.