So, you've heard about Google's free software release of its Gadgets server, and the new "Open Social API". And gosh, wouldn't it be nice if you could provide this technology to your users with your favorite free software Content Management System (CMS)? Since the documentation that comes bundled with Google's release will probably give you simultaneous whiplash and vertigo (with a large side of frustration), here's a breakdown of the problem so you'll know what you're up against, how to go about solving the problem, and plenty of free software resources to help you get there.
What you are expected to know
I'm going to assume that you are familiar (hopefully experienced) with programming in some programming language; that you know the basics of HTTP; and that you are familiar enough with your favorite Content Management System (CMS) to be able to create extensions for it (this is a tutorial for developers, not end users).
I am not going to assume you are using any particular programming language or CMS. There are plenty of other platforms out there, and I'm not about to research them all. So the important thing, is simply that you know your own CMS: you will need to be able to provide HTTP and RPC callbacks to match the given specifications.
About Google Gadgets and Open Social
Not sure what "Open Social" or "Google Gadgets" are?
Open Social adds to this mix a unified database query interface designed for "social networking" applications—that is to say, applications that keep track of "friends" or "fans" or "contacts" or other representations of relationships between people and also store various kinds of contact and personal information.
For more general information about the technologies, see:
Fitting your CMS into the Open Social picture
We are not going to discuss how to make an Open Social application, but you need to know who the client is if you're going to build a server.
The important point is that the Open Social application doesn't care about this: it just needs to find the "opensocial" and "gadgets" namespaces, and call standard functions and methods within them. It is this insulation from the environment, of course, that makes Open Social applications highly portable.
This might be a good point to mention that you are not going to create a "gadgets server", though you will need to use one.
But in order to be useful, it's going to have to look up information over the internet. It will do this by making use of web API calls, conforming to either of two possible Google-defined protocols: the "Open Social ReSTful Protocol" and the "Open Social RPC Protocol" (technically, calls made over the internet are "protocols", but the word "API" is also used for them here and in Google's documentation—they are the same thing).
It is mainly these two web protocols that you will have to create in order to make your CMS into an "Open Social Container".
Why two APIs? Well, that's got to be design by committee, in my opinion. Basically, they didn't want to restrict the application developer, so Google decided that containers must provide both. To make matters just a little more fun, messages transferred via either protocol must be available in three standard data formats: "JSON", "XML", and "AtomPub".
Also, of course, in order to be useful, you will have to map your internal object model onto a schema that makes sense to Open Social. It's probable that you aren't far off now (the principle object in Open Social is "people" which probably maps to your "users" or "members" objects), but some adaptation is inevitable. Because there are essentially six flavors of callback ("ReSTful" and "RPC" for each of "JSON", "XML", and "AtomPub"), you will clearly want to structure your program as an internal object model, with pluggable managers to provide these six different "views" of the same objects.
Hopefully, this brief introduction will reduce the vertigo. Now we'll get down to specifics.
Steps to create your Open Social container environment
I'll break the problem down into the following six steps:
1) Implement an object model that produces the correct schema for People, Activities, etc, as implied by the Open Social data model
2) Implement a Restful API (conforming to the spec), to access this information
3) Repeat for the RPC model
4) Install Shindig to provide a Gadgets server
STEP 1: Create the schema and an object model to support it
The Open Social API doesn't define an actual schema or object model that you must use. However, without one, it's hard to make sense of the concept. So I will propose a design that should make it easy to implement the API you need. You will need to adapt your CMS to provide access to the same information, so this will hopefully help you, even if you don't follow the model to the letter.
Figure 1: Entity-Relationship diagram of the Person entity. This is a breakdown of the possible contents of a Person record. Note that despite a large number of possible fields, only two are actually required
The most important entity in the schema is the "Person". Open Social is (unsurprisingly, I hope) all about people: their needs, interests, and activities. It can extend to their professional lives, personal lives, or even their sex lives, depending on the purpose of your site. Only two "fields" are absolutely required: "id" and "displayName", although there wouldn't be much purpose in using Open Social if that's all you want (this is the niche for the simpler Open ID spec).
After this, there are 21 optional (but recommended) fields, and another 38 more specialized "other" optional fields. Some of these fields can be plural. For those, the field will need to return a data structure which allows the individual elements to be read. For the singular fields, just the single element will need to be returned.
Most of the elements are just string data, but a few are themselves data entities, as illustrated in figure 1.
Full names can simply be strings provided by the "formatted" field of the "name" element, and I encourage you to do just this, unless your database already contains self-marked name elements.
Because of cultural biases, it's easy to underestimate the difficulty of parsing people's names. Many Americans are surprised to find that "Sasha" is a boy's name or that Takahashi Rumiko's family name is "Takahashi". Even lexing the words into individual names is hard as illustrated by names like "De Broglie" or "Van Rossum". So don't attempt to break names down automatically, or you'll most likely just annoy your users.
A similar situation applies to addresses, and Open Social provides a "formatted" field for both entities.
Figure 2: Person along with the other Open Social entities
Figure 2 shows the attributes of the other entities you will need to keep track of. A group is simply a tag associated with a collection of person entities, and AppData is essentially a "cookie" system: applications can store name-associated data and retrieve it through this mechanism. Messages are optional, but if you support them, they can be used to represent private messages, posts, or emails.
The most nebulous entity here is probably "Activity". It may be more instructive to think of some examples, such as: a photograph or song uploaded by the user or an article written by the user.
STEP 2: The ReSTful API
"ReST" stands for Representational State Transfer, a fairly vague and expansive term which refers to the "stateless" way in which you are accustomed to "simple" HTML web sites working: each request contains all the information needed to communicate to the server, which then returns a response based solely on that and information in a database on the server. There is no concept of "logging in" and "logging out", nor information retained between one request and the next. Authentication is repeated (albeit automatically) on every request.
This system is somewhat inefficient in its use of bandwidth, but is extremely robust, and is credited by many as one of the main design successes of the World Wide Web.
Although it is not strictly limited to such an approach, a ReSTful API can be usefully thought of as a set of URLs which respond with various kinds of information when the clients sends an HTTP GET to the URL, and this is indeed how the Open Social ReSTful API is intended to work.
The open social service types, as used in the Open Social ReSTful API
The Open Social ReSTful API consists of a set of "services" provided by specific URLs. Each one is expected to be capable of responding to requests with data in all of three different representations: JSON, Atom Pub XML, and a custom Open Social XML (a parameter in each HTTP request determines which needs to be provided). These formats are completely redundant. Three are provided in order to make the API easier to use for applications with different requirements.
The Atom Publication format ("Atom Pub") is not quite as widely supported, but similar implementation information is available from the Atom Wiki, and you will most likely find what you need there. Alternatively, you could adapt a generic XML library, as Atom Pub is an XML format.
Finally, you will probably have no choice but to use a generic XML library to read the custom Open Social XML format. An XSD schema for the format is provided as part of the ReSTful API specification, along with the rest of the details you will need to actually implement the interface.
The request sent by the container to the container server contains query information and the format that the server needs to use to represent the data
ReSTful Protocol Specification
Your plan for the code for this section is the:
ReSTful Protocol Specification
This document includes the details for the Representational State Transfer (ReST) based protocol for accessing Open Social information, including the entity, attributes, and relations that were used in the previous section. It also includes the XSD for the custom Open Social XML format, which is one of the options you must support for container queries to your server.
You will also need to refer to the following sources:
This defines the authentication method used by Open Social HTTP protocols.
JSON Developer Page
Atom Pub Developer Page and Atom Pub Specification
This provides similar developer resources for the Atom Pub format.
STEP 3: The RPC API
Now while the ReSTful interface has many advantages, particularly in the areas of robustness, cache-friendliness, and simplicity, it is still very inefficient for highly-interactive applications, and a more natural paradigm is the Remote Procedure Call or RPC. This interface "pretends" to operate just like a function call in a program, and there are various ways in which it can optimize the transfer of information, by batching queries and preserving state between calls.
Open Social RPC API Specification
The Open Social RPC API Specification will be your guide to creating the RPC interface. Much of the process is the same as for the ReSTful API, and you will have to generate responses using the same data formats.
STEP 4: The Gadget Server
Open Social is based on Google's older "Gadgets" technology, which may require a little explanation, especially for developers used to the "server side compositing" approach of most major content management systems.
Use Shindig to provide the Gadget server, while providing the container server APIs from your CMS
You have probably heard of this in the context of "AJAX" development, and of course, AJAX is based on client side compositing.
Developing a Gadgets server is a complex task which lies a bit outside the scope of this article, although it might well be a worthwhile task to implement one in your language of choice.
Fortunately, the Apache Shindig project provides the Gadget Server code in your choice of Java or PHP languages (the project welcomes additional language implementations).
Your best bet for integrating with a CMS is probably to simply install Shindig to provide the Gadget server on the same computer that your CMS is running on. The alternative would be to implement a Gadget server yourself, but there's very little reason to do that, since Gadgets are such a highly standardized service.
You will need to download and use the Shindig package, both to install the software and to study the available examples and source code.
At the time of this writing, there is no actual file release for Shindig, so you will need to use Subversion to check out a copy from the live code repository.
The Google Gadgets API Specification defines the Gadget server protocol that Shindig implements. You could of course, opt to develop your own Gadget server.
This is probably the trickiest, albeit most interesting step. How exactly will your CMS users make use of Open Social gadgets on your site? You'll need to answer this question before you can make it happen, but then again, that's what motivated you in the first place, right?
Your CMS will need to deploy the container application as a web page
Shindig includes a number of simple sample container implementations which illustrate how the container page can work. You will most likely want to implement some means of storing and deploying a user's gadgets to be rendered in the container page, using the methods that integrate best with the design of your CMS.
Opening Open Social
The internet has always been a social phenomenon. New protocols like Open Social are steps forward in giving greater fluidity to that social aspect of the internet and enabling productive contact networks to bring people closer together.
Open social provides a new enabling technology to facilitate the great human conversation that is the internet (Photo credit: Chris Metcalf / CC-By 2.0)
This is exactly the sort of software that really must be free software in order to fully work: it's hard to imagine any proprietary solution being effective across so many boundaries and for some many groups of people. The specs are still a bit raw and the software a bit shaky, but it's worth the effort to keep the internet society a free society
This work may be distributed under the terms of the Creative Commons Attribution-ShareAlike License, version 3.0, with attribution to "Terry Hancock, first published in Free Software Magazine". Illustrations and modifications to illustrations are under the same license and attribution, except as noted in their captions (all images in this article are CC By-SA 3.0 compatible).