Life is Beautiful

User Interface is an Art

Live Page-View Counter, Comet server and JSON-push

Overview

A "page-view counter" or "hit counter" is a mechanism that displays the number of page-views on an HTML page. It uses a server side of script that counts the page-views, dynamically generates an HTML page on the server side, and returns it back to the browser. Although it accurately displays the number of page-views at the point when the HTTP request was made to fetch the HTML page, it will not reflect additional changes in the page-views after that point because there is no further communication between the server and the client (unless you hit F5 to refresh the page).

This "Live Page-View Counter" is an experimental implementation of "live" version of "page-view counter", which displays the "current" page-views, dynamically updating the counter on the HTML page, utilizing Comet server and JSON, which I call "JSON-push".

[Notes] Regarding Comet server, I strongly recommend you to read "Comet: Low Latency Data for the Browser" by Alex Russell, before you proceed to the next section.

See it and believe it

Take a look at the sample implementation below, pasted to this blog entry using IFRAME. Pay attention to the counter, which will automatically increments when a new visitor comes to this blog. You just need a regular browser (it works on IE, Firefox and Safari) and don't need to hit F5.

You also see footstep icons below the page counter, which change their colors occasionally. Those changes are not initiated by Javascript on this page or the server side of script, but other users who are reading this blog while you are looking at this blog. This is a service called "Footstep live!", which I will explain later.

How It Works

Unlike a traditional "page-view counter", the "live" version returns a static HTML page instead of dynamically generated HTML page. This HTML page, however, has following line that acts as a "page-view" signal to the server.

<script type="text/javascript"
  src="http://lab.uievolution.com:8080/~onload">
</script>

"http://lab.uievolution.com:8080" specifies the Comet server and its port, and "/~onload" is an URI that indicates that the page is being loaded. When the Comet server receives this HTTP request, it increments the internal page counter, performs a few additional tasks (to be described later [*1]) and returns following script immediately (it acts like a regular HTTP server in this particular request).

onLoadCallback({"Count": N});

where N is the value of the page counter (the Comet server running behind this sample returns additional information, but they are not essential and you can ignore them for now). The "onLoadCallback" function on the HTML page displays this value in an appropriate format (the onLoadCallback function on this sample actually stores the json object in a global variable temporarily, then populates its values to the HTML page when it gets "onload" event on BODY tag).

This is how this page displays the initial page-view number when the page is loaded, which is the same value as the traditional page-view counter would display. The only difference is that the data-binding (the binding of "view" and "data") is done on the client-side, instead of the server-side.

After the page is completely loaded, the browser calls the following function.

function connectComet() {
    aObj = new JSONscriptRequest('http://lab1.uievolution.com:8080/~listen');
    aObj.buildScriptTag();
    aObj.addScriptTag();
}

This function uses JSONScriptRequest, which is in jsr_class.js written by Jason Levitt. This function dynamically creates a new script tag that is equivalent to.

<script type="text/javascript"
  src="http://lab1.uievolution.com:8080/~listen">
</script>

This is also an HTTP request to the same Comet server (ignore that fact that "lab" became "lab1" for now - I will explain it later [*2]), but the URI "/~listen" indicates that the client is now ready for JSON-push. When the Comet server receives this request, it adds the socket to the "JSON-push client list" -- the list of sockets waiting for JSON-push, instead of returning an HTTP response immediately.

JSON-push is essentially a mechanism for the Comet server to push any Javascript to those clients in the "JSON-push client list", either to a particular set of clients or to the all the clients.

For example, the Comet server may send the following script to all the JSON-push clients when it increments the counter (when the Comet server receives "/~onload" request. See [*1] above).

setCount({"Count": N});
reconnectComet();

setCount() updates the counter on the HTML page (performs the same task as onLoadCallback()), and reconnectComet() will re-establish the connection to the Comet server. Here is the implementation of reconnectComet().

function() {
    aObj.removeScriptTag();
    connectComet();
}

JSON-dispatch

Although the mechanism described above was sufficient to implement the "live" version of page-view counter, I soon encountered the backward compatibility issue. When I was testing a new feature using the same Comet server (I have no staging server to test), I have realized that I can't broadcast an arbitrary Javascript (such as "updateFootstep();") to the all the JSON-clients, because it causes script errors on existing HTML pages and will break the connections to the Comet server.

In order to work-around this problem, I added an indirection mechanism, which I call JSON-dispatch.

Instead of directly pushing function calls that may or may not exist on the client side, the Comet server wraps each function call in a function call to jsonCallback() with the function identifier as a "callback" property of the JSON object. For example, the example described above,

setCount({"Count": N});
reconnectComet();

becomes

jsonCallback({"Count": N, "callback": "counter"});
reconnectComet();

where the "counter" is the function identifier. The implementation of jsonCallback() would become like this:

function jsonCallback(json)
{
    // Dispatch JSON callback functions
    id = json.callback;
    if (id=='counter') {
        setCount(json);
    }
}

Because of this mechanism, those HTML pages can gracefully ignore the unknown JSON-push like this:

jsonCallback({"Id": 1, "Color": "red", "callback": "footstep"});

DNS hack

I have mentioned above that it uses "lab.uievolution.com:8080" for "/~onload", but "lab1.uievolution.com:8080" for "~listen" (see [*2]), although both URL are mapped to the Comet server by DNS. Originally, I was using the same URL, and the page works fine without this hack. I, however, noticed that the page counter completely stops working when I open two instances of this page at the same time. Thanks to Kenn (Kentaro Ejima, who developed the Lingr chat service, which also uses Comet server), I have learned that this problem was caused by the maximum number of simultaneously opened HTTP requests to the same server, which is "two" by default. Because the Comet server does not immediately respond to the "/~listen" request from the client and hang onto it, we hit this limitation when the user opens the second instance of the page, and any further HTTP requests (including the request to port 80, where I am running a regular Apache server) will be deferred until the Comet server responds to those requests (which could be several seconds later).

While I was tracking this problem, Kenn taught me how to work around, which is this DNS hack. Simply mapping another server name (in this case 'lab1.uievolution.com') to the same IP address, the browser is able to open more than two HTTP requests simultaneously. In other word, this is an intentional Cross Domain Comet.

The "/~listen" requests will still encounter this limitation when the user opens more than two instances of this page, but both "/~onload" requests and regular HTTP requests always works (they always immediately return). This is still not perfect, but sufficient practically.

The Service Architecture

The Comet server I am using in this experiment is the "Micro Comet Server", which I have implemented in C++ from scratch while I was learning Linux programming (I am quite new to Linux). Based on KISS principle (Keep It Simple, Stupid!), I have made it quite simple and efficient. The code size is quite small (less than 20KB at this moment - and I would like to keep it as small as possible), and it is able to handle millions of transactions (HTTP requests) per day (I don't know the exact limit yet, but I am guessing it is two to three million transactions on a single core Pentium machine). It theoretically supports up to 1024 concurrent clients, although I have tested only 200 so far. It is supposed to linearly scale (to the number of transactions - not to the number of clients), although I don't have enough data to prove it yet.

Because I have only one physical server to play with, I am running an Apache server on port 80 to process regular HTTP requests and the Micro Comet Server on port 80 to process "/~onload" and "/~listen" requests from the client. The Micro Comet Server pushes its own scripts to the clients (such as the page-view update event), but also provides a web service for the Apache server to push arbitrary javascripts to the Comet clients. It allows me to implement variety of Comet-based service applications in regular scripting languages (such as PHP, Perl and Ruby) without mixing any application specific code to the Micro Comet Server.

Footstep live!

The "Footstep Live!" is another application that demonstrates the unique user experience enabled by a Comet server. It allows users to change the colors of footstep icons by simply clicking them, and share those changes with all the other people who are looking at the same page simultaneously.

October 29, 2006 in Web/Tech | Permalink | Comments (7)

Contents 2.0

Inspired by a research paper written by IBM - the end of TV as we know it, I have decided to write up what I was thinking about web 2.0. I think the various changes and tredns we see in this "web 2.0" is not limited to the web at all.

I think it is much more appropriate to see it as a much more broader changes in the way consumers consume contents (TV programs, movies, music, books, news, games, software, ...), especially among young generatations.

Here is the list of those trends we already see today:

  • Broadcast (scheduled, real time) -> Time-Shift or On-Demand (anytime)
  • Full-length program -> Bits and Pieces
  • Original program -> Remix and Mash-up
  • CD/DVD/Paper -> Bits on HD/Flash memory
  • Living Room -> Anywhere
  • On or Off -> Always on (but loosely)
  • Professional Reviews -> Peer Reviews (Blogs and Amazon)
  • Professionally Created -> Consumer Generated
  • Passive Audience -> Active Community
  • Consume -> Participate
  • Buy -> Share or Try (then buy)
  • Copyrighted -> Public Domain, Open Source, Creative Commons

I think it is a big mistake to see these changes as the web-only phenomena. The web obviously plays a key role as a underlying technology, but the effect will not be limited to just "web users". It will not limited to "ipod" generation. It will change the way how ordinally people consume those contents in their family rooms, in their cars and on the street.

February 23, 2006 in Web/Tech | Permalink | Comments (0)

Voting application

I have created a small blog plug-in, that allows users to create a custom poll. Here are test cases I have posted to my Japanese blog:

Test 1: Click "Yes"

Test 2: Click "No"

Test 3: Click whichever you like

Test 4: Do you want to have this part with your own questions on your blog?

January 20, 2006 in Web/Tech | Permalink | Comments (1)

Web2.0: "Fortune Cookie" application

I have creaed a small web-application that randomly displays a fortune (or "kotowaza" in Japanese - although the exact meaning is slightly different) with a small twist - it also allows the user to add his/her own favorite fortunes. I also provided a short HTML source code that allows users to paste this mini-app to their blogs - positioning it as a blog plug-in.

I have published this app to my blog readers (mostly of them are Japanese) to test how such a simple application will succeed because of this very much web2.0-like small twist - "user's participation", or "user generated contents".

Here is a quick summary of the first 24 hours:

1. Visitors: ~5,000 - this is the average traffic to my blog
2. Users who pasted this mini-app to their blog: 5
3. Fortune view: ~32,000
4. Fortune posted: ~200

I think #4 is quite significant considering only 5,000 people visited this blog. It is possible that some users have posted multiple fortunes (I did not add the mechanism to track it), but I am quite sure that it was a large number of users (probably over 100) because of the timing of the posting. It is quite significant to get even 1 to 2 percent of perticipation.

I don't see any viral effect because of #2 yet, which is understandable only after 24 hours - but I already start seeing some hits from those secondary blogs.

I am quite curious how this app will perform over time. Will it become widely popular because of viral effect, or will the number of posting goes down overtime when the article goes out of the first page of my blog (even though I keep it in the side bar)? I will write up a report later again - stay tuned. 

January 13, 2006 in Web/Tech | Permalink | Comments (0)

Essence of AJAX

It is great to see Yahoo proves that creating AJAX-style web application does not necessary require Javascript or even XML (see "Yahoo launches next battle in map wars"). The essense of AJAX-style applications represented by Google Map service is not in its implementation but in its programming style and user experience.

Google happened to choose Javascript and XML over HTTP, but it can be easily done using Flash, Java or even UIEngine, just like Yahoo has proven it with Flash. The essense of this AJAX-style web-applications are (1) interactivity, (2) asynchronous data fetching (not RPC), (3) client-side data binding, (4) smart client (which performs some data processing locally).

In this sense, it is better to call it "Asynchronous Message-based Web Application" rather than AJAX applications or AJAX-style applications, because AJAX implies a very specific tool set (Javascript and XML).

The difference between "Asynchronous Message-based Web Application" and AJAX is just like the difference between Object-Oriented Programing and Java. OOP is a programming style and can be achieved using variety of tools including Java, C++ or even C. Java is a specific programming language that helps programmers to write their code in OOP style.

November 03, 2005 in Web/Tech | Permalink | Comments (4)

My Photo

Recent Posts

  • Who killed Windows?
  • Tokyo life-style vs. Fukushima life-style
  • Amazon's Cloud Reader vs. my CloudReaders (TM)
  • Facebook API on Google App Engine: JavaScript SDK vs. Python SDK
  • Debugging Facebook AppEngine application locally
  • Google App Engine: Updating Your Model's Schema (schema-versioning)
  • Handing and Testing DeadlineExceededError
  • Dear users of CloudReaders(TM)
  • Why HTML5 is a big threat to Adobe
  • Sticker shock of iLike acquisition
Subscribe to this blog's feed

Archives

  • January 2014
  • June 2012
  • August 2011
  • January 2011
  • December 2010
  • April 2010
  • October 2009
  • August 2009
  • March 2009
  • February 2009

More...

Categories

  • Business and Marketing
  • Entertainment
  • ianime.js
  • iPhone
  • News Clip
  • Technology and Business
  • Telco/Wireless Industry
  • UIE Vision
  • User Experience
  • Web/Tech