This article briefly discusses the technologies used in modern browsers to display web pages, and goes into a little more detail about the user experience on those web pages, in particular how we can get part of a web page to refresh automatically when data changes on a web server.
Browsers and HTML
I’m sure anyone who’s reading this page is aware that the web is based on a request and response process that returns web pages of data. If I click on a link my browser makes a request for the web page specified in the link, the request gets routed to the appropriate server, and the server responds with a page of HTML which my browser displays.
HTML (hypertext markup language), of course, is a simple markup language that tells a browser where to put text and images on a web page using tags (e.g. <header>). The request format is a text URL (uniform resource locator) of the kind you see all the time in your browser’s navigation bar. Furthermore, the returned text can contain additional links that the browser will show underlined and that I can click on.
Anyone who uses the internet understands this, but the success of the web is at least in part due to the simplicity of that model. The HTML is just text with a few special tags in angle brackets, and all a browser has to do is know how to send a request, handle the response, and then draw a screen using the HTML that’s returned. Similarly all a web server has to do is know how to receive a request, get the right HTML text to send back, and send it. Often web servers will simply store the text in text files on their hard drive, and just load and send the right one in response to a request depending on the text of the request.
At root it’s unbelievably simple; just look what it’s turned into.
Other Technologies Used In Web Browsers
Of course modern browsers aren’t as simple as described above and there are a number of other technologies that they understand and developers can use.
Another baseline technology for what gets displayed to the client is Cascading Style Sheets (CSS). These allow a common look and feel to be applied to a group of web pages without the need for detailed coding in each page.
Drawbacks of the Basic HTML Request/Response Page-Based Model
For a more sophisticated user experience there are a few things we might like to have:
- The ability to refresh part of a web page without reloading the entire page. Initially this could be initiated by the user clicking a button, but we want just the relevant data to update, not the entire page.
- The ability to do this refresh whilst allowing the user to continue to interact with the rest of the page. That is, the request shouldn’t block the user, it should be asynchronous.
- The ability to update the page when data changes on the server without the user having to refresh in any way.
1. Refreshing Part of a Web Page
The first problem that developers tried to solve was updating part of a web page in place without reloading the entire page. There are several ways of doing this.
However, there are some simple approaches that predate Ajax. One is to use IFrames. These are HTML elements within a page that can issue their own request/responses to a web site and render the results independently of the rest of the page. They have a SRC property that can be set to a URL. If you set the src to a different or the same URL (say on a button click) the new data will appear without a full page reload.
Many developers don’t like IFrames. Search engines can get confused by them. They may show scrollbars if your content doesn’t fit correctly. Worse if your user has scrolled to the bottom of a page and then you load a new shorter page in the same frame they may be off the bottom of it. Because of restrictions in HTML IFrames can usually only issue requests to the same site as the main site of the page they are on. All of this means people have looked for better solutions.
One advantage of this approach is that script tags can issue requests to any URL, not just the same site as the page they are on. One disadvantage of this approach is that it can lead to security vulnerabilities in the code.
2. Refreshing Part of a Web Page Asynchronously
The original call to the server will not block and will not reload the page. The user can carry on interacting with the page as usual, even if the call to the server takes some time.
One thing to note here is that the data returned is just text. It doesn’t have to be XML, in spite of the names (XMLHttpRequest, AjaX).
3. Updating a Page Automatically when Data Changes on the Server
Ajax as described so far updates a page in place, but only in response to a request from the web page. This means that the user has to click a button or something similar for the page to update.
Obviously there are situations where data is changing and we would like it to update on our web page without the need for the user to manually refresh.
There are quite a few ways of doing this, some of them direct extensions to the Ajax model described above:
The drawback to this is we may make requests when no data has changed, putting unnecessary load on our servers. Also our data on the client may well be out of date at any given time if we are between polling requests.
We really want a way for our server to send data only when it’s changed, and at the moment it has changed.
Another approach is long polling. Here the browser fires off a request with a long timeout and sets up a handler for the results using Ajax as before. However, the server itself doesn’t respond until it has data that has changed, and then it sends the data in response to the original request. The browser handles the request and immediately sets up another long timeout request for future updates.
The disadvantage of this approach is that the server has to keep the connection (a network socket) open until it has data. In general it will have as many connections as it has clients waiting for data. This obviously puts load on the server and the number of sockets that the server can possibly use becomes a limiting factor. Also this is clearly a more complex solution to implement than normal (short) polling.
In streaming the client makes a request and the server responds with an open response that keeps the communication channel open and allows subsequent responses to be sent to the client. The server may eventually time out the connection, or may keep it open indefinitely. If the connection times out the client will have to make another request to refresh the data. So this approach is like long polling with the client needing to make fewer requests.
One drawback of this approach is that many proxy servers buffer http responses until they are complete: that is, they won’t send on the message until they have the completion. This means the client won’t get timely updates. Another obvious drawback is that this is a fairly complex way of keeping data up to date.
With all of these approaches the call backs from the server tend to tie up one http communication channel. As a result many approaches to solving the problem use (at least) two channels: one for polling or streaming to update the data in place, and one for regular requests from the client to the server.
A number of commercial frameworks have been built using these techniques.
Comet is a name that’s been applied to the techniques described above to update a web page in place automatically when data changes on the server using a longlasting HTTP connection.
HTML 5 Web Sockets
HTML 5 web sockets are the new way to do bidirectional communication between a web page and a server. They don’t use the old HTTP request/response at all, but instead set up one dedicated channel for communication between client and server. This is fast, and the messages involve very little redundant header information, unlike conventional HTTP requests.
The main drawback of this new technology currently is that many browsers do not support it. For example, it doesn’t work in the last version of Internet Explorer, IE9, although it works in IE10.