Web and cloud apps have to be able to scale as close to linearly as possible, in a hostile environment. Network topology and number of users can fluctuate wildly. Contrast this with a desktop application, which just has to handle a single user. Or with a client/server app, which may have to handle a couple of hundred users usually on the same network. If millions of users start using our distributed system, the experience of each should be no different from what it would be if only 5 or 6 people were using it.
While web developers are used to dealing with concurrency, either consciously or just by knowing that calling Application.Lock() from VBScript will cripple their app, it is also becoming a wider issue for desktop developers, as chip manufacturers squeeze more and more cores on to their chips.
The arch-enemy of concurrency is shared data. Any data that two people have to update can cause concurrency problems. This is actually made slightly simpler in web programming by virtue of HTTP being stateless, and is one reason why the functional style of programming as seen on the web is becoming more popular in other applications.
If we can maintain the qualities of statelessness and atomicity in our operations, we are already well on our way to make our code safe to run concurrently. It's only when someone attempts to force a stateful model that you run into problems.
If shared data is the enemy of concurrency, locks are its atomic bombs. Locks work by serialising access to shared data, so that only one process / thread / fibre can write shared data at any one time. This means that anyone else just has to wait in line. Serialisation is the arch enemy of scalability.
So, NEVER hold editable shared data in your application layers. I will of course break this absolute rule in a future post for grandstanding purposes, but I'm allowed to - It's my blog! And of course, it's to illustrate a good point.
This issue doesn't tend to be a problem for those accustomed to web programming, since most web frameworks either enforce, or heavily imply, avoidance of global state. In most real systems today, shared state is handled by a backend RDBMS, such as Oracle or SQL Server, which offer a gigantic improvement over file-based DBs such as Access or Paradox. These do a very good job, at the moment, of handling medium to large scale apps. However, if we scale to the web, where fast global availability is required, then "Houston, we have a problem." We realise all we’ve done is offloaded the problem further down the chain. Because these client-server style databases have transactions, and therefore locks, embedded firmly at the heart of their architecture, they will not scale globally. You can partition them; you put them on ever larger machines, or even clusters of machines; but they will always require a global lock to perform a commit.
So if RDBMS' such as SQL Server don't scale well, then what does?
Whilst still a drop in the ocean in terms of commercially enterprise databases, Databases based on Distributed hash tables, such as CouchDB, Google’s BigTable, Voldemort, AWS SimpleDB and similar, are gaining a lot of traction.
Microsoft’s forthcoming Azure Table Storage falls in to this category too, unlike their slightly clumsy and poorly positioned "old man in a new hat" - SQL Azure Services.
These types of database characteristically emphasise Decentralization, Scalability and Fault Tolerance, with nodes continuously joining, leaving, and failing within the system.
Sound familiar? Well these are all databases designed to work not only on the web, but in accordance with the fundamental architecture of the web. They emphasise availability over transactional consistency which can cause serious headaches for people stuck in an RDBMS world view. (The idea that the database will sometimes return the wrong result is horrifying to them.)
Scalable Transactions are EXTREMELY complex (actually impossible, as previously discussed) in a distributed system. So, support for transactions in Distributed Databases is rare to limited. If you look at Amazon's SimpleDB or Azure Table Store, if you perform a "BatchPut" or "EntityGroupUpdate" respectively, these do succeed or fail as a unit. However, they are subject to limitations well below the expectations of developers who are used to developing against RDBMS systems. Maximum request sizes of 4MB, or number of items in a transaction (25 for SDB), seem almost laughable. However these distributed document style databases are a fundamentally different type of application from the RDMBS.
Unlike SQL Server or Oracle however, they are reliable, cheap, and MASSIVELY scalable. Therefore, while they account for a tiny fraction of the database market at the moment, they do offer a very serious cost-based competitive advantage over large scale installations of RDBMS systems. I will write a future post about lock free data management patterns and post-relational DBs in the future, but once again, for now the key takeaway for concurrency is:
Locks are mindblowingly expensive in a distributed system.
And they get more expensive the more they are used. If you really MUST use them, stand well back and wear very dark sunglasses.
Next and last up: Idempotence
Thursday, 23 July 2009
Monday, 13 July 2009
"It Worked."
In my last post I briefly discussed statelessness. The key point about statelessness is that between HTTP calls, whether to fetch web pages, or to access web services or other HTTP based services, each part of the system should be capable of being restarted without causing the others to fail.
Ideally, services should require no state. There are ways of maintaining a "session state" in many web platforms, but these should very much be seen as an optimisation or workaround, rather than a first port of call. I will look at the various session state patterns.
One side effect of statelessness is that we do not really have the context of a conversation available to us, so all the state required to complete a transaction must be available during a single request/response exchange and have no dependencies outside this. This characteristic is referred to as "Atomicity", and normally produces a particularly interesting expression on the face a developers when realised for the first time. Usually it's a result of attempting to use a component that has adopted what I would call the "Barefaced Liar" pattern - all seemed fine when the code ran locally.
This is where a component is designed to expose a simple interface, common to UI development, but actually impossible to guarantee in a distributed environment. There are lots of examples of this but often it's a Distributed Transaction Coordinator of some sort - although session state providers, and WCF’s "Bi-Directional" channel, can cause similar nauseating effects.
So... we have no state available here in which to hold stateful conversations.
The mother of all conversations, in Software Circles at least, is the "2-Phase commit". This protocol ensures that when I ask for work to be done by say three co-operating participants, then either all of it is done, or else none is. The canonical example of this is the bank transaction. We want money to be taken out of account one and put in to another, but if one of those actions fails, we don't want to revert just that one action and pretend nothing untoward has happened. We don't want both (or neither!) of the accounts to have the money.
It essentially works like this. A coordinator asks each participant to do work and waits until it has received a confirmation reply from all participants.
The participants do their work up to the point where they will be asked to commit to it. Each remembers how to undo what they've done so far. They then reply with an agreement to commit their work if the work was done, or a "failed" message if they could not perform the work.
If the coordinator received an agreement message from all participants, then it sends a commit message to them all, asking them to commit their work. On the other hand, if the coordinator received a fail message from any of the participants, then it sends a "rollback" message to all, asking them to undo their work.
Then, it waits again for responses back from all participants, confirming they've committed or rolled back. Only when all have confirmed successful committal, does the co-ordinator inform whoever is interested, that the transaction has succeeded.
This is a staple pattern of Client Server Systems, particularly of the RDBMS kind. However, it is not guaranteed to work in a stateless system as conversations require context and so are not stateless. This breaks our previous advice about maintaining disposability. You cannot invisibly dispose of a participant in a transaction who is waiting to commit with no side effects.
So... Operations exposed on the web should all be Atomic. They should fail or pass as a single unit.
To a client/server developer this of course seems at first sight to be monumentally impractical in many real world scenarios. But in fact most activities of any real-world value cannot use transactions as we know them.
In a standard web services integration, the BookFlight(), BookHotel() and BookCar() methods in a BookHoliday() operation may well be run by completely different organisations. Each of these organisations are likely to have differing policies and data platforms.
I've always found it funny that the poster boy for the 2-phase commit, The Bank Account Transfer, in reality cannot - and does not - use the 2-phase commit protocol.
So what does it use? Well the answer lies in a fairly pungent concoction of asynchronous co-ordination, reconciliation and compensation that is definitely an acquired taste. I will come back to look at specific ingredients you can use in a later post but for now, the important thing to remember is...
At the service boundary, operations should fail or pass in their entirety.
Next up: Concurrency
Ideally, services should require no state. There are ways of maintaining a "session state" in many web platforms, but these should very much be seen as an optimisation or workaround, rather than a first port of call. I will look at the various session state patterns.
One side effect of statelessness is that we do not really have the context of a conversation available to us, so all the state required to complete a transaction must be available during a single request/response exchange and have no dependencies outside this. This characteristic is referred to as "Atomicity", and normally produces a particularly interesting expression on the face a developers when realised for the first time. Usually it's a result of attempting to use a component that has adopted what I would call the "Barefaced Liar" pattern - all seemed fine when the code ran locally.
This is where a component is designed to expose a simple interface, common to UI development, but actually impossible to guarantee in a distributed environment. There are lots of examples of this but often it's a Distributed Transaction Coordinator of some sort - although session state providers, and WCF’s "Bi-Directional" channel, can cause similar nauseating effects.
So... we have no state available here in which to hold stateful conversations.
The mother of all conversations, in Software Circles at least, is the "2-Phase commit". This protocol ensures that when I ask for work to be done by say three co-operating participants, then either all of it is done, or else none is. The canonical example of this is the bank transaction. We want money to be taken out of account one and put in to another, but if one of those actions fails, we don't want to revert just that one action and pretend nothing untoward has happened. We don't want both (or neither!) of the accounts to have the money.
It essentially works like this. A coordinator asks each participant to do work and waits until it has received a confirmation reply from all participants.
The participants do their work up to the point where they will be asked to commit to it. Each remembers how to undo what they've done so far. They then reply with an agreement to commit their work if the work was done, or a "failed" message if they could not perform the work.
If the coordinator received an agreement message from all participants, then it sends a commit message to them all, asking them to commit their work. On the other hand, if the coordinator received a fail message from any of the participants, then it sends a "rollback" message to all, asking them to undo their work.
Then, it waits again for responses back from all participants, confirming they've committed or rolled back. Only when all have confirmed successful committal, does the co-ordinator inform whoever is interested, that the transaction has succeeded.
This is a staple pattern of Client Server Systems, particularly of the RDBMS kind. However, it is not guaranteed to work in a stateless system as conversations require context and so are not stateless. This breaks our previous advice about maintaining disposability. You cannot invisibly dispose of a participant in a transaction who is waiting to commit with no side effects.
So... Operations exposed on the web should all be Atomic. They should fail or pass as a single unit.
To a client/server developer this of course seems at first sight to be monumentally impractical in many real world scenarios. But in fact most activities of any real-world value cannot use transactions as we know them.
In a standard web services integration, the BookFlight(), BookHotel() and BookCar() methods in a BookHoliday() operation may well be run by completely different organisations. Each of these organisations are likely to have differing policies and data platforms.
I've always found it funny that the poster boy for the 2-phase commit, The Bank Account Transfer, in reality cannot - and does not - use the 2-phase commit protocol.
So what does it use? Well the answer lies in a fairly pungent concoction of asynchronous co-ordination, reconciliation and compensation that is definitely an acquired taste. I will come back to look at specific ingredients you can use in a later post but for now, the important thing to remember is...
At the service boundary, operations should fail or pass in their entirety.
Next up: Concurrency
Thursday, 2 July 2009
The state I am in
One of the fallacies of distributed computing states that "Network topology doesn't change."
Well, yes it does, particularly on the internet. Pieces of hardware and software are constantly being added, upgraded and removed. The network is in a constant state of flux. So how do you perform reliable operations in such a volatile environment?
Well, the answer lies in part with the idea of statelessness. HTTP is what’s known as a stateless protocol. For those coming from a non-distributed platform background, statelessness is one of the most misunderstood aspects of web development.
Partly this is due to attempts to hide statelessness by bolting on some - often clunky - state management system. This is because statelessness is often viewed as a kind of deficiency. Framework developers all over the world say, "Oh yuk. Statelessness. This is soOOo difficult to work with. I want my developers to be able to put a button on a form and say OnClick => SayHello(). It's 2009! I don't want to have to worry about state issues on a button click!".
If you're dealing with a Windows Forms style of application, your button is more or less the same “object" that is shown on the user’s screen; but in many web frameworks, a "button" and its on screen representation in a user’s browser may be thousands of miles apart. The abstraction of “a button” assumes a single object, with a state - but this is not the reality. What appears to be one button is in fact two buttons: a logical one on a server, and a UI representation in a browser, with a “click” message being sent between them. The concept of "a button" is a leaky abstraction, and the reason that it's leaky, is because it does not take in to account the network volatility that is a standard feature of the web.
So, what do I mean by statelessness?
When you use the web, it enlists the resources required to fulfil your request dynamically. Once the request has been completed, the various physical components that took part are disconnected, and neither know nor care anything for each other anymore. How sad :-(
Let's look at it in terms of a conversation,
Customer: Hello
Teller: Hello, what can I do for you today?
Customer: I'd like to know my bank balance please.
Teller: Can I have your account number please?
Customer: It's 09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0 ;-)
Teller: Well. Your balance is only $1,000,000.
Customer: Ok, Can I please transfer all of it to my Swiss bank account please?
Teller: Yeah sure, what's the account number?
Customer: 12345678.
Teller: Ok, I'll just do that now for you. Would you like a receipt?
Customer: Yes please.
Teller: Here you go => Receipt Provided.
Customer: Bye.
Teller: Bye.
This conversation contains state. Each role of Customer and Teller must know the context of the messages being transferred.
The stateless version of this would go more like...
Customer: Hello, I'd like to know the Balance for account 09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0,
Teller: It's $1,000,000
Customer: Could you please transfer $1,000,000 from account 09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0 to Swiss bank account number 12345678 and provide a receipt please.
Teller: Yup. Here you go => Receipt Provided.
The end results are the same. However in the first example, "fine-grained operations" are used. The Customer and the Teller need to know where they are in the conversation to understand the context of each statement. In the second, each individual part of the conversation contains all of the information required to perform the requested task.
If we think of these as client and server message exchanges, then in the second example it doesn't matter if the server (teller) is restarted, reinstalled or is blown-up and replaced between the two requests. This is invisible to the end user. You would never be aware of the swap.
However, in the first example, if the server was restarted it would "forget" where you were in the conversation.
Customer: Yes please.
Teller: Sorry? What?
...or in pseudocode...
Client: Yes please.
Server: Sorry? What? WTF? Ka-Boom!
This is a fundamental difference between programming in a “stateful” way, and distributed programming.
Understanding statelessness, its implications and how to manage state in a stateless environment, is a core skill of a web developer. Without understanding the implications of various state management systems, you simply cannot develop large scale applications. In web application there is ALWAYS a client, and a server, and a potentially lengthy request/response message exchange between them.
Most programmers have a fairly firm grip on Object-Oriented programming, and one of the keys to this is the packaging of state and behaviour. As a result, they get locked in to a system of thinking that says state and behaviour should always go together, hence the popularity of technologies such as CORBA, DCOM, and Remoting. These technologies make it easy for people who are used to dealing with "objects" to deal with these without worrying directly about the messaging process that goes on behind the scenes.
However, you often hear complaints about the scalability of these types of remote object technologies. The reason for this often involves assumptions about state. Since remoting interfaces are often automatically generated by whatever remoting framework you are using, and the objects used are very often designed deliberately in such a way as to be re-usable (although not necessarily performant) in both distributed and non-distributed environments, you end up with Chatty, context bound, fine-grained interfaces, which have a hard time keeping up when you have a large number of conversations going on at once.
Ceteris paribus, if you imagine the two conversations above in a remote system where each message exchange took a second, the first would take 7 seconds, the second 2.
What's also interesting is that the two message exchanges in the second conversation can be made either way round or, if required, in parallel. This of course works only if you are able to make some assumptions regarding the availability of cash to you, for example, that you have an overdraft facility of $1,000,000 which you are currently not using. I will come back to this when I discuss Concurrency and Data Consistency, but it is perfectly reasonable to run these operations in parallel. This, along with the rapid growth in multi-core and cloud computing, is one reason for the recent resurgence of interest in functional/stateless programming languages such as Erlang and F#.
The important thing to remember about statelessness is that it is all about maintaining a high degree of disposability. You should be able to throw everything away between message exchanges without a user noticing a server restart. Doing this will enable your application to handle server restarts and changes in network topology gracefully.
Next up: Atomicity.
Well, yes it does, particularly on the internet. Pieces of hardware and software are constantly being added, upgraded and removed. The network is in a constant state of flux. So how do you perform reliable operations in such a volatile environment?
Well, the answer lies in part with the idea of statelessness. HTTP is what’s known as a stateless protocol. For those coming from a non-distributed platform background, statelessness is one of the most misunderstood aspects of web development.
Partly this is due to attempts to hide statelessness by bolting on some - often clunky - state management system. This is because statelessness is often viewed as a kind of deficiency. Framework developers all over the world say, "Oh yuk. Statelessness. This is soOOo difficult to work with. I want my developers to be able to put a button on a form and say OnClick => SayHello(). It's 2009! I don't want to have to worry about state issues on a button click!".
If you're dealing with a Windows Forms style of application, your button is more or less the same “object" that is shown on the user’s screen; but in many web frameworks, a "button" and its on screen representation in a user’s browser may be thousands of miles apart. The abstraction of “a button” assumes a single object, with a state - but this is not the reality. What appears to be one button is in fact two buttons: a logical one on a server, and a UI representation in a browser, with a “click” message being sent between them. The concept of "a button" is a leaky abstraction, and the reason that it's leaky, is because it does not take in to account the network volatility that is a standard feature of the web.
So, what do I mean by statelessness?
When you use the web, it enlists the resources required to fulfil your request dynamically. Once the request has been completed, the various physical components that took part are disconnected, and neither know nor care anything for each other anymore. How sad :-(
Let's look at it in terms of a conversation,
Customer: Hello
Teller: Hello, what can I do for you today?
Customer: I'd like to know my bank balance please.
Teller: Can I have your account number please?
Customer: It's 09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0 ;-)
Teller: Well. Your balance is only $1,000,000.
Customer: Ok, Can I please transfer all of it to my Swiss bank account please?
Teller: Yeah sure, what's the account number?
Customer: 12345678.
Teller: Ok, I'll just do that now for you. Would you like a receipt?
Customer: Yes please.
Teller: Here you go => Receipt Provided.
Customer: Bye.
Teller: Bye.
This conversation contains state. Each role of Customer and Teller must know the context of the messages being transferred.
The stateless version of this would go more like...
Customer: Hello, I'd like to know the Balance for account 09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0,
Teller: It's $1,000,000
Customer: Could you please transfer $1,000,000 from account 09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0 to Swiss bank account number 12345678 and provide a receipt please.
Teller: Yup. Here you go => Receipt Provided.
The end results are the same. However in the first example, "fine-grained operations" are used. The Customer and the Teller need to know where they are in the conversation to understand the context of each statement. In the second, each individual part of the conversation contains all of the information required to perform the requested task.
If we think of these as client and server message exchanges, then in the second example it doesn't matter if the server (teller) is restarted, reinstalled or is blown-up and replaced between the two requests. This is invisible to the end user. You would never be aware of the swap.
However, in the first example, if the server was restarted it would "forget" where you were in the conversation.
Customer: Yes please.
Teller: Sorry? What?
...or in pseudocode...
Client: Yes please.
Server: Sorry? What? WTF? Ka-Boom!
This is a fundamental difference between programming in a “stateful” way, and distributed programming.
Understanding statelessness, its implications and how to manage state in a stateless environment, is a core skill of a web developer. Without understanding the implications of various state management systems, you simply cannot develop large scale applications. In web application there is ALWAYS a client, and a server, and a potentially lengthy request/response message exchange between them.
Most programmers have a fairly firm grip on Object-Oriented programming, and one of the keys to this is the packaging of state and behaviour. As a result, they get locked in to a system of thinking that says state and behaviour should always go together, hence the popularity of technologies such as CORBA, DCOM, and Remoting. These technologies make it easy for people who are used to dealing with "objects" to deal with these without worrying directly about the messaging process that goes on behind the scenes.
However, you often hear complaints about the scalability of these types of remote object technologies. The reason for this often involves assumptions about state. Since remoting interfaces are often automatically generated by whatever remoting framework you are using, and the objects used are very often designed deliberately in such a way as to be re-usable (although not necessarily performant) in both distributed and non-distributed environments, you end up with Chatty, context bound, fine-grained interfaces, which have a hard time keeping up when you have a large number of conversations going on at once.
Ceteris paribus, if you imagine the two conversations above in a remote system where each message exchange took a second, the first would take 7 seconds, the second 2.
What's also interesting is that the two message exchanges in the second conversation can be made either way round or, if required, in parallel. This of course works only if you are able to make some assumptions regarding the availability of cash to you, for example, that you have an overdraft facility of $1,000,000 which you are currently not using. I will come back to this when I discuss Concurrency and Data Consistency, but it is perfectly reasonable to run these operations in parallel. This, along with the rapid growth in multi-core and cloud computing, is one reason for the recent resurgence of interest in functional/stateless programming languages such as Erlang and F#.
The important thing to remember about statelessness is that it is all about maintaining a high degree of disposability. You should be able to throw everything away between message exchanges without a user noticing a server restart. Doing this will enable your application to handle server restarts and changes in network topology gracefully.
Next up: Atomicity.
Monday, 29 June 2009
If you're not confused, you're not paying attention
In the last post we looked at the myths of infinite latency and bandwidth. I'd like quickly to cover some of the other fallacies before coming back to these, and how they result in some fundamental architectural constraints for distributed systems.
So let’s quickly check off a couple of these fallacies and their solutions.
The network is secure - It's not. The internet is a public network. Anything sent over it by default is equivalent to writing postcards. You wouldn't write your bank account details on a post card. Don't send them openly over the web.
Solution: Secure Sockets Layer (SSL), the famous “padlock” icon in your browser. Now my postcard is locked in a postcard sized safe. It can still go missing, but no-one can open it without the VERY long secret combination.
There is one administrator. - There is not. Lots of people, all speaking different languages, with different skills, running different platforms with entirely different needs, requirements and implementations. The people deploying code to the web are often entirely separate from those responsible for keeping that code running, often these groups have nothing to do with each other.
Solution: When designing software for this type of environment, you should make absolutely no assumptions about access to or availability of particular resources. Always write code using a "lowest common denominator" approach. Try to ensure that when your system must fail or degrade, it does so gracefully. I will come back to this in a future post.
These problems resulting from the heterogeneity of distributed systems are mostly solved or worked around by the adoption of the various web standards. The point of the standards is that messages can be exchanged without the requirement for a specific platform, or all the baggage that goes along with that.
Right... take a deep breath. Some of the jargon used in the next couple of paragraphs may cause eye/brain-strain, but do not fear! All will become clear, hopefully in 6 posts’ time. The next 3 posts will cover what I would regard as the most important considerations in building a large web based system, and are directly related to the remaining fallacies.
Following that, I’ll discuss some of characteristics and issues that emerge from systems built within these constraints. Specifically, Concurrency, Horizontal Scalability and Distributed Data Consistency (or inconsistency, as more accurately describes the issue).
So next time... Statelessness.
So let’s quickly check off a couple of these fallacies and their solutions.
The network is secure - It's not. The internet is a public network. Anything sent over it by default is equivalent to writing postcards. You wouldn't write your bank account details on a post card. Don't send them openly over the web.
Solution: Secure Sockets Layer (SSL), the famous “padlock” icon in your browser. Now my postcard is locked in a postcard sized safe. It can still go missing, but no-one can open it without the VERY long secret combination.
There is one administrator. - There is not. Lots of people, all speaking different languages, with different skills, running different platforms with entirely different needs, requirements and implementations. The people deploying code to the web are often entirely separate from those responsible for keeping that code running, often these groups have nothing to do with each other.
Solution: When designing software for this type of environment, you should make absolutely no assumptions about access to or availability of particular resources. Always write code using a "lowest common denominator" approach. Try to ensure that when your system must fail or degrade, it does so gracefully. I will come back to this in a future post.
These problems resulting from the heterogeneity of distributed systems are mostly solved or worked around by the adoption of the various web standards. The point of the standards is that messages can be exchanged without the requirement for a specific platform, or all the baggage that goes along with that.
Right... take a deep breath. Some of the jargon used in the next couple of paragraphs may cause eye/brain-strain, but do not fear! All will become clear, hopefully in 6 posts’ time. The next 3 posts will cover what I would regard as the most important considerations in building a large web based system, and are directly related to the remaining fallacies.
| Fallacy | Consideration/Constraint |
| Network Topology doesn't change. | Statelessness |
| Transport cost is zero. | Atomic/Transactionless Interfaces |
| The network is reliable. | Idempotence (yes. really!) |
Following that, I’ll discuss some of characteristics and issues that emerge from systems built within these constraints. Specifically, Concurrency, Horizontal Scalability and Distributed Data Consistency (or inconsistency, as more accurately describes the issue).
So next time... Statelessness.
Wednesday, 17 June 2009
Faster than a speeding bullet
Over the next couple of posts I'm intending to go in to just a little more depth about the fallacies of network computing I mentioned last time. During the past week I have been doing some work using Amazon’s Web Services Platform to speed up one of our websites.
While demonstrating the delivery speed gained by using Amazon’s CloudFront web service, I encountered a surprising (to me at least) reaction. Though impressed by the visible improvement, many non-web developers were raising points such as, "Why does where our site is hosted matter? I thought the web is globally available", "I thought we were paying for unlimited bandwidth with our ISP?"...
Well, this reaction fell straight in to the path of fallacies number 2 and 3!
Latency is zero.
Bandwidth is infinite.
While demonstrating the delivery speed gained by using Amazon’s CloudFront web service, I encountered a surprising (to me at least) reaction. Though impressed by the visible improvement, many non-web developers were raising points such as, "Why does where our site is hosted matter? I thought the web is globally available", "I thought we were paying for unlimited bandwidth with our ISP?"...
Well, this reaction fell straight in to the path of fallacies number 2 and 3!
Latency is zero.
Bandwidth is infinite.
WRONG!!!
Many people assume that bandwidth and latency are the same thing, however the common metaphor is of a system of roads. The number of lanes on the road is equivalent to its bandwidth, and its length is equivalent to its latency.
A high-bandwidth connection usually has lower latency, since it has less chance to get congested at peak times. However, latency and bandwidth are NOT directly related. They normally go together (i.e. high bandwidth usually means better ping times), but they don't have to.
Even if we could buy "unlimited" bandwidth from our supplier, then if our website is served up from a single location, no matter how fast the connection, it will still suffer from the effects of network latency/lag when viewed at a global level.
It's all to do with the speed of light. No, really! It's Physics time. Hooray!
With the exception some new fangled weird physics theories, as far as normal physics goes, the speed of light is constant. Nothing can travel faster than it, and this "nothing" rule therefore applies to the messages you send around the internet.
Light goes very "fastly" (as my 2 year old son says) - about 300K km/second. This is roughly a million times faster than sound, and fast enough to zoom around the Earth more than 7 times in one second. This sounds really quick, and it is for day to day things like phoning someone (or using smoke signals); you wouldn't perceive it. However, over distances of thousands of miles, the assumption that you never need to worry about the speed starts to break down.
You can sometimes see this effect if you have satellite TV and the same programme is being broadcast on both terrestrial and satellite channels. If you flick between the channels you'll see the satellite picture arrives a moment later than the one broadcast from the ground.
Broadcasting uses "unreliable" protocols, so if messages are held up, they are assumed missing in action and your signal/picture will break up. TCP/IP however is a "reliable" protocol, so if messages are held up, you have to wait for them to arrive, and your experience is that your signal will appear to slow down.
If you're a computer trying to send a message to the other side of the world, it takes about 100 milliseconds to "ping" a computer in Australia from here in the UK (assuming your message travels at the speed of light).
Try this!... Ping a well known server in Australia:
> ping wwf.org.au
...and a well know server here in the UK:
> ping direct.gov.uk
Depending upon where you are located, you'll probably see very different times. I'm seeing ping times of between 340 and 360 ms to Australia, and just 14 to 21 pinging locally (from Scotland to London).
So what does this mean to me as a developer? Well if my users are in London and my site is hosted in Australia, it's going to take a long time to load.
The most common way to mitigate this problem is to use the smallest possible message size, compress your graphics, and so on. However, for rich content like large flash or silverlight files, or streaming video, this is often impossible. An alternative is to put your content as close as possible, in network terms, to your end user.
Whatever your application does, there are some important design decisions to be made in terms of how to partition, deploy and manage it to ensure an optimum experience for end users.
The most common design practices to work around these 2 assumptions are...
1. Optimise your message sizes carefully.
2. Compress any large content.
3. Make a clear separation between types of content, so that your application can be partitioned either logically or geographically if necessary.
At a basic level you might separate static vs. dynamically generated content, but could also be partitioning based on expected usage, data size, or resource locale, for example.
Whatever you need to take in to account for your specific application, the one rule to rule them all is
"Never assume that the network is fast enough by default."
Friday, 5 June 2009
The limits of my language are the limits of my world
In my last couple of posts, I described the web and how it is built. The reason for this is to introduce the web to non-web developers - primarily Windows ones, as Windows developers "see" the world differently from web developers. This is as a result of the underlying operating environment.
I am about to make some massive generalisations for the purposes of illustrative clarity.
Windows developers tend to see things in terms of "objects", "containers" and "references" which all communicate via "events" that are raised and responded to. Web developers tend to see things in terms of "resources", "relations" and "links" which all communicate via "messages" that are sent and received.
Windows developers "create new objects" where web developers "post resources". This may all be argued to be a case of "You say tomato", but they represent fundamental differences between traditional Windows development and web development.
An "event" is an abstraction that performs poorly on the web. A windows developer new to web development will see an event in a way that is appropriate to windows development, but if you deal with events in a web application the same way you deal with events in a windows application, you will run in to trouble. However, If you see don't see an event, but a pair of messages, you can store, edit, copy, cache and queue a message. If you think of things in terms of events, it's difficult conceptually to get your head around the idea of a "cached event".
This mismatch of understanding between non-distributed and distributed development results in the famous "Fallacies of Network Computing".
These fallacies are a list of flawed assumptions made by programmers when first developing web, or in fact any distributed, applications.
They are...
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
Over the next few posts I will explain these incorrect assumptions, the problems they have caused (and continue to cause) for people new to web development, and the considerations, patterns and constructs that should be taken in to account to work around or overcome them.
I am about to make some massive generalisations for the purposes of illustrative clarity.
Windows developers tend to see things in terms of "objects", "containers" and "references" which all communicate via "events" that are raised and responded to. Web developers tend to see things in terms of "resources", "relations" and "links" which all communicate via "messages" that are sent and received.
Windows developers "create new objects" where web developers "post resources". This may all be argued to be a case of "You say tomato", but they represent fundamental differences between traditional Windows development and web development.
An "event" is an abstraction that performs poorly on the web. A windows developer new to web development will see an event in a way that is appropriate to windows development, but if you deal with events in a web application the same way you deal with events in a windows application, you will run in to trouble. However, If you see don't see an event, but a pair of messages, you can store, edit, copy, cache and queue a message. If you think of things in terms of events, it's difficult conceptually to get your head around the idea of a "cached event".
This mismatch of understanding between non-distributed and distributed development results in the famous "Fallacies of Network Computing".
These fallacies are a list of flawed assumptions made by programmers when first developing web, or in fact any distributed, applications.
They are...
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
Over the next few posts I will explain these incorrect assumptions, the problems they have caused (and continue to cause) for people new to web development, and the considerations, patterns and constructs that should be taken in to account to work around or overcome them.
Monday, 1 June 2009
The journey of of 1000 miles (part 2)
In the previous post, We looked briefly at the protocols that make the web work. A browser starts by making an HTTP request. This is resolved by DNS, sent over the network via routers to reach a destination which then produces a response message and sends that back.
Now we know how messages are sent around the web, what kind of things are we able to send? Well... Enter HYPERTEXT! (Wow!). The web was designed to send "Hypertext" around? But what is Hypertext? Essentially it's a body of text with an in place reference to another body of text. If you ever had one of those adventure game books as a child (perhaps giving away my long held nerdiness there) they work as a form of hypertext. "If you want to open the treasure chest, go to page 134, if not, go to page 14."
So, this is not necessarily a computery idea. In fact, the idea of hypertext is generally agreed as originating from an idea by an engineer with the best name in computing history... Vannevar Bush, way back in 1945. Whilst there are some very well know implementations of Hypertext (Adobe PDF and Microsoft's original Encarta) it was the creation of the web in 1992 that eventually led to wide scale adoption of hypertext.
The core language of the web, HTML (Hypertext Markup Language) provides a way of "Marking up" text to contain references to another body of text. It does this using a mechanism called a "hyperlink", which describes the reference to another piece of text.
For example...I write a document and say in it...
The Vancouver Sun's statement that "The 2011 movie TRON 2 could become the most expensive film ever made with a reported budget of £300 million dollars." has been debunked.
I can augment this information by pointing at references, without breaking the flow of the text. I do this using an HTML "element" (a label with angled brackets around it) to point at a "hypertext reference". HTML elements can contain "attributes" which are values associated with the elements.
So for example... The "A" (anchor) element has an "href" (hypertext reference) attribute.
The basic format of this is...
The Vancouver Sun's statement that "The 2011 movie TRON 2 could become the most expensive film ever made with a reported budget of £300 million dollars" has been <a href="http://www.slashfilm.com/2009/04/13/">debunked</a>
A web browser knows not to display the anchor element directly, but to underline the contained text and allow a user to "click" on it. The browser knows when the user clicks, this means GET the document specified in the HREF attribute".
This Request/Response messaging pattern is core not only to the web but to the underlying TCP protocol over which the web (HTTP) runs.
Unlike the internet protocols used for something like Skype, or RealPlayer,
for web a browser to receive any information, it must first ask for it.
EVERYTHING you do on the web is affected by this important architectural constraint and this is one of the most important factors affecting the nature of the web and the design of applications that run on it.
This system of documents, links, requests and responses provide the fundamental application plaform on which every single web application is built.
Interestingly, the bright sparks who designed HTTP decided that whilst HTML could provide a natural format for hypertext documents, HTTP should not REQUIRE documents to be in HTML format. They can in fact be in any format including many that you'd maybe not think of as even being documents... text, images, videos, data and files containing programming code. Not only can documents be "linked" they can also be "embedded" so for example...
Using HTML you can link to an image for example, using the IMG "tag" (which has a src "attribute"). Instead of forcing the end user to request the resource the browser will fetch this resource inline and place it inline where the tag is.
so this....
<img src="http://pbskids.kids.us/images/sub-square-barney.gif" />
...will render as...
GOSH!
Interesting Historic note: The ability to "embed" was actually opposed by Tim Berners Lee (inventor of the web) presumably because he'd been "barneyed". The IMG was a custom tag used only by Marc Andressens "Mosaic" browser - "the pre-pre-precursor to todays FireFox, and it's certainly my view that if the IMG had not been included in HTML you'd not be reading this now.
In addition to linking and embedding documents HTML has all sorts of tags for formatting and describing documents.
There are about 100 tags in version 4 of HTML (HTML5 will introduce about another 25-30) which enable all kinds of display, embedding and linking of document elements.
If you view the source code of this page, you will see a huge collection of tags. This is the "markup" used to describe how this page should be displayed.
Of course, complex designs, necessitate complex descriptions, so you'll witness some fairly complicated looking HTML out there in the wild. There is no fast way to learn all these tags, but conceptually, as long as you understand the concepts of documents, elements and attributes then this should be able to build on this.
What I've tried to do over the last 2 posts is to try and get everyone to a common definition of the web. So, we all now understand that the web "is made of"...
Cool. So now we're all on the same page. Onwards and upwards! Only 998 miles to go!
Now we know how messages are sent around the web, what kind of things are we able to send? Well... Enter HYPERTEXT! (Wow!). The web was designed to send "Hypertext" around? But what is Hypertext? Essentially it's a body of text with an in place reference to another body of text. If you ever had one of those adventure game books as a child (perhaps giving away my long held nerdiness there) they work as a form of hypertext. "If you want to open the treasure chest, go to page 134, if not, go to page 14."
So, this is not necessarily a computery idea. In fact, the idea of hypertext is generally agreed as originating from an idea by an engineer with the best name in computing history... Vannevar Bush, way back in 1945. Whilst there are some very well know implementations of Hypertext (Adobe PDF and Microsoft's original Encarta) it was the creation of the web in 1992 that eventually led to wide scale adoption of hypertext.
The core language of the web, HTML (Hypertext Markup Language) provides a way of "Marking up" text to contain references to another body of text. It does this using a mechanism called a "hyperlink", which describes the reference to another piece of text.
For example...I write a document and say in it...
The Vancouver Sun's statement that "The 2011 movie TRON 2 could become the most expensive film ever made with a reported budget of £300 million dollars." has been debunked.
I can augment this information by pointing at references, without breaking the flow of the text. I do this using an HTML "element" (a label with angled brackets around it) to point at a "hypertext reference". HTML elements can contain "attributes" which are values associated with the elements.
So for example... The "A" (anchor) element has an "href" (hypertext reference) attribute.
The basic format of this is...
The Vancouver Sun's statement that "The 2011 movie TRON 2 could become the most expensive film ever made with a reported budget of £300 million dollars" has been <a href="http://www.slashfilm.com/2009/04/13/">debunked</a>
A web browser knows not to display the anchor element directly, but to underline the contained text and allow a user to "click" on it. The browser knows when the user clicks, this means GET the document specified in the HREF attribute".
This Request/Response messaging pattern is core not only to the web but to the underlying TCP protocol over which the web (HTTP) runs.
Unlike the internet protocols used for something like Skype, or RealPlayer,
for web a browser to receive any information, it must first ask for it.
EVERYTHING you do on the web is affected by this important architectural constraint and this is one of the most important factors affecting the nature of the web and the design of applications that run on it.
This system of documents, links, requests and responses provide the fundamental application plaform on which every single web application is built.
Interestingly, the bright sparks who designed HTTP decided that whilst HTML could provide a natural format for hypertext documents, HTTP should not REQUIRE documents to be in HTML format. They can in fact be in any format including many that you'd maybe not think of as even being documents... text, images, videos, data and files containing programming code. Not only can documents be "linked" they can also be "embedded" so for example...
Using HTML you can link to an image for example, using the IMG "tag" (which has a src "attribute"). Instead of forcing the end user to request the resource the browser will fetch this resource inline and place it inline where the tag is.
so this....
<img src="http://pbskids.kids.us/images/sub-square-barney.gif" />
...will render as...
GOSH!
Interesting Historic note: The ability to "embed" was actually opposed by Tim Berners Lee (inventor of the web) presumably because he'd been "barneyed". The IMG was a custom tag used only by Marc Andressens "Mosaic" browser - "the pre-pre-precursor to todays FireFox, and it's certainly my view that if the IMG had not been included in HTML you'd not be reading this now.
In addition to linking and embedding documents HTML has all sorts of tags for formatting and describing documents.
There are about 100 tags in version 4 of HTML (HTML5 will introduce about another 25-30) which enable all kinds of display, embedding and linking of document elements.
If you view the source code of this page, you will see a huge collection of tags. This is the "markup" used to describe how this page should be displayed.
Of course, complex designs, necessitate complex descriptions, so you'll witness some fairly complicated looking HTML out there in the wild. There is no fast way to learn all these tags, but conceptually, as long as you understand the concepts of documents, elements and attributes then this should be able to build on this.
What I've tried to do over the last 2 posts is to try and get everyone to a common definition of the web. So, we all now understand that the web "is made of"...
DNS - The Domain name system and protocol. Used to convert named servers "www.myserver.com" in to IP addresses 127.0.0.1
TCP/IP - The Transmission Control and Internet Protocol Suite. - Used to send messages around the network
HTTP - The Hypertext Transfer Protocol - Used to specify the purpose of a message sent over TCP/IP
HTML - Hypertext Markup Language - Used to define the messages sent.
Cool. So now we're all on the same page. Onwards and upwards! Only 998 miles to go!
Subscribe to:
Posts (Atom)