Notes : Chapter 03 of Web Scalability For Startup Engineers, P1: Managing state
According to the autho, frontend layer include the client, components between the client and data center, parts of data center that respond directly to the client, this layer has most of the traffic, high throughput.
We have 3 types of frontend applications :
- Multipage applications where we send a request to the server each time we want to go to specific page.
- Single page applications great for apps with a lot of user interactions, with SPAs, users don’t have to wait long to see changes on the page, instead of reloading the whole page every time, the app sends smaller requests to the server, asking only for the needed information, the server sends back just that information, and the app updates only the part of the page that needs to change, not the entire page.
- Hybird applications which a mix both where some requests reload the entire page while other request data from server and update only some parts of the page.
Note: the author only covered how to manage state of the multipage and hybird applications.
Keeping the frontend with no state is going to enable us to scale better by simply adding more clones.
Statelness: is property related to a service, object, or a server, meaning that it does not hold any data, so any instance can replace another without affecting the outcome and the user experience (the instance does not store data about his previous interactions or states), these instances delegate the state management to other services.
A frontend server instance that contains user session data, or other in memory data , local files is a stateful server , because it contains data that other instances of the same type can not access
Managing HTTP Sessions
Session: is responsible for managing communication between two resources, it also the time between when communication start and get closed, session layer ensure that this session stay open when it is needed to transfer data and closed it to avoid wasting resources.
Http are stateless but we can build a session storage to manage and recognize that a request is coming from a specific user.
The http contract specify that all active cookies should be included in the request sent to the server that set them, this feature allow the server to know which request belong to particular user even if he is connected using multiple browser using the same ip address.
It is better if we can keep the session storage outside of it’s main server so it is going to be accessible by any other server that we are using.
There is some solution for this , for example we can:
- Store session state in cookies: by storing session data in requests cookies,are not storing it in our data center , the server handle the creation of session cookies and updating them , we can call the application in this case as stateless in the context of http sessions, this approach will make our requests exchange some heavy session data each time a request is sent even when we try to load css files , images …, base64 encryption increase the size of the encrypted bytes by ⅓ , if request is 1kb it becomes 1.3kb after encryption.
- Use an external data store: it is when we have only the session id or key in the cookies data , the server will then read this session id and connect with the session storage to retrieve current session data, we need to keep latency as low as possible by using stores like memcashed , redis, dynamodb, cassandra, it is good if our data storage can handle data partitioning automatically as the horizontal scaling is going to be difficult if we do it alone. It is possible to use technologies that facilitate object clustering (managing and sync objects in multiple nodes in a distributed system to ensure that all objects work together seamlessly ) like ( Terracotta: allow the modification of data in the distributed system like they are in the same server hiding complexity of distributed system, ensure synchronization of data between nodes, manage access to object to insure data integrity when multiple nodes try to update the same object, )
- Use a load balancer that support sticky sessions: session management is going to be handled by load balancer and not the server , the load balancer will inspect the request header to ensure that the session cookie goes to the server who initiated the session , this approach break the principle of statelessness of http request, auto scale without breaking users data because there data is going to be bounded a single user machine.
Managing Files
The usage of amazon S3 or similar services will provide a better way to scale things with a usage of a cdn.
The user will upload file to a server , the server will push it to File storage service (ex: S3), and request files that are not present in CDN.
If we are enable to use cloud providers , consider partitioning files and storing them in multiple random server, if we need more server we can use a weighted server selection approach to allow to set the percentage of files that can be stored in each server , by storing a files into multiple radom server and saving the location into a metadata database.
In each server and incase we don’t need a high throughput we can setup a regular server with a RAID controllers a piece of hardware that handle the RAID processing it handle this task instead of a cpu, this approach is going to save us a lot of time and money.
We also need to use redundancy on the server level where we can store a file in multiple physical servers.
The situation become more complex when we want to perform multiple reads and write on the same files in this case we can either introduce SSD or partition our servers into a much more smaller servers to get a higher throughput and lower random access times.
For availability we can use RAID controller and for a higher availability we can use replication of files in multiple servers, we can make our app copy the file into the different server or use Resync or something similar to keep each master server in sync with slave (s).
Building a personal complete and scalable file storage service is hard, so instead we can go to an open source solution for this.
For example mongodb allow us to store files in a mongodb cluster using GRIDFS , this tool split files into a smaller chunks and store them into a mongo db collections as if they were regular documents with this approach we can scale only a one system and benefit from the advantages of replication and partitioning that mongo db offer to us. There is also some other solutions like e Astyanax Chunked Object released by netflix , which use cassandra as a data store and allow you to leverage cassandra features including transparent partitioning, redundancy, and failover. Instead of building our own partitioning , redundancy, and failover.
The implementation of data file storage with cassandra help in avoiding hotspots by randomizing file chunks download and downloading them from different servers/node in the cluster.
Hotspots: in distributed systems hotspots happen when a one resource experience high overload or when many clients try to access same node.
We should always go for a cloud based file storage, if not we can look for an open source alternative, and our last option should be building a file storage from scratch before doing this we should learn from google file storage system, ClusterFS, and hadoop distributed file system.
Managing other types of state
We might face some cases where local server cache, application in memory state, and resource locks prevent us from scaling our applications.
An example of this if we want to build an ecommerce application and we need to display auctions in a real time we might want to cache auction data to improve performance, if we store the cached data in our web server memory we will face a problem when we want to revalidate the cache we have to go to each server and do that, caching the data in a shared cache object is better where we can have only a one copy of an object, but in other cases like building web blogging app we might cache the users number of followers and names , in server memories where clients might see different number of followers based on the server they access, so it depend on our business if it accept data inconsistency or not
Another type of server state, is locks which is used to prevent race condition and to sync access to shared resources, locks are used in the frontend layer and some of the application use them locally , which cause a problem where locks are locked independently in each server without any sync, the solution to this is to handle lock outside of the server.
keeping the lock and the auction modifier in the same server will cause an issue if we create clone of our server, where a two threads will try to access the biding website and update the same auction without knowing that there is another one updating it.
To solve this we can use functional partitioning and clone our services , and then separate the lock from the service and share it globally, with this we can ensure that our servers does not hold any local state and can be scaled and cloned independently.
Isolating a functionality that require a global state into a separate service, so it become easier for us to scale and it also hide the shared state behind a layer of abstraction, but this method might increase latency where the app need to do more network calls to a service , that was available locally , by creatin a separate component we might end up with a more complexity to manage and maintain.
Implementing locking depend on the programming language that we are using , in case of using java we can use Zookeeper with Curator (netflix), or use the atomicity of the nosql data stores to implement locks this solution is much more simple and can be used with scripting languages like php , ruby.
We should keep our services stateless so we can scale them easily by adding more clones.
