svn

How deployment system works

The overall scheme how deployment system works is follows:

Developers commit their changes into SVN repository.

When a commit is inserted into SVN repository, the deployment system is notified about new changes using hooks.

The deployment server then retrieves files of the project from the SVN repository, builds the application, and pushes it to servers of the cluster.

The whole process is completely automated. Developer just commits changes into SVN repository and deployment system will do the rest to update cluster.

Deployment clusters and servers

The picture above shows only one cluster.

In the real world, however, things are a little more complicated, and developers may need to deploy their applications to many clusters.

In the simplest example this may be a production and a demo cluster. There may also be a cluster for each developer (for them to be able to test their features and not to disturb other developers), a staging cluster to show current most stable application release in opposite to demo cluster that contains the latest version of the application.

Take no notice that a demo cluster may contain only a single server. A cluster of a single server is still a cluster.

So, there are two entities considered in the deployment process: a cluster, and a server.

Cluster is a set of servers.

A server is not always a separate computer. For example, we may install demo and staging servers at a single computer.

Each server belongs to exactly one cluster.

Deployment branches

Each cluster is connected to a branch inside an SVN repository.

All deployment branches are stored under servers/ directory in the SVN repository. The trunk/ directory is sometimes an exclusion from this rule - there may be a cluster that is deployed directly from trunk/.

Deployment branches are named after the cluster designation. Example names are:

• servers/demo/ - for demo cluster

• servers/production/ - for production cluster

• servers/staging/ - for staging cluster

• servers/yuri.popoff/ - for developer

When a developer commits into a branch, connected cluster gets updated.

More then one cluster can be connected to a branch. In this case, all these clusters are updated when a commit is made into this branch.

The deployment process steps

Deployment process consists of the following steps:

1. Servers of the cluster are scheduled for update.

   The reason for this step is that if we have a very large cluster (1000 computers), then the single deployment server may hang trying to serve all those servers in parallel.

   All servers are starting at this step and are eventually moved to the next step when there are no more then a predefined limit of servers operating at step two.

2. Changed files are uploaded to servers.

   At this step files are copied to a temporary directory allowing users to access the cluster without noticing that the cluster is being updated.

3. Servers are locked from access.

   The reason to lock servers is that we are preparing to update database structure. If someone accesses the application while database is in an intermediate state, this may cause errors and unexpected behaviour.

   When servers are locked, all bots stop their processing, and users can not access the cluster. If they try, they'll see a message like “We are updating the software, please, try againg in several seconds.”.

   Also, when a server is locked, new files are moved from a temporary directory to their persistant place.

4. Database structure is updated

5. The lock is released

   After the lock is released, all caches are flushed.

All servers of the cluster perform update at the same step. For example, no servers will be locked if there is at least one server that has not yet completed files update. Database will not be updated until all servers of the cluster get locked. Lock will not be released from any server until database update process is completed.

To move servers between steps correctly, servers and clusters are assigned a state at each moment of time. The states are described in the articles below.

See also

Последняя модификация: 21.03.13 14:16