XWiki in the cloud

Posted on 06 Apr 2012 | Back to Bitloom home

The XWiki Platform Cloud is a project I contributed to develop in the context of research activities we are doing at XWiki.

It is an extenstion that allows XWiki to be deployed on cloud environments.

There are two main ideas behind this extension:

The first point is about the retrieval of configuration parameters that are usually passed using environment variables, either through the JVM or the operating system.

This is the case of Amazon Elastic Beanstalk (see later), but also of CloudFoundry and Heroku

Amazon Elastic Beanstalk allows the user to specify up to eight environment variables (i.e., AWS_ACCESS_KEY_ID, AWS_SECRET_KEY_ID, JDBC_CONNECTION_STRING, PARAM1 to PARAM5).

CloudFoundry uses environment variables to pass JSON encoded datastructures containing information about the services bound to the application, and their configuration parameters. Here, for example, you can look at a dump of the environment for a hello world application. If you search for VCAP_SERVICES you will see the JSON describing the services available to the application which include a mysql-5.1 service, running on a given hostname, exposing a database with a given name, that can be accessed by a user with a given username and a given password.

Heroku’s approach is similar to Amazon’s.

XWiki dynamic configuration

XWiki Platform Cloud introduces two components that allow to take advantage of environment variable for configuration:

The configuration data source is able to retrieve configuration keys also from the JVM and operating system environments. It is also able to perform variable remapping, which is the equivalent of having pointers in configuration files. We’ll see later how this is useful.

The Hibernate configuration provider API has a default implementation that dynamically build the Hibernate configuration file using the previously mentioned data source. It overrides all the properties found in the bundled hibernate.cfg.xml by using the values coming from environmental variables (retrieved using the configuration data source)

The remapping feature is useful in some cases when we have different properties with the same semantics.

This is the case of Amazon Elastic Beanstalk. The JDBC_CONNECTION_STRING is used to indicate the JDBC endpoint to connect to. This is exactly what connection.url property in the hibernate.cfg.xml file specifies. By using remapping we can tell to remap connection.url to JDBC_CONNECTION_STRING so that when the hibernate configurator will dynamically build the hibernate.cfg.xml to initialize XWiki, it will take the value of the connection.url property from the value of the JDBC_CONNECTION_STRING provided by the cloud environment.

The remapping.properties file shows the remapping that are used in the distribution built to be deployed on the Amazon Elastic Beanstalk

By defining appropriate remapping properties we can build distributions for different cloud environments, provided that there is a one-to-one correspondence between the semantics of an environment variable and an hibernate.cfg.xml property. This is not the case for CloudFoundry where an explicit hibernate configurator capable to parse the JSON file and extract the needed information should be provided.

The major advantage here is that your Web Application WAR always stays the same for a given cloud environment. You won’t need to touch any configuration file. What changes, in fact, are the parameters you provide to your application at startup using the cloud service’s UI and APIs.

Cloud services

Currently the XWiki Platform Cloud project provides the experimental support for storing attachments on a Amazon S3 compatible storage.

Deploying on Amazon Elastic Beanstalk

In the following sections I will detail the steps to deploy an XWiki using the Amazon AWS infrastructure and, in particular, the Elastic Beanstalk service. If you want to play with it, Amazon provides a free usage tier that allows user to instantiate virtual machines, databases, and so on for free, provided that certain constraints on their usage are met.

Create the application

The first step consists in uploading the XWiki WAR and create an application in the elastic beanstalk service.

The following pictures show what you are expected to see during these phases (click on them for a larger version).

Create a new application
Don't launch a new environment
Your WAR will be available on S3

Setup the database

In order to make XWiki store its data, we will use the Amazon RDS service which allows to instantiate and manage a relational database. We will use MySQL.

Create the parameter group

We need to tweak the default MySQL configuration in order to be able to upload and store large files. In particular we need to set the max_allowed_packet parameter to a larger value, as indicated in the XWiki MySQL installation guide

Amazon RDS allows users to provide custom configurations by using parameter groups. Unfortunately there is no UI for doing this in the web management console, so we’ll need to use a terminal.

In the following ACCESS_KEY and SECRET_KEY are your Amazon AWS keys that you can retrieve on the security credentials page

Setup the security group for RDS instances

RDS uses Amazon EC2 to provide the actual infrastructure. EC2 virtual machines are configured using security groups that allow a system administrator to define which ports should be open to the outside world.

In order to allow connections to the DB server we need to open the MySQL port, which normally is 3306. To do this we need to perform the following steps:

At this point, by default, RDS database instances will use the elasticbeanstalk-default security group, defined in EC2. Now we need to open the MySQL port by modifying the elasticbeanstalk-default security group:

The following pictures show what you are expected to see during these phases (click on them for a larger version).

Setup security groups in RDS
Add the MySQL port to EC2 security group
MySQL port added to EC2 security group

Create and start the database server

The following pictures show what you are expected to see during these phases (click on them for a larger version).

MySQL selection
DB instance details
Additional configuration

Management options
Configuration summary
Database endpoint

Launch a new environment

Now we can start a new environment for our XWiki.

We are not yet ready. We must configure the XWiki instance so that it will correctly connect to the RDS database we created earlier. For some reason it’s not possible to specify these parameters in the previous step. Instead, to do so, we need to edit the configuration of an already started environment:

The following pictures show what you are expected to see during these phases (click on them for a larger version).

Launch new environment
Environment configuration
Environment configuration summary

Environment started
Edit configuration
Environment parameter setup

Connect to XWiki

The environment will restart and after a while you can connect to the URI you chose when you initially launched the environment and you will finally have an XWiki up an running on Amazon Elastic Beanstalk. You can now import the initial XAR and start working.

XWiki redirect page
XWiki main page

Differently from a standard XWiki deployment, the welcome page shows a link click here to go to the main wiki page. This done in order to avoid an issue with the elastic beanstalk health check.

This service, in fact, tries to ping an URI of your application (you can setup which one when you launch your environment) in order to understand if it is alive or not. Since XWiki takes some time to initialize, the first time the health check service will ping the URI it would believe that XWiki is dead and will put the whole application in an error state and shut it down.

By replacing the standard home page (which by default is an immediate redirect to the main xwiki page) with a static page we can delay XWiki initialization and make the health checker happy.

Using the S3 storage

The XWiki Platform Cloud provides the experimental support for storing attachments on a Amazon S3 compatible storage.

In order to use it you can perform the previously described steps with the following modifications: