SlipStream™ Tutorial

SlipStream™, by SixSq, is a framework for the automatic deployment and testing of n-tier and distributed software systems. It makes extensive use of cloud computing technologies to provide dynamic near-production environments for realistic runtime creation of systems, permitting software stakeholders to assert their ability to release the software into production on a continuous basis. SlipStream™ relies on virtualization for deploying machines and uses a cloud computing back-end to manage the life-cycle of virtual images. SlipStream™ currently can use two cloud backends: StratusLab and Amazon EC2. SlipStream will continue to support more cloud back-ends, as needed and requested.

SlipStream™ is available as public services for both StratusLab and Amazon EC2. Before you can successfully use these services, you will need to have valid accounts with them.

This tutorial document is composed of several tutorials of increasing complexity, designed to get you started on SlipStream™ as fast and simply as possible. The first Simple Client/Server Tutorial uses the Apache2 web server to demonstrate a "hello world" of distributed system tests using SlipStream™. In this first tutorial, we show you how to use existing base images to create a deployment to test a simple client/server pair running on two different virtual machines in the cloud.

Reusing the first tutorial, we then move to a slightly more complex scenario where we show you how with a few button clicks the first tutorial can be turned into a powerful performance system test. This second case deploys several clients in parallel to show how to produce increased load on the web server.

We execute the tutorials in this document daily to make sure that the fundamentals of these tutorials are sound. However, if you experience problems going through these tutorials, you'll find a special appendix to this document with troubleshooting tips. If you think that any of these tutorials are not clear, confusing, missing insights or could simply be made better, please help us improving them by sending your feedback to <slipstream-support@sixsq.com>.

You can find the solutions to the SlipStream™ tutorials on the server in the Public/Tutorials project area. All of the required projects are publicly readable, you therefore only need a standard SlipStream™ account to view them.

Before we start the tutorial, here are a few important SlipStream™ concepts that will come up often. SlipStream™ is accessed via a web application. SixSq runs a public SlipStream™ Service on Amazon EC2 and a SlipStream™ Service on StratusLab. Your organization may run it's own SlipStream™ service; if so, contact your administrator to find the address of the server. If you have your own SlipStream™ instance and you want to use it for this tutorial, please make sure that your instance of SlipStream™ includes the Tutorial SlipStream™ Projects, which ships with the product by default.

Metadata about your images and deployments are organized into projects in SlipStream™. Each project then consists of a number of modules. The modules may be Project, Machine Image or Deployment modules. Here is a short description of these module types:

The web interface allows the parameters for each of these modules to be defined, edited, versioned and saved.

We also have two main workflows in SlipStream™: Image Creation and Deployment. Here is a short explanation of what they are:

These terms will be referred to throughout this document.

SlipStream™ is a secure web application requiring a personal account. To register for an account, fill in the registration form. Make sure you provide a valid email address as it will be used to validate your account. Once you've received the email, hit the confirmation link to activate your account. From that point-on, you are good to go on SlipStream™.

By default, SlipStream™ is configured to use the StratusLab reference cloud service for "cloud" resources. In order to build images and execute deployments, you must have an StratusLab account and you must fill your SlipStream™ profile with your StratusLab username and password. The simplest way to request a StratusLab account is to send an email to the StratusLab support mailing list. Once registered and activated, access the SlipStream™ application of your choice (e.g. SlipStream™ Service on Amazon EC2, SlipStream™ Service on StratusLab) and click on Start or the Start button, or the login link at the top right of the screen. Once logged in, go to you account. Once there, click the Edit button and select the "Properties" tab. Then fill the "StratusLab" or "Amazon" section of the form (see the comments next to the fields for guidance). Save the changes. Now you should be good to go!

The goal of this tutorial is to show you how to build a simple distributed test with a client and a server running on separate machines. We are going to use an Apache server as the server, put some data into it, and then from a separate machine test that the data can be accessed remotely. The end result, or the solution, of this tutorial can be found in SlipStream™ in the public project Public/Tutorials/HelloWorld.

To host the different modules we will create in this tutorial, first create a new top level project. Simply click on the "Start" button on the main page. If you haven't logged-in, you will be asked to do so. You will then be shown the root projects page, which will only list the projects you have read access to. To create a new project, click the "New Project" button. Give this project a unique name. Fill-in the form and save it. That's it; you have now created a new project. In this new project, create another project called, say, "HelloWorld" in which we will perform this tutorial.

The "Authorization" tab gives you access to the authorization for this module. As you can tell, by default, the project is only visible to you, as this is the default for each module you create. (Groups are not yet supported.) This means that nobody else will see this project in the root project list. To share this project with others, simply hit the Edit button, and check the checkbox corresponding to Read and Public.

Now that we have a project to work in, we need to create a couple of machine image modules: one for the server and another for the test suite. You can do this by navigating to the new project you've created in the previous step and clicking the "New Machine Image" button. This will add a new image to the current project.

As with most forms in the application, you will see a summary tab, followed by different more specialized tabs. In view mode, only the parts that are defined are displayed, whereas in edit mode, all the available tabs are displayed. For example, Machine Images have two specialized tabs: Creation and Deployment. Call this new machine image apache. The following subsections provide information for the other parts of the form.

As briefly explained in the introduction section, SlipStream™ supports two main workflows: "image creation" and "deployment execution". The advantage of creating new specific virtual machine is that during execution, these machine will be up and running very quickly, at the price of creating once a new machine, a process that can take up to 40 minutes. Therefore, for this tutorial, we will not create new machines but simply rely on the deployment instructions (called Targets). We do however encourage you to strongly consider creating new machine images whenever appropriate to improve the performance of you deployments. For this refer to the SlipStream™ Reference Manual.

#!/bin/sh -xe
apt-get update -y
apt-get install -y apache2

echo 'Hello from Apache deployed by SlipStream!' > /var/www/data.txt

service apache2 stop
port=$(ss-get port)
sed -i -e 's/^Listen.*$/Listen '$port'/' /etc/apache2/ports.conf
sed -i -e 's/^NameVirtualHost.*$/NameVirtualHost *:'$port'/' \
          /etc/apache2/ports.conf
sed -i -e 's/^<VirtualHost.*$/<VirtualHost *:'$port'>/' \
          /etc/apache2/sites-available/default
service apache2 start
ss-set ready true

#!/bin/sh -xe
apt-get update -y
apt-get install -y apache2

echo 'Hello from Apache deployed by SlipStream!' > /var/www/data.txt

service apache2 stop

port=$(ss-get port)

sed -i -e 's/^Listen.*$/Listen '$port'/' /etc/apache2/ports.conf
sed -i -e 's/^NameVirtualHost.*$/NameVirtualHost *:'$port'/' \
          /etc/apache2/ports.conf
sed -i -e 's/^<VirtualHost.*$/<VirtualHost *:'$port'>/' \
          /etc/apache2/sites-available/default

service apache2 start
ss-set ready true

#!/bin/sh -x
cp /var/log/apache2/access.log $SLIPSTREAM_REPORT_DIR
cp /var/log/apache2/error.log $SLIPSTREAM_REPORT_DIR

Now that we have created an Apache machine image module to act as our web server in our client/server tutorial, we next need to create the corresponding client to contact this server.

To go with the Apache server machine image we have created in the the previous section, we will now build a client machine image module containing a simple test suite client.

Since we now have a bit more experience with building images, here is a more condensed set of instructions for the client image. Following the same steps as for the Apache server, create in the same project a new Machine Image. Let us call this new image "testclient".

#!/bin/sh -xe
# Wait for the metadata to be resolved
web_server_ip=$(ss-get --timeout 360 webserver.hostname)
web_server_port=$(ss-get --timeout 360 webserver.port)
ss-get --timeout 360 webserver.ready

# Execute the test
ENDPOINT=http://${web_server_ip}:${web_server_port}/data.txt
wget -O /tmp/data.txt ${ENDPOINT} && \
    ss-set statecustom "OK: $(cat /tmp/data.txt)" || \
    ss-abort "Could not get the test file: ${ENDPOINT}"

#!/bin/sh -x
cp /tmp/data.txt $SLIPSTREAM_REPORT_DIR

Now that we have two machine image modules to work with, we will create a deployment module to run these images to form our simple client/server system test.

Go back to your project and click on the New Deployment button. Give it a sensible name, say client_server. You will see that deployments have the concept of Node. A Node in SlipStream™ is a running instance of an image. For this tutorial, we need to add two nodes corresponding to our Apache server and our test suite. To do this follow these simple steps in edit mode:

We are now ready to create our new deployment test.

To launch the distributed test, simply click on the "Run" button of the deployment you have created above. Sit back, relax and enjoy the show!

The execution instance created for each submission allows you to track the progress of the distributed test. Once the test is completed, you can access the reports generated by the different nodes, by clicking on the "Results" link.

You can access all your execution by clicking on the "dashboard" link at the top right of every page. To only list the execution of a given deployment module, select the "Runs" tab.

Thanks for taking the time to learn about SlipStream™ and please if there's anything we can do to improve this tutorial or the application itself, don't hesitate in getting back to us at <slipstream-support@sixsq.com>.

The goal of this tutorial is to demonstrate how SlipStream™ can be used to automate performance testing, reusing existing functional system test modules. A single parameter is required to instruct SlipStream™ to instantiate several instances of a single image, without needing any duplication of metadata in SlipStream™. We will use this feature to define a performance test, where several test suites will be launched together to create a higher load on the Apache server we already created in the "Simple Client/Server" tutorial. With a slight modification to the test suite, such that instead of downloading a document once, we will loop over a few times such that we can study the behaviour of Apache (or any server) under load.

If you have not performed the "Simple Client/Server" tutorial, you can find a solution of that tutorial in the area Public/Apache. However, we assume that you have acquired the knowledge presented in that first tutorial.

In your home project (refer to the "Simple Client/Server" tutorial), create another project called, say, "Performance_Tutorial". It is in this project that we will execute this tutorial.

Create a new deployment module in the "Performance_Tutorial" project called, say, "deployment". In this deployment we need to add two nodes corresponding to our Apache server and a test suite. To start with, we will reuse the "Simple Client/Server" tutorial test suite.

In this tutorial, we introduce the concept of multiplicity. This means that by setting a multiplicity value on a node, we instruct SlipStream™ to instantiate at runtime several times the same virtual machine, but with a distinct image names. When instantiating nodes with multiplicity, SlipStream™ creates entries on the execution instance of the form <nodename>-<i> where i is an index ranging from 1 to the value given to the multiplicity field.

To do this follow these steps:

We are now ready to submit our new deployment test.

To launch the distributed test, simply click on the "Run" button of the deployment you have created above.

The execution instance created for each submission allows you to track the progress of the distributed test. When the test is completed, you can access the reports generated by the different nodes, by clicking on the "Results" link.

In the report, you will now see an entry for each test suite, containing the specific report of the test suite instance.

While this is interesting, we can do better. Next, we will modify the test suite to increase the load.

Instead of simply reusing the test suite from the simple client/server tutorial, we could upgrade the test suite, in which we could for example loop a few times over getting the data, and record the access time for each retrieval. We could also perform this data retrieval from several threads, to make sure we saturate the network, and this by increasing the threads over time. By resubmitting this deployment and varying the multiplicity number, we should see how the number of clients and requests affect the web server's response time. Finally, we could also increase the size of the retrieved document, to increase again the load of the server and the network. We leave this exercise to the user and are looking forward to hearing your solutions.