Skip to main content

UPDATE: mitigation instructions have been improved (don't update openswan) and we forgot to mention rebooting.
UPDATE: Links to updated System VM templates are now below

Yesterday, a buffer overflow vulnerability was announced in glibc that affects most current Linux distributions. In CloudStack, the system VMs contain a vulnerable version of glibc.

CloudStack community members have built an updated system VM template, which ShapeBlue is hosting at http://packages.shapeblue.com/systemvmtemplate/ (More information on the packages at http://shapeblue.com/packages).

For instructions on how to update the SystemVM template in CloudStack, see here.

For those who wish to patch their running system VMs, ssh into each one and run:

apt-mark hold openswan apt-get clean apt-get update && apt-get upgrade

After updating glibc, the system will need to be rebooted.

Information about how to connect to your System VMs is available here.

Other CloudStack-related systems may be affected!

Please review security updates from Linux distributions you use on your management server, storage systems, hypervisors, as well as other Linux VMs and bare-metal systems running in your environments. This post provides instructions for determining if a system is vulnerable, as well as patching directions for common Linux distributions.

The Apache CloudStack project is pleased to announce the 4.3.2 release of the CloudStack cloud orchestration platform. This is a minor release of the 4.3 branch which released on March 25, 2014. The 4.3.2 release contains more than 100 bug fixes since the 4.3.1 release. As a bug fix release, no new features are included in 4.3.2.

As a minor release it is a simple upgrade from 4.3.0 or 4.3.1 with no architectural changes.

Documentation

The 4.3.2 release notes includes full list of corrected issues as well as upgrade instructions from previous versions of Apache CloudStack. Please see the Release Notes for a full list of corrected issues and upgrade instructions.

http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.3.2/

The official installation, administration and API documentation for each release are available on our Documentation Page.

http://docs.cloudstack.apache.org/

Downloads

The official source code for the 4.3.2 release can be downloaded from our Downloads Page.

http://cloudstack.apache.org/downloads.html

About Apache CloudStack

Apache CloudStack is an integrated Infrastructure-as-a-Service (IaaS) software platform that allows users to build feature-rich public and private cloud environments. CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources. The project became an Apache top level project in March 2013.

For additional marketing or communications information, please contact the marketing mailing list: marketing@cloudstack.apache.org

To learn how to join and contribute to the Apache CloudStack community please visit our website: cloudstack.apache.org

CVE-2014-7807: Apache CloudStack unauthenticated LDAP binds

CVSS:
7.5, AV:N/AC:L/Au:N/C:P/I:P/A:P

Vendors:
The Apache Software Foundation
Citrix, Inc.

Versions Afffected:
Apache CloudStack 4.3, 4.4

Description:
Apache CloudStack may be configured to authenticate LDAP users. When so configured, it performs a simple LDAP bind with the name and password provided by a user. Simple LDAP binds are defined with three mechanisms (RFC 4513): 1) username and password; 2) unauthenticated if only a username is specified; and 3) anonymous if neither username or password is specified. Currently, Apache CloudStack does not check if the password was provided which could allow an attacker to bind as an unauthenticated user.

Mitigation:
Users of Apache CloudStack 4.4 and derivatives should update to the latest version (4.4.2)

An updated release for Apache CloudStack 4.3.2 is in testing. Until that is released, we recommend following the mitigation below:

By default, many LDAP servers are not configured to allow unauthenticated binds. If the LDAP server in use allow this behaviour, a potential interim solution would be to consider disabling unauthenticated binds.

Credit:
This issue was identified by the Citrix Security Team.

Command Line Interface Tool Simplifies Apache CloudStack Configuration and Management

11 November 2014 —Apache CloudStack, the mature, turnkey Open Source cloud computing software platform used for creating private, public, and hybrid cloud environments, today announced Apache CloudMonkey v5.3.0, the latest feature release of its command line interface tool.

CloudMonkey is written in Python, and can be used both as an interactive shell and as a command line tool that simplifies CloudStack configuration and management.

Apache CloudMonkey v5.3.0 is the latest feature release of the 5.x line that was first released in September 2013. Some of the new features and changes include:

  • Unicode support in CloudMonkey;
  • Better autocompletion for API arguments, filter arguments and config options;
  • Current server profile is displayed on the prompt;
  • Changing server profile prints masked values of passwords and keys;
  • New command line argument -d for display options such as default, json and table;
  • New config option “verifysslcert” that enables/disables SSL certificate checking when making HTTP API calls;
  • CloudMonkey outputs without color on terminal in non-interactive mode;
  • Better error handling, errors written to stderr and non-zero exit codes in case of error;
  • Several bugfixes related to networking, server profiles and unicode string handling

Downloads and Documentation

The official source code for CloudMonkey v5.3.0 can be downloaded from http://cloudstack.apache.org/downloads.html. A community-maintained distribution is available at the Python Package Index (PyPi) at http://pypi.python.org/pypi/CloudMonkey/

CloudMonkey's usage is documented at https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+CloudMonkey+CLIPackage documentation can be found at http://pythonhosted.org/cloudmonkey/

Availability and Oversight

As with all Apache products, CloudMonkey is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. The Apache CloudStack Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases.

About Apache CloudStack

Apache CloudStack is a mature, turnkey integrated Infrastructure-as-a-Service (IaaS) Open Source software platform that allows users to build feature-rich public and private cloud environments. Hailed by Gartner Group as "a solid product", CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources. CloudStack entered the Apache Incubator in April 2012 and became an Apache Top-level Project in March 2013. For downloads, documentation, and ways to become involved with Apache CloudStack, visit http://cloudstack.apache.org/ and https://twitter.com/CloudStack

Mature, easy-to-deploy Open Source Cloud computing software platform boasts improved efficiency and performance.

The Apache CloudStack project announced the immediate availability of Apache CloudStack v4.4.1, the latest version of the turnkey Open Source cloud computing software platform used for creating private-, public-, and hybrid cloud environments.

Apache CloudStack clouds enable billions of dollars' worth of business transactions annually across their clouds, and its maturity and stability has led it to has become the Open Source platform for many service providers to set up on-demand, elastic public cloud computing services, as well as enterprises and others to set up a private or hybrid cloud for use by their own employees.

"We are delighted to be releasing version 4.4.1 of Apache CloudStack," said Giles Sirett, member of the Apache CloudStack Project Management Committee. "This latest version of CloudStack reflects months of hard work by our diverse developer community and brings even more features to help our service-provider and enterprise users enhance their cloud platforms. Apache CloudStack continues to grow in both deployments and developer community size, and is the platform of choice for thousands of organisations that need to build IaaS environments quickly and securely with a proven, production-grade, technology."

Lauded by Gartner Group, CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources.

CloudStack v4.4.1 reflects dozens of new features and improvements, including:

  • Improved Storage Management
  • Virtual Private Cloud tiers can now span guest networks across availability zones
  • Support for VMware Distributed Resource Scheduler
  • Improved Support for Hyper-V Zones, VPC and Storage Migration

A complete overview of all new enhancements can be found in the project release notes at http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.4.1/

CloudStack has been used by thousands of organizations worldwide and is in use/production at Alcatel-Lucent, Autodesk, BT Cloud, China Telecom, DATACAENTER Services, DataPipe, Edmunds.com, Exoscale, GreenQloud, Hokkaido University, IDC Frontier, Ikoula, KDDI, KT/Korea Telecom, LeaseWeb, NTT, Orange, PCextreme, Schuberg Philis, Shopzilla, Slovak Telekom, SunGard AS, Taiwan Mobile, Tata, Trader Media Group, TomTom, University of Melbourne, University of Sao Paolo, Verizon, WebMD and Zynga, among others.

CloudStack originated at Cloud.com, which was acquired by Citrix in 2011. CloudStack was submitted to the Apache Incubator in April 2012 and graduated as an Apache Software Foundation Top-level Project in March 2013.

Availability

CloudStack v4.4.1 is available immediately as a free download from http://cloudstack.apache.org/downloads.html. Apache CloudStack software is released under the Apache License v2.0.

Governance and Oversight

Apache CloudStack is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases.

Get Involved!

Apache CloudStack welcomes contribution and community participation through mailing lists as well as attending face-to-face MeetUps, developer trainings, and user events. Catch CloudStack in action at the CloudStack Collaboration Conference, the official user/developer conference of the Apache CloudStack community, 19-21 November 2014 in Budapest, Hungary @CCCEU14 and http://cloudstackcollab.org

About Apache CloudStack

Apache CloudStack is a mature, turnkey integrated Infrastructure-as-a-Service (IaaS) Open Source software platform that allows users to build feature-rich public and private cloud environments. Hailed by Gartner Group as "a solid product", CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources. CloudStack entered the Apache Incubator in April 2012 and became an Apache Top-level Project in March 2013. For downloads, documentation, and ways to become involved with Apache CloudStack, visit http://cloudstack.apache.org/ and https://twitter.com/CloudStack

© The Apache Software Foundation. "Apache", "CloudStack", "Apache CloudStack", the Apache CloudStack logo, and the Apache CloudStack Cloud Monkey logo are registered trademarks or trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

As previously mentioned, the realhostip.com dynamic DNS service is being retired at the end of September.

Citrix is reporting that they are still seeing DNS queries against the domain; Those who have not reconfigured their CloudStack installations will find part of their installations breaking once the realhostip service is retired on September 30th.

If you are running a version of CloudStack older than 4.3 and you have not reconfigured your installation to not use realhostip.com, please take the time to do so now before users are affected. Instructions are available in the CloudStack Wiki as well as other blogs on the Internet.

Command Line Interface Tool Simplifies Apache CloudStack Configuration and Management

28 August 2014 —Apache CloudStack, the mature, turnkey Open Source cloud computing software platform used for creating private, public, and hybrid cloud environments, today announced Apache CloudMonkey v5.2.0, the latest feature release of its command line interface tool.

CloudMonkey is written in Python, and can be used both as an interactive shell and as a command line tool that simplifies CloudStack configuration and management.

Apache CloudMonkey v5.2.0 is the latest feature release of the 5.x line that was first released in September 2013. Some of the new features and changes include:

  • Multiple server profiles where users can use CloudMonkey against different CloudStack management servers and switch between them using a profile option;
  • A default profile under the section [local] is added with default values;
  • Some bugfixes related to network requests, error handling, JSON decoding and shell interactivity;
  • Every time 'set' is called, CloudMonkey will write the config and reload config file;
  • Configuration options 'protocol', 'host', 'port', 'path' are deprecated now but setting them is still allowed which sets a single "url" option, in the config file the [server] section is deprecated now and CloudMonkey won’t read values from this section anymore but instead read from current server profile;
  • Missing key/values are automatically set with defaults by CloudMonkey;
  • During installation and upgrades, it will detect the platform to install either pyreadline (Windows) or readline (OSX and Linux);

Downloads and Documentation

The official source code for CloudMonkey v5.2.0 can be downloaded from http://cloudstack.apache.org/downloads.html. A community-maintained distribution is available at the Python Package Index (PyPi) at http://pypi.python.org/pypi/CloudMonkey/

CloudMonkey's usage is documented at https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+CloudMonkey+CLIPackage documentation can be found at http://pythonhosted.org/cloudmonkey/

Availability and Oversight

As with all Apache products, CloudMonkey is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. The Apache CloudStack Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases.

About Apache CloudStack

Apache CloudStack is a mature, turnkey integrated Infrastructure-as-a-Service (IaaS) Open Source software platform that allows users to build feature-rich public and private cloud environments. Hailed by Gartner Group as "a solid product", CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources. CloudStack entered the Apache Incubator in April 2012 and became an Apache Top-level Project in March 2013. For downloads, documentation, and ways to become involved with Apache CloudStack, visit http://cloudstack.apache.org/ and https://twitter.com/CloudStack

As mentioned previously, the realhostip.com dynamic DNS resolver service is being retired this summer. During testing of Apache CloudStack version 4.3, we found a few more issues related to realhostip.com that have been addressed for the 4.4 release.

In order to give everybody a reasonable window to update their CloudStack installations to use the updated code, the retirement date for the realhostip.com service has been pushed back to September 30th, 2014. This provides an additional 3 months from the original June 30th date.

Any questions related to the retirement of the realhostip.com service and it's affect on CloudStack installations should be send to the CloudStack Users or Development mailing lists. Further information about how to subscribe and interact with the mailing lists is available at https://cloudstack.apache.org/mailing-lists.html.

Update: The email infrastructure is back to normal, although it may take some time for the queued messages to completely flush out of the backlog. See the linked post from the ASF infrastructure team below for more details.

There are ongoing problems with the ASF's email infrastructure that mean no mail delivery is happening for our project's lists. Please check the official ASF infrastructure blog post and twitter feed for updates.

UPDATE 1: The instructions below are incomplete. The proper upgrade command is "apt-get install openssl libssl1.0.0". If you've just updated openssl, please go back and update libssl as well.
UPDATE 2: Adding instructions for VMWare-hosted System VMs (Thanks to Geoff Higginbottom at ShapeBlue)
UPDATE 3: Added instructions for verification of correct versions installed
UPDATE 4: Apache CloudStack 4.0-4.1 not vulnerable, they use older Debian/openssl.

Earlier this week, a security vulnerability was disclosed in OpenSSL, one of the software libraries that Apache CloudStack uses to encrypt data sent over network connections. As the vulnerability has existed in OpenSSL since early 2012, System VMs in Apache CloudStack versions 4.1.1-4.3 are running software using vulnerable versions of OpenSSL. This includes CloudStack's Virtual Router VMs, Console Proxy VMs, and Secondary Storage VMs.

We are actively working on creating updated System VM templates for each recent version of Apache CloudStack, and for each of the hypervisor platforms which Apache CloudStack supports. Due to our testing and QA processes, this will take several days. In the meantime, we want to provide our users with a temporary workaround for currently running System VMs.

If you are running Apache CloudStack 4.0.0-incubating through the recent 4.3 release, the following steps will help ensure the security of your cloud infrastructure until an updated version of the System VM template is available:

For KVM/Xen hosted systems

  1. As an administrator in the CloudStack web UI, navigate to Infrastructure->System VMs
  2. For each System VM listed, note the host it is running on, and it’s “Link Local IP address."
  3. With that data, perform the following steps for each System VM:
    1. ssh into that host as root
    2. From the host, ssh into the SSVM via it’s link local IP address: (e.g. ssh -i /root/.ssh/id_rsa.cloud -p 3922 169.254.3.33)
    3. On the System VM, first run "apt-get update"
    4. Then run apt-get install "openssl libssl1.0.0". If a dialog appears asking to restart programs, accept it’s request.
    5. Next, for Secondary Storage VMs, run /etc/init.d/apache2 restart
    6. Log out of the System VM and host server
  4. Back in the CloudStack UI, now navigate to Infrastructure->Virtual Routers. For each VR, host it's running on and it's link local IP address, and then repeat steps a-f above.

For VMWare hosted systems

  1. As an administrator in the CloudStack web UI, navigate to Infrastructure->System VMs
  2. For each System VM listed, note it's management IP address
  3. With that data, perform the following steps for each System VM:
  4. From the Management Server, ssh to the System VM via it's management IP: (eg ssh -i /var/lib/cloud/management/.ssh/id_rsa -p 3922 root@10.40.50.8)
  5. On the System VM, first run "apt-get update"
  6. Then run apt-get install "openssl libssl1.0.0". If a dialog appears asking to restart programs, accept it’s request.
  7. Next, for Secondary Storage VMs, run /etc/init.d/apache2 restart
  8. Log out of the System VM
  9. Back in the CloudStack UI, now navigate to Infrastructure->Virtual Routers. For each VR, host it's running on and it's link local IP address, and then repeat steps a-f above.

Verification

On each System VM, you can test if it has non-vulnerable openssl packages installed by listing installed packages and looking at the installed versions of openssl and libssl. As in the example below, for a system to be non-vulnerable, the packages need to be at or above version 1.0.1e-2+deb7u6:

root@v-14-VM:~# dpkg -l|grep ssl ii  libssl1.0.0:i386                     1.0.1e-2+deb7u6                  i386         SSL shared libraries ii  openssl                              1.0.1e-2+deb7u6                  i386         Secure Socket Layer (SSL) binary and related cryptographic tools

We realize that for larger installations where System VMs are being actively created and destroyed based on customer demand, this is a very rough stop-gap. The Apache CloudStack security team is actively working on a more permanent fix and will be releasing that to the community as soon as possible.

For Apache CloudStack installations that secure the web-based user-interface with SSL, these may also be vulnerable to HeartBleed, but that is outside the scope of this blog post. We recommend testing your installation with [1] to determine if you need to patch/upgrade the SSL library used by any web servers (or other SSL-based services) you use.

http://filippo.io/Heartbleed/