Skip to main content

Big news this week: Apache CloudStack has passed major milestones towards graduation, major progress on bugs against 4.1.0, discussions about integrating the Palo Alto firewall with CloudStack, and getting Eclipse and Maven to play nice for developers. Some interesting posts from members of the Apache CloudStack community as well.

Want to keep reading the CloudStack Weekly News? See the next section for information on how to contribute.

Contributing to the Apache CloudStack Weekly News

If you have an event, discussion, or other item to contribute to the Weekly News, you can add it directly to the wiki by editing the issue you want your item to appear in. (The next week's issue is created before the current issue is published - so at any time there should be at least one issue ready to edit.) Alternatively, you can send to the cloudstack-dev mailing list with a subject including News: description of topic or email the newsletter editor directly (jzb at apache.org), again with the subject News: description of topic. Please include a link to the discussion in the mailing list archive or Web page with details of the event, etc.

Major Discussions

CloudStack Getting Closer to Graduation

Chip Childers notified the project that the graduation resolution has passed the IPMC vote. With the votes passed, it's now up to the Apache Software Foundation board to discuss. The next board meeting is on March 20th, though it's unclear whether the IPMC vote was finalized in time for the matter to be discussed during this meeting.

Eclipse and Maven Working Together

Alex Huang wrote last week that he's made some changes to "help developers with better productivity." To put it succinctly, Alex found that there are better ways to build CloudStack using Eclipse and Maven together, and has written a script to help developers speed up their tools. The procedure and new impatient profile should be documented on the wiki soon.

Palo Alto

Will Stevens brought up a topic about integrating the Palo Alto firewall with CloudStack:

The problem I am running into right now is that Palo Alto does not allow any two interfaced to have the same IP (even if they are in different zones, vrs, vsys and vlans). This is an issue because CloudStack supports each account having their own private IP ranges and two accounts can use the same private IP range. For example, by default if you create a network with source nat and you do not specify any gateway or subnet data, it will give you 10.1.1.0/24 as an IP range. This means it will be very likely that two CloudStack accounts will be using the same private IP space.

Alex replied, "There is a mode in CloudStack that only allows non-intersecting cidrs for guest networks. It was introduced specifically because many physical network devices do not expect cidrs to intersect even when it's on different VLANs."

Stuck on a technical issue? Ask on the -dev list, and it's quite likely you'll not only get an answer – odds are someone's had the problem you've got before, and thought about a solution.

CloudStack Planet - Posts from the CloudStack Community

Security in the Cloud and the CCSK

Sebastien Goasguen wrote about security in the cloud this week, and the Certificate of Cloud Security Knowledge (CCSK).

SELinux + KVM + CloudStack

David Nalley blogged about getting an SELinux policy to work for CloudStack because, "I really dislike advocating for people to turn off a security mechanism to get software to work. Additionally I really want some of the advantages of sVirt."

HOWTO: Using CloudStack Resources with Puppet (Part 1)

David also started writing about using CloudStack resources with Puppet. David says he's written and talked about CloudStack resources previously, but "while cheerleading and telling people it is awesome should be enough - it really doesn't tell you how to actually use it."

LDAP Authentication in CloudStack (v4.0.1)

Kirt Jantzer wrote about LDAP Authentication in CloudStack (v4.0.1) this week.

PCExtreme Case Study

Posted to the wiki this week, a case study about PCExtreme's use of Apache CloudStack: "PCExtreme Achieves Business Agility with Apache CloudStack"

Upcoming Events

Jira

Things are looking up this week, bug-wise. Bugs have dropped in all categories, and all of the blocker and critical bugs have been assigned.

New Committers and PPMC Members

No new committers or PPMC members were announced last week.

acwn-icon.pngThis week the project started the vote to determine whether the Apache CloudStack project feels ready to self-govern, discussions were held about implementing BVT (and how), and the PPMC announced its consensus for the proposed PMC chair. In addition, several technical discussions for features and processes within the project, and discussion on the -marketing list about the design of the Web site.

Want to keep reading the CloudStack Weekly News? See the next section for information on how to contribute.

Contributing to the Apache CloudStack Weekly News

If you have an event, discussion, or other item to contribute to the Weekly News, you can add it directly to the wiki by editing the issue you want your item to appear in. (The next week's issue is created before the current issue is published - so at any time there should be at least one issue ready to edit.) Alternatively, you can send to the cloudstack-dev mailing list with a subject including News: description of topic or email the newsletter editor directly (jzb at apache.org), again with the subject News: description of topic. Please include a link to the discussion in the mailing list archive or Web page with details of the event, etc.

Major Discussions

Some of the major discussions and issues that have taken place on cloudstack-dev, cloudstack-marketing and cloudstack-users in the past week. This is by no means exhaustive, if you need to be up-to-date on all development issues in the project, you'll definitely want to be subscribed to the mailing lists!

VOTE: Graduate Apache CloudStack from the Incubator

Chip Childers has started the vote on whether to graduate Apache CloudStack from the Incubator. The vote lasts 72 hours (vote started Monday morning). If successful, the IPMC will need to vote on the resolution and recommend graduation to the Apache Board of Directors.

PPMC Will Recommend Chip Childers as Chair

In conjunction with the graduation discussion, the CloudStack PPMC has reached consensus on recommending Chip Childers as chair, once the project graduates. See How the ASF Works: Roles for more on roles within Top Level Projects (TLPs).

Meet Cato_Fong

If you haven't already, Prasanna Santhanam invites everyone to meet Cato_Fong, a handy IRC bot that keeps an eye on the Jenkins builds for the project. Says Prasanna, "if the build is broken and the executor on builds.a.o has not yet queued the job be prepared to be ambushed by Cato_Fong!"

Syslog Enhancements

Anshul Gangwar has revived a discussion from December about using log4j to send syslog messages. See the FS on the wiki, and join in the discussion on the -dev mailing list if you have anything to add.

Getting Rid of KVM Patchdisk Raises Discussion of New System VM Programming Model

Marcus Sorensen started a discussion last week about getting rid of the patchdisk that KVM creates on primary storage when bringing up a VM. Marcus says, "This patch disk has been bugging me for awhile, as it creates a volume
that isn't really tracked anywhere or known about in cloudstack's database. Up until recently these would just litter the KVM primary
storages, but there's been some triage done to attempt to clean them up when the system vms go away. It's not perfect. It also can be
inefficient for certain primary storage types, for example if you end up creating a bunch of 10MB luns on a SAN for these."

Later in the discussion, Edison Su responded to Marcus' approach to handling the issue. Edison says "we put a lot of logic into init scripts inside system VM, which has unnecessarily complicated the system VM programming." Ultimately, Edison suggest that we may need "to start working on a new system VM programming model now? Better to just put a Python daemon inside system VM, and provide restful API through link local IP address(or private IP if it's vmware), then mgt server or hypervisor agent code can just send commands to the Python daemon through HTTP, instead of SSH."

Build Verification Test (BVT) for CloudStack Checkins

Alex Huang proposed building a BVT system to "ensure that checkins do not break the master branch."

After a fair amount of discussion, Chip Childers responded, saying that the first step to getting Gerrit is "for us to agree to using it and to be able to clearly articulate why. Without being able to explain our issue, we'll be questioned about jumping to a tool-based solution by the infra team."

The entire discussion is worth a read for anyone involved in day-to-day CloudStack development. (Indeed, one hopes that all developers have paid close attention to this thread, not only the handful of active voices who have jumped into the discussion.)

Proposal to Refactor Appliance Building Code Under tools/appliance

Rohit Yadav has proposed to "fix/refactor devcloud appliance building scripts from tools/devcloud/sr to tools/appliance" and "to create a new devcloud that has all the build tools, IDE and possibly a lightweight desktop environment or tiling window manager."

Incremental Build

Edison points out that Maven 3 doesn't support incremental builds, "so it takes a long time (4-7 minutes) to make a build" even with small changes in the source code. Animesh Chaturvedi points out a tool that might help add incremental build support to Maven, but there was no enthusiasm for changing tools away from Maven after the effort that went into switching from Ant.

Marketing: Website Discussions

There's been a fair amount of discussion this week about improving the Apache CloudStack Web site. Sebastien Goasguen started the discussion on March 5th, pointing to a site that he found "simple, yet modern." The discussion has moved on to work on the sitemap and flow of the site.

No firm proposals or decisions have come out of the discussion as of yet.

There's also discussion about the elevator pitch and top N features of CloudStack that should be on the landing page.

LDAP Setup Assistance

Kirk Jantzer came to the -users list looking for help in setting up LDAP. Ilya Musayev shared an earlier note to -dev about using Microsoft's Active Directory (LDAP) working with CloudStack, and Kirk reported that he was able to get it working by adding users with dummy passwords.

System VM Templates

Following a discussion on -users, Chip noted that 4.1.0-incubating will use the same system VM templates used by 4.0.0-incubating and CloudStack releases from Citrix prior to its donation to Apache. However, Chip adds, "we're working on an automated system VM build process that should be part of 4.2.0-incubating (allowing us to refresh the image that's used)."

CloudStack Planet - Posts from the CloudStack Community

  • David Nalley wrote about efficient use of CloudStack using Puppet.

Upcoming Events

Jira

The number of issues for 4.1.0 has crept back up in the last week, likely because QA had been blocked from testing by some bugs previously - with those out of the way, QA is able to find more issues in the release.

Note that some of the issues are also less dire than they seem: CLOUDSTACK-1584 is a blocker for release, but it's only to ensure that an important task (adding links to the documentation for downloads) is completed before the release.

New Committers and PPMC Members

No new committers or PPMC members were announced last week.

acwn-icon.pngThis week, a vote on updating the project bylaws, fixing the "Tomcat situation" after 4.1, and discussions around the support lifecycle. Some respectable progress in knocking out major and blocker bugs for 4.1.0 as well. The project also welcomes two new PPMC members and three new committers.

Major Discussions

Some of the major discussions and issues that have taken place on cloudstack-dev and cloudstack-users in the past week. This is by no means exhaustive, if you need to be up-to-date on all development issues in the project, you'll definitely want to be subscribed to the mailing lists!

Summary of Why Where and How Development Happens Matters

Chip Childers kicked off a thread about why it matters where and how development happens, based on a discussion that had taken place on cloudstack-private:

It's largely a re-hash of things that have already been discussed, but we wanted to get this summary moved into the dev list so that the points are available for reference. We should probably distill this into a wiki page somewhere, but that's not done quite yet (volunteer?). Keep in mind
that I've tried to pull the important parts of the conversation into this email... it was a fairly long thread of discussion and debate.
I'd suggest reading it all the way to the end to form your own understanding of why we have to be careful about how we work as a community.

The TL;DR version:

The issue that we ran into with several features being developed "outside the community" for 4.1 was a major deal, and it had several
implications. First, doing that effectively hurts our community. The other issue is related to the legal right of the project to accept the
code developed elsewhere.

Rather than summarizing it here, I'd recommend that readers spend the time to read Chip's initial email and the replies in the thread.

Using DIY System VMs

Rohit Yadav shared "that the do-it-yourself systemvm appliance feature works for me, for Xen,":

There is one catch though, VirtualBox exports VHD appliance which is said to be compliant with HyperV. I thought we may need to do something for Xen separately, so I followed and found a way. The "way" is to export a raw disk image and convert it to a VHD 1 but the problem is the VHD created from that "way" fails when vhd-util tries to scan for any dependent VHDs (parents etc.), I don't know what's the reason.

Read the rest of the thread if you have an interest in creating custom system VMs for CloudStack.

Fixing the Tomcat Situation, Post 4.1

Noa Resare has admitted being intensely frustrate with "the current tomcat situation" when working on packaging CloudStack. The current setup, says Noa, has many problems. In response, Noa has started a proof-of-concept "of a replacement for this whole mess yesterday, a few tens of lines of code setting up an embedded jetty web container. A few lines of code to parse a config file, set up logging and spawn an embedded servlet container."

Changing Project Bylaws to Modify PMC Chair Voting Process and Term

Chip kicked off a VOTE thread last week to modify the project bylaws slightly:

As previously discussed, we'd like to make a change to our bylaws to modify the method of selecting a PMC chair. We also want to add a term for the chair.

The text was clarified slightly during the vote. The vote remains open until March 5.

Support Lifetime

With the 4.1 release getting closer, David Nalley raised the topic of the support lifetime for releases. Chip proposed a model that would consist of only bug fix releases for:

  • The latest feature release of our active major version number (i.e.: 4.x)
  • The latest feature release of our last major version number (doesn't exist today, but will be 4.x when / if we bump to 5.0)

Joe Brockmeier replied with a +1, saying that with the current level of participation in the bug-fix releases "this is the most realistic approach that's good for the community."

David asked whether an end of support means an end to bug fixes and security fixes at the same time. "Wearing your enterprise software consumer hat - does a supportlifetime of approximately 12 months make sense?" Instead, David proposed "we should add a month (so that EOL is one month after 4.n+2 releases, with the understanding that 4.n is likely to only receive security fixes if any during that extra one month window.)"

The discussion continues, so developers and users that have a vested interest in the support lifetime for ACS releases would do well to follow and respond to the discussion.

API Throttling

Parth Jagirdar has started a discuss thread about API throttling. "API throttling number can be set to anything at this point. Suggestions here is to have this number set to a value that is 'greater than' number of API that can be fired by any potential action on UI." (Note, Parth then sent out a follow-up email to correct the initial subject line from [DISCUSS} to DISCUSS, but all relevant discussion has happened in the original thread. It's probably not necessary to send a follow-up in those situations and may fragment the conversation.)

Branch Stability Status

Sudha Ponnaganti posted a report to the -dev mailing list about branch stability for QA testing:

4.1 Branch:
*Xen and VMWare are blocked with the following two issues. KVM is working fine (agent issue-1469 has been fixed by Hugo).
CLOUDSTACK-1252
Failed to download default template in VMware
CLOUDSTACK-1470
Xen - unhandled exception executing api command: deployVirtualMachine

Master:
There are no blockers now on Master / 4.2. Below are the only blockers and you
can continue to use older templates to make progress on master till new one gets
hardened.
CLOUDSTACK-1462
Used Master Branch System VM Template: Volume of System VM Failed to Create on
the XenServer due to IOError

Summary of IRC Meeting for 27 February 2013

The ASFBot shot a summary of the weekly IRC meeting to the -dev mailing list. If you missed it or need a reminder of what happened, check it out.

Report from the Doc Sprint on Friday

Joe sent out a short report about the doc sprint that took place on Friday, March 1st. Another is planned for Friday, March 8th.

CloudStack Planet - Posts from the CloudStack Community

Provisionr - Automated Deployment of Massive Infra in the Clouds

David reported on a talk at ApacheCon North America about Andrei Savu's demo of "software that Axemblr had been working on around deploying pools of interrelated virtual machines called Provisionr."

ApacheCon North America Report: Troubleshooting CloudStack

Joe Brockmeier wrote a report on two talks at ApacheCon North America by Kirk Kosinski on troubleshooting CloudStack. The talks covered common networking issues and how to use log files to troubleshoot CloudStack.

Cloud Talks from ApacheCon North America

Mark Hinkle has put up a post about Cloud talks at ApacheCon North America with links to presentations.

Upcoming Events

  • Doc Sprint (IRC): The doc team is running another sprint on Friday, March 8th from 16:00 to 23:00 UTC (that's 08:00 to 15:00 Pacific time, 11:00 to 18:00 Eastern in the USA) in #cloudstack-dev. All CloudStack contributors who have an interest in making the docs awesome are encouraged to attend and help out. See the wiki for more info.

Jira

Big bug-fixing week for Apache CloudStack! The community knocked out 5 blocker bugs, 3 critical bugs, 9 major bugs, and 3 minor bugs. Still plenty of work left to do before 4.1.0 is ready for prime-time, though.

It's worth noting that the doc sprint helped quite a bit, bringing the total number of docs bugs for 4.1.0 to 44 (down from 64 just two weeks ago). Of the remaining bugs for 4.1.0, 80 are currently unassigned, down from 99 last week.

New Committers and PPMC Members

A big week for new committers and PPMC members.

Contributing to the Apache CloudStack Weekly News

If you have an event, discussion, or other item to contribute to the Weekly News, you can add it directly to the wiki by editing the issue you want your item to appear in. (The next week's issue is created before the current issue is published - so at any time there should be at least one issue ready to edit.) Alternatively, you can send to the cloudstack-dev mailing list with a subject including News: description of topic or email the newsletter editor directly (jzb at apache.org), again with the subject News: description of topic. Please include a link to the discussion in the mailing list archive or Web page with details of the event, etc.

This week, interesting discussions on the Java version(s) to be supported by CloudStack, updates on translation, and database changes. Also discussion on the next CloudStack Collaboration Conference, and more. Work on 4.1.0 continues, and there's much to be done before it's ready to ship.

Major Discussions and Issues

Some of the major discussions and issues that have taken place on cloudstack-dev and cloudstack-users in the past week. This is by no means exhaustive, if you need to be up-to-date on all development issues in the project, you'll definitely want to be subscribed to the mailing lists!

Schedule Reminder

Chip Childers has sent out a friendly reminder about the schedule:

Thursday is the last day of this
phase of QA / bug-fix work. We defined it as:

2013-02-28
Docs Completion Target (except release notes and translations) (Docs
may be included in the release after this date, after consensus on
each addition that the inclusion does not reduce release quality).

Release Branch moves to limited updates only (only commits allowed
in would be release blockers fixes, translation updates, etc...)

I'd like to get as many bugs resolved as possible (as well as ensure
that the blockers that Sudha has shared this morning are addressed as
quickly as possible).

After Thursday, we're going to want to move to a very limited amount of
change within the 4.1 branch. Given that, now's the time to knock down
the blockers... but also as many of the other priority bugs as
possible.

If you have 4.1 bugs on your plate, please be sure to try to either resolve them or at least triage/report status.

CloudStack Collaboration Summit

Mark Hinkle started a discussion about the next CloudStack Collaboration Conference, offering to work on a proposal for an event in the spring and fall. Lots of folks have expressed interest, and suggested holding the spring event in Europe rather than North America. The discussion hasn't been resolved yet, so no dates or location have been announced as of yet.

QA Scrum Meeting Minutes

The QA Scrum meeting minutes for 18 February 2013 sent to the mailing list.

Weekly IRC Meeting Minutes

The minutes for the weekly CloudStack meeting have been posted to the list. Note that the community has a weekly meeting every Wednesday at 17:00 UTC in #cloudstack-meeting on Freenode.

Supported Java Version?

Wido den Hollander noted that the master branch wouldn't build on his systems over the last few days. The culprit? Seems to be that some changes have snuck in that want a later version of Java than is in Ubuntu 12.04.

No decision has been reached so far, but there is a case to be made that changing the Java version is a significant disruption to users.

Translation Update

Sebastien Goasguen has sent out an update

Discussion on Database Changes

Last week Rohit Yadav had a Google+ hangout with Alex and Abhi to discuss future of database deployment and upcoming work on creating a new tool called DatabaseCreator that will make database deployments easier. It was decided and enforced on 4.1 and master branch that:

  • create-schema.sql ought not be changed from the version 4.0 schema
  • Any new additions should go into their correct upgrade paths, for 4.2 that would mean schema-410to420.sql and schema-410to420-cleanup.sql
  • It would do rolling upgrade from 4.0, this was enforced in DatabaseUpgradeChecker

Once DatabaseCreator is implemented correctly for 4.2 release, a sysadmin will have power to work on their own upgrade strategies. The workflow would be:

  • System admin uses the tool to take a db dump, upgrade the database, at this stage the new database should be backward compatible with old db.
  • Next all CloudStack management servers are upgraded.
  • The tool is called again to do sanity checks and cleanup any db schema.
Development on DIY SystemVM templates

This week Rohit Yadav and Chiradeep were finally able to configure a veewee project so systemvm template can be built on one's own box using veewee and VirtualBox, Rohit also created a systemvm building jenkins job. The source code exists in tools/appliance and has a README for folks to get started. The default template is based on Debian Wheezy but anyone can change the definition.rb, preseed.cfg and postinstall.sh script to fork their own systemvm templates based on Ubuntu, Fedora etc.

Rohit shared a post with the issues and challenges of setting an automated jenkins build job that would create systemvm appliances and export them to various virtual disk image formats. Using his approach anyone should be able to replicate an automated appliance build job.

Upcoming Events

Jira

The count of bugs for 4.1.0 has actually increased since last week. The overall tally of blocker and critical bugs has increased, with two additional blocker bugs and four more critical bugs (though perhaps not the same bugs as last week). A lot of work is needed to get 4.1.0 into shape before we will be able to release.

Of the remaining bugs for 4.1.0, 99 are currently unassigned.

New Committers and PPMC Members

No new committers or PPMC members announced this week.

Contributing to the Apache CloudStack Weekly News

If you have an event, discussion, or other item to contribute to the Weekly News, you can add it directly to the wiki by editing the issue you want your item to appear in. (The next week's issue is created before the current issue is published - so at any time there should be at least one issue ready to edit.) Alternatively, you can send to the cloudstack-dev mailing list with a subject including News: description of topic or email the newsletter editor directly (jzb at apache.org), again with the subject News: description of topic. Please include a link to the discussion in the mailing list archive or Web page with details of the event, etc.

Another busy week on -dev. This week, discussions on whether to graduate to a TLP, the whens and hows of merging, database changes for 4.1 and master, and documentation.

Also worth noting, we now have a marketing mailing list. To sign up and participate in promoting Apache CloudStack, please subscribe to the mailing list (cloudstack-marketing-subscribe@incubator.apache.org) and help out!

Major Discussions and Issues

Some of the major discussions and issues that have taken place on cloudstack-dev and cloudstack-users in the past week. This is by no means exhaustive, if you need to be up-to-date on all development issues in the project, you'll definitely want to be subscribed to the mailing lists!

QA Scrum Meeting Minutes

The QA Scrum meeting minutes for 12 February 2013 sent to the mailing list.

IRC Meeting Minutes

Summary of the weekly IRC Meeting for 13 February 2013.

Note that last week's meeting went in reverse alpha-order, but it was largely agreed during the meeting that we should have an agenda for the meeting. Please submit agenda items by Wednesday at 16:00 UTC.

Graduation to a Top-Level Project (TLP)?

Is Apache CloudStack ready to leave the incubator? Chip Childers raised the discussion on the -dev list on February 13th.

My general impression is that we have come a long way as a community since CloudStack entered the incubator. While there are still rough edges for us to work through over time, we are dealing with our problems quite well as a community. The simple reason that I believe we are in a position to ask to graduate, is that we are no longer getting value from the incubation process! That's a good thing, because it means that we have managed to learn quite a bit about the ASF processes, rules, methods and preferences.

Marcus Sorensen agreed that there are rough edges but, "the incubation process itself may not teach us anything further about these things, and they're just things we need to iron out over time."

David Nalley wrote:

I don't see us actively receiving any benefit from continuing in incubation. We are far from perfect, but the project seems to be policing itself. So I am not seeing a huge incentive to staying.

There are also some downsides to remaining in incubation. First there's the label 'incubation' that follows almost everything we do, and is potentially off-putting to potential community members. Second as a community there are a number of things we can't do for ourselves, and thus have to ask permission or for help - this includes votes on releases, creating new user accounts, etc. I think of this as the overhead of being in the incubator.

Discussion on Database Changes and Merges

Rohit Yadav put out the word that there were database changes ahead for the 4.1 branch and master.

Rohit also brought up a merge for database schemas for 4.1.0 and pointed to the current differences in the database schema for 4.1 over 4.0.

Documentation for 4.1

Sebastien Goasguen started a thread on documentation for 4.1, which is a must-read for anyone working on docs.

How to Treat Broken Builds

David Nalley expressed some frustration with the community's reaction to broken builds.

In general it seems we don't care, and this makes it more difficult to fix problems. Jenkins reporting a broken build (be it a broken run of RAT, failure to compile, failure of a unit test, building docs, etc.) should be our Andon cord. We should all stop commits that aren't fixing the broken build. To illustrate why this is a problem, RAT failures started occurring recently, this keeps us from testing whether CloudStack builds, because each build is conditioned on the successful completion of the test before it. That in turn keeps apidocs from building, which keeps marvin from building, which keeps documentation from building. We essentially are blind until it gets fixed.

Marcus Sorensen asked if there was a Jenkins report committers could subscribe to in order to avoid the problem. David replied, "Yes there are - the commits mailing list receives the notifications from both jenkins.cs.o and builds.a.o. ... You can subscribe by sending an email to cloudstack-commits-subscribe@incubator.apache.org."

Wait Before you Merge!

How long should you wait until you merge into master or a branch? Chip Childers brought this up in response to a merge that was announced on February 12th and then committed on February 13th. This discussion raised the fact that there's not a formal merge process documented. Chip volunteered to take a crack at the document and asked for help "in getting it in shape to reflect consensus on the topic."

Android and iOS CloudStack Clients

Abhinandan Prateek asked about the utility of a Android and/or iOS client for CloudStack. David pointed out that an Android client called Cumulus exists that is "pretty useful as an end-user, but less so as an admin."

Pranav Saxena replied that there was already some work underway off-list on a client using PhoneGap, and it might be possible to collaborate. (Ed. Note: This is why you should bring ideas to the list sooner rather than later, so there's not a wasted effort when two or more community members have the same idea and try to implement it separately.)

New Dependency on OWASP ESAPI for Java

Likitha Shetty announced a new dependency on the OWASP ESAPI for Java in master.

Upcoming Events

Jira

Bug count for 4.1.0 hasn't significantly decreased since last week, though major bugs have dropped slightly while there's been an increase of blocker and critical bugs.

Of the remaining bugs for 4.1.0, 95 are currently unassigned. Doc bugs are by far the largest component needing help, with 64 bugs remaining.

New Committers and PPMC Members

David Nalley announced that Likitha Shetty has been asked to become a committer, and Likitha has accepted. Please join us in congratulating Likitha!

Contributing to the Apache CloudStack Weekly News

If you have an event, discussion, or other item to contribute to the Weekly News, you can add it directly to the wiki by editing the issue you want your item to appear in. (The next week's issue is created before the current issue is published - so at any time there should be at least one issue ready to edit.) Alternatively, you can send to the cloudstack-dev mailing list with a subject including News: description of topic or email the newsletter editor directly (jzb at apache.org), again with the subject News: description of topic. Please include a link to the discussion in the mailing list archive or Web page with details of the event, etc.

The Apache CloudStack project is pleased to announce the 4.0.1-incubating release of the CloudStack Infrastructure-as-a-Service (IaaS) cloud orchestration platform. This is a minor release in the 4.0.0 branch, which contains fixes for more than 30 bugs.

Apache CloudStack is an integrated software platform that allows users to build a feature-rich IaaS. CloudStack includes an intuitive user interface and rich API for managing the compute, networking, accounting, and storage for private, hybrid, or public clouds. The project entered the Apache Incubator in April 2012.

The 4.0.1-incubating release includes fixes for a number of issues, including a minor security vulnerability (CVE-2012-5616), problems adding KVM hosts, fixes to documentation, and more. Please see the CHANGES file in the release for a full list of issues that are fixed in this release. As a bugfix release, no new features are included in 4.0.1-incubating, and it should be a simple upgrade from 4.0.0-incubating.

Downloads

The official source code release can be downloaded from:

http://incubator.apache.org/cloudstack/downloads.html

In addition to the official source code release, individual contributors have also made convenience binaries available on the Apache CloudStack download page.

Incubating

Apache CloudStack is an effort undergoing incubation at The Apache Software Foundation (ASF). Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

In the past week, the 4.0.1-incubating release passed its VOTE on the general@incubator.apache.org list, work continued on 4.1.0, and there were active discussions on using Gerrit, cloud-init, and whether memory usage has increased following the adoption of the Spring framework.

Major Discussions and Issues

Some of the major discussions and issues that have taken place on cloudstack-dev and cloudstack-users in the past week. This is by no means exhaustive, if you need to be up-to-date on all development issues in the project, you'll definitely want to be subscribed to the mailing lists!

Gerrit proposed as review mechanism

Alex Huang kicked off a discussion about using Gerrit. David Nalley pointed out that many of the problems Alex looks to solve with Gerrit are "social/cultural problems" that a technical solution alone won't fix.

Hugo Trippaers also voiced concerns about implementing Gerrit, saying that "a formal process like Gerrit" isn't necessary.

The committer status is granted based on the trust we put in a certain individuals to take care of the CloudStack project, for me that included taking the responsibility that any contributions are up to spec. I want to trust my fellow committers that they know what they are doing and i don't feel the need to second guess that by wanting to look over their code before they can commit it.

Cloud-init vs. homegrown scripts

David Nalley raised a question from the Ghent Build-a-Cloud-Day about "whether or not to package the SSH key reset and password reset utilities, or whether we should focus our PW/SSH efforts on cloud-init."

Wido den Hollander voiced support for cloud-init, saying "We want CloudStack to be accepted by more and more users and they probably want to use cloud-init. cloud-init has cool Puppet and Chef plugins as well which make it very
easy to get it all up and running."

Chiradeep Vittal says he likes cloud-init, but worried that adopting cloud-init would be "hurting the investment of existing CloudStack users who have built hundreds of templates with the extant scripts."

Wido says that "the scripts should keep working" but preferred to avoid packaging the homegrown scripts and encouraging new users/new setups to go with cloud-init.

Chip says that "I'm in agreement that the existing scripts would need to remain as a supported option for quite some time. I consider things like that to actually be related to our semver version numbering, although it's not specifically our CS query API, it's a similar interface expectation."

Changes to CPU and memory overcommit feature

Abhinandan Prateek forwarded a note from Bharat Kumar about changes to the way capacity is calculated in CloudStack.

I have made changes to the way capacity is calculated in CloudStack ,
please review and comment.

I will illustrate this with an example.

let us say we have a host with
Actual Total capacity=4GB ram,

and the overcommit ratio be 2.

Current way
Total capacity= 4*2= 8GB.
Values after deploying 4 VMs with 1GB in service offering.
Allocated per vm =1GB.
Used=4GB
Available=8-4=4GB

now if we decrease the overcommit ratio to 1
Total Capacity = 4*1=4GB.
used Capacity = 4GB.
Available = 4-4=0. (implies 100% usage. can also go to more than 100%)

JVM Memory usage increased with latest master

The latest master is consuming nearly 900MB of memory, according to Koushik Das.

There's some discussion as to the cause of this, and CLOUDSTACK-1168 ("Memory usage is very high") has been created. If you're testing master/4.1 and run into this problem, it'd be helpful to update the ticket with your experience and the environment you're seeeing the issue in.

Blockers to run automated tests

Sudha Ponnaganti brought up a couple of issues on February 8th that are blocking automated testing. CLOUDSTACK-1216 has been fixed, CLOUDSTACK-1200 (Unknown column 'vm_instance.disk_offering_id' in table vm_instance, db exception shown in MS log) is still unresolved.

CloudStack 4.0.1-incubating passes VOTE

Joe Brockmeier announced on Friday that the first point release from the CloudStack 4.0 branch has passed its vote on general@incubator.apache.org with +1 votes from Jim Jagielski, Olivier Lamy, and Alex Karasulu. No -1 or +0 votes were cast.

The release has been pushed to the mirrors and will be announced on Tuesday, February 12.

ACS 4.1 Release Schedule Reminder

Chip Childers sent out a reminder about the release schedule for CloudStack 4.1.0 on Monday.

2013-02-28 is the end of our test phase, and should be the end of the doc finalization for the release.

We'll shift into a mode where the 4.1 branch should only be updated by me after that date (after you ask for a cherry-pick for a critical bug or translation update).

Chip also noted that there are 15 features/improvements listed as "open" for 4.1, and asked that the reporter/assignee update to "In Progress" or move the target release to 4.2 or Future, as appropriate.

Upcoming Events

John Kinsella has announced the first Bay Area CloudStack Meetup to be held in San Jose on February 20th.

Jira

Status for 4.1 as of Monday, 11 February - by priority:

Of the remaining bugs for 4.1.0, 103 are currently unassigned.

New Committers and PPMC Members

No new committers or PPMC members were announced this week.

Contributing to the Apache CloudStack Weekly News

If you have an event, discussion, or other item to contribute to the Weekly News, you can add it directly to the wiki by editing the issue you want your item to appear in. (The next week's issue is created before the current issue is published - so at any time there should be at least one issue ready to edit.) Alternatively, you can send to the cloudstack-dev mailing list with a subject including News: description of topic or email the newsletter editor directly (jzb at apache.org), again with the subject News: description of topic. Please include a link to the discussion in the mailing list archive or Web page with details of the event, etc.

Another busy week in Apache CloudStack land! Javelin has been merged with master, the 4.1 branch has been created, and the 4.0.1-incubating release has passed its second round vote to go on to the IPMC vote.

Major Discussions and Issues

Javelin Merged

The Javelin branch has been merged into master by Kelven Yang. Kelven has noted that developers should increase the amount of memory available to the JVM if running the management server under Maven:

For linux/Mac users

MAVEN_OPTS="-XX:MaxPermSize=512m -Xmx2g"
export MAVEN_OPTS

For Windows users

Add similar settings to windows environment settings (via Windows control panel)

4.1 Branch Created

Following the Javelin merge, Chip Childers has created the 4.1 branch, and has asked committers "to respect the feature and improvement freeze in that branch. Bug fixes, doc updates and other release stabilization activities are obviously expected." Chip also says that committers should continue committing directly to the 4.1 branch until code freeze. (Code freeze, excepting fixes for blocker bugs and so forth, is on February 28th.)

If you're a non-committer and wish to send in a patch against 4.1.0, send in a patch built against the 4.1 branch. Chip says:

Committers taking these fixes should also consider applying them to master. If there are conflicts in master (which may happen, as there were a
couple of code-base refactoring activities, like switching packages from com.cloud to org.apache.cloudstack), apply the fix into 4.1 anyway, and inform the submitter that the patch has conflicts with master to get that sorted out (or you can fix it yourself).

4.0.1-incubating Vote

The 4.0.1-incubating release has passed the vote on cloudstack-dev, and is now in the 72-hour IPMC vote. Going into the vote on general@incubator.apache.org, the release artifacts already have 3 +1 mentor votes, so it should pass barring any -1 votes.

Preparing for Translation

With the feature freeze in place, it's time to get docs into shape and start getting things ready for translations. Sebastien Goasguen has created a new /pot directory with the .pot (portable object templates).

Sebastien notes that contributors need to remember the 50-character limit for XML filenames, as Transifex doesn't support longer file names. Contributors will also need to run publican update_pot when updating doc files and/or creating new files.

Packaging in 4.1

Hugo Trippaers sent out an update about the discussion on packaging for 4.1. Hugo says the main goal for redoing the current way of packaging CloudStack is to get "rid of ant and waf completely." The secondary goal "is to create a reference set of packages which in itself should be enough to get anyone going with CloudStack, but will hardly depend on the underlying distro. Real distro specific stuff should be handled by packagers from those distros. We just aim to provide a reference implementation."

Hugo also says that the plan is to rename the packages "to make it perfectly clear what somebody is installing." That's going to change the location of a number of files, but Hugo says "we intend to include symlinks for backwards compatibility."

The planned packages for now are cloudstack-management, cloudstack-agent, cloudstack-common, cloudstack-cli, cloudstack-docs, cloudstack-awsapi and
cloudstack-usage. You might already have seen these names in some of the checkins.

One of the side-effects of the new packaging plan is that CloudMonkey may not be installed with the RPMs, but instead require that admins use PIP to grab CloudMonkey.

Support VM Snapshot Held to 4.2

Mice Xia proposed merging the VM Snapshot branch to master ahead of the 4.1 branching/feature freeze on January 28th. However, there were a number of questions about the feature's implementation and consensus was not reached in time for the 4.1 branch's creation.

On the plus side, this is why we have time-based releases. While it will delay inclusion into CloudStack slightly, the feature will be well-poised to go into 4.2 and the additional attention that the discussion has raised will likely make it that much better a feature when merged to master. It also means that a single feature won't delay the 4.1 release, which already has quite a lot of improvements and new features that CloudStack users will want to have ready when it's scheduled for release.

Proposal to Deprecate the non-RESTful APIs and API Server

Rohit Yadav has proposed deprecating CloudStack's non-RESTful APIs and API server over a reasonable timeline. The reaction has been positive, and Min Chen has offered to share an architecture design that's in-progress, when it's ready.

Help Reviewing Runbook

David Nalley has sent out a request for help reviewing his runbook, which is an opinionated guide to setting up a simple CloudStack install on a single CentOS 6.3 server using KVM. Other CloudStack users and contributors have expressed interest in creating similar runbooks for Ubuntu-based systems.

Upcoming Events

  • India User Group Meet-up: Citrix, ShapeBlue, and CloudByte are combinedly organizing a CloudStack India User Group Meetup in Bangalore (5 February, 2013).
  • Build a Cloud Day (BACD): Full day BACD at SCALE on Friday (22 February 2013).
  • CloudStack Booth at SCALE: Meet with CloudStack folks at SCALE (23-24 February 2013), also Joe Brockmeier is speaking about CloudStack at SCALE.

Jira

Be sure to check the list of 4.1.0 bugs to see if there are any open issues assigned to you, or any unassigned issues that you could take on.

New Committers and PPMC Members

The Apache CloudStack project is pleased to welcome Koushik Das as a committer this week. Well done, Koushik!

As the 4.1.0 feature freeze approaches, the mailing list has been extremely active. The activity on -dev is off the charts, with (according to MarkMail) more than 4,400 messages sent to -dev in January - and the month isn't over yet! This eclipses the previous record set in October 2012 of 3,109 messages. Major discussions this week include the Javelin merge and IP clearance issues.

The feature freeze is 31 January 2013. Testing and bugfixing will carry through from 1 February 2013 to 28 February 2013, then docs freeze is on 28 February (excepting release notes and translations). See the full schedule on the wiki.

Major Discussions and Issues

As usual, the -dev mailing list has been hopping for the past week. Here's a summary of some of the major issues and discussions that have taken place over the last week.

Counting down to feature freeze for 4.1.0-incubating

Reminder on 21st January that there were 10 days left to feature freeze on Apache CloudStack 4.1.0-incubating. Status at the time of the reminder:

Out of 95 total proposed features / improvements, their status is:

3 Closed
12 Resolved
5 Reopened
21 In Progress
54 Open

IP Donations

This week we had several VOTE results around IP donations from Citrix.

Note that Chip Childers sent an email about incoming IP Clearance VOTEs to incubator-general@apache.org about the 7 IP clearance votes that would be sent to the incubator folks.

Javelin Merge

Alex Huang started a discussion about merging the Javelin branch into master. "The content of the merge is the storage framework refactoring and converting everything use Spring." There's some decisions yet to be resolved around the storage piece of this that needs to be resolved before the request to merge Javelin is decided.

LICENSE and NOTICE files for 4.1.0

Chip Childers has raised a question about changes to the LICENSE and NOTICE files for CloudStack. This would remove all binary dependency notice info from the top level LICENSE and NOTICE files in the source tree, and create two copies of the Whisker descriptor.xml files, one that generates the source distro's files and one that generates the files for packaged versions of Apache CloudStack.

VOTE called for 4.0.1-incubating

The VOTE for 4.0.1-incubating has been called as of January 25 and will run for 72 hours or until it gets the required votes, or is -1'ed and restarted. Note that, if successful, the release still needs to be voted on by the IPMC and receive at least three +1 votes. The release could be out as early as next week, if all goes well.

Writing unit tests after Javelin is merged

Alex Huang has written up a page on how to write unit tests after the Spring injection framework is merged into Master.

CloudStack San Francsico Bay Area Meetup group created

A new meet group has been created for San Francsico Bay Area http://www.meetup.com/CloudStack-SF-Bay-Area-Users-Group/. First meetup is expected in February, so stay tuned.

Bang-up job on translations

According to Sebastien Goasguen

The mad race to CloudStack documentation translation has started.
Japan is leading the way with 52% of the translation complete.
China is close second with 40%
Brazil is third with 11%

See more about working on translations on the wiki.

Hyper-V Plugin

Donal Lafferty has submitted a review request for a Hyper-V plugin for CloudStack.

Upcoming Events

Speaking at events doesn't happen by accident, or without preparation. For example, if you want to speak at a major event like OSCON, you need to submit a talk well in advance.

CFPs to be aware of

Mark Hinkle sent a note to -dev reminding that the CFP for OSCON 2013 will close on 4 February. If we want to see CloudStack talks at OSCON (and we do, yeah?) then get those proposals in!

The Linux Collaboration Summit CFP also closes on 4 February 2013 in case you were looking to get a CloudStack or related talk in.

What's on tap for the coming weeks

Jira

New Committers and PPMC Members

As usual, a lot of activity on the -dev mailing list. Several VOTEs this week around features donated by Citrix, and quite a few upcoming events. Speaking of events, videos from the CloudStack Collaboration Conference are up on YouTube!

Major Discussions and Issues

Upcoming Events

Jira

New Committers and PPMC Members

For this week's news only we're mentioning new committers and PPMC members for this month (Jan 2013);

  • Sebastien Goasguen (committer)
  • Joe Brockmeier (PPMC)
  • Kelcey Damage (committer)