FreeBSD HAST project completed

Pawel Jakub Dawidek was awarded a grant by the FreeBSD Foundation last year to implement storage replication software that will enable users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes.

The HAST (High Availability Storage Project) is now completed. Pawel reports:

I’m very happy to report to FreeBSD users that the HAST project I was working on for the last three months is ready for testing and already committed to the HEAD branch.

I’ll describe what HAST does in few words. HAST allows for synchronous block-level replication of any storage media (called GEOM providers, using FreeBSD nomenclature) over a TCP/IP network for fast failure recovery. HAST provides storage using the GEOM infrastructure, meaning it is file system and application independent and can be combined with any existing GEOM class. In case of a primary node failure, the cluster will automatically switch to the secondary node, check and mount the UFS file system or import the ZFS pool, and continue to work without missing a single bit of data.

I must admit the project was quite challenging, not only from the technical point of view, but also because it was sponsored by the FreeBSD Foundation. The FreeBSD Foundation has a great reputation and is known to select the projects it funds very carefully. I felt strong pressure that should I fail, the FreeBSD Foundation’s reputation might be hurt. Of course, not a single dollar would be spent on a failed project, but the FreeBSD community’s expectations were very high and I really wanted to do a good job.

During the work a number of people contacted me privately offering help, explaining how important HAST is for FreeBSD and giving me the motivation to soldier on.

I hope that HAST will meet the community’s expectations and I myself am looking forward to using it

The final commit notice from Pawe? Jakub Dawidek briefly describes the project. A detailed discussion of the project is available form the FreeBSD Wiki.

This is a good example of how the wider FreeBSD community can financially support further development of FreeBSD, and it also shows the value that the FreeBSD Foundation brings to the community. To see more of these sort of projects started, funded and completed, why not support and donate to the Foundation so they can continue sponsoring more projects. (disclaimer: I am not affiliated with the FreeBSD Foundation)

How the FreeBSD Project’s Processes help companies build products

George Neville-NeilGeorge Neville-Neil has written the lead article for the January 2010 issue of the Open Source Business Resource (OSBR) and the FreeBSD Foundation is the sponsor for this month’s issue. The entire issue is available as a PDF and George’s article is also available in HTML.

From the article’s abstract:

The processes that open source projects use to produce new work and maintain the quality of their code base is a subject that comes up infrequently in discussions of open source. One reason for this is that engineers and programmers are usually loathe to deal with issues that are not directly related to the piece of code or technology that they are working on.

Successful businesses know that good processes lead to continued success. The attributes that attract a business to an open source project are stability, reliability, and longevity. Stability gives a business the confidence to invest time into developing products on the project’s platform, safe in the knowledge that the next incremental step in development won’t be torpedoed by some unforeseen change. Reliability is often not associated with open source and many projects are perceived as being too cutting edge for a business to build upon. Longevity is of value as many businesses are inherently conservative in their approaches, attempting to reduce the risks of adopting any technique or technology. One way to reduce risk is to work with an open source project that has a proven track record of delivering quality products, on schedule.

This article attempts to dispel the myth of the perceived tension between a formally run business and the apparently less formally run open source projects with which a business interacts. We describe how one particular open source project has developed processes which provide its users, customers, and partners with a product that is stable, reliable, and long lived.

Read George’s articlle in full

FreeBSD fund raising drive – final plea

The FreeBSD Foundation has not reached its fund raising goal for 2009 yet ($46,000 short of their $300,000 goal). The Foundation is hoping to to double spending on development in 2010 and needs our help:

Why make a donation? Right now we’re putting together our 2010 budget. Our goal for next year is to double our project development spending, continue sponsoring BSD-related conferences, sponsor more developers to travel to these conferences, and spend more on needed equipment for the project.

I’ve made my donation this year.  You can too. You may donate to the Foundation here

Read the final plea here

The Foundation announced earlier this week it’s receiving a cheque for $500 from the Bad Code Offsets project. I’ve never heard of this project:

The project is a way to undo the bad code other people have written without actually replacing the bad code. Much like carbon offsets, money used to buy Bad Code Offsets goes towards open-source projects which not only produce good code, but produce software that helps developers build good software

Read the announcement

FreeBSD Foundation end-of-year newsletter (2009)

FreeBSD foundation logoDeb Goodkin announced the publication of the annual FreeBSD Foundation’s End-of-Year Newsletter (2009).

Table of contents:

  • Letter From the President
  • End-of-Year Fundraising Update
  • New Console Driver
  • Flattened Device Tree Project
  • Improvements to the FreeBSD TCP Stack
  • Highly Available Storage Project
  • FreeBSD Developer Summit, Cambridge, UK
  • EuroBSDCon 2009
  • KyivBSD 2009
  • 2009 Grant and Travel Grant Recipients
  • FreeBSD Testimonial from iXsystems
  • Financials

The full newsletter can be read here.

Previous years’ newsletters: 2008200720062005

Update on FreeBSD HAST Project

Pawel Jakub DawidekBack in October we mentioned that the FreeBSD Foundation decided the fund the HAST project (new FreeBSD Project: HAST)

Pawel Jakub Dawidek, the sponsored FreeBSD developer, has made quite a bit of progress since and finished the development of stage one:

I want to report that first milestone of the HAST project is complete.

Summary of the work that have been done:

  • Implementation of hastd daemon.
  • Implementation of hastctl utility to manage hastd daemons.
  • GEOM_GATE class was extended so that the caller can specify the name of GEOM provider. Before only /dev/ggateX names were supported. HAST will use /dev/hast/
  • Implementation of communication protocol. There is abstraction layer on top and below there are three protocols implemented currently:
  • proto_tcp4 – It is used for communication between primary and secondary nodes.
  • proto_uds – (UDS – UNIX Domain Socket) It is used for communication between hastctl and hastd.
  • proto_socketpair – It is used for communication between main hastd daemon and worker processes forked from it.
  • Implementation of nv (name-value) API, which allows to easy create packets containing name-value pairs. It is used for entire communication through the protocols above. It is also responsible for managing correct byte-order.
  • Implementation of ebuf (extendable buffer) API, which provides a way to extend given buffer by adding data at the back, but also at the front without reallocating it and copying the data very often (or never).
  • Implementation of logging API (pjdlog). The API decides if messages should be logged on standard output/error (before going into background) or to syslog (when we daemonize). It also provides some shortcuts for logging a message and exiting, etc. It supports notion of debug level and can skip messages intended for higher debug level than requested.
  • Implementation of configuration file parser in lex/yacc. Configuration file is designed in a way that it can be kept identical on both nodes.
  • Checksumming and compression for the data is not one of the project’s goal, but the stubs are there, so this can be added easly.
  • A lot of care was taken to be able to handle more nodes in the future. This is not implemented and in not project goal, but I wanted to make it ready for future improvements (source)

HAST enables users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes. The software will allow for synchronous block-level replication of any storage media (GEOM providers, using FreeBSD nomenclature) over the TCP/IP network and for fast failure recovery. HAST will provide storage using GEOM infrastructure, which means it will be file system and application independent and could be combined with any existing GEOM class. In case of a master node failure, the cluster will be able to switch to the slave node, check and mount UFS file system or import ZFS pool and continue to work without missing a single bit of data.

In order to fund more of these sort of projects, the Foundation is reliable on our donations and support. The Foundation hasn’t reached its 2009 goal yet. You can help by donating.

(BTW, I’m in no way affiliated with the work of the FreeBSD Foundation;  I only want to see FreeBSD progress to become even better)

FreeBSD end-of-year fund raising drive (update)

freebsd_foundation 100.x100FreeBSD is free; it can be downloaded, used and adapted without paying any fees, unlike other major operating systems. This is why the FreeBSD Foundation needs donations to be able to fund new projects and conferences.

The FreeBSD Foundation is collecting donations for the next year. The Foundation is a regular sponsor of many events, funds trips (including mine) to FreeBSD conferences and occasionally funds projects for various enhancements to FreeBSD. What else does the Foundation do?

The Foundation writes on its blog:

We would like to thank everyone who has donated to the FreeBSD Foundation this year. We have raised $183,888 towards our 2009 goal of $300,000! We are almost 2/3 of the way to reaching our goal! Oh, and BTW, we have had 671 donors this year. This is compared to just over 300 this time last year. This is important not only to help us keep our Public Charity Status, but it shows there are many users who are passionate about FreeBSD and want to show their support.

With the weakened economy we have been very conservative with our spending this year. But, like each previous year we have increased the amount we have spent on the FreeBSD Project and community. We were blown away with the number of project proposals we received this year. We were able to fund 7 projects this year. Unfortunately we didn’t have the budget to fund all the proposals we received.

This coming year we want to double the amount we spend on project development. In order to accomplish this, we need to meet our fund-raising goal.

Continues

Colin Percival has decided to donate all income made from his online backup service, Tarsnap, to the Foundation:

As a FreeBSD user and developer, I obviously care about the success of FreeBSD. I make a small contribution towards this success via my role as Security Officer; but the time I spend working on my Tarsnap online backup service prevents me from making as much of a direct contribution as I would like. Fortunately the FreeBSD Foundation does an excellent job of supporting FreeBSD development; but like most such organizations, they are funded entirely by donations and are always in need of more. In light of this, I am pleased to announce that I will be donating all of the profits made by Tarsnap for the month of December to the FreeBSD Foundation.

Continues

In memory of donations
After the passing away of John Birrell, the Foundation has made it possible to donate-in-memory-of:
It is now possible to make a donation to the FreeBSD Foundation in memory of someone. Select the “in memory of” button on the donation page and type in the name of the person the donation is being made in memory of.
Apart from my own donation, I donate 10% of the income (adverts, donations etc) from this blog.
Why not be generous, donate and help the Foundation reach it’s goal of $300.000 ?

Support FreeBSD and the FreeBSD Foundation

FreeBSD foundation logoMany people have already donated since my last post about the FreeBSD Foundation‘s end of year fund raising drive.

So far almost 600 people  (and companies) have donated to the FreeBSD Foundation, but the  $300.000 target hasn’t been achieved yet: the counter stands at $166.381.

What does the FreeBSD Foundation do?

We fund and manage projects, sponsor FreeBSD events, Developer Summits and provide travel grants to FreeBSD developers. We provide and help maintain computers and equipment that support FreeBSD development and improvements. We also represent the Project in executing contracts, license agreements, copyrights, trademarks, and other legal arrangements that require a recognized legal entity.

FreeBSD 8.0 has been released now. If you’re a happy FreeBSD user and want to support the development of FreeBSD 9, why not make a (small) donation? More people have donated this year than before. Any amount counts.

Donate quickly and securely

medium_dsc_8084