Update on FreeBSD HAST Project

Pawel Jakub DawidekBack in October we mentioned that the FreeBSD Foundation decided the fund the HAST project (new FreeBSD Project: HAST)

Pawel Jakub Dawidek, the sponsored FreeBSD developer, has made quite a bit of progress since and finished the development of stage one:

I want to report that first milestone of the HAST project is complete.

Summary of the work that have been done:

  • Implementation of hastd daemon.
  • Implementation of hastctl utility to manage hastd daemons.
  • GEOM_GATE class was extended so that the caller can specify the name of GEOM provider. Before only /dev/ggateX names were supported. HAST will use /dev/hast/
  • Implementation of communication protocol. There is abstraction layer on top and below there are three protocols implemented currently:
  • proto_tcp4 – It is used for communication between primary and secondary nodes.
  • proto_uds – (UDS – UNIX Domain Socket) It is used for communication between hastctl and hastd.
  • proto_socketpair – It is used for communication between main hastd daemon and worker processes forked from it.
  • Implementation of nv (name-value) API, which allows to easy create packets containing name-value pairs. It is used for entire communication through the protocols above. It is also responsible for managing correct byte-order.
  • Implementation of ebuf (extendable buffer) API, which provides a way to extend given buffer by adding data at the back, but also at the front without reallocating it and copying the data very often (or never).
  • Implementation of logging API (pjdlog). The API decides if messages should be logged on standard output/error (before going into background) or to syslog (when we daemonize). It also provides some shortcuts for logging a message and exiting, etc. It supports notion of debug level and can skip messages intended for higher debug level than requested.
  • Implementation of configuration file parser in lex/yacc. Configuration file is designed in a way that it can be kept identical on both nodes.
  • Checksumming and compression for the data is not one of the project’s goal, but the stubs are there, so this can be added easly.
  • A lot of care was taken to be able to handle more nodes in the future. This is not implemented and in not project goal, but I wanted to make it ready for future improvements (source)

HAST enables users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes. The software will allow for synchronous block-level replication of any storage media (GEOM providers, using FreeBSD nomenclature) over the TCP/IP network and for fast failure recovery. HAST will provide storage using GEOM infrastructure, which means it will be file system and application independent and could be combined with any existing GEOM class. In case of a master node failure, the cluster will be able to switch to the slave node, check and mount UFS file system or import ZFS pool and continue to work without missing a single bit of data.

In order to fund more of these sort of projects, the Foundation is reliable on our donations and support. The Foundation hasn’t reached its 2009 goal yet. You can help by donating.

(BTW, I’m in no way affiliated with the work of the FreeBSD Foundation;  I only want to see FreeBSD progress to become even better)

FreeBSD end-of-year fund raising drive (update)

freebsd_foundation 100.x100FreeBSD is free; it can be downloaded, used and adapted without paying any fees, unlike other major operating systems. This is why the FreeBSD Foundation needs donations to be able to fund new projects and conferences.

The FreeBSD Foundation is collecting donations for the next year. The Foundation is a regular sponsor of many events, funds trips (including mine) to FreeBSD conferences and occasionally funds projects for various enhancements to FreeBSD. What else does the Foundation do?

The Foundation writes on its blog:

We would like to thank everyone who has donated to the FreeBSD Foundation this year. We have raised $183,888 towards our 2009 goal of $300,000! We are almost 2/3 of the way to reaching our goal! Oh, and BTW, we have had 671 donors this year. This is compared to just over 300 this time last year. This is important not only to help us keep our Public Charity Status, but it shows there are many users who are passionate about FreeBSD and want to show their support.

With the weakened economy we have been very conservative with our spending this year. But, like each previous year we have increased the amount we have spent on the FreeBSD Project and community. We were blown away with the number of project proposals we received this year. We were able to fund 7 projects this year. Unfortunately we didn’t have the budget to fund all the proposals we received.

This coming year we want to double the amount we spend on project development. In order to accomplish this, we need to meet our fund-raising goal.

Continues

Colin Percival has decided to donate all income made from his online backup service, Tarsnap, to the Foundation:

As a FreeBSD user and developer, I obviously care about the success of FreeBSD. I make a small contribution towards this success via my role as Security Officer; but the time I spend working on my Tarsnap online backup service prevents me from making as much of a direct contribution as I would like. Fortunately the FreeBSD Foundation does an excellent job of supporting FreeBSD development; but like most such organizations, they are funded entirely by donations and are always in need of more. In light of this, I am pleased to announce that I will be donating all of the profits made by Tarsnap for the month of December to the FreeBSD Foundation.

Continues

In memory of donations
After the passing away of John Birrell, the Foundation has made it possible to donate-in-memory-of:
It is now possible to make a donation to the FreeBSD Foundation in memory of someone. Select the “in memory of” button on the donation page and type in the name of the person the donation is being made in memory of.
Apart from my own donation, I donate 10% of the income (adverts, donations etc) from this blog.
Why not be generous, donate and help the Foundation reach it’s goal of $300.000 ?

Support FreeBSD and the FreeBSD Foundation

FreeBSD foundation logoMany people have already donated since my last post about the FreeBSD Foundation‘s end of year fund raising drive.

So far almost 600 people  (and companies) have donated to the FreeBSD Foundation, but the  $300.000 target hasn’t been achieved yet: the counter stands at $166.381.

What does the FreeBSD Foundation do?

We fund and manage projects, sponsor FreeBSD events, Developer Summits and provide travel grants to FreeBSD developers. We provide and help maintain computers and equipment that support FreeBSD development and improvements. We also represent the Project in executing contracts, license agreements, copyrights, trademarks, and other legal arrangements that require a recognized legal entity.

FreeBSD 8.0 has been released now. If you’re a happy FreeBSD user and want to support the development of FreeBSD 9, why not make a (small) donation? More people have donated this year than before. Any amount counts.

Donate quickly and securely

medium_dsc_8084

New FreeBSD Foundation Project: Flattened Device Tree

FreeBSD foundation logoThe FreeBSD Foundation has announced another funded project!

“Rafal Jaworowski and Semihalf has been awarded a grant to provide FreeBSD with support for the flattened device tree (FDT) technology. This project allows for describing hardware resources of a computer system and their dependencies in a platform-neutral and portable way.

The main consumers of this functionality are embedded systems whose hardware resources assignment cannot be probed or self-discovered.

The FDT idea is inherited from Open Firmware IEEE 1275 device-tree notion (part of the regular Open Firmware implementation), and among other deployments is used as a basis for Power.org’s embedded platform
reference specification (ePAPR).

Rafal JaworowskiThanks to this project, embedded FreeBSD platforms will grow in a uniform and extensible way of representing hardware devices, compliant with industry standards (ePAPR, Open Firmware), independent of architecture and platform (portable across ARM, MIPS, PowerPC etc.),

said Rafal Jaworoski, FreeBSD Developer.

Semihalf is a privately owned company, based in Krakow, Poland. They specialize in embedded systems design and development, with expertise in both software and hardware. Among their portfolio are FreeBSD ports to high-end embedded processors (including multi-core) with a wide range of peripheral drivers (storage, networking, pattern matching, security engines etc.); most of this work is publicly available from the FreeBSD repository.

You can find out more about the project at http://wiki.freebsd.org/FlattenedDeviceTree.

This project will complete by February 2010.”

If you want, you can support this project too.

New FreeBSD Foundation Project: HAST

FreeBSD foundation logoThe FreeBSD Foundation has announced that is funding a new funded project: HAST

“Pawel Jakub Dawidek has been awarded a grant to implement storage replication software that will enable users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes. The project is partly being funded by OMCnet Internet Service and TransIP BV.

The software will allow for synchronous block-level replication of any storage media (GEOM providers, using FreeBSD nomenclature) over the TCP/IP network and for fast failure recovery. HAST will provide storage
using GEOM infrastructure, which means it will be file system and application independent and could be combined with any existing GEOM class. In case of a master node failure, the cluster will be able to
switch to the slave node, check and mount UFS file system or import ZFS pool and continue to work without missing a single bit of data.

High-availability is the number one requirement for any serious use of any operating system,

Pawel Jakub Dawideksaid Pawel Jakub Dawidek, FreeBSD Developer.

Highly available storage is one of the key components in such environments. I strongly believe there are many FreeBSD users that have been waiting a long time for this functionality. I’ll do my best to deliver software that matches FreeBSD quality and that will satisfy the needs of our users.

Pawel has been an active FreeBSD committer since 2003. During this period, he has touched almost every part of the kernel. But, his main interest in FreeBSD is storage and security related topics. Pawel is the author of various GEOM classes (eli, mirror, gate, label, journal, hsec, etc.), geom(8) utility, various opencrypto improvements as well as port of the ZFS file system from OpenSolaris to FreeBSD.

The project will complete by February 2010.”

If you want, you can support this project too.

FreeBSD Foundation call for donations

FreeBSD foundation logoWe’re over half way through 2009, but the FreeBSD Foundation has not reached half of their 2009 fundraising goal. Justin Gibbs, founder and president of the FreeBSD Foundation, is calling on people’s generosity to support.

The Foundation is (part)funding some of the (Free)BSD conferences, sends developers to attend and funds new projects.

Justin writes:

Millions of systems run FreeBSD.  Hundreds of volunteers contribute to FreeBSD’s success.  But what is the size of FreeBSD’s user base?  This simple question is very hard to answer, but its answer is vital to the cause of promoting FreeBSD.  It is extremely difficult to convince
businesses to invest time and money to add FreeBSD support to their products based solely on vague estimates of the size of our community.
We should know – working to make FreeBSD a more widely supported platform is a task the FreeBSD Foundation has worked on since its inception.

Please help us in our fight to promote FreeBSD.  A donation to the FreeBSD Foundation helps fund our work, but it also gives us strength in numbers.  Our count of unique donors is a vital indication of the size and buying power of our community.  However, we have never broken even one thousand donors in any year.  We know in our hearts that this is a small fraction of our user base and of those who want to help expand FreeBSD’s presence.

So stand up and be counted!  Make a donation.  Encourage other FreeBSD users to donate as well.  No donation amount is too large or too small.  Just by becoming a donor you are making a powerful statement about the strength of FreeBSD!

You can make a donation by going to: http://www.freebsdfoundation.org/donate/.

To find out more about The FreeBSD Foundation, please visit http://www.freebsdfoundation.org.

FreeBSD foundation newsletter – June 2009

The FreeBSD Foundation have released their quarterly update. It gives a nice overview of the projects and conferences that are funded by the Foundation.
Table of contents:
  • Letter From the President
  • 2009 Fundraising Drive
  • Dru Lavigne Helping Foundation
  • Safe Removal of Active Disk Devices
  • Wireless Mesh Support
  • Improvements to the FreeBSD TCP Stack
  • AVR32 Support
  • Problem Reporting Prototype
  • FreeBSD Powers Long Distance Wireless Link
  • DCBSDCon 2009
  • AsiaBSDCon 2009
  • Foundation at BSDCan and Developer Recognition
  • 2009 Grant and Travel Grant Recipients
  • BSDCan Spotlight
  • Financials

Read the whole issue here.

The foundation is still way away from their donation target. To support FreeBSD and the FreeBSD Foundation, why not make a donation on their website? If you ever decide to support my website, 10% of your donation will be given to the Foundation.

Since recent times, the Foundation have their own blog and twitter account.