Update on FreeBSD HAST Project

Pawel Jakub DawidekBack in October we mentioned that the FreeBSD Foundation decided the fund the HAST project (new FreeBSD Project: HAST)

Pawel Jakub Dawidek, the sponsored FreeBSD developer, has made quite a bit of progress since and finished the development of stage one:

I want to report that first milestone of the HAST project is complete.

Summary of the work that have been done:

  • Implementation of hastd daemon.
  • Implementation of hastctl utility to manage hastd daemons.
  • GEOM_GATE class was extended so that the caller can specify the name of GEOM provider. Before only /dev/ggateX names were supported. HAST will use /dev/hast/
  • Implementation of communication protocol. There is abstraction layer on top and below there are three protocols implemented currently:
  • proto_tcp4 – It is used for communication between primary and secondary nodes.
  • proto_uds – (UDS – UNIX Domain Socket) It is used for communication between hastctl and hastd.
  • proto_socketpair – It is used for communication between main hastd daemon and worker processes forked from it.
  • Implementation of nv (name-value) API, which allows to easy create packets containing name-value pairs. It is used for entire communication through the protocols above. It is also responsible for managing correct byte-order.
  • Implementation of ebuf (extendable buffer) API, which provides a way to extend given buffer by adding data at the back, but also at the front without reallocating it and copying the data very often (or never).
  • Implementation of logging API (pjdlog). The API decides if messages should be logged on standard output/error (before going into background) or to syslog (when we daemonize). It also provides some shortcuts for logging a message and exiting, etc. It supports notion of debug level and can skip messages intended for higher debug level than requested.
  • Implementation of configuration file parser in lex/yacc. Configuration file is designed in a way that it can be kept identical on both nodes.
  • Checksumming and compression for the data is not one of the project’s goal, but the stubs are there, so this can be added easly.
  • A lot of care was taken to be able to handle more nodes in the future. This is not implemented and in not project goal, but I wanted to make it ready for future improvements (source)

HAST enables users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes. The software will allow for synchronous block-level replication of any storage media (GEOM providers, using FreeBSD nomenclature) over the TCP/IP network and for fast failure recovery. HAST will provide storage using GEOM infrastructure, which means it will be file system and application independent and could be combined with any existing GEOM class. In case of a master node failure, the cluster will be able to switch to the slave node, check and mount UFS file system or import ZFS pool and continue to work without missing a single bit of data.

In order to fund more of these sort of projects, the Foundation is reliable on our donations and support. The Foundation hasn’t reached its 2009 goal yet. You can help by donating.

(BTW, I’m in no way affiliated with the work of the FreeBSD Foundation;  I only want to see FreeBSD progress to become even better)

FreeBSD Mall now shipping FreeBSD 8

freebsd_8_cdThe FreeBSD Mall is now shipping FreeBSD® Version 8.0. The four-disc CD set or dual-sided DVD is available for purchase, either individually or on a subscription basis at a discounted price. The FreeBSD Mall has a long tradition of providing a reliable source of software, documentation, and support to the open source community.

We are pleased to be the primary distributor of the FreeBSD 8.0 release. FreeBSD Mall takes its commitment to customer service very seriously, and will continue its current tradition of providing outstanding FreeBSD services and software through this latest release.

says Theresa Garner, General Manager, FreeBSD Mall, Inc.

FreeBSD Version 8.0 marks the beginning of the new 8-STABLE branch, which improves upon the functionality of FreeBSD Version 7.X and introduces new features. Key focuses of the FreeBSD 8.0 release include wireless networking, virtualization, and storage technology.

The release features the addition of Virtual Access Points (VAP) support to 802.11 wireless networking, which allows mulitiple wireless networks to be hosted from a single access point. Draft 802.11 mesh networking support allows FreeBSD-based devices to dynamically link together to create a larger wireless network.

In addition, virtual machine administrators can now create their own nested jails, and FreeBSD now supports host and guest modes in Virtual Box. FreeBSD 8.0 can also run as a 32-bit Xen Dom U guest.

Other notable features of FreeBSD 8.0 include:

  • NFSv3 GSSAPI support, experimental NFSv4 client and server.
  • 802.11s D3.03 wireless mesh networking and Virtual Access Point support.
  • ZFS is no longer in experimental status.
  • Ground-up rewrite of USB, including USB target support.
  • Continued SMP scalability improvements in many areas, especially VFS.
  • Revised network link layer subsystem.
  • Experimental MIPS architecture support.

pfSense on FLOSS (audio interview)

pfsense logo 100x100Scott Ullrich and Chris Buechler, the guys behind the pfSense project, have been interviewed by Randal Schwartz and Leo Laporte on FLOSS 101. (via)

FLOSS Weekly is a podcast covering free and open source software.

Will Backman has interviewed Randal Schwartz on BSDTalk (24 mins). They talk about Randal’s early experiences with BSD, permissive licenses, OpenBSD, OpenSolaris, perl, the BSDFund credit card, and the Floss Weekly podcast.

New version of mergemaster with type checking

I think the commit message explains this pretty well, but I wanted to give a heads up message because this is very likely to ring a few bells the first time through. I personally noticed it as result of the recent termcap change, and I’m sure I’m not the only one with stale stuff in /etc/ in particular.

Please make sure that you employ (what should be) the usual precaution of backing up /etc before running this version of mergemaster. I also encourage regular use of the -P option (or PRESERVE_FILES=yes in ~/.mergemasterrc or /etc/mergemaster.rc) however while I believe that this new code is -P safe backing up /etc first is preferred this time around.

Author: dougb
Date: Sat Dec 12 02:19:41 2009
New Revision: 200425
URL: http://svn.freebsd.org/changeset/base/200425

Over time things that used to be files/directories/links can change to something else. So add code to detect when things don't match and give the user choices about how to fix it.

If we're using -P and something in the above check needs to be moved we need to have the directory there for it, so create it at the beginning and delete empty versions of it at the end.

The case where something used to be a file or link and now is supposed to be a directory (e.g., /etc/security) is especially dangerous, so make failure to install a necessary directory in $DESTDIR a fatal error.


Describe an OS

Laszlo explains why his little company is running FreeBSD:

I have a small company, where we decided to choose Freebsd as the server platform. We don’t have too much money to spend so the price was one of the main reasons. We needed a server platform which can host web services reliably.

Although my partners and me also are IT experts, we wanted to spend the least possible time with the operating system. We wanted to concentrate on the web development instead. When we started the company, Windows wasn’t reliable at all. We hadn’t got enough time to support it – not mentioning the price of a Windows server with Microsoft or Oracle SQL server. LINUX was an obvious choice.

After fighting with the problems in several LINUX distributions (SUSE, Slackware), one of my friends asked: Why don’t you try Freebsd? I never had any problems with it! Upon his suggestion we installed Freebsd on a server without any previous experience. We experienced that Freebsd completely fulfilled our needs, it adequately performs as a web server which needs no maintenance. We usually reinstall our servers when we need a full version upgrade, apart from it our Freebsd servers were running all time without interruption. These “servers” were desktop computers without any server features! On a workshop SamoaTel ISP wrote: “And, in case you missed this, FreeBSD is extremely stable, particularly under heavy load. ”

I think Freebsd have more advantages than disadvantages. Once it is set up properly – which is not always easy in case of exotic hardware devices – it runs till the end of the hardware’s life.

Read the whole post – describe an OS (via)

Video for Linux (v4l) support in FreeBSD

freebsd_webcamAlexander Leidinger announced that he’s working on Video4Linux support for FreeBSD. The Linux headers can be used without having to accept the GPL.  This makes it potentially available to all the BSDs:

“Yesterday I committed the v4l support into the linuxulator (in 9-current). Part of this was the import of the v4l header from linux. We have the permission to use it, it is not licensed via GPL. This means we can use it in FreeBSD native drivers, and they are even allowed to be com piled into GENERIC (but I doubt we have a driver which could provide the v4l interface in GENERIC).

The code I com mitted is “just” the glue-code which allows to use FreeBSD native devices which provide a v4l inter face (e.g. multimedia/pwcbsd) from linux programs.

If someone is willing to write the glue-code for the v4l2 interface please contact me. We have the permission to use the v4l2 header too, we just need some one doing the coding.

In a similar way, if someone is willing to add v4l2 inter face support to FreeBSD native drivers (I do not know any FreeBSD driver which provides a v4l2 interface), just tell me and I import the v4l2 header into FreeBSD.”

The whole post is still available through Google Cache

Why the BSDs get no love

Jack Wallen has updated the Techrepublic blog with a post about how old-fashioned and how so 90′s some of the BSD operating systems are with regards to their installation:

Over the weekend, I installed OpenBSD. And then I installed FreeBSD. And then I quickly realized why the BSDs are getting no love. For some odd reason, the BSDs refuse to join the rest of the modern world. Instead they have decided that they (the BSD communities) are going to rebel and remain in the 90s with the text-based installation and their cryptic install instructions. But then the BSD community complains that they get no love…no press…no user-base. Oh sure, the silver-back geeks and the server farms will run one or more flavors of BSD (it IS insanely stable and secure). More than likely, those are the users that have been running BSD since their days in high-school computer club.

I write these words in hopes that a member of the BSD clan will get word back to their high counsel. If BSD wants to gain any respect among the masses, they have to modernize, join the new world order, and (at the very least) add a GUI installation tool – or, heaven forbid, a Live CD. I did find a Live BSD project that hadn’t been updated since 2004. After much digging, I did finally come up with the BSD Anywhere project that attempts to modernize the BSD (OpenBSD at least), but shoots itself in the foot by including only the IceWM window manager with a default configuration that looks, surprisingly, very 90s! Go figure.

Jack updated his post with a comment acknowledging that PC-BSD has a graphical installer and is easy to install.

Does he know that the upcoming PC-BSD 8.0 will be able to install plain FreeBSD as well as PC-BSD. All this graphically, intuitively and easily, and with a lot of goodies, incl. ZFS

pkgin 0.3.0 for FreeBSD 8.0

pkgin_logoPkgin (not pidgin ;-) is a tool developed by a NetBSD developer, to conveniently handle and manage pkgsrc binary packages and is aimed to be an apt / yum like tool for managing pkgsrc binary packages. Pkgin relies on pkg_summary for installation, removal and upgrade of packages and associated dependencies, using a remote repository.

Many GNU/Linux distributions provide a convenient way of searching, installing and upgrading software by using binary archives found on “repositories”. NetBSD and operating systems relying on pkgsrc have tools like pkg_add and pkg_delete, but they are unable to correctly handle binary upgrades, and sometimes even installation itself.

Baptiste Daroussin has started porting pkgin to FreeBSD a couple of weeks ago. Although pkgin relies on pkgsrc packages (NetBSD, DragonFlyBSD), Baptiste has patched it so it can handle FreeBSD ports binaries.

More about this can be read here:  EN or FR.