FreeBSD Foundation funded projects (Jail, DTrace)

One of the ways the FreeBSD Foundation supports FreeBSD, is by funding FreeBSD development (e.g. the Jail based virtualisation project). The Foundation has agreed to fund two more projects:

I Resource Containers Project

This project will be undertaken by Edward Tomasz Napierala. ”Unlike Solaris zones, the current implementation of FreeBSD Jails does not provide per-jail resource limits. As a result, users are often forced to replace jails with other virtualization mechanisms. The goal of this project is to create a single, unified framework for controlling resource utilisation, and to use that framework to implement per-jail resource limits. In the future, the same framework might be used to implement more sophisticated resource controls, such as Hierarchical Resource Limits, or to implement mechanisms similar to AIX WLM. It could also be used to provide precise resource usage accounting for administrative or billing purposes.”

“It’s great that the Foundation decided to fund this project. It will make jail-based virtualization a much better choice in many scenarios, for example for Virtual Private Server providers.”

Edward noted.

II DTrace Userland Project

Rui Paulo has been awarded a grant to add DTrace userland support to FreeBSD.

DTrace is a general purpose and lightweight tracing framework that allows administrators, developers and users to investigate causes of system failure or performance bottlenecks. The FreeBSD operating system has had support for kernel-only DTrace since FreeBSD 8.0, but DTrace userland support was missing. Having userland support in DTrace allows inspection of userland software itself and its correlation with the kernel, thus allowing a much better picture of what exactly is going on behind the scenes.

This project will first concentrate on adding libproc support for symbol to address mapping, address to symbol mapping, breakpoint setup and the rtld interactions with DTrace. Next it will focus on DTrace process control, importing the pid provider and adapting it to FreeBSD and porting the userland statically defined probe provider (usdt). Finally it will bring in the plockstat provider.

“By having userland DTrace support, companies can make their products perform much better on FreeBSD due to the fact that they now have access to this amazing tool. When we mix the userland support with the kernel side DTrace support, we can also make FreeBSD a better operating system because we can investigate performance bottlenecks much easier.”

said FreeBSD developer Rui Paul

Well done to Rui and Edward. We’re looking forward to testing the results of their work at the end of the year. If you wish to see more of these sort of projects, you can donate to the FreeBSD Foundation.

FreeBSD quick news and links (28/06/2010)

PC-BSD Installer

The new PC-BSD installer (available as GUI and text installer), which is also able to install plain FreeBSD, has now been committed to the FreeBSD source tree. This video goes into the details of the installer.

Kris Moore: PC-SYSINSTALL – A new system installer backend for PC-BSD and FreeBSD

BSD Certification

The BSD Certification Group needs reviewers for the BSDA exam objectives.

BSDA Certification Exam can be taken at MeetBSD 2010, Cracow, Poland: BSDA Certification, Cracow

Help test ZFS v15

ZFS v15 brings in user and group quotas and help is needed to test, before it’s imported.

I would like to do a call for testing for my ZFS v15 patch.

As the user/group quotas feature is too much attractive for my needs, I couldn’t resist and have created (and debugged + tested) a ZFS v15 patch for head (applies cleanly against stable/8 as well).

It is a backport of several onnv-revisions, always consulting pjd’s p4 tree and includes four post-9396 related user/groupquota bugfixes. The bootcode (zfsimpl.h) is properly updated to support v15 as well, the python part is modified (paths, smb support, ioctls).  Continues

Nvidia Releases a Much Improved Video Driver

Nvidia announced on June 22nd the final and stable version of the 256.x proprietary driver for Nvidia graphics cards. Nvidia 256.35 incorporates lots of fixes and improvements, over previous releases. Unofficial GLX support was also added for a few OpenGL extensions, as well as Thermal Settings reporting improvements, Compiz fixes, many VDPAU improvements, and many more. Without further introduction, let’s take a look at some of the most important changes brought by the Nvidia 256.35 video driver (via)

Press Release

FreeBSD Developer position

iXsystems is looking for a Senior Software Engineer:

SR. SOFTWARE ENGINEER in San Jose, CA. FreeBSD driver & kernel dev; prog in C/C++, PHP, SQL/x86 assembly lang; eng group; tools/env & tech writing skills; Reqs: BSCS + 5 yrs. exp. (source)

iXsystems is the all-around FreeBSD company that builds FreeBSD certified servers and storage solutions, runs the FreeBSD Mall, and is the corporate sponsor of PC-BSD and FreeNAS.

New FreeBSD Committers

The following people have been awarded commit privileges in recent weeks:

  • Andrey V. Elsukov (source)
  • Matthew Fleming (source)
  • Ashish Shukla (ports)
  • Brendan Fabeny (ports)

BSD Can sponsored trip reports

The FreeBSD Foundation kindly sponsored a number of FreeBSD developers to attend BSDCan 2010 (videos here). These are links to some of the reports:

And one NetBSD link:

Testing NetBSD: Easy Does It

In a software project as large as NetBSD the interactions between different software components are not always immediately obvious to even the most skilled programmers. Tests help ensure that the system functions according to the desired criteria. Periodic automated runs of these tests with results visible on the web ensures both that tests are run in a regular fashion and that the results are available to all interested parties.

This short article explains the NetBSD test strategies and provides a brief overview of the enabling technologies. It also details how effortless it is to run the test suite and why doing so is in every developer’s, patch submitter’s and system administrator’s best interest. The intended audience is people with a keen interest in testing and quality assurance, and a desire to reduce personal headache. The article is written against NetBSD-current as of June 2010 and applies to what will eventually become NetBSD 6.

Read more: Automated Testing Framework (ATF)

FreeBSD Foundation sponsors BSNMP enhancements

The FreeBSD Foundation has announced that Shteryana Shopova has been awarded a grant to make improvements to Bsnmp (snmp category).

This project includes several enhancements to the existing FreeBSD SNMP framework, including SNMPv3-compliant user authentication, packet encryption and view-based access control. In addition, the project also includes a new module that will allow full SNMP management and monitoring of the FreeBSD wireless networking stack. When the
project is completed, FreeBSD should be the OS of choice when building open source-based embedded wireless appliances due to the advanced capabilities of its wireless network stack, and the light-weight, secure and complete management solution that bsnmpd(1) will provide out of the box. Existing FreeBSD installations that use bsnmpd(1) for
monitoring will also benefit from the added security and finer-grained access-control to SNMP data.

“SNMP is the defacto standard for network monitoring,”

said Shteryana Shopova, FreeBSD developer. She also added,

“SNMP is used everywhere – in network servers, switches, routers, firewalls, workstations, ip phones, printers, UPSs, all sorts of embedded appliances. I am happy to have the opportunity to work on several additions to bsnmpd(1) that have been requested by the FreeBSD community.”

This project will be completed in October 2010.

Source: Bsnmp enhancements announcement (FreeBSD Announcement List)

FreeBSD Foundation announces jail based virtualization project

The FreeBSD Foundation has selected another project for funding: FreeBSD Jail based virtualisation project, to be undertaken by Bjoern A. Zeeb:

“We are pleased to announce that Bjoern A. Zeeb has been awarded a grant to improve FreeBSD’s jail based virtualization infrastructure and to continue to work on the virtual network stack. His employer, CK Software GmbH is matching the Foundation’s funding with hours.

FreeBSD has been well known for its jail based virtualization during the last decade. With the import of the virtual network stack, FreeBSD’s operating system level virtualization has reached a new level.

This project includes cleanup of two years of import work and development and, more notably, brings the infrastructure for a network stack teardown. Cleanly shutting down a network stack in FreeBSD will be the major challenge in the virtualization area to get the new feature to production ready quality for the 9.x release lifecycle.

Further, the project includes generalization of the virtual network stack framework, factoring out common code. This will provide an infrastructure and will ease virtualization of further subsystems like SYSV/Posix IPC with minimal overhead. All further virtualized subsystems will immediately benefit from shared debugging facilities, an essential
feature for early adopters of the new technology.

“Improved jail based virtualization support, that continues to be very lightweight and as easily manageable as classic jails, will be a killer feature for the next few years,”

said Bjoern A. Zeeb, FreeBSD developer. He also added,

“It will allow people to partition their FreeBSD server, run simulations without racks of hardware, or provide thousands of virtual instances in hosting environments fairly easy and efficiently. While this follows the trend of green computing, it also adds to FreeBSD’s virtualization portfolio with Xen or other more heavyweight hypervisor support, which can be mixed with jails as needed.”

While work in this area will have to continue, the funding for this project will end mid-July 2010.”

I wish Bjoern all the best and hopefully we can try and use his work in FreeBSD 9.

If you want to see the FreeBSD Foundation support and fund more of these sort of projects, you can help them by donating to the Foundation (I’m not affiliated to the Foundation)

FreeBSD Google SoC Projects started

The FreeBSD Project  received many applications from students  wanting to participate in Google’s Summer of Code program. This year 18 student proposals to work with the FreeBSD Project were accepted as part of this program.

For those with projects that were not accepted this year, the FreeBSD Project is always willing to help mentor students so they can learn more about operating system development through our normal community mailing lists and development forums. The FreeBSD Foundation can also be approached for project funding.

Read the official announcement (FreeBSD GSoC) for more information. The complete list of student projects selected for funding can be found in the FreeBSD Summer of Code wiki.

  • May 24: Start of coding
  • July 12-16: Mid-term Evaluations
  • August 9: End of coding
  • August 16-20: Final Evaluations

(FreeBSD Events Calendar)

FreeBSD Foundation accepting project proposals

Love FreeBSD? Love coding, taking FreeBSD to new hights and getting paid for your work?

If this is you, why not submit a proposal to the FreeBSD Foundation?

The FreeBSD Foundation is pleased to announce we are soliciting the submission of proposals for work relating to any of the major subsystems or infrastructure within the FreeBSD operating system. Proposals will be evaluated based on desirability, technical merit and cost-effectiveness.

To find out more about the proposal process please download the application document (pdf).

Note, the deadline for submitting project proposals is March 1.

If you need some inspiration as to where you can help out, take a look at the FreeBSD Ideas page to see where FreeBSD can be improved and extended.

FreeBSD related appointments

Erwin Lansing has joined the FreeBSD Foundation board of Directors (announcement)

The FreeBSD Foundation is pleased to announce that Erwin Lansing has joined the Board of Directors. For those of you who haven’t met Erwin, here is his bio:

Erwin previously worked for an rapidly expanding webhosting startup and now holds a position as Network Systems Engineer at the Danish incumbent ISP, TDC. He joined the FreeBSD Ports Development Team in 2003 and has been a member of the Ports Management Team since 2005. He is mainly working on the package building cluster, creating and distributing ready-to-install binary packages of 3rd party software for FreeBSD, in addition to regression testing the integration of FreeBSD with 3rd party software projects.

Thomas Abthorpe is the new FreeBSD portmgr secretary (source)

On behalf of portmgr, I am pleased to announce that portmgr has found a new secretary: Thomas Abthorpe. Thomas has been a FreeBSD ports committer since 2007 and has made more than 1000 commits since. He has previously served on the ports-security team and is currently a member of the KDE and donation teams. He has also mentored several new ports committers over the years.

In his role as portmgr secretary, Thomas will help portmgr keep track of ongoing issues, keeps the portmgr, and other bookkeeping work like organizing votes and stay in touch with other FreeBSD teams.

FreeBSD HAST project completed

Pawel Jakub Dawidek was awarded a grant by the FreeBSD Foundation last year to implement storage replication software that will enable users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes.

The HAST (High Availability Storage Project) is now completed. Pawel reports:

I’m very happy to report to FreeBSD users that the HAST project I was working on for the last three months is ready for testing and already committed to the HEAD branch.

I’ll describe what HAST does in few words. HAST allows for synchronous block-level replication of any storage media (called GEOM providers, using FreeBSD nomenclature) over a TCP/IP network for fast failure recovery. HAST provides storage using the GEOM infrastructure, meaning it is file system and application independent and can be combined with any existing GEOM class. In case of a primary node failure, the cluster will automatically switch to the secondary node, check and mount the UFS file system or import the ZFS pool, and continue to work without missing a single bit of data.

I must admit the project was quite challenging, not only from the technical point of view, but also because it was sponsored by the FreeBSD Foundation. The FreeBSD Foundation has a great reputation and is known to select the projects it funds very carefully. I felt strong pressure that should I fail, the FreeBSD Foundation’s reputation might be hurt. Of course, not a single dollar would be spent on a failed project, but the FreeBSD community’s expectations were very high and I really wanted to do a good job.

During the work a number of people contacted me privately offering help, explaining how important HAST is for FreeBSD and giving me the motivation to soldier on.

I hope that HAST will meet the community’s expectations and I myself am looking forward to using it

The final commit notice from Pawe? Jakub Dawidek briefly describes the project. A detailed discussion of the project is available form the FreeBSD Wiki.

This is a good example of how the wider FreeBSD community can financially support further development of FreeBSD, and it also shows the value that the FreeBSD Foundation brings to the community. To see more of these sort of projects started, funded and completed, why not support and donate to the Foundation so they can continue sponsoring more projects. (disclaimer: I am not affiliated with the FreeBSD Foundation)