Update on FreeBSD Jail Based Virtualization Project

Bjoern Zeeb has provided a summary regarding the completion of the funded portion of the FreeBSD Jail Based Virtualization Project:

“I am happy to report that the funded parts of the FreeBSD Jail Based Virtualization project are completed. Some of the results have been shipping with 8.1-RELEASE while others are ready to be merged to HEAD.

Jails have been the well known operating system level virtualization technique in FreeBSD for over a decade. The import of Marko Zec’s network stack virtualization has introduced a new way for abstracting subsystems. As part of this project, the abstraction framework has been generalized. Together with Jamie Gritton’s flexible jail configuration syscalls, this will provide the infrastructure for, and will ease the virtualization of, further subsystems without much code duplication. The next subsystems to be virtualized will likely be SYSV/Posix IPC to help, for example, PostgreSQL users. This will probably be followed by the process namespace.”

The full post can be read on the FreeBSD Foundation’s blog: Update on FreeBSD Jail Based Virtualization Project

FreeBSD Foundation turns to NYI.net for East Coast US Mirror

The following is a press release issued by the FreeBSD Foundation and New York Internet; Bill Lessard from nyi.net forwarded this to me.
If you have any FreeBSD related products or services you want to generate interest in, why not contact me
too?

FreeBSD Foundation turns to NYI.net for East Coast US Mirror

Deployment Adds Enterprise-Grade Redundancy for Improved Reliability, Reduced Latency, High-Speed Backups and Other Efficiencies

The FreeBSD Foundation, a 501(c)(3) non-profit organization dedicated to supporting the FreeBSD Project and community, today announced that NYI, a New York City-based, mission-critical data services provider, will be mirroring key West coast infrastructure at NYI’s 999 Frontier Road data center in Bridgewater, New Jersey, a recently opened 40,000 square foot facility.

In addition to providing enterprise-grade redundancy and reliability for the Project’s infrastructure, the East coast mirror will reduce latency during heavy download times, distribute load between the two coasts, and allow for up-to-date backups of all Project data that can be synchronized via high-speed Internet connections.

“Having a well-connected, secondary site with NYI’s amenities to host FreeBSD project infrastructure means that we can move services between sites when doing scheduled maintenance to improve reliability for FreeBSD developers and users,”

said Simon L. Nielsen, FreeBSD.org administrative team. He added,

“The new site also enables us to expand significantly the available hardware for FreeBSD package building, allowing the FreeBSD ports team to perform QA test builds and quickly produce binary FreeBSD packages for end-users.”

“We are long-time open-source advocates. The FreeBSD Foundation in particular represents everything that got us into technology in the first place. With this deployment, we take our commitment to a new level in the hope that what we are doing lays the foundation for next-generation data centers built around FreeBSD. As many people in the community know, NYI’s 999 Frontier Road facility features many of the Project’s efforts, as everything from PDUs to the servers run FreeBSD.”

said Phillip Koblence, VP Operations, NYI.

The East coast mirror at 999 Frontier is also notable because it replaces aging and inadequate hardware; provides dual-configuration so that experimental vs. production runs can be separated out, allowing changes to the ports system to be evaluated continuously rather the interrupting production flow; deploys to multiple sites, providing resiliency in the event of a failure; provides build capacity required to support the ports ABI changes required to improve the foundations for binary package support while maintaining ports-stable regression testing.

The FreeBSD Foundation is pleased to have been able to fund the purchase of the hardware. Brad Davis, Mark Linimon, and Simon Nielsen from the FreeBSD Project worked on the configuration, along with key members of the NYI team.

About The FreeBSD Foundation
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting the FreeBSD Project and community. The Foundation gratefully accepts donations from individuals and businesses, using them to fund and manage projects, sponsor FreeBSD events, Developer Summits and provide travel grants to FreeBSD developers. In addition, the Foundation represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. The FreeBSD Foundation is entirely supported by donations. More information about The FreeBSD Foundation is available on the web.

About NYI

Established in 1996, NYI is headquartered in the heart of the Wall Street area and owns and maintains its own data centers, including 999 Frontier, a newly opened 40,000 square foot facility in Bridgewater, New Jersey. The company’s core services include colocation, dedicated servers, web and email hosting, and managed services, as well as turnkey disaster recovery and business continuity solutions from its Bridgewater location. With high-bandwidth connectivity partners AboveNet, Verizon Business, Optimum Lightpath, and AT&T, NYI specializes in mission-critical data services for the financial services industry, in addition to customers from a broad range of industries, including media, law, fashion, architecture, life sciences and real estate. NYI is SAS 70 Type II-compliant, in additon to being both PCI and HIPAA compliant.

FreeBSD events and conferences (KyivBSD, MeetBSD, EuroBSD)

Here is some info and details of upcoming FreeBSD related conferences and events.

KyivBSD 2010 Conference

On 25 Septempter the annual KyivBSD Conference will be held in Kiev, Ukraine. It’s mainly aimed at FreeBSD and PC-BSD users and developers.

More info: http://ru.kyivbsd.org.ua (RU) (EN translation) (via)

BSD-Day 2010

BSD-Day@2010 will be held at Eötvös Loránd University, Budapest, Hungary on 20 November 2010.

The purpose of this one-day event is to gather Central European developers of today’s open-source BSD systems, popularize their work, and provide a real-life communication interface between developers and users. There are no formalities, no papers, and no registration or participation fee, however the invited developers are encouraged to give a talk on their favorite BSD-related topic. The goal is to motivate potential future developers and users, especially undergraduate university students to work with BSD systems.

EuroBSDCon 2010 Travel Grants

The FreeBSD Foundation is now accepting travel grant applications for EuroBSDCon 2010 (Karlsruhe, Germany from October 8-10′. More details: EuroBSDCon 2010 Travel Grants.

For those interested in open source firewalls, there will be a pfSense tuturial at EuroBSDCon 2010.

pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router. In addition to being a powerful, flexible firewalling and routing platform, it includes a long list of related features and a package system allowing further expandability without adding bloat and potential security vulnerabilities to the base distribution.

MeetBSD 2010 (California)

Registration is now open for MeetBSD 2010 (Mountain View, California, 5-6 November): www.meetbsd.com

NYCBUG presentation

Ivan Ivanov presented “Examples in Cryptography with OpenSSL”. Download/listen the MP3.

Usenix Security Symposium 2010 (Capsicum)

Robert Watson will present Capsicum (coming in FreeBSD 9.0) at Usenix Security Symposium.

Capsicum is a lightweight operating system capability and sandbox framework planned for inclusion in FreeBSD 9. Capsicum extends, rather than replaces, UNIX APIs, providing new kernel primitives (sandboxed capability mode and capabilities) and a userspace sandbox API. These tools support compartmentalisation of monolithic UNIX applications into logical applications, an increasingly common goal supported poorly by discretionary and mandatory access control. We demonstrate our approach by adapting core FreeBSD utilities and Google’s Chromium web browser to use Capsicum primitives, and compare the complexity and robustness of Capsicum with other sandboxing techniques.

These and other conferences can be found on my FreeBSD Events and Conferences Calandar.

FreeBSD foundation newsletter – July 2010

The FreeBSD Foundation has published their semi-anual newsletter with updates on projects receiving support and funding, their donation goal for 2010 and how FreeBSD and commercial vendors can co-operate.

Projects funded by the FreeBSD Foundation

The FreeBSD Foundation funds various projects each year and there are several interesting projects in progress:

The FreeBSD Foundation is able to fund these projects due to the generosity of user donations. If you haven’t donated yet this year, consider doing so here. No donation amount is too small–in fact, the more individual donations the better, as this helps the Foundation to meet the IRS’ non-profit requirements. Also, your name is added to the website when you donate, allowing you to show your support for FreeBSD.

Source: Dru’s blog (reposted with permission)

FreeBSD DAHDI driver project announcement

The FreeBSD Foundation has announced it will sponsor Max Khon to finish the DAHDI FreeBSD driver port.

DAHDI (Digium/Asterisk Hardware Device Interface) is the open source device interface technology used to control Digium and other legacy telephony interface cards.

“The purpose of DAHDI/FreeBSD project is to make it possible to use FreeBSD as a base system for software PBX solutions.

DAHDI (Digium/Asterisk Hardware Device Interface) is an open-source device driver framework and a set of HW drivers for E1/T1, ISDN digital and FXO/FXS analog cards (http://www.asterisk.org/dahdi/). Asterisk is one of the most popular open-source software PBX solutions.

The project includes porting DAHDI framework and HW drivers for E1/T1, FXO/FXS analog and ISDN digital cards to FreeBSD. This also includes TDMoE support, software and HW echo cancellation (Octasic, VPMADT032) and HW transcoding support (TC400B). The work is ongoing in the official DAHDI SVN repository with the close collaboration with DAHDI folks at Digium.

The project is nearing its completion: DAHDI framework and HW drivers telephony cards has been ported and tested. There are a number of success stories from early adopters who use E1/T1 and FXO/FXS cards on FreeBSD for several months.”

Congratulations, Max, for receiving the grant. It’s great to see FreeBSD branching out into another specialist area. AskoziaPBX used to be based on FreeBSD, but hardware support issues made the team decide to move to a Linux based PBX solution. Hopefully we will soon see another fully FreeBSD based PBX system (Askozia, Michael Iedema?)

FreeBSD Foundation funded projects (Jail, DTrace)

One of the ways the FreeBSD Foundation supports FreeBSD, is by funding FreeBSD development (e.g. the Jail based virtualisation project). The Foundation has agreed to fund two more projects:

I Resource Containers Project

This project will be undertaken by Edward Tomasz Napierala. “Unlike Solaris zones, the current implementation of FreeBSD Jails does not provide per-jail resource limits. As a result, users are often forced to replace jails with other virtualization mechanisms. The goal of this project is to create a single, unified framework for controlling resource utilisation, and to use that framework to implement per-jail resource limits. In the future, the same framework might be used to implement more sophisticated resource controls, such as Hierarchical Resource Limits, or to implement mechanisms similar to AIX WLM. It could also be used to provide precise resource usage accounting for administrative or billing purposes.”

“It’s great that the Foundation decided to fund this project. It will make jail-based virtualization a much better choice in many scenarios, for example for Virtual Private Server providers.”

Edward noted.

II DTrace Userland Project

Rui Paulo has been awarded a grant to add DTrace userland support to FreeBSD.

DTrace is a general purpose and lightweight tracing framework that allows administrators, developers and users to investigate causes of system failure or performance bottlenecks. The FreeBSD operating system has had support for kernel-only DTrace since FreeBSD 8.0, but DTrace userland support was missing. Having userland support in DTrace allows inspection of userland software itself and its correlation with the kernel, thus allowing a much better picture of what exactly is going on behind the scenes.

This project will first concentrate on adding libproc support for symbol to address mapping, address to symbol mapping, breakpoint setup and the rtld interactions with DTrace. Next it will focus on DTrace process control, importing the pid provider and adapting it to FreeBSD and porting the userland statically defined probe provider (usdt). Finally it will bring in the plockstat provider.

“By having userland DTrace support, companies can make their products perform much better on FreeBSD due to the fact that they now have access to this amazing tool. When we mix the userland support with the kernel side DTrace support, we can also make FreeBSD a better operating system because we can investigate performance bottlenecks much easier.”

said FreeBSD developer Rui Paul

Well done to Rui and Edward. We’re looking forward to testing the results of their work at the end of the year. If you wish to see more of these sort of projects, you can donate to the FreeBSD Foundation.

FreeBSD quick news and links (28/06/2010)

PC-BSD Installer

The new PC-BSD installer (available as GUI and text installer), which is also able to install plain FreeBSD, has now been committed to the FreeBSD source tree. This video goes into the details of the installer.

Kris Moore: PC-SYSINSTALL – A new system installer backend for PC-BSD and FreeBSD

BSD Certification

The BSD Certification Group needs reviewers for the BSDA exam objectives.

BSDA Certification Exam can be taken at MeetBSD 2010, Cracow, Poland: BSDA Certification, Cracow

Help test ZFS v15

ZFS v15 brings in user and group quotas and help is needed to test, before it’s imported.

I would like to do a call for testing for my ZFS v15 patch.

As the user/group quotas feature is too much attractive for my needs, I couldn’t resist and have created (and debugged + tested) a ZFS v15 patch for head (applies cleanly against stable/8 as well).

It is a backport of several onnv-revisions, always consulting pjd’s p4 tree and includes four post-9396 related user/groupquota bugfixes. The bootcode (zfsimpl.h) is properly updated to support v15 as well, the python part is modified (paths, smb support, ioctls).  Continues

Nvidia Releases a Much Improved Video Driver

Nvidia announced on June 22nd the final and stable version of the 256.x proprietary driver for Nvidia graphics cards. Nvidia 256.35 incorporates lots of fixes and improvements, over previous releases. Unofficial GLX support was also added for a few OpenGL extensions, as well as Thermal Settings reporting improvements, Compiz fixes, many VDPAU improvements, and many more. Without further introduction, let’s take a look at some of the most important changes brought by the Nvidia 256.35 video driver (via)

Press Release

FreeBSD Developer position

iXsystems is looking for a Senior Software Engineer:

SR. SOFTWARE ENGINEER in San Jose, CA. FreeBSD driver & kernel dev; prog in C/C++, PHP, SQL/x86 assembly lang; eng group; tools/env & tech writing skills; Reqs: BSCS + 5 yrs. exp. (source)

iXsystems is the all-around FreeBSD company that builds FreeBSD certified servers and storage solutions, runs the FreeBSD Mall, and is the corporate sponsor of PC-BSD and FreeNAS.

New FreeBSD Committers

The following people have been awarded commit privileges in recent weeks:

  • Andrey V. Elsukov (source)
  • Matthew Fleming (source)
  • Ashish Shukla (ports)
  • Brendan Fabeny (ports)

BSD Can sponsored trip reports

The FreeBSD Foundation kindly sponsored a number of FreeBSD developers to attend BSDCan 2010 (videos here). These are links to some of the reports:

And one NetBSD link:

Testing NetBSD: Easy Does It

In a software project as large as NetBSD the interactions between different software components are not always immediately obvious to even the most skilled programmers. Tests help ensure that the system functions according to the desired criteria. Periodic automated runs of these tests with results visible on the web ensures both that tests are run in a regular fashion and that the results are available to all interested parties.

This short article explains the NetBSD test strategies and provides a brief overview of the enabling technologies. It also details how effortless it is to run the test suite and why doing so is in every developer’s, patch submitter’s and system administrator’s best interest. The intended audience is people with a keen interest in testing and quality assurance, and a desire to reduce personal headache. The article is written against NetBSD-current as of June 2010 and applies to what will eventually become NetBSD 6.

Read more: Automated Testing Framework (ATF)