The Ports+ Architecture: uyota 匠の一手

The Ports+ has front-end and back-end processes. The front-end helps users to run "make config" commands before build begins. The core is implemented in the back-end.

The back-end implements most of the functionalities. It is written in GNU make and uses GNU awk. GNU make has its own extension such that it can remake MAKEFILES targets and reload changes back to adjust make rules dynamically. Generated "make" rules uses special characters such as dash (-), period (.), and probably some more. There is no specific document or language specification in any versions or branches of the "make" problems but, at least, GNU make allows them. Another GNU extension used is multi-level variable expansions. GNU awk has extension of handling of -v argument first, and then, execution of BEGIN blocks. Some awks implement the other way, BEGIN block first and then -v argument. That prevents any preprocessing in the BEGIN block based on -v arguments.

The back-end is a big wrapper around the FreeBSD ports system. A single "make" process maintains all dependencies and processing of all commands. The ports system is a collection of "make" files. That makes difficult to build concurrently even if each port is independent of each other. The Ports+ extracts all dependency information and determines the order of builds for given targets.

The back-end uses INDEX-? file for the package name resolutions and dependencies for ones not installed. "?" is the version number of FreeBSD. In addition, it uses /var/db/pkg database to determine dependencies and dynamic libraries for installed packages and and /var/db/ports to process OPTIONS. OPTIONS can change add or remove package dependencies and needs extra steps to resolve. After getting all dependencies, "make" rules are written and new rules are reloaded by GNU make. Once that's done, GNU make generates dependency trees and starts executing.

The front-end is just a simple helper. When a user wants to install or upgrade a few packages, config-recursive is likely a preferred way. On the other hand, when many or all packages are being upgraded, config-conditional is most likely enough. One "make" file being an entire system won't be an easy one to use for most of people, the front-end is expected to give some user friendly actions.

The original plan started to support only upgrading installed packages. It was found that sometime upgrading installs a new package as a new dependency. In order to implement that, all packages had to be loaded to complete the equations. That led to more expensive operations required by the Ports+. Yet, Ports+ is magnitude faster than portupgrade or portmaster for resolving dependencies of many packages.

Ports+ is best used with pkgdb and portupgrade. It assumes that all dependency information is correct in /var/db/pkg. Pkgdb fixes if there are problems. Also, ports+ cannot handle special cases noted in UPDATING file.

This is ideal for multi-core systems. For example, if the traditional way of upgrading a few hundreds of packages via ports takes eight hours for a modern 8-core system, the Ports+ may finish in about one hour. Finish majority of upgrades with ports+ and complete the details by portupgrade.