This post is obsolete

This blog post is quite old, and a lot has changed since then. Meanwhile, I have changed employer, blogging platform, software stack, infrastructure, interests, and more.
You will probably find more recent and relevant information about the topics discussed here elsewhere.
Still, this content is provided here for historical reasons, but please don’t expect it to be current or authoritative at this point.
Thanks for stopping by!

OpenSolaris Home Server Scripting 3: Managing Package Repositories

Package Scripting

OpenSolaris OS comes with the Image Packaging System (IPS) (no link, opensolaris.org no longer exists) for managing installation and additional software. By default, it is configured to pull packages from the /release (no link, opensolaris.org no longer exists) repository, but of course there are many other interesting repositories with software to install from.

Chris Gerhard recommends adding at least the /extras repository (free, registration required (no link, sun.com no longer exists)) so you can easily install VirtualBox, Adobe Flash, TrueType Fonts etc.

If you have a support contract or are an Oracle/Sun employee, you may want to switch your preferred repository to the supported one. The more adventurous may want to switch to the development repository (no link, opensolaris.org no longer exists).

Then, the Software Porting Community on OpenSolaris.org (no link, opensolaris.org no longer exists) has a hierarchical system for making open source software available on OpenSolaris through two repositories: /pending and /contrib.

And then there are a couple of more repositories from user groups, private people, companies offering commercial packages and so on.

One way to register package repositories with the IPS on your home server is by using the package management GUI, or by using the pkg(1) (no link, opensolaris.org no longer exists) command (See also: “How to Add or Update a Publisher (no link, sun.com no longer exists)”).

But of course we want to stick to our “Script Everything” philosophy and write ourselves a small script that takes care of all of our package configuration needs.

Scripting Package Configuration

We’ll use a simple configuration file that contains all the parameters necessary for configuring our favorite package repositories, one line per repository.

For each repository, we’ll need:

  • Its URL for pkg(1) to pull packages from.

  • A name for the repository.

  • An optional certificate file and a key file for repositories that want them (We’ll just store the base name and assume they’ll end with key.pem or .certificate.pem).

  • An optional flag that indicates whether we want this repository to be our preferred one.

To keep things nice and simple, we’ll separate the fields with spaces and add a few comments for documentation. Here’s an example, taken from my current home server configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
admin@krengi:/krongi/config/pkg$ more pkg.config 
#
# Package configuration
#
# Fields are separated by a space.
# Fields: URL publishername preferred keyfile
# "preferred" is either empty or not.
# Keyfile is the basename of the key/certificate name
https://pkg.sun.com/opensolaris/support standardsupport OpenSolaris_standard_support preferred
http://pkg.opensolaris.org/contrib/ contrib
https://pkg.sun.com/opensolaris/extra/ extras OpenSolaris_extras
https://pkg.sun.com/opensolaris/ha-cluster/ hacluster Open_HA_Cluster_2009.06
http://pkg.opensolaris.org/release/ opensolaris.org

Certificate and key files are stored together with our configuration file in the same directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
admin@krengi:/krongi/config/pkg$ ls -al
total 14
drwxr-xr-x 2 admin staff   9 2010-03-20 14:48 .
drwxr-xr-x 6 admin staff   6 2010-03-05 13:49 ..
-rw-r--r-- 1 admin staff 758 2010-03-20 14:40 Open_HA_Cluster_2009.06.certificate.pem
-rw-r--r-- 1 admin staff 888 2010-03-20 14:41 Open_HA_Cluster_2009.06.key.pem
-rw-r--r-- 1 admin staff 750 2010-03-20 14:41 OpenSolaris_extras.certificate.pem
-rw-r--r-- 1 admin staff 888 2010-03-20 14:41 OpenSolaris_extras.key.pem
-rw-r--r-- 1 admin staff 758 2010-03-20 14:41 OpenSolaris_standard_support.certificate.pem
-rw-r--r-- 1 admin staff 888 2010-03-20 14:41 OpenSolaris_standard_support.key.pem
-rw-r--r-- 1 admin staff 535 2010-03-20 14:48 pkg.config

Resetting to Default State

As always, we’ll start with a function that resets everything to a default state: Only the /release repository, configured as the preferred one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Hard coded constants
DEFAULT_PUB="opensolaris.org"
DEFAULT_PUB_URL="http://pkg.opensolaris.org/release/"

# Pfexec versions of commands
PFPKG="pfexec pkg"

# Clear repositories from pkg and revert to default
function clear_publishers {
        # Set default publisher.
        echo "Setting default publisher $DEFAULT_PUB at $DEFAULT_PUB_URL."
        $PFPKG set-publisher -P -O $DEFAULT_PUB_URL $DEFAULT_PUB

        # List all publishers we know except the default one.
        PUBLISHERS=$( pkg publisher -Ha | sed -e 's/\s\s*/ /g' | \
                cut '-d ' -f1 | grep -v $DEFAULT_PUB )

        # Clear all publishers.
        if [ "$PUBLISHERS" ] ; then
                echo "Clearing all other publishers."
                $PFPKG unset-publisher $PUBLISHERS
        fi

        echo "Done."
}

Going Through the List of Repositories

Now the more interesting part follows: We’ll feed the configuration file line by line into a while loop that puts together the parameters for the right pkg(1) call.

We’ll also copy key and certificate files into /var/pkg/ssl which is the same location that the package manager GUI uses.

Here’s the piece of code for parsing the configuration file and configuring package repositories:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
KEY_DIR="/var/pkg/ssl"
PFMKDIR="pfexec mkdir"
PFCHOWN="pfexec chown"
PFCP="pfexec cp"

# Print an error message and exit.
function emit_error {
    echo "Error: $1"
    exit 1
}

# Set up repositories for pkg
# The argument is a file containing information about publishers and their URLs
# with possible keyfiles and "preferred" option.
function add_publishers {
    CONFIG_FILE=$1
    CONFIG_DIR=$(dirname $1)
    [ $CONFIG_DIR ] || CONFIG_DIR="."

    # Make sure the directory for storing keys is set up correctly.
    if [ ! -d $KEY_DIR ] ; then
        echo "Creating directory $KEY_DIR."
        $PFMKDIR -m 0755 -p $KEY_DIR
        $PFCHOWN root:root $KEY_DIR
    fi

    # File format:
    # One line per package repository.
    # Lines containing "#" are ignored.
    # Fields in a line are separated by space.
        # Fields: "URL publisher keyfile preferred".
    # "preferred" can be any string. If present, this publisher will be
    # marked as preferred.
    # Keyfile will be expanded to "keyfile.key.pem" and
    # "keyfile.certificate.pem".
    # We assume that keyfiles are in the same directory as the config file.
    # "keyfile" and "preferred" can be left out.
    cat $CONFIG_FILE | grep -v '#' | while read url pub key pref; do
        # Handle the keyfile field.
        key_string=""
        key_opt=""
        if [ $key ] ; then
            echo "Storing key and certificate for $key in $KEY_DIR."
            $PFCP $CONFIG_DIR/$key.key.pem $KEY_DIR
            [ $? -eq 0 ] || \
                emit_error \
                "Can't copy $CONFIG_DIR/$key.key.pem."
            key_opt="-k $KEY_DIR/$key.key.pem"
            $PFCP $CONFIG_DIR/$key.certificate.pem $KEY_DIR
            [ $? -eq 0 ] || \
                emit_error \
                "Can't copy $CONFIG_DIR/$key.certificate.pem."

            key_opt="$key_opt -c $KEY_DIR/$key.certificate.pem"
            key_string=" with key $key"
        fi

        # Handle the preferred field.
        pref_string=""
        pref_opt=""
        if [ $pref ] ; then
            pref_string=" preferred"
            pref_opt="-P"
        fi

        # Put everything together into a pkg command.
        echo "Adding$pref_string publisher $pub at $url$key_string."
        $PFPKG set-publisher $pref_opt $key_opt -O $url $pub
        [ $? -eq 0 ] || \
            emit_error "Couldn't add publisher $publisher."
    done
}

This script is slightly more chatty than previous scripts. You may want to modify it to be more silent or use a log file.

Conclusion

Download the full config_pkg (no link, page no longer exists) script including a help function and argument parsing and feel free to modify it to your needs. Over time, you’ll accumulate quite a few repositories, some with key/certificate files and some not. Then this is a handy tool to configure them all at once whenever needed.

Your Take

What are your favourite repositories? Do you use alternative package management systems, if so which ones? What packages do you install on a regular basis after a fresh install of OpenSolaris?

Comments

Commenting is currently not available, because I’d like to avoid cookies on this site. I may or may not endeavor into building my own commenting system at some time, who knows?

Meanwhile, please use the Contact form to send me your comments.

Thank you!

Welcome

This is the blog of Constantin Gonzalez, a Solutions Architect at Amazon Web Services, with more than 25 years of IT experience.

The views expressed in this blog are my own and do not necessarily reflect the views of my current or previous employers.

Copyright © 2022 – Constantin Gonzalez – Some rights reserved.
By using this site you agree to not hold the author responsible for anything related to this site. See Site Info/Imprint for details and our information policy.

This site was built using Pelican, which is written in Python, using a homegrown theme that borrows heavily from Elegant. It is hosted on Amazon S3 and distributed through Amazon CloudFront.