I recently started using Amazon Linux AMI as the main platform for deployment. So far I used Debian and Ubuntu distributions for long time, and very soon I was disappointed how outdated some software, provided by standard Amazon Linux AMI and EPEL repositories is.
Then I decided to to find out, how easy is to build your own software for Amazon Linux AMI. Luckily, it turned out to be easier, when you don't start from scratch, but use existing packages as a leverage. Here and in the Part 2. Publishing your own work I share my experience.
These instructions should work for all RHEL-based distributions, and to build rpm packages for CentOS in particular.
Yeah, the software in Amazon Linux AMI sometimes is too old. In particular, what I found outdated:
Not to mention that default Python is still 2.6 (released in 2008), and Python 3 is not in repository yet (released 2008-2009).
Actually, it's not that bad, after all. For example nginx is only two months old, and so on, but if you got accustomed to the luxury of being on the "bleeding edge", you'd feel slightly uncomfortable in a new environment, as I did.
I discovered that people tend to resort to following three decisions:
Two last options are kind of "improper". The rightful way would be to build a new rpm package, and to distribute it to your machines with a yum repository.
First, I consider how we can make it properly and at the same time with no big hassle. In the second part I describe more advanced techniques: how to use mock with SCM and how to publish your own yum repository on Amazon S3.
Usually all the software is already packaged. The only problem is that is is outdated sometimes. Here I describe how I re-built the latest version of python-pip package from Fedora repository.
I use Fedora Mock to build packages in a chrooted environment.
# yum install --enablerepo=epel mock
Create a separate user for builds
# useradd -s /bin/bash mockbuild # usermod -a -G mock mockbuild # newgrp mock
Then try to reset your privileges to a mockbuild and initialize a new chroot environment.
$ mock -r default --init
The -r option means "root" and defines the way how chroot will be created. Descriptions of roots you may find /etc/mock directory.
Assuming that you work in a Linux AMI instance, and your instance is a 64-bit, the directory /var/lib/mock/epel-6-x86_64/root/ contains the virtual root, and the directory /var/lib/mock/epel-6-x86_64/result/ will contain the output srpm and rpm files (not now, currently it should be empty).
Rawhide is the version of Fedora, containing the most recent versions of the packages. These packages aren't meant to work in Amazon Linux AMI, and to be honest, sometimes aren't meant to work at all (it's a very unstable branch, actually). Nonetheless, as it's the "cheapest" way for us to build a package, we start with it.
Find out the Fedora rawhide repository file, and add it to the list of your repositories. Download whatever you like from rpmfind.net.
# cd /tmp # wget ftp://rpmfind.net/linux/fedora/linux/development/rawhide/x86_64/os/Packages/f/fedora-release-rawhide-19-0.1.noarch.rpm # rpm2cpio fedora-release-rawhide-19-0.1.noarch.rpm | cpio -idmv # cp etc/yum.repos.d/fedora-rawhide.repo /etc/yum.repos.d/
This set of command downloads the rpm package for you and extract it to current directory. With the last command you copy the repository description to the standard repos path.
You may accidentally turn your system to garbage with the simple yum install, unless you disable rawhide repository. We will only enable rawhide temporary, to download srpms.
sed -i 's/enabled=1/enabled=0/g' /etc/yum.repos.d/fedora-rawhide.repo
Then find which version of the package Rawhide provides. Below is the python-pip example. It's cool that you can enable and disable yum repositories on the fly, by the way.
$ yum info --enablerepo='rawhide*' python-pip
Then download the srpm. Script yumdownloader is the part of yum-utils package which should be already installed in your system:
$ yumdownloader --enablerepo='rawhide*' --source python-pip
Rebuild the package
$ mock -r default --rebuild ./python-pip-1.2.1-2.fc19.src.rpm
If the build has passed successfully, you may find your packages in the /var/lib/mock/epel-6-x86_64/result/ directory. You may install it immediately with rpm -ihv <path/to/python-pip.rpm> or yum install <path/to/python-pip.rpm>
How kosher is the way of building rpm packages I described here? Not so much, actually. First, it's hard to distribute your package. Second, you don't have any control over the package contents, and unfortunatelu, the successful rebuild don't guarantee that it will work. And lastly, the only small subset of packages you can rebuild without any changes this way.
In the part 2 I try to explain, how you can re-build packages "consciously", and make them availabe via yum repository on github.