Argent Portage Central Configuration

From Argent Linux Wiki
Jump to navigationJump to search

System Administration

This is the system administration sequence, starting with the heart of every Gentoo based system: /etc/portage.

First, something you have to know: Argent is a complete Gentoo. Same portage, same ebuilds, same profiles, same USE flags, same /etc/portage, with every regular Gentoo feature. What Argent adds on top is a central configuration, a binhost built with that configuration, and tools like epkg and avasile to keep your system in line with them.

So everything the Gentoo documentation says applies to Argent too, and you should read it before you touch /etc/portage:

This page only explains what is different on Argent.

How /etc/portage works on Argent

On Argent, /etc/portage is a git directory. Actuall a symlink to a git repo:

/etc/portage -> /opt/argentws-build/conf/intel/portage

and /opt/argentws-build is a git clone of our central configuration repository:

https://gitlab.com/argent/argentws-build.git

avasile sets this up for you when you switch modes (--usermode, --devmode, --srcmode). The only thing that changes between modes is which make.conf is active:

/etc/portage/make.conf/00-main.conf -> make.conf.amd64-user (user mode)

/etc/portage/make.conf/00-main.conf -> make.conf.amd64-devel (devmode)

/etc/portage/make.conf/00-main.conf -> make.conf.amd64-srcmode (srcmode)

To see which mode you are on:

readlink -f /etc/portage/make.conf/*

Why is /etc/portage under git

Because the binary packages on our binhost are built with exactly this configuration. The USE flags, keywords, masks and make.conf you have are the same ones our build server used. If your configuration drifts away from it, the binaries stop matching your system, and portage starts compiling things you didn't ask for (or, in user mode, silently ignores what you changed).

So git gives us:

  • every Argent machine has the same configuration, the one the binaries were built with
  • one epkg update brings you the new configuration together with the new packages
  • every change we make is public, with history, on gitlab (for now)
  • switching modes is just changing one symlink

What epkg update does to /etc/portage

At every epkg update, /opt/argentws-build is reset to exactly what is on our server:

git fetch --depth=1 origin master && git reset --hard origin/master && git clean -fd

This means:

  • any change you make to an Argent file is lost at the next update
  • any new file of yours that git doesn't ignore is deleted at the next update
  • files that git ignores are left alone, and this is where your own configuration goes

epkg devsync does a plain git pull instead. If you changed an Argent file, the pull stops with a conflict, and you have to restore the file (see below) before it can continue.

Argent files and your files

Every directory in /etc/portage has a .gitignore that works as a whitelist: it ignores every file with a dot in its name (*.*), except the Argent files, which are listed one by one.

The Argent files start with 00-, like 00-argent.package.use. Don't edit them.

Your files:

  • must have a dot in their name, otherwise git doesn't ignore them and epkg update deletes them
  • should start with a bigger number than 00-, so they are read after ours and win over them

The convention is 100-my.<something>:

/etc/portage/package.use/100-my.package.use

/etc/portage/package.accept_keywords/100-my.package.keywords

/etc/portage/package.mask/100-my.package.mask

/etc/portage/package.unmask/100-my.package.unmask

/etc/portage/package.env/100-my.package.env

/etc/portage/make.conf/01-my.make.conf

Don't name a file 00-my.package.keywords, that name is on the Argent whitelist, so git doesn't protect it.

Keep a copy of your files before you switch modes. avasile --usermode, --devmode, --srcmode and --custom delete the whole /opt/argentws-build and clone it again, and your files go away with it. epkg update doesn't do this, only a mode switch does.

make.conf

make.conf is the main configuration of portage. It's where Gentoo keeps the settings for the whole system:

  • how packages are compiled: COMMON_FLAGS, CFLAGS, CXXFLAGS, MAKEOPTS
  • the global USE flags: USE, and the USE flags written as variables, like VIDEO_CARDS, L10N, PYTHON_TARGETS
  • how portage behaves: FEATURES, EMERGE_DEFAULT_OPTS
  • where things are kept: PKGDIR (binary packages), DISTDIR (sources), PORTDIR (the portage tree)
  • which licenses you accept: ACCEPT_LICENSE, and where sources are downloaded from: GENTOO_MIRRORS

It's written like a shell script, VARIABLE="value", and a variable can use another one, like CFLAGS="${COMMON_FLAGS}". The Gentoo documentation is here, and all the variables are in:

man make.conf

make.conf as a directory

On Gentoo, /etc/portage/make.conf can be a single file or a directory. When it's a directory, portage reads every file in it in alphabetical order, as if they were one big make.conf, and when the same variable is set twice, the last one wins.

Argent uses it as a directory. The main file is a symlink to our make.conf for the mode you are on:

/etc/portage/make.conf/00-argent.conf -> /opt/argentws-build/conf/intel/portage/make.conf.amd64-devel

(or 00-main.conf, same thing)

The main file must be called 00-argent.conf, 00-main.conf or main.conf. These are the names epkg looks for, if it can't find one of them, epkg fails to run. Don't rename it, and don't replace it with your own.

The three Argent make.conf files are the same, except for how they use the binhost:

  • make.conf.amd64-user: --getbinpkgonly, only binaries
  • make.conf.amd64-devel: --getbinpkg and FEATURES=buildpkg, binaries when they match, compile and save the binary when they don't
  • make.conf.amd64-srcmode: no binhost, compile everything and save the binaries

Your own make.conf

Next to ours, you can keep your own:

/etc/portage/make.conf/01-my.make.conf

It's read after ours, so whatever you set there wins. The whole make.conf directory is ignored by git, so epkg update leaves it alone. For example: 

MAKEOPTS="-j16"
VIDEO_CARDS="amdgpu radeonsi"
L10N="en en-US ro ro-RO"
USE="${USE} -wayland"

Things to know:

  • MAKEOPTS, COMMON_FLAGS and the other compiler settings only matter for what you compile yourself. If you use -march=native, what you compile (and what gets saved in /var/cache/binpkgs) runs only on your CPU
  • everything that changes USE flags (USE, VIDEO_CARDS, L10N, PYTHON_TARGETS, ...) means compiling in devmode, and has no effect in user mode, see USE flags below
  • don't change PORTDIR or GENTOO_DISTRO, epkg depends on them

The moment you do this, you are deriving away from Argent. Your system is no longer the one our binaries were built for, and we can't help you anymore with the issues specific to your setup. But that's what Gentoo is in the end: your system, your configuration, and back to reading the documentation.

make.conf.local

/etc/portage/make.conf.local is not a portage file, portage never reads it. Only epkg reads it, right after the main make.conf, so whatever you put there overrides the main make.conf, for epkg only. avasile and epkg create it empty.

Today it matters only for the few variables epkg reads directly from make.conf when it starts, like PORTDIR and GENTOO_DISTRO. PKGDIR is not one of them, epkg asks portage for it, so setting it in your 01-my.make.conf is enough. Most users should leave make.conf.local empty.

USE flags

The global USE flags live in the active make.conf (make.conf.amd64-user, make.conf.amd64-devel, ...), grouped by topic: SYSTEM_USE, CODECS_USE, MEDIA_USE, SOUND_USE and so on. The per package USE flags live in /etc/portage/package.use/00-*.package.use.

If USE flags are new to you, read the Gentoo Handbook on USE flags first, the rest of this section assumes you know what they are.

In user mode

Don't change USE flags in user mode. User mode installs only binary packages (--getbinpkgonly), and in this case portage doesn't check if the USE flags of the binary match yours. You get our binaries, built with our USE flags, whatever you set, so your change simply does nothing.

If you need different USE flags, switch to devmode first:

avasile --devmode

In devmode

Here you can change USE flags. When your USE flags are the same as ours, portage installs the binary from our binhost. When they differ, portage compiles that package for you, and saves the result in /var/cache/binpkgs so you don't compile it again.

To see which USE flags a package has, and which ones are active:

equery uses <package>

To change them for one package, add a line to your own file:

echo "media-video/mpv lua -wayland" >> /etc/portage/package.use/100-my.package.use

To change them globally, use your own make.conf file (see make.conf above):

echo 'USE="${USE} -wayland"' >> /etc/portage/make.conf/01-my.make.conf

Keep the ${USE} in there, otherwise you throw away all our USE flags, not only the one you wanted to change.

Then see what would be rebuilt, and do it:

epkg upgrade

epkg upgrade already rebuilds every package whose USE flags changed. For only one package:

epkg src-newuse <package>

Every USE flag you change is one more package you compile yourself, and one more package that no longer follows the binhost. Change only what you really need.

Looking into the git

Since /etc/portage is a git repository, you can use git to see what is going on:

cd /etc/portage

By default you have only the last commit (epkg update fetches with --depth=1). You can see the whole history on gitlab, or bring it locally with:

epkg force-sync

Then, what we changed lately:

git log --oneline

What we changed in one file, and why:

git log -p package.use/00-argent.package.use

The next epkg update makes it shallow again.

If you changed any Argent file by mistake:

git status

git diff

To restore an Argent file you changed:

git checkout -- package.use/00-argent.package.use

To see your own files (the ones git ignores):

git status --ignored

Never commit anything here. At the next epkg update your commit is thrown away by the reset, together with everything in it.

Leaving the central configuration

If you want /etc/portage fully yours, disconnected from Argent, avasile has the --custom and --decentralize-* modes. They are for advanced Gentoo users, read avasile --help carefully before using them. --decentralize-gentoo is a one way ticket.


Portage

Portage on Argent is the regular Gentoo portage. epkg is only a wrapper around it, so you can always use emerge, equery, qlist and the rest of the Gentoo tools directly. The mode you are on still applies: in user mode, emerge installs only binaries too, because --getbinpkgonly is set in the make.conf.

For how portage itself works, the Gentoo Handbook: Portage is the reference. Below is only what Argent does differently.

Repositories

Argent uses two repositories, both defined in /etc/portage/repos.conf/:

argent-ws in /var/db/repos/argent-ws, synced from https://gitlab.com/argent/argent-ws.git

gentoo in /var/db/repos/gentoo, synced from https://gitlab.com/argent/portage.git

argent-ws is the main repository (main-repo = argent-ws), with our own packages, our fixes and our profiles. It has a higher priority (50) than gentoo (1), so when both have the same version of a package, ours wins.

The gentoo repository is our snapshot of the Gentoo tree, not the live Gentoo one, so the ebuilds you have match the binaries you get. Don't point it to the Gentoo servers, you will get ebuilds that are newer than our binaries and you'll end up compiling half of the system.

To see which one is the main repository on your system:

portageq repositories_configuration / | grep main-repo

In user mode, both repositories are minimal: only profiles, metadata and eclass, no ebuilds. You don't need ebuilds to install binaries. In devmode and srcmode you get the full trees.

Your own overlays

You can add any other overlay the Gentoo way, for example with eselect repository:

eselect repository enable guru

It writes /etc/portage/repos.conf/eselect-repo.conf, which has a dot in its name, so it survives epkg update. If you write the repos.conf file by hand, follow the same rule, like /etc/portage/repos.conf/100-my-overlay.conf.

Keep in mind:

  • epkg update syncs only gentoo and argent-ws, use epkg devsync or emaint sync -r <overlay> for yours
  • our binhost has nothing from your overlays, everything from there gets compiled, so overlays make sense only in devmode or srcmode

Example: your own kernel

A good reason for your own overlay is your own kernel. Say you want linux-argent with a different kernel config. You don't edit our ebuilds (epkg update resets them), you clone them into your overlay under a new name.

You need devmode for this, in user mode argent-ws has no ebuilds to clone from:

avasile --devmode

1. Create your overlay

eselect repository create myoverlay

It lives in /var/db/repos/myoverlay. Our kernel ebuilds use the argent-kernel eclass from argent-ws, so your overlay has to see it. Edit /var/db/repos/myoverlay/metadata/layout.conf: 

masters = gentoo argent-ws
thin-manifests = true

2. Clone the two ebuilds under a new name

The name matters: linux-mykernel builds a kernel called 6.12.110-mykernel, so it installs next to 6.12.110-argent, not over it. The sources package must be called mykernel-sources, the same way ours are linux-argent and argent-sources.

cd /var/db/repos/myoverlay

mkdir -p sys-kernel/linux-mykernel/files sys-kernel/mykernel-sources/files

cp /var/db/repos/argent-ws/sys-kernel/linux-argent/linux-argent-6.12.110.ebuild sys-kernel/linux-mykernel/linux-mykernel-6.12.110.ebuild

cp /var/db/repos/argent-ws/sys-kernel/linux-argent/files/* sys-kernel/linux-mykernel/files/

cp /var/db/repos/argent-ws/sys-kernel/argent-sources/argent-sources-6.12.110.ebuild sys-kernel/mykernel-sources/mykernel-sources-6.12.110.ebuild

3. Get a kernel config to start from

Our kernel config is inside our kernel tarball. Download it, and take the config out of it:

epkg fetchsourcepkg =sys-kernel/argent-sources-6.12.110

tar -xJOf /var/cache/distfiles/linux-6.12.110-argent.tar.xz --wildcards '*/argent/config/argent-6.12-amd64.config' > /root/mykernel-6.12-amd64.config

Change it the way you want. The easiest way is to install argent-sources, copy the config in there as .config, run make menuconfig, and take the .config back.

Then put it in both packages, with the name the eclass looks for (<name>-<major.minor>-amd64.config):

cp /root/mykernel-6.12-amd64.config sys-kernel/linux-mykernel/files/

cp /root/mykernel-6.12-amd64.config sys-kernel/mykernel-sources/files/

4. Edit the ebuilds

Both of them still download our kernel tarball (K_ROGKERNEL_SELF_TARBALL_NAME="argent" stays), but they use your config instead of ours.

sys-kernel/linux-mykernel/linux-mykernel-6.12.110.ebuild: 

EAPI=8

K_ROGKERNEL_SELF_TARBALL_NAME="argent"
K_REQUIRED_LINUX_FIRMWARE_VER="20260622"
K_ROGKERNEL_FORCE_SUBLEVEL="110"
K_ROGKERNEL_FORCE_UPPERLEVEL="6.12"
K_ROGKERNEL_PATCH_UPSTREAM_TARBALL="0"
K_KERNEL_NEW_VERSIONING="1"

inherit argent-kernel

KEYWORDS="amd64"
DESCRIPTION="My own kernel image, based on linux-argent"
RESTRICT="mirror"
IUSE="grub2"

src_prepare() {
	argent-kernel_src_prepare
	cp "${FILESDIR}/mykernel-6.12-amd64.config" "${S}/argent/config/" || die
}

sys-kernel/mykernel-sources/mykernel-sources-6.12.110.ebuild: 

EAPI=8

K_ROGKERNEL_FORCE_SUBLEVEL="110"
K_ROGKERNEL_FORCE_UPPERLEVEL="6.12"
K_ROGKERNEL_NAME="mykernel"
K_ROGKERNEL_URI_CONFIG="yes"
K_ROGKERNEL_SELF_TARBALL_NAME="argent"
K_ONLY_SOURCES="1"
K_KERNEL_NEW_VERSIONING="1"

inherit argent-kernel

KEYWORDS="amd64"
DESCRIPTION="My own kernel sources, based on argent-sources"
RESTRICT="mirror"
IUSE="grub2 sources_standalone"

DEPEND="${DEPEND}
	sources_standalone? ( !=sys-kernel/linux-mykernel-${PVR} )
	!sources_standalone? ( =sys-kernel/linux-mykernel-${PVR} )"

src_prepare() {
	argent-kernel_src_prepare
	cp "${FILESDIR}/mykernel-6.12-amd64.config" "${S}/argent/config/" || die
}

What changed compared to ours:

  • K_ROGKERNEL_NAME is mykernel in the sources ebuild (in linux-mykernel it comes from the package name by itself)
  • the sources depend on linux-mykernel, not on linux-argent
  • src_prepare() puts your config next to ours, inside the unpacked tarball, where the eclass looks for it

5. Manifest and install

ebuild sys-kernel/linux-mykernel/linux-mykernel-6.12.110.ebuild manifest

ebuild sys-kernel/mykernel-sources/mykernel-sources-6.12.110.ebuild manifest

epkg install sys-kernel/mykernel-sources

This pulls in linux-mykernel too, and compiles both, there is no binary for your kernel on our binhost. With USE=grub2, the grub menu is regenerated at the end, otherwise do it yourself:

grub-mkconfig -o /boot/grub/grub.cfg

And rebuild the external modules (nvidia, virtualbox, zfs, ...) for your new kernel:

epkg module-rebuild

6. Keeping it up to date

Your kernel doesn't follow ours anymore. When we move linux-argent to a new version, you copy the new ebuilds again, rename them, and put your changes back in. If the major version changes (6.12 to 6.18 for example), your config needs a new name too, mykernel-6.18-amd64.config, and a look with make oldconfig.

Keep a copy of /etc/portage/repos.conf/eselect-repo.conf, a mode switch with avasile removes it (your overlay in /var/db/repos/myoverlay stays).

Binhost

The binary packages come from our binhost, defined in:

/etc/portage/binrepos.conf/binhost-main.conf

The mode decides how portage uses it:

  • user mode: --getbinpkgonly, only binaries, never compiles
  • devmode: --getbinpkg, binaries when they match your configuration, compiles when they don't, and saves what it compiled in /var/cache/binpkgs (FEATURES=buildpkg)
  • srcmode: compiles everything

If you have your own binhost (for example a second machine that compiles for you), add it in its own file, like /etc/portage/binrepos.conf/100-my-binhost.conf, with its own priority. See the Gentoo binary package guide for the format.

Profile

Argent uses the Gentoo profile:

default/linux/amd64/23.0/desktop/plasma/systemd

avasile sets it at every mode switch, and our binaries are built for it. You can see it with:

eselect profile show

You can change it with eselect profile set, as on any Gentoo, but then your system no longer matches the binhost, and the next avasile mode switch sets it back.

On top of the profile, we have our own overrides in /etc/portage/profile/ (use.mask, package.use.mask, package.provided). Those are Argent files too.

Keywords, masks and unmasks

Same as on Gentoo, same rule as for USE flags: your own files, next to ours.

To accept the testing version of a package:

echo "app-editors/neovim ~amd64" >> /etc/portage/package.accept_keywords/100-my.package.keywords

To stop a package (or a version) from being installed:

echo ">=media-video/obs-studio-32" >> /etc/portage/package.mask/100-my.package.mask

To allow something that we masked:

echo "=media-video/obs-studio-32.0.1" >> /etc/portage/package.unmask/100-my.package.unmask

Before you unmask anything of ours, look in /etc/portage/package.mask/00-*.package.mask why we masked it, there is usually a comment. In user mode, an unmasked or keyworded version that is not on our binhost can't be installed.

Per package environment

/etc/portage/package.env/ together with /etc/portage/env/ lets you compile some packages with different settings (CFLAGS, FEATURES, MAKEOPTS, ...). We use it for a few packages already, see /etc/portage/package.env/00-argent.package.env. The Gentoo documentation for it is here.

Your own, for example to compile a big package with fewer jobs:

echo 'MAKEOPTS="-j2"' > /etc/portage/env/100-my-lowjobs.conf

echo "www-client/chromium 100-my-lowjobs.conf" >> /etc/portage/package.env/100-my.package.env

This only matters when a package gets compiled, so devmode or srcmode.

Sets

Argent ships its own package sets in /etc/portage/sets/: @basesystem, @plasma-full, @plasma-tools, @multimedia, @tools and so on. To see what is in one:

cat /etc/portage/sets/basesystem

To install one:

epkg install @multimedia

You can make your own set, it's a text file with one package per line. Careful with the name: our set names have no dot, but yours must have one, otherwise epkg update deletes it:

/etc/portage/sets/my.tools

and then:

epkg install @my.tools

The world file

/var/lib/portage/world is the list of packages you asked for. Everything else on your system is there because something in that list needs it.

  • epkg install adds to it
  • epkg oneshot and epkg recompile don't
  • epkg remove takes out of it
  • epkg clean removes everything that nothing in it needs anymore

To take a package out of it without removing the package itself (it goes away at the next epkg clean, if nothing else needs it):

emerge --deselect <package>

Retrieved from "https://wiki.argentlinux.io/index.php?title=Argent_Portage_Central_Configuration&oldid=60"