SBnews - SlackBuilder News - current_development

Originally written: 2019-10-27
Last updated:

Current Development


Development has shifted gears from 14.2 to current through most of 2019. Other than upgrades to the noarch repository work on Slackbuilder-14.2 has been put on hold while all effort is put into preparing for the next Slackware release ( whenever that may be ). Official Slackware patches will still be available via the pkgd / Pkg-applet upgrade system though.

Some interesting new features have been introduced that are worth noting.

Split packages

For several years now I have been contemplating splitting Slackware packages into series of sub-packages. I've decided to go ahead and implement that idea. Going forward Slackbuilder will provide 4 sub-pkgs ( if applicable ) for each 1 pkg of Slackware's it harbors in its repositories. This also applies to any Slackbuilds.org packages etc... The break down is as follows...

For a given pkgname

  • pkgname: the main runtime executables and libraries etc...
  • pkgname-doc: any documention files...doc, info, man, gtk-doc...
  • pkgname-dev: includes, headers, libs, package-config .pc files etc...needed for compilation of third party apps.
  • pkgname-nls: native language support files or the 'locales'.

Note: If you don't want to deal with the split packages. The same pkgslists can still be used to install with. Just point them to a standard repository.

So why would anyone do this? Slackware prides itself on simplicity and one package is all you need. Right?

Well...sometimes an install everything approach is not feasible. While I appreciate Slackware's philosophy Slackbuilder's is to start small and build up to a complete install if desired.

Which leads to the next point.

The Installer

There have been a few new developments and some restructuring in the installer. ie:

  • A new pkglist convention.
  • A new API for settings configurations to facilitate cross arch targets etc...
  • cross architecture ( x86_64, i686, armv7, aarch64 ) installations all from one x86_64 workstation or usbboot.img.
  • not only cross arch but multiple versions can be installed ( for now just 14.2 and current ). most likely only the recent release version and current will be available at any given time but it is possilbe to provide more if desired.
  • new tools to accomodate the cross arch installs.
  • There is also a new hybrid setting SlackM soon to be renamed to Muscles that is comprized of converted Alpine to Slackware packages along with some Slackware source and Slackbuilds.org packages. Everything to get a Slackbuilder system up and running. For those that are unaware, Alpine is a musl libc distribution.

Package lists

The installer uses a SRC file that has a list of defined sources with one or more pkslists attached to them. Take a look in the tutorial here for a more complete description. The default pkgslists directory defined by the environment varialbe PKG_LST_DIR is /etc/slkn-setup/pkgslists.

The tendancy has been to create a new line of pkgslists for each new setting. This has proven cumbersome and unnecessarily redundant. So in the intrest of simplicity a single line of pkgslists are now used to define all Slackware source installations ( which at the moment is all but one ). So any official Slackware repository for any given architecture can utilize the same set of pkgslists. There are a few packages in the lists available in some repos but not others. This isn't a big deal. If a package is unavailable it is simply ignored.

The lists for all current Slackbuilder derivatives are...
Where SVRSN is the Slackware version (current only for now) and ext is either pkgs or lst.

  • A00_rootfs_SVRSN.ext: a minimal list of packages that will provide a working chroot environment
  • A01_boot_SVRSN.ext: a list of packages that will make the rootfs bootable ( minus the kernel pkgs )
  • A01_arm_extras: packages only found in arm/aarch64 repos...
  • A02_Xorg_SVRSN.ext: minimal X pkgs to provide a seccessful xinit/xterm login...no window manager etc...
  • A03_Desktop_base_SVRSN.ext: desktop support packages...things a window manager and other xap's need (minus the window manager)
  • A04_Audio_SVRSN.ext: alsa/pulseaudio and their dependancies etc...
  • A05_Dev_SVRSN.ext: development pkgs for non split pkg installations sutiable for compiling kernels etc...Note: this is not an all inclusive list. you may still need to install pkgs to meet any specific requirements.
  • A05_Dev-all_SVRSN.ext: all pkgs in A05_Dev_SVRSN plus all the pkgname-dev pkgs required for a development system etc...
  • A10_kernel_ARCH_SVRSN.ext: the kernel and modules required for the architecture and/or particular setting in question.
  • B00_Sbuilderbox_SVRSN.ext: ARCH specific pkgs for Slackbuilder's openbox desktop. There are variouse other B line pkgslists like for touchscreens etc... but there's no need to list them all here.
  • C00_SBtools_SVRSN.ext: this the heart of Slackbuilder. ie: pkg-tools, slkn-setup ( the installer ),sb-apps and others...essentially the pkgs that make Slackbuilder what it is.
  • C01_Sbox_SVRSN.ext: like SBtools these are the specific configurattion and support packages that make Slackbuilder's openbox desktop uniq. Similar to the B line there are other pkgslist in the C that don't need to be specifally listed here. ie: config pkgs for laptops, touchscreens. chromebooks etc...

Note: Typically the A line of pkgslists represent Slackware repositories. But in the case of split packages, can also be used for Slackbuilder repos. And not all Slackware packages are split. Just the ones used in Slackbuilder. B is for Slackbuilder-ARCH-SVRSN and C is for Slackbuilder-noarch-SVRSN repositories.

Settings API

I've been grappling with the best way to deal with installations outside the norm for a while now. Like for varying types of architectures or other non standard requirements. In the past I had to edit the installation program itsetlf. But that is prohibitive to individual solutions. The idea behind the SETTINGS is to allow for individuals to create their own installation specifications. In the beginning it was to just save simple SRC and postinstall script settings for a standard x86 Slackware installation. But as time marched on I have found that easily changing the standard routines on the fly is desireable.

A little background

I recently got my Samsung Chromebook (2012) back from my wife. She had been using it for the past 6 years. It was showing its age and She demanded an upgrade. Long story short. I now had the opportunity to develop an installation setting for it. In the coarse of this venture many new obtacles presented themselves. To solve these problems I needed a simple way of wresting control of the installation process at crucial points.

The result of this effort is a new Exynos_smsng_crbk_SVRSN istallation setting and a new API. I'll spare the details concerning the setting here. There is an Intallation.txt file in the docs/ sub directory of the setting for more info. But I'll give a little run down of the new API.

The basic structure

So first of all in order for your setting to show up in the installer dialog you simply create a directory under /etc/slkn-setup/settings/. It can be empty and it will still be presented in the settings dialog. Of coarse for it to have any meaning there has to at least be a SRC-ARCH or SRC-tpl file present with sources and package lists defined. Next should be the scripts file with a list of postinstall scripts to run. The new settings directory with these two files are the foundation of a setting. So far nothing is new here.

What is new are the other 5 files. The following is a list of the files the installer will act upon in order...

  1. CONF: If this file exists is pulled in at the beginning. Near the top of slkn-setup/setup.cfg. This is where you can define/redefine environment variables.
  2. SRC-ARCH: ARCH=(x86_64, i686, armv7, aarch64, etc). The installer defaults to this file first and is the one saved when creating a setting by the installer.
    Go here for more info on SETTINGS.
  3. SRC-tpl: if no SRC-armv7 etc... exists and this one does then this file is used. It is parsed and place holders of SVRSN, ARCH etc... are converted to whatever your running system is. Useful for maintaining one file for all architectures. The idea is that after the installer saves your setting you then convert it to the SRC-tpl format for production use.
  4. 0-init: If excecutable this file will run any initial setup that may be required. This is where we download a rootfs, qemu-ARCH-static and setup binfmt_misc etc...
  5. 1-target: If executable this file runs after selecting a DEVICE to partition. It overrides the default partitioning routine.
  6. 2-preinstall: If executable will perform any necessary preinstall tasks. It runs before getpkgs. This is where unpacking a rootfs and copying qemu-ARCH-static into it is done for example.
  7. 3-postinstall: Runs if executable before the scripts and overrides the default postinstall but not the scripts. This is a good place to run any special boot config routines etc...
  8. scripts: A simple list of postintstall configuration scripts to run.

These 8 files are how you can communicate with the installer. What is in them is entirely up to you. There are no special hooks or enforced structures to follow except for the SRC and scripts files. You can use as many or as few as required. The CONF, SRC and scripts files aren't required to be executable but the numbered 0-3 are if they are to be executed.

You can take a look in the SbArmfs, slarm64 and Exynos settings to see how I put them together. The SbArmFs setting is the base for the others. I just symlinked the CONF, and 0-3 scripts along with some other bin/ stuff back to SbArmFs. I created an environment varialbe to pull in a boot.cfg uniq to the setting in question and other stuff. I'll spare the details here. Basically your setting is yours to do with as you please.

There is a couple of things to keep in mind though. The CONF and 0-3 files are included at their described locations into the installer. They are not executed as a command per se. This gives you acces to the functions and variables of the installer. Look in the slkn-setup/setup.cfg file for whats available. There are some dialog functions for displaying messages, yes/no boxes, lists and menus etc..

Another important point to note here is the use of two function commands Refreshmsg and getconfig. The thing about dialog is that each presented dialog runs as its own script and it has a tendancy to drop environment variables. So we have to run Refreshmsg quite often in order to pass changes on to the rest of the program. Also if you notice variables in your CONF file are missing later on. You may need to call getconfig to grab them again.

Cross Arch and more

By now you probably noticed all the references to cross architecture installations. I want to mention a little more about this. It is only possible if your target device can boot from an sdcard or usb disk/drive. And the usbboot-x86_64-SVRSN.img image obviously cannot boot on an arm, aarch64 etc... device. But from an x86_64 host either form the usbboot-x86_64-SVRSN.img or your Slackbuilder workstation you can install onto an sdcard or usb-disk targeted for the arm device in question. Insert the sdcard in the arm device and boot up into your new Slackbuilder system.

Qemu, binfmt_misc

There are some support tools available to acheive this. In the /etc/slkn-setup/binfmt_tools directory you will find a command Archit. It is used to download the appropriate qemu-ARCH-static file and then register the ARCH with binfmt_misc. To use it first change to the binfmt_tools/ directory. Then call like so...

./Archit arm
or
./Archit aarch64

Or whatever your target arch is. Look in binfmt.conf for a complete list of supported architectures.

Using root file sytems

While I was designing the arm related settings. The idea was to just install packages normally. Once a chrootable environment becomes available in the target, getpkg begins installing packages via chroot. But I noticed that Slackware's installpkg was not able to install packages correctly for other architectures. For some reason it can't re-establish the symbolic links. At least for arm variants from an x86_64 host anyway.

Therefore we need to download an appropriate rootfs. Unpack it to the target and copy the binfmt_tools/qemu-ARCH-static to the rootfs/usr/bin/ directory before we can start installing packages. There are three rootfs archives thus far in the repository for this purpose. They all start with the prefix sbfs for Slackbuilder File System. One for each of the following architectures: x86_64, armv7, aarch64. The future should see more as more architechures are supported. There really should be one for i686 but I've been neglecting that architecture for some time now. I need to make sure the repository is brought up-to-date first.

The rootfs could be used as an installation medium in it's own right. Everything is provided in it to perform a complete installation. It just needs to be prepped a little.

You can run Archit from the rootfs to setup cross arch intalls. But I wouldn't recommend this approach. Every system call inside the rootfs has to go through the qemu-ARCH-statc binary. This creates a huge bottleneck. But for a rootfs the same ARCH as the host, it is a viable solution and can be applied on most linux systems.

There is a slkn-setup/settings/SbArmFs/tools/root-tools.txz archive that if unpacked to the rootfs/ will provide some tools in the root user home directory. After unpacking it and chrooting to the rootfs/ there will be an alias available defined in the root/.profile file of hello. Running this command will mount dev/ and sys/ inside the rootfs and update mtab so pkg-tools will work. Use the alias logout to quit. It will call mout ( which unmounts all the dev and sys stuff ) then calls exit.

Note: to upgrade the rootfs pkg-upgrade needs to be called with the -f option to force an upgrade. Otherwise it will fail due to the ps command crashing from lack of dependencies and other missing stuff ( small size has its price ). Normally pkg-upgrade tests if there is another instance of itself running. Inside the rootfs/ this is not necessary.

Chromebook Recovery

I have nuked the Chromeos partitions on several occasions on at least three Chromebooks over the years. For convenience I am shipping the offical Google linux_recovery.sh script in /etc/slkn-setup/chromebook_recovery/. Just in case you do the same. Or you can download it directly from Goolge at https://dl.google.com/dl/edgedl/chromeos/recovery/linux_recovery.sh if you'd rather.

Muscles

A couple of years ago I got interested in Alpine Linux. Mainly for its use of musl libc, it's small size, quick speed and the many rootfs offerings it provides. I really favor the rootfs distribution medium. It works well with Slackbuilder's installer.

So for fun I created an installation setting I called Alphie. Where it downloaded a rootfs ran some chroot commands that installed Alpine with its apk package manager then installed Sbtools to finish the job. It was a fun experiment and it worked. But Alpine's rapid distribution cycle proved to be to volitile for a long term solution. Their updates began breaking things so Alphie fell by the wayside and my musl libc Slackbuilder dreams were put on hold.

Along came Slackmlinux which I discovered in the slackware.uk repositories. I quickly put together an installation setting for it. Only to find out it is not a complete distribution. It failed to install. But I was bitten by the musl bug again so I revisited Alphie.

Package Conversion

There are several issues when trying to twist a foreign distro into a Slackbuilder system. The main deal breaker is the unreliablity of the up stream repositories. And in the case of Alpine, it's authoritarian package manager.

To get around both of these problems. I built a package conversion engine that converts Alpine packages to Slackware ones. At the same time it reduces the many Alpine sub-packages ( qemu has 80! ) down to three main categories pkgname, pkgname-dev, pkgname-doc ( the implementaion of the fourth pkgname-nls came later. All future repos will include it ). This quiets things down a bit. We gain stability and regain our freedom at the same time. We can now manage our system the Slackware way.

Its all in a name

This new approach supercedes and obsoletes the Alphie method and so required a new name. At first I dubbed it SlackM and that's what it is at the time of this writing. But I soon discovered that SlackM was already in use by SlackRadio as the name of their media player or something. That's no good! So the new name will be Muscles and the arm variants ArmMuscles.

Summing up

This article has gotten a bit long at this point so I'll close it out now. There has been alot of work done on Slackbuilder over the last few years and I don't get around to documenting it's progress often enough so I appologize for it's windiness.

I hope you find Slackbuilder as enjoyable as I do!

Slackerson

2019-10-30 current_development
2018-11-17 new_additions
2017-05-11 grub_bug_squashed
2017-05-02 pkginfo_to_pkg-tools
2017-03-05 tutorial_updated
2016-11-12 source_changes

Comment in the Forum.