getpkg
getpkg is a non interactive command that takes one or more package names or pkgs.lst files as an argument. It will verify if the package(s) need to be installed, upgraded or is up-to-date. If the package is to be installed/upgraded it will download the package to a cache directory and tell slackware's installpkg to install it, then remove the downloaded package from the cache. After processing all command line arguments it will silently update the package database.
Note: getpkg does not resolve dependencies.
Examples of use
--> getpkg kodi root / %%%%%%%%%%% kodi install at the location /var/pkgcache/Slackbuilder-x86_64-14.2 24550K .......... .......... ........ 100% 2.58M=12s Verifying package kodi-16.1-x86_64-4_Skn.txz. Installing package kodi-16.1-x86_64-4_Skn.txz: PACKAGE DESCRIPTION: # kodi (Multi-platform Media Center) # # XBMC is an award winning media center application for Linux, Mac OS X, # Windows and XBox. The ultimate hub for all your media, XBMC is easy to # use, looks slick, and has a large helpful community. XBMC supports # viewing and playing a vast library of audio, video and image formats. # XBMC has a sophisticated library management system that allows you to # organize all your media to give you quick and immediate access. # # Homepage: http://kodi.tv # Executing install script for kodi-16.1-x86_64-4_Skn.txz. Package kodi-16.1-x86_64-4_Skn.txz installed.
If already installed then .....
--> getpkg kodi root / %%%%%%%%%%% kodi is up-to-date
If not an available pkg
--> getpkg nada root /
The root / line indicates the mountpoint mntpt variable which in this case is the default root directory /. mntpt is used instead of something like ROOT so as not to muck up other environment variables from other commands. Namely slackware's pkgtools. ROOT_DEV is used in others. I have not yet discovered mntpt used anywhere else.
An example of a different mntpt....Note: The mntpt directory must already exist. It is not created automatically.
--> mntpt=/mnt/installdir getpkg nada root /mnt/installdir Error: incomplete SQL: INSERT INTO PKGS (ID, NAME, FILE, LOCATION, SIZE_COMP, SIZE_UNCOMP) VALUES
The error is because there is no slackware installation at /mnt/installdir and 'nada' is not a real pkg. getpkg runs a database update after it attempts to install a package(s). There are actually two databases created. One for availabe packages and one for installed packages. Since there is no installed packages yet there is a database error.
Installing a real package...
--> mntpt=/mnt/installdir getpkg kodi root /mnt/installdir %%%%%%%%%%% kodi install at the location /var/pkgcache/Slackbuilder-x86_64-14.2 24550K .......... .......... ........ 100% 2.20M=11s Verifying package kodi-16.1-x86_64-4_Skn.txz. Installing package kodi-16.1-x86_64-4_Skn.txz: PACKAGE DESCRIPTION: # kodi (Multi-platform Media Center) # # XBMC is an award winning media center application for Linux, Mac OS X, # Windows and XBox. The ultimate hub for all your media, XBMC is easy to # use, looks slick, and has a large helpful community. XBMC supports # viewing and playing a vast library of audio, video and image formats. # XBMC has a sophisticated library management system that allows you to # organize all your media to give you quick and immediate access. # # Homepage: http://kodi.tv # Executing install script for kodi-16.1-x86_64-4_Skn.txz. Package kodi-16.1-x86_64-4_Skn.txz installed.
This works because 'kodi' is a real pkg.
Then if you try ....
--> mntpt=/mnt/installdir getpkg nada root /mnt/installdir
There is no error because now the installed package database is not empty.
Note: The above example is just for demonstration purposes and is not recommended in practice since the CACHE environment variable has not been defined for the mntpt. At this point the host database has been altered to coincide with the mntpt. Just run pkg-info -u to correct the problem. In short the installdir package cache needs to be isolated from the host's. This is achieved by changing the CACHE variable. The cache directory will be created automatically if it does not exist.
Intended Use
getpkg can of course be used directly but is meant as a building block for other more sophisticated commands. It is used in getpkgs which is inturn used in the installer 'slkn-setup', slkn-usbboot's 'mkimg' and slackpacks's 'sbuilder' commands among others.
The example above shows how to change the install directory by changing the mntpt environment variable. There are other variables that change the default behavior too. Like the ARCH, SVRSN ( slackware version ), SRC and CACHE.
Note: the default CACHE directory is /var/pkgcache. This needs to be changed for a different mntpt location so as not to corrupt the host's database. Specifically for installed packages.
The following is an example of how to script your own command to install a package and it's recursive dependencies into a mntpt. The mntpt can be any mounted device or directory...
Building a custom script
Assuming you already have a mntpt directory created ( we'll use the example above ) and a package of interest to install ( continuing with the example we'll use kodi ) the next step is to get a list of dependencies if any exists. The ldeps command is suited for this purpose...
--> ldeps -s kodi libdvdcss libmicrohttpd tinyxml yajl crossguid dcadec libmpeg2 libass jdk
the -s option returns a simple flat list of recursive dependencies. Lets go ahead and store that in a file for use later. We need to add kodi to the list as well.
(ldeps -s kodi ; echo kodi) > kodi.lst
The name of the file is arbitrary.
You can install this list from the command line without a script like this...first update the database...
--> SRC=HTTP:localhost/Slackbuilder-x86_64-14.2 mntpt=/mnt/installdir CACHE=/tmp/my_cache pkg-info -u
SRC ==> localhost/Slackbuilder-x86_64-14.2/PACKAGES.TXT
50K .......... 100% 315M=0s
Then...
--> SRC=HTTP:localhost/Slackbuilder-x86_64-14.2 mntpt=/mnt/installdir CACHE=/tmp/my_cache getpkg kodi.lst root /mnt/installdir %%%%%%%%%%% libdvdcss install at the location /tmp/my_cache/Slackbuilder-x86_64-14.2 100K .......... .......... ..... 100% 377M=0s Verifying package libdvdcss-1.4.0-x86_64-1_Skn.txz. Installing package libdvdcss-1.4.0-x86_64-1_Skn.txz: PACKAGE DESCRIPTION: # libdvdcss (DVD decryption library) # # libdvdcss is a cross-platform library for transparent DVD device # access with on-the-fly CSS decryption. It is used by libdvdread # and most DVD players such as VLC because of its portability and # because, unlike similar libraries, it does not require your DVD # drive to be region locked. # # Home Page: http://developers.videolan.org/libdvdcss/ # Executing install script for libdvdcss-1.4.0-x86_64-1_Skn.txz. Package libdvdcss-1.4.0-x86_64-1_Skn.txz installed. %%%%%%%%%%% libmicrohttpd install at the location /tmp/my_cache/Slackbuilder-x86_64-14.2 150K .......... .......... .......... ....... 100% 292M=0.001s Verifying package libmicrohttpd-0.9.48-x86_64-1_Skn.txz. Installing package libmicrohttpd-0.9.48-x86_64-1_Skn.txz: PACKAGE DESCRIPTION: # libmicrohttpd (HTTP server C library) # # GNU libmicrohttpd is a small C library that is supposed to make it # easy to run an HTTP server as part of another application. # # GNU libmicrohttpd is free software and part of the GNU project. # Executing install script for libmicrohttpd-0.9.48-x86_64-1_Skn.txz. Package libmicrohttpd-0.9.48-x86_64-1_Skn.txz installed. %%%%%%%%%%% tinyxml install . . . . . %%%%%%%%%%% kodi install at the location /tmp/my_cache/Slackbuilder-x86_64-14.2 24550K .......... .......... ........ 100% 245M=0.3s Verifying package kodi-16.1-x86_64-4_Skn.txz. Installing package kodi-16.1-x86_64-4_Skn.txz: PACKAGE DESCRIPTION: # kodi (Multi-platform Media Center) # # XBMC is an award winning media center application for Linux, Mac OS X, # Windows and XBox. The ultimate hub for all your media, XBMC is easy to # use, looks slick, and has a large helpful community. XBMC supports # viewing and playing a vast library of audio, video and image formats. # XBMC has a sophisticated library management system that allows you to # organize all your media to give you quick and immediate access. # # Homepage: http://kodi.tv # Executing install script for kodi-16.1-x86_64-4_Skn.txz. Package kodi-16.1-x86_64-4_Skn.txz installed.
You don't have to define the SRC if your ok with the settings in /etc/pkg-tools/RC. But you can for whatever reason you may have need of.
Putting it in a script
#!/bin/bash
#
# install_pkgs....
# arguments can be a file or standard input or a combination of both
# ----------------------------------------------------------------
eval $(svrsn)
# make sure there is a space at the end of each SRC entry
SRC+="HTTP:mirrors.kernel.org/slackware/slackware${SUFFIX}-${SVRSN} "
SRC+="HTTP:slackonly.com/pub/packages/${SVRSN}-${RCH} "
SRC+="HTTP:localhost/Slackbuilder-${ARCH}-${SVRSN} "
export CACHE="/tmp/my_cache"
export mntpt="/mnt/installdir"
export SRC="${SRC}"
while :; do
case $1 in
-u) pkg-u # shorthand for pkg-info -u
shift
;;
esac
break
done
# input should be package name(s) or package list file(s)
getpkg $@
Call it like so...with a file and/or a package as arguments...we need to update the database first..but this can be done all at once with the -u option...
--> ./install_pkgs -u kodi.lst chkrootkit
Downloading data files...
HTTP:mirrors.kernel.org/slackware/slackware64-14.2
700K .......... ....... 100% 1.69M=0.5s
0K .......... .......... .......... .......... ...... 100% 426K=0.1s
0K 100% 24.9M=0s
HTTP:slackonly.com/pub/packages/14.2-x86_64
3100K .......... .......... .......... ......... 100% 881K=3.2s
HTTP:localhost/Slackbuilder-x86_64-14.2
850K .......... .......... ......... 100% 167M=0.006s
Building sqlite database...
root /mnt/installdir
%%%%%%%%%%% libdvdcss is up-to-date
%%%%%%%%%%% libmicrohttpd up-to-date
%%%%%%%%%%% yajl is up-to-date
%%%%%%%%%%% crossguid is up-to-date
%%%%%%%%%%% dcadec is up-to-date
%%%%%%%%%%% libmpeg2 is up-to-date
%%%%%%%%%%% libass is up-to-date
%%%%%%%%%%% jdk is up-to-date
%%%%%%%%%%% kodi is up-to-date
%%%%%%%%%%% chkrootkit install
at the location /tmp/my_cache/14.2-x86_64
300K .. 100% 4239G=0.3s
Verifying package chkrootkit-0.50-x86_64-1_slonly.txz.
Installing package chkrootkit-0.50-x86_64-1_slonly.txz:
PACKAGE DESCRIPTION:
# chkrootkit (Check Rootkit)
#
# chkrootkit is a common unix-based program intended to help system
# administrators check their system for known rootkits. It is a shell
# script using common UNIX/Linux tools like the strings and grep
# commands to search core system programs for signatures and for
# comparing a traversal of the /proc filesystem with the output of
# the ps (process status) command to look for discrepancies.
#
# Homepage: http://www.chkrootkit.org
#
Package chkrootkit-0.50-x86_64-1_slonly.txz installed.
Notice that since I already installed the packages in kodi.lst it shows everything is up-to-date, But the extra package on the end gets installed.
svrsn is a command that is part of the slackerson-utils package. It provides ARCH SVRSN SUFFIX and RCH environment variables and is used throughout the SlackBuilder tools. Here is what it provides...
--> svrsn ARCH=x86_64 SVRSN=14.2 SUFFIX=64 RCH=x86_64
RCH provides the variable for the slackonly SRC line. This variable is needed for sources that use x86 to represent a 32bit ARCH. You can define the ARCH variable on the command line. Notice how everything changes for a 32bit architecture....
--> ARCH=i686 svrsn ARCH=i686 SVRSN=14.2 SUFFIX= RCH=x86
As you have probably noticed I have been using a x86_64 ARCH as my host system for all my examples. But you can install a 32bit system from a 64bit host by changing the ARCH variable. Of course you can change any of the other variables if need be.
Summary
With this script we now have a no nonsense command that can install multiple package list files and/or single packages as arguments. By creating another script that calls it with a predefined set of package list files you can install a complete OS with one command. Of course you would still have to configure the installation and such. This is how 'getpkgs' sort of works but it uses packages lists attached to a SRC defined in /etc/slkn-setup/SRC.
A new command is born
The 'install_pkgs' command above has limited use and is not as robust as using 'getpkgs' for an installation method. It is just a teaser to demonstrate the concept on how to script with the package manager tool. But with a little modification it can be turned into an actual useful command for the system.
pkg-get is an interactive command that will take a single package name as an argument. It will install the package and its dependencies after verifying each one to determine its installation status ( to install, upgrade, or pkg is up-to-date ). Then asks if you want to continue.
It would be nice to have an alternative to 'pkg-get' that will..
- take multiple arguments ( but no files )
- and for each one install it along with its recursive dependencies
- without having to prepare a pkg.lst file first
- and will do so no questions asked
We can eliminate the extra task of preparing a file by incorporating the 'ldeps' command in the script. I also want to remove the ability to supply a file as an argument to avoid any overhead of trying to process too many dependencies. In case a package is listed in the file that has already been installed as a dependency of a previously listed package. Package lists can get pretty long so no need for any of that. In that same spirit we should also eliminate the same effect from CLI provided arguments that may share dependencies as well.
Normally getpkg updates the database after completing its task. This happens after each call to it. Not after each package has been installed. Consequently if the same package is supplied more than once it will get installed that many times. We need to prevent that by sanitizing the packs list.
Here is the new command....
#!/bin/bash
#
# pkg-getd....
# takes multiple pkgs as arguments and installes each pkg with
# its recursive dependencies
# ----------------------------------------------------------------
while :; do
case $1 in
-u) pkg-u
shift
;;
esac
break
done
echo "Checking for dependencies..."
for input in $@; do
[[ -f $input ]] && continue
Packs+="$(ldeps -s $input) "
Packs+="$input "
done
# if two or more pkgs share dependecies we don't want to install the same pkg multiple
# times so...remove multiple occurances and rebuild list...
echo "sanitizing dependency list..."
Packs=$(echo $Packs | tr ' ' '\n' | sed -e '/^$/d' -e 's/^/%/;s/$/%/')
for pkg in ${Packs}; do
Packs=$(echo $Packs | sed -e 's/'"$pkg"'//g')
Packs+="$pkg "
done
packs=$(echo $Packs | sed -e 's/%%/ /g;s/%//g')
for pks in $packs; do
echo $pks
done
[[ $packs ]] && getpkg $packs
It doesn't really need the while loop to update. But it's nice to have if you want to change the environment variables mntpt, SRC or CACHE and an update is necessary.
Here it is in action with the call to 'getpkg' commented out....
--> time ./pkg-getd kodi Checking for dependencies... libdvdcss libmicrohttpd tinyxml yajl crossguid dcadec libmpeg2 libass jdk kodi real 0m0.324s user 0m0.226s sys 0m0.099s
Versus 'pkg-get'
--> time pkg-get kodi Pkg-get... -------------------------------------------------------------------------------- Dependency-Check => kodi Finding recursive dependencies for....kodi Done! There are 10 dependencies to verify libdvdcss is up-to-date libmicrohttpd is up-to-date tinyxml is up-to-date yajl is up-to-date crossguid is up-to-date dcadec is up-to-date libmpeg2 is up-to-date libass is up-to-date jdk is up-to-date kodi is up-to-date There are 0 pkgs to be installed/upgraded.... -------------------------------------------------------------------------------- real 0m0.763s user 0m0.449s sys 0m0.227s
Notice it is a little over twice as fast collecting the dependency list. But 'pkg-get' pre preps the list and will only feed 'getpkg' packages that need to be installed or upgraded. The new 'pkg-getd' will send 'getpkg' everything and let it do the verifying which will actually take longer when complete.
Here it is with 'getpkg' line uncommented to demonstrate the time difference.
--> time ./pkg-getd kodi Checking for dependencies... libdvdcss libmicrohttpd tinyxml yajl crossguid dcadec libmpeg2 libass jdk kodi root / %%%%%%%%%%% libdvdcss is up-to-date %%%%%%%%%%% libmicrohttpd is up-to-date %%%%%%%%%%% tinyxml is up-to-date %%%%%%%%%%% yajl is up-to-date %%%%%%%%%%% crossguid is up-to-date %%%%%%%%%%% dcadec is up-to-date %%%%%%%%%%% libmpeg2 is up-to-date %%%%%%%%%%% libass is up-to-date %%%%%%%%%%% jdk is up-to-date %%%%%%%%%%% kodi is up-to-date real 0m4.441s user 0m2.737s sys 0m0.741s
Since there are no packages to install 'pkg-get' completes its job faster. If there is a long list of packages to install then 'pkg-getd' at this point can be very time consuming.
Improving performance
I'll go ahead and add a loop to verify the packages to ease the burden on 'getpkg'. There is a command verifypkg shipped with the pkg-tools package just for this purpose. And I might as well add in a usage bit too while I'm at it.
#!/bin/bash
#
# pkg-getd....
# takes multiple pkgs as arguments and installes each pkg with
# its recursive dependencies
# ----------------------------------------------------------------
USAGE="
$(basename $0) [Option] [pkg(s)]
(Requires at least one pkg as an argument)
[Options]
-u (update the pkg database)
Install pkg(s) and its recursive dependencies
";
[[ "$#" -eq 0 ]] && {
echo "$USAGE";
exit
}
# update database
while :; do
case $1 in
-u) pkg-u
shift
;;
esac
break
done
# get deps
echo ""
echo "Checking for dependencies..."
for input in $@; do
[[ -f $input ]] && continue
Packs+="$(ldeps -s $input) "
Packs+="$input "
done
# if two or more pkgs share dependecies we don't want to install the same pkg multiple
# times so...remove multiple occurances and rebuild list...
echo "sanitizing dependency list..."
Packs=$(echo $Packs | tr ' ' '\n' | sed -e '/^$/d' -e 's/^/%/;s/$/%/')
for pkg in ${Packs}; do
Packs=$(echo $Packs | sed -e 's/'"$pkg"'//g')
Packs+="$pkg "
done
Packs=$(echo $Packs | sed -e 's/%%/ /g;s/%//g')
# verify packages
echo "verifying packages..."
hr="-----------------------------------------------------------------"
echo $hr
for pkg in ${Packs}; do
method=`verifypkg ${pkg}`
case ${method} in
upgrade)
packs+="$pkg ";
echo "$pkg will be upgraded";
;;
install)
packs+="$pkg ";
echo "$pkg will be installed";
;;
uptodate)
echo "$pkg is up-to-date";
;;
nosrc)
echo "$pkg has nosrc"
;;
nopkg)
echo "No such package $pkg";
;;
esac
done
echo $hr
[[ $packs ]] && getpkg $packs
'pkg-get' still performs better due to a more sophisticated dependency checking routine. But we have gotten quite a bit more umph over 'getpkg' alone.
--> time ./pkg-getd kodi Checking for dependencies... sanitizing dependency list... verifying packages... ----------------------------------------------------------------- libdvdcss is up-to-date libmicrohttpd is up-to-date tinyxml is up-to-date yajl is up-to-date crossguid is up-to-date dcadec is up-to-date libmpeg2 is up-to-date libass is up-to-date jdk is up-to-date kodi is up-to-date ----------------------------------------------------------------- real 0m1.324s user 0m1.130s sys 0m0.643s
Wrapping up
As you can see it is a fairly trivial matter to slap a command together to meet our whim. By incorporating 4 pkg-tools commands ( pkg-info, ldeps, verifypkg, getpkg ) and a little logical glue we have managed to create a rather sophisticated command that rivals the most popular package managers in function.
Performance? Uhh. Not so sure. With 10 or less packages supplied as arguments, yes. But if you throw tens of packages at it, each with moderately long dependency lists, the verification process can be quite time consuming and more than a little annoying. Escpecialy when those packages start sharing dependencies. This is why I limited 'pkg-get' to 1 argument in the first place.
A different verification paradigm would need to be implemented to deal with very large package lists which could reach into the 100's or even 1000's. In that event bulk file processing is the best choice.
Suggested use
pkg-getd is best used with 10 or less packages supplied as arguments.
Note: This command gets right to work no questions asked. Use it responsibly. It is not a search tool !