The automatic package manager is an approach to reduce manual packaging work. The new implementation is intended to become a flexible and valuable packaging tool.
New automatic packager.
New build system in far future.
Universal tool, both for advanced users and beginners (from verbose+confirm everything to automatic guesses).
See also my OpenOffice Presentation for SuSE Engineering Workshop Nuernberg 2003
Create a packutils design framework.
Write packutils modules (usable separately).
Create a design for connection between autopack and build system.
Write autopack system.
Extend packutils modules.
Write new build systems.
Open development after writing functional core.
Write the whole system to be as much as possible independent on packaging and build systems.
Autopack should work in phases. Phase is an array of actions. Actions are suggested after package inspection. Any phase action can issue another action to any phase or provide a phase change. Any part of array can be filled or unfilled by user via package meta-description.
Phase arrays are empty by default. All autopack intelligence goes from guesses and tests.
Analyze home page or download URL and try to find all download-able files.
Implementation needed: spider
Compare version and try to find latest one. An extra part will take advance of downloading of incremental or cross version patches or xdeltas.
Implementation needed: cmpversions (compare versions and find latest), findunpack (create unpack sequence), deltaserver and deltaclient (be able to download via delta server)
Find mirror based on configured mirrors or downloaded mirror list
Download sources.
Verify src (MD5, PGP signatures, digests).
Unpack complete sources.
Should contain misc code inspectors.
Implementation needed: getdesc (be able to analyze spec, ebuild, debian and lsm script to find needed information) and others.
Patch sources.
Configure sources.
Ugly hack on configured packages.
Well-known phases.
Phase for analyzing logs.
By default, autopack actions should be empty. There will be plug-in programs, which will perform one of possible actions:
Add action to build process (configure, build, install, strip)
Perform some test (FHS conformance, split pack-devel)
Ask user for answer and put result to registry (what to do with this file)
Autopack can have some questions and its urgency. For example:
"File AUTHORS is changed since last version." Class license urgency high
"File main.c contains cast warning." Class portability urgency low
"Make failed, error not resolved." Class build urgency critical
Registry is a set of answers to questions. It will store extra info for future rebuilds and versions.
Answer can be classified as forever, for this (major, minor) version etc.
Example:
file: glib-config, question: is -devel file, guess: no, user answer: yes forever
file: mplayer, question: dist menu location, guess: misc, answer: multimedia/video
Application registry (part of registry) should track MIME types, GUI type, extensions and menu location for all binaries (incl. negative answers).
In difference of standard systems package systems, autopack package description should contain only such info, which cannot be deduced from sources:
URL (home page)
Download URL (download page or download file URL)
Download policy (default is spider, sourceforge-style etc.)
Release policy (what versions are stable, unstable etc.)
Differences for package descriptions, authors etc.
Needed patches.
It is store place for special information needed for clean update. For example MD5 of AUTHORS, LICENSE, info about successes or fails for old build etc.
Created by autopack after interaction with user.
There are big advantage of shell interface - simply writing more tests and simple utility connection.
Patches should be classified to simplify version upgrade.
Fix or fix from CVS.
Local fix or enhancement (paths etc).
Automatic systematic local fix (CFLAGS etc.).
in live system
in build chroot
in build root (install root)
in build directory (make root, for automake based projects)
Install and access wrappers
Purpose:
Make file lists
Make dependency lists
Binary is renamed and its arguments are traced.
Plus: Very simple to implement. Can trace any binary calls.
Minus: Low effectivity
Pre-loading special shared library (as ltrace does).
Plus: Simple to implement. Can trace library issues.
Minus: Cannot trace SUID, login shells etc.
Uses kernel tracing features (as strace does).
Plus: Simple to implement.
Minus: Cannot trace SUID.
Kernel logging hacks
Plus: 100% work
Minus: Hard to implement.
Uses mtime utime of filesystem.
Plus: Simple to implement.
Minus: Cannot trace if utime is used. System cannot be live.
Changes UID.
Creates complete chroot environment for build.
Install root changes root of install targets. Works for most modern make tools.
Compiles package in different directory than source resides. Works for most modern make tools.
Changes UID and adds wrapper, so build process still thinks, that it runs as root. Traces and simulates actions, which cannot be done by ordinary users.
Wrapper checks write accesses outside from permitted areas.
Prohibiter makes certain files invisible for build process.
Dynamically check all unsuccesfull file accesses and classifies it.
Debugging and install tools.
Virtual file system implemented in kernel, wrapper or library. All write accesses are realized in different layer than read accesses. Upper layer hides lower layer file.
Wrapper implementation, inspired by upper mentioned projects.
Special checks for:
strange /dev accesses
fake /proc accesses (neutral cpuinfo)
accesses to instances of package itself (illegal self-dependencies)
package maintainer can give permission to permit special access
Implement a set of build guesses
create a set of guesses for automatization of build process
Implement a set of tests of package correctness
make these tests optional
Package build (needed for build) tree creation
Package reverse (target to rebuild) tree creation
Scalability of package build tree
Tool to simplify lists of files and dependencies.
Needed for build "treefication".
Why: Better orientation in dependencies.
Optional and required dependencies
Open problem: Distinguish between required for build and optional for build.
Open problem: How to distinguish "implicit dependency" from "explicit dependency", if this dependency is "required" for sub-package.
Possible solution: Call analysis - if call is done from inside linker.
Possible solutions:
Simple for pkgconfig
Hack automake
Try to recompile without it
problem: required for whole tree
possible solution: remove all "devel" files while building.
There are more ways. All of them will require a lot of coding to be done, some of them also extra stuff in builds.
I can imagine three possible ways:
Full targeting - any recompilation of any library causes recompilation of all dependent packages. It's most safe and most expensive way, which can simply lead to loops.
Upgrade targeting - any upgrade (either package or build) causes recompilation of all dependent packages. It's relatively safe, but still expensive way. In extreme cases it can lead to problems (rarely found).
Rebuild triggers - package can trigger self for recompilation. For example user of static library can trigger itself for recompilation after static library update.
Symbol targeting - upgrade which causes library soname change (or file list change) will target all dependent packages for recompilation. Symbol targeting needs to have also support from build - chance to explicitly declare binary incompatibility for both dependency exporter and importer (perl declares upgrade incompatible for its modules {done by exporter}, galeon targets itself to recompile after mozilla update {done by importer, if exporter don't do it}). Most expensive for target evaluation (and probably coding), but as small as possible for recompilation. In certain cases it can cause troubles, which needs to be solved by other way (for example statically linked libraries).
Deactivation - package targets some library to recompile. If binary image of this library is not changed.
Manual classification - change submitter can guarantee, that this change does not progress any change to upper layer. Manual classification can block recompilation targeting.
Manual activity - manual request for rebuild.
Package meta-description standard.
Integration with autostuff (to generate tags as "make submit", "make release").
My own old project OSBuilder: Build own GNU-Linux
Port systems for meta distributions with optional dependencies and excellent dependency system.