Automatic package manager project

The automatic package manager is an approach to reduce manual packaging work. The new implementation is intended to become a flexible and valuable packaging tool.

Basic ideas and background

Targets of the project

See also my OpenOffice Presentation for SuSE Engineering Workshop Nuernberg 2003

Steps to this goal

Development policy

Ideas for autopack

Phases

Autopack should work in phases. Phase is an array of actions. Actions are suggested after package inspection. Any phase action can issue another action to any phase or provide a phase change. Any part of array can be filled or unfilled by user via package meta-description.

Phase arrays are empty by default. All autopack intelligence goes from guesses and tests.

websearch

Analyze home page or download URL and try to find all download-able files.

Implementation needed: spider

cmpversions

Compare version and try to find latest one. An extra part will take advance of downloading of incremental or cross version patches or xdeltas.

Implementation needed: cmpversions (compare versions and find latest), findunpack (create unpack sequence), deltaserver and deltaclient (be able to download via delta server)

findmirror

Find mirror based on configured mirrors or downloaded mirror list

download

Download sources.

verifysrc

Verify src (MD5, PGP signatures, digests).

unpack

Unpack complete sources.

inspect

Should contain misc code inspectors.

Implementation needed: getdesc (be able to analyze spec, ebuild, debian and lsm script to find needed information) and others.

patch

Patch sources.

configure

Configure sources.

postconfhack

Ugly hack on configured packages.

build, preinst, install, postinst, preun, postun, clean, trigger

Well-known phases.

analyze

Phase for analyzing logs.

Actions

By default, autopack actions should be empty. There will be plug-in programs, which will perform one of possible actions:

Question and its classification

Autopack can have some questions and its urgency. For example:

"File AUTHORS is changed since last version." Class license urgency high

"File main.c contains cast warning." Class portability urgency low

"Make failed, error not resolved." Class build urgency critical

Registry

Registry is a set of answers to questions. It will store extra info for future rebuilds and versions.

Answer can be classified as forever, for this (major, minor) version etc.

Example:

file: glib-config, question: is -devel file, guess: no, user answer: yes forever

file: mplayer, question: dist menu location, guess: misc, answer: multimedia/video

Application registry

Application registry (part of registry) should track MIME types, GUI type, extensions and menu location for all binaries (incl. negative answers).

Package meta-description

In difference of standard systems package systems, autopack package description should contain only such info, which cannot be deduced from sources:

Automatic package description

It is store place for special information needed for clean update. For example MD5 of AUTHORS, LICENSE, info about successes or fails for old build etc.

Created by autopack after interaction with user.

What language

There are big advantage of shell interface - simply writing more tests and simple utility connection.

Patches

Patches should be classified to simplify version upgrade.

Build types

Required subprojects

Install and access wrappers

Purpose:

Possible implementations

Binary wrapper

Binary is renamed and its arguments are traced.

Plus: Very simple to implement. Can trace any binary calls.

Minus: Low effectivity

DL wrapper

Pre-loading special shared library (as ltrace does).

Plus: Simple to implement. Can trace library issues.

Minus: Cannot trace SUID, login shells etc.

Kernel tracing

Uses kernel tracing features (as strace does).

Plus: Simple to implement.

Minus: Cannot trace SUID.

Kernel level

Kernel logging hacks

Plus: 100% work

Minus: Hard to implement.

u/mtime

Uses mtime utime of filesystem.

Plus: Simple to implement.

Minus: Cannot trace if utime is used. System cannot be live.

Wrapper techniques

Normal user

Changes UID.

Build chroot (SuSE autobuild)

Creates complete chroot environment for build.

Install root (automake/autoconf)

Install root changes root of install targets. Works for most modern make tools.

Build dir (automake/autoconf)

Compiles package in different directory than source resides. Works for most modern make tools.

Fake root (Debian)

Changes UID and adds wrapper, so build process still thinks, that it runs as root. Traces and simulates actions, which cannot be done by ordinary users.

Sandbox (Gentoo)

Wrapper checks write accesses outside from permitted areas.

Prohibiter (not yet implemented)

Prohibiter makes certain files invisible for build process.

Dynamic checker/installer (not yet implemented)

Dynamically check all unsuccesfull file accesses and classifies it.

ltrace, strace, installwatch etc.

Debugging and install tools.

Multi layer file system

Virtual file system implemented in kernel, wrapper or library. All write accesses are realized in different layer than read accesses. Upper layer hides lower layer file.

Plans for autopack

Wrapper implementation, inspired by upper mentioned projects.

Special checks for:

Implement a set of build guesses

Implement a set of tests of package correctness

Package dependency builder

Package build (needed for build) tree creation

Package reverse (target to rebuild) tree creation

Scalability of package build tree

Abbreviator

Tool to simplify lists of files and dependencies.

Needed for build "treefication".

Why: Better orientation in dependencies.

Topics for new build system

Optional and required dependencies

Open problem: Distinguish between required for build and optional for build.

Open problem: How to distinguish "implicit dependency" from "explicit dependency", if this dependency is "required" for sub-package.

Possible solution: Call analysis - if call is done from inside linker.

Possible solutions:

problem: required for whole tree

possible solution: remove all "devel" files while building.

Rebuild classification

There are more ways. All of them will require a lot of coding to be done, some of them also extra stuff in builds.

I can imagine three possible ways:

Long term goals

Resources

My own old project OSBuilder: Build own GNU-Linux

BSD port system

Gentoo portage

Port systems for meta distributions with optional dependencies and excellent dependency system.

uPM - The micro Package Manager

GAR

SourceMage Linux