Project

General

Profile

Task #7681

bung: create .deb

Added by Charles Atkinson almost 3 years ago. Updated 8 months ago.

Status:
Closed
Priority:
Normal
Start date:
15/02/2020
Due date:
% Done:

100%

Close

Task_7681.png View (92.6 KB) Charles Atkinson, 03/07/2019 07:30

5646

Subtasks

Task #9720: bung: productionise binary .deb creationClosedCharles Atkinson

History

#1 Updated by Charles Atkinson over 2 years ago

  • Status changed from New to In Progress

Understanding .deb architecture by similar example

Knowing what should be in the .deb will:
  • Allow us to check the .deb
  • Provide useful background
  • Help choose between toolsets

For full Debian compliance a source .deb is also required (for the .deb build system; bung has no compiled components)

The .deb will install in the same way as the existing bung 2.0 (currently beta1) with all static files under /opt/bung-<version>. So it will differ from a .deb from the Debian maintainers.

Which .deb to explore? Best:
  • One from the Debian maintainers because it should be Debian policy compliant.
  • A small one for ease of exploration
  • Ideally one without compiled components
  • A familiar one

Binary .deb

Chose logcheck and extracted it:
c@CW10:~$ mkdir /tmp/tmp && cd /tmp/tmp
c@CW10:/tmp/tmp$ apt-get download logcheck
...
c@CW10:/tmp/tmp$ ll
total 76K
-rw-r--r-- 1 c users 74K Oct 25  2014 logcheck_1.3.17_all.deb
root@CW10:/tmp/tmp# ar xv logcheck_1.3.17_all.deb 
x - debian-binary
x - control.tar.gz
x - data.tar.xz
c@CW10:/tmp/tmp$ mkdir control && tar -xvf control.tar.gz -C control
./
./md5sums
./control
./postrm
./postinst
./conffiles
root@CW10:/tmp/tmp# mkdir data && tar -xvf control.tar.gz -C data
./
./var/
./var/lib/
./var/lib/logcheck/
./usr/
./usr/share/
./usr/share/lintian/
./usr/share/lintian/overrides/
./usr/share/lintian/overrides/logcheck
...
./usr/sbin/
./usr/sbin/logcheck
./etc/
./etc/cron.d/
./etc/cron.d/logcheck
./etc/logcheck/
./etc/logcheck/ignore.d.server/
./etc/logcheck/ignore.d.paranoid/
./etc/logcheck/cracking.ignore.d/
./etc/logcheck/logcheck.logfiles
./etc/logcheck/ignore.d.workstation/
./etc/logcheck/cracking.d/
./etc/logcheck/violations.d/
./etc/logcheck/logcheck.conf
./etc/logcheck/violations.ignore.d/
Explored:
  • control/conffiles: list of files (not dirs) under /etc installed by package
  • control/control: same as shown by aptitude show <package>
  • control/md5sums: md5sum for each file in the package
  • control/postinst: bash script including /etc/logcheck tree ownership and permissions setting
  • control/postrm: bash script. When called with purge, removes user logcheck's home directory, /var/lib/logcheck
  • control/postrm: bash script. When called with purge, removes user logcheck's home
  • data: tree to be copied to / with required ownerships and perms. Includes empty directories

Copied so not lost by reboot:

root@CW10:~/tmp/logheck-package# rsync -a /tmp/tmp/ 

Source .deb

Doc: https://wiki.debian.org/Packaging/SourcePackage

root@CW10:/etc/apt# diff sources.list{.bu4,}
33a34
> deb-src http://ftp.de.debian.org/debian/ jessie main contrib non-free
root@CW10:/etc/apt# apt update
root@CW10:~/tmp/logheck-package-source# apt-get source logcheck
Explored. Complex. Possibly outdated. Included tests. Decided to defer source .deb which will anyway need knowledge gained during binary .deb manual build > automated build.

#2 Updated by Charles Atkinson over 2 years ago

Planning the bung binary .deb

Docs

Not for now. Specifics about source package work:

debian-binary

The content of this file is "2.0\n". This states the version of the deb file format. For 2.0 all other lines get ignored.

control.tar.gz

control

Doc: https://www.debian.org/doc/manuals/developers-reference/ch06.en.html#bpp-debian-control

conffiles

Not needed because bung does not install any conffiles. https://www.debian.org/doc/manuals/maint-guide/index.en.html confirms it is optional

md5sums

Generate when building the .deb

postinst

Hand craft

postrm

Hand craft

data.tar.xz

Generate when building the .deb. Same files as when creating the tarball, same ownerships and perms as when installing

Testing

lintian <whatever>.deb

#3 Updated by Charles Atkinson over 2 years ago

Manual binary .deb build

git directories

master c@CW8:~/d/Projects/bung/git$ mkdir -p apt/binary-deb/control

debian-binary

master c@CW8:~/d/Projects/bung/git$ echo 2.0 > apt/binary-deb/debian-binary

control

master c@CW8:~/.../apt/binary-deb/control$ cat control 
Package: bung
Version: 2.0.0
Architecture: all
Maintainer: Charles Atkinson <bung@charlesmatkinson.org>
Installed-Size: 1756
Depends: default-mta | mail-transport-agent, cron | cron-daemon, rsyslog | system-log-daemon
Recommends: rsync, mysql-server | postgresql, openldap, ssh-client
Suggests: lsof
Section: admin
Priority: optional
Homepage:
Description: backup system
 backup scripts next generation (bung) is a suite of bash scripts that runs
 standard backup utilities such as mysqldump, pg_dump, rsync, slapcat and tar.
 .
 bung adds support for hotplug storage devices (such as USB disks), LVM
 snapshots, file system mounting, logging, on-screen notifications and report
 emails

postinst

Started creating. End of session commit fee58ab3

Next

  • Add yad | xenity to control
  • Continue with postinst

#4 Updated by Charles Atkinson over 2 years ago

  • Added yad | xenity to control
  • Completed creation of postinst
  • Started creating of postrm. Need to re-create /opt if it has disappeared after removal.

End of session commit 6b7f3a01

#5 Updated by Charles Atkinson over 2 years ago

Updated postrm to ensure that /opt exists.

Command to populate data directory, based on the tarball creation and installation procedures. To be run by root in the root of the git tree:

version=2.0.1
binary_deb_dir=apt/binary-deb
control_dir=$binary_deb_dir/control
data_dir=$binary_deb_dir/data
bung_dir=$data_dir/opt/bung-$version

mkdir -p $bung_dir/{lib,share/cron,share/doc/examples/{cron,'hook scripts',rsync.exclude,script.conf},share/logcheck,share/man/man{5,8}} \
    && cp -p opt/*.sh $bung_dir/ \
    && cp -p opt/lib/*.{fun,scrippet,sh} $bung_dir/lib/ \
    && cp -p opt/share/cron/cron.weekly $bung_dir/share/cron/ \
    && cp -p opt/share/doc/{AUTHORS,BUGS,COPYING,NEWS,README} $bung_dir/share/doc/ \
    && cp -p opt/share/doc/*.{odt,pdf,txt} $bung_dir/share/doc/ \
    && cp -p opt/share/doc/examples/cron/cron.daily $bung_dir/share/doc/examples/cron/ \
    && cp -p opt/share/doc/examples/'hook scripts/'*.sh \
        $bung_dir/share/doc/examples/'hook scripts/'  \
    && cp -p opt/share/doc/examples/rsync.exclude/*.exclude \
        $bung_dir/share/doc/examples/rsync.exclude/ \
    && cp -p opt/share/doc/examples/script.conf/{*.conf,*.cnf,*.rules} \
        $bung_dir/share/doc/examples/script.conf/ \
    && cp -p opt/share/logcheck/bung $bung_dir/share/logcheck/ \
    && cp -p opt/share/man/man5/*.5.gz $bung_dir/share/man/man5/ \
    && cp -p opt/share/man/man8/*.8.gz $bung_dir/share/man/man8/ \
    && chown -R root:root $bung_dir \
    && find $bung_dir -type f -execdir chmod 644 {} \+ \
    && find $bung_dir -type f -name '*.sh' -execdir chmod 755 {} \+ \
    && find $bung_dir -type d -execdir chmod 755 {} \+ \
    && chmod 744 $bung_dir/share/cron/cron.weekly \
    && chown root:logcheck $bung_dir/share/logcheck/bung

#6 Updated by Charles Atkinson over 2 years ago

md5sums

Command to generate control/md5sums:

version=2.0.1
binary_deb_dir=apt/binary-deb
control_dir=$binary_deb_dir/control
data_dir=$binary_deb_dir/data
bung_dir=$data_dir/opt/bung-$version

( cd $data_dir \
    && find -type f -exec md5sum {} + \
        | sort --key=2 \
        | sed 's|  \./|  |' 
) > $control_dir/md5sums

bung_2.0.1-1.deb

Commands to generate:

version=2.0.1
build=1
deb=bung_$version-$build.deb
binary_deb_dir=apt/binary-deb
control_dir=$binary_deb_dir/control
data_dir=$binary_deb_dir/data
bung_dir=$data_dir/opt/bung-$version

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# tar -czf $binary_deb_dir/control.tar.gz --directory $control_dir control md5sums postinst postrm \
    && tar -czf $binary_deb_dir/data.tar.gz --directory $data_dir . \
    && \
    (cd $binary_deb_dir \
        && ar rc ${deb##*/} debian-binary control.tar.gz data.tar.gz
    )

Test

version=2.0.1
build=1
binary_deb_dir=apt/binary-deb
deb=$binary_deb_dir/bung_$version-$build.deb
control_dir=$binary_deb_dir/control
data_dir=$binary_deb_dir/data
bung_dir=$data_dir/opt/bung-$version
tmp_dir=/tmp/tmp

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --info $deb
[OK]
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --contents $deb
[OK]
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg-deb --show $deb
bung    2.0.0-1
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg-deb --control $deb $tmp_dir
[no output, /tmp/tmp had control/ files; OK]

git

End of session commit f483990a

Next

Test by installation, removal and purge

#7 Updated by Charles Atkinson over 2 years ago

Test

Preparation

root@CW10:~# mv /opt{,.aside}
root@CW10:~# mkdir /opt
root@CW10:~# ll -d /opt
drwxr-xr-x 2 root root 4096 Jun 30 18:00 /opt

Test

version=2.0.1
build=1
binary_deb_dir=apt/binary-deb
deb=$binary_deb_dir/bung_$version-$build.deb
control_dir=$binary_deb_dir/control
data_dir=$binary_deb_dir/data
bung_dir=$data_dir/opt/bung-$version
tmp_dir=/tmp/tmp

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# apt install $deb
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package apt/binary-deb
root@CW10:/home/c/d/Auroville/Blue Light/bung/git/apt/binary-deb# dpkg --install bung_2.0.1-1.deb
Selecting previously unselected package bung.
(Reading database ... 204982 files and directories currently installed.)
Preparing to unpack bung_2.0.1-1.deb ...
Unpacking bung (2.0.0-1) ...
Setting up bung (2.0.0-1) ...
/var/lib/dpkg/info/bung.postinst: 25: /var/lib/dpkg/info/bung.postinst: [[: not found
/var/lib/dpkg/info/bung.postinst: 26: /var/lib/dpkg/info/bung.postinst: [[: not found
/var/lib/dpkg/info/bung.postinst: 33: /var/lib/dpkg/info/bung.postinst: [[: not found

Fix

Changed postinst and preinst (and /var/lib/dpkg/info/bung.postrm) shebang line from /bin/sh (symlink to dash) to /bin/bash and rebuilt the .deb

Retest

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --install $deb
[OK]
root@CW10:~# dpkg --remove bung
(Reading database ... 205121 files and directories currently installed.)
Removing bung (2.0.1-1) ...
root@CW10:~# ll /opt
total 0
root@CW10:~# ll /etc/opt
total 4
drwxr-xr-x 3 root root 4096 Jun 23 11:31 bung
root@CW10:~# dpkg --purge bung
(Reading database ... 204980 files and directories currently installed.)
Removing bung (2.0.1-1) ...
Purging configuration files for bung (2.0.1-1) ...
rmdir: failed to remove ‘/etc/bung’: No such file or directory

Fix

Changed postrm's /etc/bung to /etc/opt/bung and rebuilt the .deb

Re-test

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --install $deb   
Selecting previously unselected package bung.
(Reading database ... 204982 files and directories currently installed.)
Preparing to unpack .../binary-deb/bung_2.0.1-1.deb ...
Unpacking bung (2.0.1-1) ...
Setting up bung (2.0.1-1) ...
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# ll /opt
total 4
lrwxrwxrwx 1 root root   15 Jun 30 19:03 bung -> /opt/bung-2.0.1
drwxr-xr-x 4 root root 4096 Jun 30 19:03 bung-2.0.1
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# ll /etc/opt
total 4
drwxr-xr-x 3 root root 4096 Jun 23 11:31 bung
root@CW10:~# dpkg --purge bung
(Reading database ... 205121 files and directories currently installed.)
Removing bung (2.0.1-1) ...
Purging configuration files for bung (2.0.1-1) ...
rmdir: failed to remove ‘/etc/opt/bung’: Directory not empty
root@CW10:~# mv /etc/opt/bung{,.aside}
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --install $deb
Selecting previously unselected package bung.
(Reading database ... 204982 files and directories currently installed.)
Preparing to unpack .../binary-deb/bung_2.0.1-1.deb ...
Unpacking bung (2.0.1-1) ...
Setting up bung (2.0.1-1) ...
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# ll /etc/opt/bung /opt
/etc/opt/bung:
total 0

/opt:
total 4
lrwxrwxrwx 1 root root   15 Jun 30 19:06 bung -> /opt/bung-2.0.1
drwxr-xr-x 4 root root 4096 Jun 30 19:06 bung-2.0.1
root@CW10:~# dpkg --purge bung
(Reading database ... 205121 files and directories currently installed.)
Removing bung (2.0.1-1) ...
Purging configuration files for bung (2.0.1-1) ...
root@CW10:~# ll /etc/opt/bung /opt
ls: cannot access /etc/opt/bung: No such file or directory
ls: cannot access /opt: No such file or directory

Fix

Added debugging in postrm where it should ensure /opt and rebuilt the .deb.

Created /opt:

root@CW10:~# mkdir /opt

Re-test

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --install $deb
...
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --purge bung
(Reading database ... 205121 files and directories currently installed.)
Removing bung (2.0.1-1) ...
postrm checking /opt existence
Purging configuration files for bung (2.0.1-1) ...
postrm checking /opt existence
Added more dubugging and found /opt existed when postrm was run:
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --purge bung
(Reading database ... 205121 files and directories currently installed.)
Removing bung (2.0.1-1) ...
postrm remove: drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
postrm other cases: drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
postrm checking /opt existence
drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
Purging configuration files for bung (2.0.1-1) ...
postrm purge: drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
postrm remove: drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
postrm other cases: drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
postrm checking /opt existence
drwxr-xr-x 2 root root 4096 Jun 30 19:32 /opt
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# ll /opt
ls: cannot access /opt: No such file or directory

Fix

From https://unix.stackexchange.com/questions/222562/when-will-dpkg-delete-directories:

Nowadays removal is handled in remove.c, and directories are removed only if they are empty and the current package (being removed) is the only one using them

Edited postinst so it creates /opt/.placeholder if it does not exist

Re-test (OK)

root@CW10:/home/c/d/Auroville/Blue Light/bung/git# mkdir /opt
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --install $deb
...
postinst checking for /opt/.placeholder
postinst creating /opt/.placeholder
root@CW10:~# ll -A /opt
...
-rw-r--r-- 1 root root 102 Jun 30 19:48 .placeholder
lrwxrwxrwx  1 root root   15 Jun 30 19:48 bung -> /opt/bung-2.0.1
drwxr-xr-x  4 root root 4096 Jun 30 19:48 bung-2.0.1
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --purge bung
...
dpkg: warning: while removing bung, directory '/opt' not empty so not removed
root@CW10:~# ll -A /opt
total 4
-rw-r--r-- 1 root root 102 Jun 30 19:48 .placeholder

Install for production

root@CW10:~# rsync -a /opt.aside/ /opt/
root@CW10:/home/c/d/Auroville/Blue Light/bung/git# dpkg --install $deb
...
root@CW10:~# ll /opt
...
lrwxrwxrwx 1 root root    15 Jun 30 19:56 bung -> /opt/bung-2.0.1
...
drwxr-xr-x 4 root root  4096 Jun 30 19:56 bung-2.0.1
...

Next

  • Add /opt/bung/log as a symlink to /var/log/bung
  • postrm with purge: remove /var/log/bung if empty
  • postrm: when removing directories generate standard message about them not being empty
  • Test installed package by usage
  • Test installation

#8 Updated by Charles Atkinson over 2 years ago

Building .deb on CW8

Good to run the process again for greater clarity.

Had to install logcheck so chown root:logcheck $bung_dir/share/logcheck/bung did not generate an error.

root@CW8.iciti:~# cd /home/c/d/Projects/bung/git \
    version=2.0.1; \
    build=1; \
    binary_deb_dir=apt/binary-deb; \
    deb=$binary_deb_dir/bung_$version-$build.deb; \
    control_dir=$binary_deb_dir/control; \
    data_dir=$binary_deb_dir/data; \
    bung_dir=$data_dir/opt/bung-$version; \
    tmp_dir=/tmp/tmp
root@CW8.iciti:.../Projects/bung/git# mkdir -p $bung_dir/{lib,share/cron,share/doc/examples/{cron,'hook scripts',rsync.exclude,script.conf},share/logcheck,share/man/man{5,8}} \
    && cp -p opt/*.sh $bung_dir/ \
    && cp -p opt/lib/*.{fun,scrippet,sh} $bung_dir/lib/ \
    && cp -p opt/share/cron/cron.weekly $bung_dir/share/cron/ \
    && cp -p opt/share/doc/{AUTHORS,BUGS,COPYING,NEWS,README} $bung_dir/share/doc/ \
    && cp -p opt/share/doc/examples/cron/cron.daily $bung_dir/share/doc/examples/cron/ \
    && cp -p opt/share/doc/examples/'hook scripts/'*.sh \
        $bung_dir/share/doc/examples/'hook scripts/'  \
    && cp -p opt/share/doc/examples/rsync.exclude/*.exclude \
        $bung_dir/share/doc/examples/rsync.exclude/ \
    && cp -p opt/share/doc/examples/script.conf/{*.conf,*.cnf,*.rules} \
        $bung_dir/share/doc/examples/script.conf/ \
    && cp -p opt/share/logcheck/bung $bung_dir/share/logcheck/
root@CW8.iciti:.../Projects/bung/git# for section in 5 8
do
    for f in opt/share/man/man$section/*.$section
    do   
        gzip --to-stdout $f > $bung_dir/share/man/man$section/${f##*/}.gz
    done
done
Manually copied /usr/share/doc/bung/*{.odt,.pdf,.txt} from a production system to $bung_dir/share/doc
root@CW8.iciti:.../Projects/bung/git# chown -R root:root $bung_dir \
    && find $bung_dir -type f -execdir chmod 644 {} \+ \
    && find $bung_dir -type f -name '*.sh' -execdir chmod 755 {} \+ \
    && find $bung_dir -type d -execdir chmod 755 {} \+ \
    && chmod 744 $bung_dir/share/cron/cron.weekly \
    && chown root:logcheck $bung_dir/share/logcheck/bung
root@CW8.iciti:.../Projects/bung/git# ( cd $data_dir \
    && find -type f -exec md5sum {} + \
        | sort --key=2 \
        | sed 's|  \./|  |' 
) > $control_dir/md5sums
root@CW8.iciti:.../Projects/bung/git# tar -czf $binary_deb_dir/control.tar.gz --directory $control_dir control md5sums postinst postrm \
    && tar -czf $binary_deb_dir/data.tar.gz --directory $data_dir . \
    && \
    (cd $binary_deb_dir \
        && ar rc ${deb##*/} debian-binary control.tar.gz data.tar.gz
    )
That created bung_2.0.1-1.deb in /home/c/d/Projects/bung/git/apt/binary-deb.

Would be better to set the variables by sourcing a file and to build the .deb outside the git tree.

Testing the .deb

Installed on CW8:

root@CW8.iciti:.../Projects/bung/git# dpkg --install $deb
Selecting previously unselected package bung.
(Reading database ... 186621 files and directories currently installed.)
Preparing to unpack .../binary-deb/bung_2.0.1-1.deb ...
Unpacking bung (2.0.1-1) ...
Setting up bung (2.0.1-1) ...
Tested by plugging USB HDD. Pop-up notifications, log and email OK.

Installed on others:

master c@CW8:~/d/Projects/bung/git$ version=2.0.1; \
    build=1; \
    binary_deb_dir=apt/binary-deb; \
    deb=$binary_deb_dir/bung_$version-$build.deb; \
    control_dir=$binary_deb_dir/control; \
    data_dir=$binary_deb_dir/data; \
    bung_dir=$data_dir/opt/bung-$version; \
    tmp_dir=/tmp/tmp
master c@CW8:~/d/Projects/bung/git$ ssh_opts='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet'
master c@CW8:~/d/Projects/bung/git$ for h in LS1.avenir.av pc{1,3,4,5,8,9}.iciti.av
do
    echo ==== $h ====
    scp -p $deb root@$h:Downloads
done
[OK except for pc3]
c@CW8:~$ for h in LS1.avenir.av pc{1,3,4,5,8,9}.iciti.av
do
    ssh $ssh_opts root@$h
done
On each computer, ran dpkg --install Downloads/bung_2.0.1-1.deb

Next

  • Decided not to add /opt/bung/log as a symlink to /var/log/bung because that would differ from how bung 2.0.1 is installed from tarball. Nice for the next version
  • Allow time for test by usage on LS1.avenir.av pc{1,3,4,5,8,9}.iciti.av
  • Decide how to streamline .deb build:
    • Temporary directory outside git tree
    • *.odt, .pdf and .txt files. In git tree? If not, where to get them from?

#10 Updated by Charles Atkinson over 2 years ago

Streamlining the manual build

Planning

Before starting to use dh, best streamline the manual build:
  • *.odt files and their derivatives in git so all files required for the binary.deb file are in git
  • Tokenise the bung version number in control/control and control/postinst
  • Tokenise the .deb build number in control/control
  • Build in a temporary directory, not in the git tree
  • Create .deb in $PWD

Action

  • Added 2.0.x .odt files and their derivatives to git. Commit 37d5f9cc
  • Tokenised bung version number and .deb build number in control/control and control/postinst as bung_version and deb_build_version Commit d7456a6c

Build script

Completed build_deb.sh debugging. Commit 6e92ac19

#11 Updated by Charles Atkinson over 2 years ago

Creating the source .deb

Must use a snapshot of git as source (because have not mastered branching yet). The 2.0.1 tarball is suitable.

Research, research, research ...

Which of many examples to follow?

From https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial:

  • Two kinds of packages: (if unsure, use non-native)
    • Native packages: normally for Debian specific software (dpkg, apt)
    • Non-native packages: software developed outside Debian
  • Main file: .dsc (meta-data)
  • Other files depending on the version of the source format
    • 1.0 or 3.0 (native): package_version.tar.gz
    • 1.0 (non-native): pkg_ver.orig.tar.gz: upstream source
    • pkg_debver.diff.gz: patch to add Debian-specific changes

From the dh_make man page:

dh_make must be invoked within a directory containing the source code, which must be named <packagename>-<version>

dh_make uses envars:

Invocation (indep for all architectures):

dh_make --indep

Tree

+-- bung-2.0.1/
|   |
|   +-- debian/
|   |   |
|   |   + changelog, compat, control, copyright, docs, rules
|   |
|   +-- opt/ and subdirs
|
+-- bung_2.0.1.orig.tar.gz
|
|   |
|   |

debuild -us -uc

Learning by doing

Mostly following https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial

Package installations

root@CW10:~# apt install dh-make
...
The following NEW packages will be installed:
  dh-make
...

JDI

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 
tmp_dir=/tmp/bung_source_deb
rm -fr "${tmp_dir:-UNSET_GUARD}" \
    && mkdir -p "$tmp_dir"/bung-2.0.1 \
    && cp -p /root/Downloads/bung-2.0.1.tgz "$tmp_dir/bung_2.0.1.orig.tar.gz" \
    && cd "$tmp_dir" \
    && tar -xf bung_2.0.1.orig.tar.gz --directory bung-2.0.1 \
    && cd bung-2.0.1 \
    && dh_make --copyright gpl --indep

Next

Populate the debian directory

#12 Updated by Charles Atkinson over 2 years ago

Fixing the copyright

Yesterday's procedure had created a GPLv3 copyright but bung uses GPLv2. Changed to:

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 
tmp_dir=/tmp/bung_source_deb
rm -fr "${tmp_dir:-UNSET_GUARD}" \
    && mkdir -p "$tmp_dir"/bung-2.0.1 \
    && cp -p /root/Downloads/bung-2.0.1.tgz "$tmp_dir/bung_2.0.1.orig.tar.gz" \
    && cd "$tmp_dir" \
    && tar -xf bung_2.0.1.orig.tar.gz --directory bung-2.0.1 \
    && cd bung-2.0.1 \
    && dh_make --copyright gpl2 --indep

Populating the debian directory

.ex and .EX template file removal

root@CW10:/tmp/bung_source_deb/bung-2.0.1/debian# rm *.{ex,EX}

changelog

root@CW10:/tmp/bung_source_deb/bung-2.0.1# dch --increment
After removing the new stanza:
root@CW10:/tmp/bung_source_deb/bung-2.0.1# cat debian/changelog 
bung (2.0.1-1) unstable; urgency=low

  * Initial release

 -- Charles Atkinson <bung@charlesmatkinson.org>  Sun, 07 Jul 2019 07:43:49 +0530

control

Created manually based on the newly created sample with 3.9.5 changed to 3.9.6 as later prompted by lintian:

root@CW10:/tmp/bung_source_deb/bung-2.0.1/debian# cat control
Source: bung
Section: admin
Priority: optional
Maintainer: Charles Atkinson <bung@charlesmatkinson.org>
Build-Depends: debhelper (>= 9)
Standards-Version: 3.9.6
Homepage: 

Package: bung
Architecture: all
Depends: default-mta | mail-transport-agent, cron | cron-daemon, rsyslog | system-log-daemon
Description: backup system
 backup scripts next generation (bung) is a suite of bash scripts that runs
 standard backup utilities such as mysqldump, pg_dump, rsync, slapcat and tar.
 .
 bung adds support for hotplug storage devices (such as USB disks), LVM
 snapshots, file system mounting, logging, on-screen notifications and report
 emails.

copyright

Edited from the newly installed template:

root@CW10:/tmp/bung_source_deb/bung-2.0.1/debian# cat copyright
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: bung
Source: 2019 Charles Atkinson <bung@charlesmatkinson.org>

Files: *
Copyright: 2019 Charles Atkinson <bung@charlesmatkinson.org>
License: GPL-2.0+

Files: debian/*
Copyright: 2019 Charles Atkinson <bung@charlesmatkinson.org>
License: GPL-2.0+

License: GPL-2.0+
 This package is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
 the Free Software Foundation; either version 2 of the License, or
 (at your option) any later version.
 .
 This package is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU General Public License for more details.
 .
 You should have received a copy of the GNU General Public License
 along with this program. If not, see <http://www.gnu.org/licenses/>
 .
 On Debian systems, the complete text of the GNU General
 Public License version 2 can be found in "/usr/share/common-licenses/GPL-2".

postinst and postrm

Copied from git:

root@CW10:/tmp/trash.d/bung-2.0.1/debian# cp -p '/home/c/d/Auroville/Blue Light/bung/git/apt/binary-deb/control/'{postinst,postrm} .

rules

Docs:

Viewed root@CW10:~/tmp/logheck-package-source/logcheck-1.3.17/debian/rules

Hmm ... several examples but much doubt about what is needed. What has to be done by the rules file? According to #7681#note-8 above, the tasks to build the binary .deb were:
  • Set ownerships and permissions (the logcheck file is an exception)
  • Generate compressed man pages. No -- they are already compressed because installed from the tarball.

Tried (the spaces shown before dh are a tab):

root@CW10:/tmp/bung_source_deb/bung-2.0.1/debian# cat rules
#!/usr/bin/make -f

# Uncomment  this to turn on  verbose  mode.
# Doc: debhelper(7)
#export  DH_VERBOSE =1

%:
    dh $@

Maybe clean should remove the compressed man pages

README files

root@CW10:/tmp/bung_source_deb/bung-2.0.1/debian# rm README.Debian
root@CW10:/tmp/bung_source_deb/bung-2.0.1/debian# cat README.source 
bung for Debian
---------------

This source .deb was created from bung-2.0.1.tgz which was not publicly
available.

bung-2.0.1.tgz was created from the bung git by:

version=$(<lib/version.scrippet) \
&& version=${version#*=} \
&& tar cvz \
    -f bung-$version.tgz \
    *.sh \
    lib/*.{fun,scrippet,sh} \
    share/cron/cron.weekly \
    share/doc/{AUTHORS,BUGS,COPYING,INSTALL,NEWS,README} \
    share/doc/*.{odt,pdf,txt} \
    share/doc/examples/cron/cron.daily \
    share/doc/examples/'hook scripts/'*.sh \
    share/doc/examples/rsync.exclude/*.exclude \
    share/doc/examples/script.conf/{*.conf,*.cnf,*.rules} \
    share/logcheck/bung \
    share/man/man5/*.5.gz \
    share/man/man8/*.8.gz

The bung git was not publicly available (there were plans to change that).

For a copy of the tarball and for news of git access, write to the author,
address below.

 -- Charles Atkinson <bung@charlesmatkinson.org>  Sun, 07 Jul 2019 09:01:42 +0530

source directory

Probably not needed because no patching to be done. TODO: try removing it

Build (failed)

root@CW10:/tmp/bung_source_deb/bung-2.0.1# debuild -uc -us
...
Now running lintian...
warning: the authors of lintian do not recommend running it with root privileges!
W: bung source: debhelper-but-no-misc-depends bung
W: bung: new-package-should-close-itp-bug
W: bung: empty-binary-package
Finished running lintian.
Fixed the first lintian warning by:
root@CW10:/tmp/bung_source_deb/bung-2.0.1# diff debian/control{.bu0,}
11c11
< Depends: default-mta | mail-transport-agent, cron | cron-daemon, rsyslog | system-log-daemon
---
> Depends: default-mta | mail-transport-agent, cron | cron-daemon, rsyslog | system-log-daemon, ${misc:Depends}
The second warning is inevitable given that this build does not fix any ITP (intend to build) bugs.

Found what had been built:

root@CW10:/tmp/bung_source_deb/bung-2.0.1# ll ..
total 680
drwxr-xr-x 5 root root   4096 Jul  7 09:01 bung-2.0.1
-rw-r--r-- 1 root root   1894 Jul  7 09:53 bung_2.0.1-1_all.deb
-rw-r--r-- 1 root root   1563 Jul  7 09:53 bung_2.0.1-1_amd64.build
-rw-r--r-- 1 root root   1376 Jul  7 09:53 bung_2.0.1-1_amd64.changes
-rw-r--r-- 1 root root   1948 Jul  7 09:53 bung_2.0.1-1.debian.tar.xz
-rw-r--r-- 1 root root    754 Jul  7 09:53 bung_2.0.1-1.dsc
-rw-r--r-- 1 root root 667922 Jun 29 09:14 bung_2.0.1.orig.tar.gz
The .deb should be ~668 kB

#13 Updated by Charles Atkinson over 2 years ago

Evolved build tree creation commands

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 
bung_version=2.0.1
tmp_dir=/tmp/bung_source_deb
rm -fr "${tmp_dir:-UNSET_GUARD}" \
    && mkdir -p "$tmp_dir"/bung-$bung_version \
    && cp -p /root/Downloads/bung-$bung_version.tgz "$tmp_dir/bung_$bung_version.orig.tar.gz" \
    && cd "$tmp_dir" \
    && tar -xf bung_$bung_version.orig.tar.gz --directory bung-$bung_version \
    && cd bung-$bung_version \
    && dh_make --copyright gpl2 --indep \
    && rm debian/{*.{ex,EX},README*} \
    && cp -p ~/tmp/{changelog,control,copyright,README.source,rules} debian \
    && cp -p '/home/c/d/Auroville/Blue Light/bung/git/apt/binary-deb/control/'{postinst,postrm} debian \
    && sed -i "s/%bung_version%/$bung_version/g" debian/{postinst,postrm}

Build

root@CW10:/tmp/bung_source_deb/bung-2.0.1# debuild -uc -us
...
Now running lintian...
warning: the authors of lintian do not recommend running it with root privileges!
W: bung source: maintainer-script-lacks-debhelper-token debian/postrm
W: bung source: maintainer-script-lacks-debhelper-token debian/postinst
W: bung: new-package-should-close-itp-bug
W: bung: empty-binary-package
Fixed the maintainer-script-lacks-debhelper-token warning by (and similar for postrm)
root@CW10:/tmp/bung_source_deb/bung-2.0.1# diff debian/postinst{.org,}
55c55
< 
---
> #DEBHELPER#
The .deb was bigger than before but still too small:
root@CW10:/tmp/bung_source_deb/bung-2.0.1# ll ..
total 680
drwxr-xr-x 5 root root   4096 Jul  7 18:41 bung-2.0.1
-rw-r--r-- 1 root root   3040 Jul  7 18:50 bung_2.0.1-1_all.deb
-rw-r--r-- 1 root root   1563 Jul  7 18:50 bung_2.0.1-1_amd64.build
-rw-r--r-- 1 root root   1376 Jul  7 18:50 bung_2.0.1-1_amd64.changes
-rw-r--r-- 1 root root   2936 Jul  7 18:50 bung_2.0.1-1.debian.tar.xz
-rw-r--r-- 1 root root    754 Jul  7 18:50 bung_2.0.1-1.dsc
-rw-r--r-- 1 root root 667922 Jun 29 09:14 bung_2.0.1.orig.tar.gz

Try: removing the source directory

root@CW10:/tmp/bung_source_deb/bung-2.0.1# rm -r debian/source/
root@CW10:/tmp/bung_source_deb/bung-2.0.1# debuild -uc -us
[same]
Found in ../bung_2.0.1-1_amd64.build:
dpkg-source: warning: no source format specified in debian/source/format, see dpkg-source(1)
dpkg-source: info: using source format `1.0'

Try: verbose

root@CW10:/tmp/bung_source_deb/bung-2.0.1# export DH_VERBOSE=1
That made little difference. Tried:
root@CW10:/tmp/bung_source_deb/bung-2.0.1# diff debian/rules{.org,}
5c5
< #export  DH_VERBOSE =1
---
> export DH_VERBOSE=1
root@CW10:/tmp/bung_source_deb/bung-2.0.1# debuild -uc -us
[more output but did not find an error which explained the empty .deb]

#14 Updated by Charles Atkinson over 2 years ago

Evolved build tree creation commands

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 
bung_version=2.0.1
tmp_dir=/tmp/bung_source_deb
rm -fr "${tmp_dir:-UNSET_GUARD}" \
    && mkdir -p "$tmp_dir"/bung-$bung_version \
    && cp -p /root/Downloads/bung-$bung_version.tgz "$tmp_dir/bung_$bung_version.orig.tar.gz" \
    && cd "$tmp_dir" \
    && tar -xf bung_$bung_version.orig.tar.gz --directory bung-$bung_version \
    && cd bung-$bung_version \
    && dh_make --copyright gpl2 --indep \
    && rm debian/{*.{ex,EX},README*} \
    && cp -p ~/tmp/{changelog,control,copyright,postinst,postrm,README.source,rules} debian

Build (new warnings to investigate)

Try: changing the rules file

From https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial:

* Required targets:

  • build, build-arch, build-indep: should perform all the configuration and compilation
  • binary, binary-arch, binary-indep: build the binary packages
    • dpkg-buildpackage will call binary to build all the packages, or binary-arch to build only the Architecture: any packages
  • clean: clean up the source directory

How debhelper (dh_*) scripts use the rules file: https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial#Outline0.2.5.21

Investigation

Saw dh_fixperms had worked on files under ./debian/bung, not on {.,./lib,./share}. Maybe OK.

./debian/bung/ should be populated by dpkg-buildpackage

An example of packaging a single python script (https://github.com/phusion/debian-packaging-for-the-modern-developer/tree/master/tutorial-2#debianrules) showed the source code not under . as done until now but under ./debian/<package name> which is consistent with dh_fixperms working on files under ./debian/bung.

./debian/bung/ should be populated by dpkg-buildpackage

Changed the rules file to do so:

root@CW10:~/tmp# cat rules
#!/usr/bin/make -f

# Uncomment this to turn on verbose mode
# Doc: debhelper(7)
export DH_VERBOSE=1

clean:
    @# Do nothing

build:
    @# Do nothing

binary:
    mkdir -p debian/bung
    cp -pr *.sh lib/ share/ debian/bung
    dh_gencontrol
    dh_builddeb

#15 Updated by Charles Atkinson about 2 years ago

Orientation

There are now separate git repos for bung and bung_packaging. Ref Task #8681
"Redmine: create bung repo for packaging".

In the conventional upstream > packagers way, the bung git is used to produce a bung tarball which is an input to the .deb creation procedure.

The last procedure, to create bung_2.0.2-2.deb was logged in "Nextcloud/bung/2.0+/Backup scripts next generation 2.0.x+ Programmer's Guide.odt":

As unprivileged user (only git_dir was specific to the user):

c@CW8:~$ git_dir=/home/c/d/Projects/bung/bung_packaging; version=2.0.2; build=1 
c@CW8:~$ binary_deb_dir=$git_dir/apt/binary-deb; \
    deb=$binary_deb_dir/bung_$version-$build.deb; \
    control_dir=$binary_deb_dir/control; \
    data_dir=$binary_deb_dir/data; \
    bung_dir=$data_dir/opt/bung-$version; \
    mkdir -p $bung_dir $control_dir $data_dir \
    && tar -xf /tmp/bung-2.0.2.tgz -C $bung_dir
As root:
root@CW8.iciti:~# git_dir=/home/c/d/Projects/bung/bung_packaging; version=2.0.2; build=2 
root@CW8.iciti:~# binary_deb_dir=$git_dir/apt/binary-deb; \
    deb=$binary_deb_dir/bung_$version-$build.deb; \
    control_dir=$binary_deb_dir/control; \
    data_dir=$binary_deb_dir/data; \
    bung_dir=$data_dir/opt/bung-$version; \
    chown -R root:root $bung_dir \
    && find $bung_dir -type f -execdir chmod 644 {} \+ \
    && find $bung_dir -type f -name '*.sh' -execdir chmod 755 {} \+ \
    && find $bung_dir -type d -execdir chmod 755 {} \+ \
    && chmod 744 $bung_dir/share/cron/cron.weekly \
    && chown root:logcheck $bung_dir/share/logcheck/bung \
    && sed -e "s/%bung_version%/$version/" -e "s/%deb_build_version%/$build/" \
        $control_dir/control.template > $control_dir/control \
    && sed "s/%bung_version%/$version/" \
        $control_dir/postinst.template > $control_dir/postinst \
    && ( cd $data_dir \
    && find -type f -exec md5sum {} + \
           | sort --key=2 \
           | sed 's|  \./|  |' 
       ) > $control_dir/md5sums \
    && tar -czf $binary_deb_dir/control.tar.gz --directory $control_dir \
        control md5sums postinst postrm \
    && tar -czf $binary_deb_dir/data.tar.gz --directory $data_dir . \
    && \
    (cd $binary_deb_dir \
        && ar rc ${deb##*/} debian-binary control.tar.gz data.tar.gz
    )
That created /home/c/d/Projects/bung/bung_packaging/apt/binary-deb/bung_2.0.2-2.deb

Build trees

Source

Current understanding:
Directory or file Usage Notes
bung_2.0.2.orig.tar.gz Renamed tarball
bung_2.0.2/ Tarball unpack destination
bung_2.0.2/debian/
bung_2.0.2/debian/changelog Log of changes to the package Create with dch?
bung_2.0.2/debian/compat Content: 10
bung_2.0.2/debian/control Information for users
bung_2.0.2/debian/copyright
bung_2.0.2/debian/rules Makefile
bung_2.0.2/debian/source/format Content:3.0 (quilt)
bung_2.0.2/debian/<whatever> Tree to be copied under /

Binary

Current understanding:
Directory or file Usage Notes
binary-deb/ .deb created here
binary-deb/control/ Files for the .deb
binary-deb/control/control Information for users
binary-deb/control/md5sums Validation
binary-deb/control/postinst Script to run after files copied
binary-deb/control/postrm Script to run after files removed
binary-deb/data/ Tree to be copied under /

Next

  • Develop a process to create the source .deb. The process should put the tarball and files from git into a temporary directory then use standard Debian tools -- maybe only debuild -- to build the source .deb
  • Figure out how to use the source .deb to produce the binary .deb

#16 Updated by Charles Atkinson about 2 years ago

Developing a process to create the source .deb

Command to try:

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 

bung_ver=2.0.2
git_dir='/home/c/d/Auroville/Blue Light/bung/bung_packaging git/apt'
tarball_url='https://redmine.auroville.org.in/attachments/download/5933/bung-2.0.2.tgz'
tarball=bung_$bung_ver.orig.tar.xz 

root_dir=$(mktemp --directory /tmp/XXXXX)
if (($?==0)); then
    mkdir -p "$root_dir/bung-$bung_ver/debian/source/" \
        && cd "$root_dir/" \
        && wget "$tarball_url" --output-document="$tarball" \
        && tar -xf "$tarball" \
        && cd bung-$bung_ver/ \
        && dh_make --copyright gpl2 --indep --single \
        && rm debian/{*.{ex,EX},README*} \
        && cp -p "$git_dir"/debian/{changelog,control,postinst,postrm,README.source,rules} debian \
        && sed -i "s/%bung_version%/$bung_version/g" debian/{postinst,postrm} \
        && debuild -uc -us
fi

#17 Updated by Charles Atkinson about 2 years ago

"command to try"

The "command to try" above did not work well:

Skipping creating ../bung_2.0.2.orig.tar.xz because it already exists
Currently there is no top level Makefile. This may require additional tuning.
You already have a debian/ subdirectory in the source tree.
dh_make will not try to overwrite anything

Command from Jul 2019

Retrieved the command run back in July from CW10's root' bash history. Added | grep ^E because there were any warnings generated by installing in /opt which is contrary to Debian policy:

root@CW10:~# export DEBEMAIL="bung@charlesmatkinson.org" 
root@CW10:~# export DEBFULLNAME="Charles Atkinson" 
root@CW10:~# bung_version=2.0.1
root@CW10:~# tmp_dir=/tmp/bung_source_deb
root@CW10:~# rm -fr "${tmp_dir:-UNSET_GUARD}" \
    && mkdir -p "$tmp_dir"/bung-$bung_version \
    && cp -p /root/Downloads/bung-$bung_version.tgz "$tmp_dir/bung_$bung_version.orig.tar.gz" \
    && cd "$tmp_dir" \
    && tar -xf bung_$bung_version.orig.tar.gz --directory bung-$bung_version \
    && cd bung-$bung_version \
    && dh_make --copyright gpl2 --indep \
    && rm debian/{*.{ex,EX},README*} \
    && cp -p ~/tmp/{changelog,control,copyright,postinst,postrm,README.source,rules} debian \
    && debuild -uc -us | grep ^E
Skipping creating ../bung_2.0.1.orig.tar.gz because it already exists
Currently there is no top level Makefile. This may require additional tuning.
Done. Please edit the files in the debian/ subdirectory now. You should also
check that the bung Makefiles install into $DESTDIR and not in / .
E: bung source: debian-rules-missing-required-target binary-indep
E: bung source: debian-rules-missing-required-target binary-arch
E: bung: debian-changelog-file-missing
E: bung: no-copyright-file
E: bung: non-standard-toplevel-dir share/

Evolved command

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 

build_version=1
bung_version=2.0.2
git_dir='/home/c/d/Auroville/Blue Light/bung/bung_packaging git/apt'
tarball_url='https://redmine.auroville.org.in/attachments/download/5933/bung-2.0.2.tgz'
tarball=bung_$bung_ver.orig.tar.xz 

root_dir=$(mktemp --directory /tmp/XXXXX)
if (($?==0)); then
    mkdir -p "$root_dir/bung-$bung_version" \
        && cd "$root_dir/" \
        && wget "$tarball_url" --output-document="$tarball" \
        && echo ==== After downloading tarball ==== \
        && pwd \
        && ls -l \
        && tar -xf "$tarball" \
        && cd bung-$bung_version/ \
        && echo ==== Running dh_make ==== \
        && dh_make --copyright gpl2 --indep --packagename bung --yes \
        && rm debian/{*.{ex,EX},README*} \
        && cp -p "$git_dir"/debian/{changelog,control,postinst,postrm,rules} debian \
        && sed -i "s/%bung_version%/$bung_version/g" debian/{changelog,control,postinst,postrm} \
        && sed -i "s/%deb_build_version%/$deb_build_version/g" debian/{changelog,control} \
        && sed -i "s/%date_rfc2822%/$(date --rfc-2822)/g" debian/changelog \
        && echo ==== Running debuild ==== \
        && debuild -uc -us
fi

Next

Why can't the evolved command find the tarball? Why does it want to unpack it anyway? Should the tarball have been unpacked already?

#18 Updated by Charles Atkinson about 2 years ago

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 

build_version=1
bung_version=2.0.2
git_dir='/home/c/d/Auroville/Blue Light/bung/bung_packaging git/apt'
tarball_url='https://redmine.auroville.org.in/attachments/download/5933/bung-2.0.2.tgz'
tarball=bung_$bung_version.orig.tar.xz 

root_dir=$(mktemp --directory /tmp/XXXXX)
if (($?==0)); then
    mkdir -p "$root_dir/bung-$bung_version" \
        && cd "$root_dir/" \
        && wget "$tarball_url" --output-document="$tarball" \
        && echo ==== After downloading tarball ==== \
        && pwd \
        && ls -l \
        && tar -xf "$tarball" \
        && cp -p "$tarball" bung-$bung_version/ \
        && cd bung-$bung_version/ \
        && echo ==== Running dh_make ==== \
        && dh_make --copyright gpl2 --indep --packagename bung --yes \
        && rm debian/{*.{ex,EX},README*} \
        && cp -p "$git_dir"/debian/{changelog,control,postinst,postrm,rules} debian \
        && sed -i "s/%bung_version%/$bung_version/g" debian/{changelog,control,postinst,postrm} \
        && sed -i "s/%deb_build_version%/$deb_build_version/g" debian/changelog \
        && sed -i "s/%date_rfc2822%/$(date --rfc-2822)/g" debian/changelog \
        && echo ==== Running debuild ==== \
        && debuild -uc -us
fi

#19 Updated by Charles Atkinson about 2 years ago

Hacking scripts installed by package

Added debug print to:
  • /usr/bin/dpkg-source
    root@CW10:/tmp/5TvQZ/bung-2.0.2# diff /usr/bin/dpkg-source{.org,}
    427a428
    >     info(_g("CHARLES After run_vendor_hook"));
    428a430,431
    >     # DEVEL
    >     info(_g("CHARLES Running $srcpkg->build($dir)"));
    429a433
    >     info(_g("CHARLES After $srcpkg->build($dir)"));
    
  • /usr/share/perl5/Dpkg/Source/Package.pm
    root@CW10:/tmp/888JP/bung-2.0.2# diff /usr/share/perl5/Dpkg/Source/Package.pm{.org,}
    471a472
    >     info(_g("CHARLES Package.pm: sub extract: Copy orig tarballs"));
    535a537
    >     info(_g("CHARLES Package.pm: started sub before_build"));
    539a542
    >     info(_g("CHARLES Package.pm: started sub build"));
    540a544
    >     info(_g("CHARLES Package.pm: sub build: running eval"));
    542a547
    >         info(_g("CHARLES Package.pm: sub build: running run_exit_handlers"));
    545a551
    >     info(_g("CHARLES Package.pm: sub build: ending"));
    552a559
    >     info(_g("CHARLES Package.pm: started do_build"));
    

Fixed one bug

Fixed this by changing tarball=bung_$bung_version.orig.tar.xz to tarball=bung_$bung_version.orig.tar.gz:

dpkg-source: info: building bung using existing ./bung_2.0.2.orig.tar.xz
unxz: (stdin): File format not recognized
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

New command one and its bug

New command:

export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 

build_version=1
bung_version=2.0.2
git_dir='/home/c/d/Auroville/Blue Light/bung/bung_packaging git/apt'
tarball_url='https://redmine.auroville.org.in/attachments/download/5933/bung-2.0.2.tgz'
tarball=bung_$bung_version.orig.tar.gz 

root_dir=$(mktemp --directory /tmp/XXXXX)
if (($?==0)); then
    mkdir -p "$root_dir/bung-$bung_version" \
        && cd "$root_dir/" \
        && wget "$tarball_url" --output-document="$tarball" \
        && echo ==== After downloading tarball ==== \
        && pwd \
        && ls -l \
        && tar -xf "$tarball" \
        && cp -p "$tarball" bung-$bung_version/ \
        && cd bung-$bung_version/ \
        && echo ==== Running dh_make ==== \
        && dh_make --copyright gpl2 --indep --packagename bung --yes \
        && rm debian/{*.{ex,EX},README*} \
        && cp -p "$git_dir"/debian/{changelog,control,postinst,postrm,rules} debian \
        && sed -i "s/%bung_version%/$bung_version/g" debian/{changelog,control,postinst,postrm} \
        && sed -i "s/%deb_build_version%/$deb_build_version/g" debian/changelog \
        && sed -i "s/%date_rfc2822%/$(date --rfc-2822)/g" debian/changelog \
        && echo ==== Running debuild ==== \
        && debuild -uc -us
fi
New bug:
dpkg-source: info: building bung using existing ./bung_2.0.2.orig.tar.gz
dpkg-source: warning: ignoring deletion of file mysql_bu.sh, use --include-removal to override
[snip many similar]
dpkg-source: warning: ignoring deletion of file share/logcheck/bung, use --include-removal to override
dpkg-source: error: cannot represent change to bung_2.0.2.orig.tar.gz: binary file contents changed
dpkg-source: error: add bung_2.0.2.orig.tar.gz in debian/source/include-binaries if you want to store the modified binary in the debian tarball
dpkg-source: info: CHARLES Package.pm: sub build: running run_exit_handlers
dpkg-source: error: unrepresentable changes to source
...

New command two and its bug

Tried removing these lines (debug and possibly unwanted tarball extraction) ...

        && echo ==== After downloading tarball ==== \
        && pwd \
        && ls -l \
        && tar -xf "$tarball" \
        && cp -p "$tarball" bung-$bung_version/ \
... leaving:
export DEBEMAIL="bung@charlesmatkinson.org" 
export DEBFULLNAME="Charles Atkinson" 

build_version=1
bung_version=2.0.2
git_dir='/home/c/d/Auroville/Blue Light/bung/bung_packaging git/apt'
tarball_url='https://redmine.auroville.org.in/attachments/download/5933/bung-2.0.2.tgz'
tarball=bung_$bung_version.orig.tar.gz 

root_dir=$(mktemp --directory /tmp/XXXXX)
if (($?==0)); then
    mkdir -p "$root_dir/bung-$bung_version" \
        && cd "$root_dir/" \
        && wget "$tarball_url" --output-document="$tarball" \
        && cd bung-$bung_version/ \
        && echo ==== Running dh_make ==== \
        && dh_make --copyright gpl2 --indep --packagename bung --yes \
        && rm debian/{*.{ex,EX},README*} \
        && cp -p "$git_dir"/debian/{changelog,control,postinst,postrm,rules} debian \
        && sed -i "s/%bung_version%/$bung_version/g" debian/{changelog,control,postinst,postrm} \
        && sed -i "s/%deb_build_version%/$deb_build_version/g" debian/changelog \
        && sed -i "s/%date_rfc2822%/$(date --rfc-2822)/g" debian/changelog \
        && echo ==== Running debuild ==== \
        && debuild -uc -us
fi
Better; new bug:
echo ==== copying files ===
==== copying files ===
cp -pr *.sh lib/ share/ debian/bung
cp: cannot stat ‘*.sh’: No such file or directory
cp: cannot stat ‘lib/’: No such file or directory
cp: cannot stat ‘share/’: No such file or directory
debian/rules:14: recipe for target 'binary' failed
make: *** [binary] Error 1
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
"==== copying files ===" is not in the command so probably in the makefile. More later

#20 Updated by Charles Atkinson about 2 years ago

Searched for "dpkg-source: warning: ignoring deletion of file" several hits which did not lead me to a solution.

Found tarball gone and not unpacked:

root@CW10:/tmp/OrKB2/bung-2.0.2# tree
.
└── debian
    ├── bung
    ├── changelog
    ├── compat
    ├── control
    ├── copyright
    ├── docs
    ├── postinst
    ├── postrm
    ├── rules
    └── source
        └── format
Changed to minimal makefile:
master c@CW10:~/.../bung_packaging git/apt/debian$ cat rules
#!/usr/bin/make -f

# Uncomment this to turn on verbose mode
# Doc: debhelper(7)
export DH_VERBOSE=1

%:
    dh $@
Output:
...
dpkg-source: info: building bung using existing ./bung_2.0.2.orig.tar.gz
dpkg-source: warning: ignoring deletion of file mysql_bu.sh, use --include-removal to override
[snip many similar, one for each file in the tarball]
...
dpkg-source: info: building bung in bung_2.0.2-.debian.tar.xz
...
dpkg-source: info: building bung in bung_2.0.2-.dsc
...
   dh_auto_install
    install -d debian/bung
   dh_installdocs
    install -g 0 -o 0 -d debian/bung/usr/share/doc/bung
    install -g 0 -o 0 -m 644 -p debian/copyright debian/bung/usr/share/doc/bung/copyright
   dh_installchangelogs
    install -o 0 -g 0 -p -m644 debian/changelog debian/bung/usr/share/doc/bung/changelog.Debian
...
dpkg-deb: building package `bung' in `../bung_2.0.2_all.deb'.
 dpkg-genchanges  >../bung_2.0.2-_amd64.changes
dpkg-genchanges: warning: missing Section for source files
dpkg-genchanges: including full source code in upload
dpkg-genchanges: error: cannot fstat file ../bung_2.0.2-_all.deb: No such file or directory
...
Better; that left:
root@CW10:/tmp/eP7zA/bung-2.0.2# tree
.
└── debian
    ├── bung
    │   ├── DEBIAN
    │   │   ├── control
    │   │   ├── md5sums
    │   │   ├── postinst
    │   │   └── postrm
    │   └── usr
    │       └── share
    │           └── doc
    │               └── bung
    │                   ├── changelog.Debian.gz
    │                   └── copyright
    ├── bung.debhelper.log
    ├── bung.substvars
    ├── changelog
    ├── compat
    ├── control
    ├── copyright
    ├── docs
    ├── files
    ├── postinst
    ├── postrm
    ├── rules
    └── source
        └── format
Ignoring the "warning: ignoring deletion of file" messages for now, focussed on "dpkg-genchanges >../bung_2.0.2-_amd64.changes". The build number is missing and the architecture should be all, not amd64.

Build version is set by build_version=1 but used only in the sed commands, not with dh_make or debuild.

Found the debuild options confusing so changed the command to clarify:

...
dpkg_build_opts=''
dpkg_buildpackage_opts='-a all'    # Architecture all
dpkg_buildpackage_opts+=' -uc'     # Do not sign source package
dpkg_buildpackage_opts+=' -us'     # Do not sign .changes file
...
        && debuild $dpkg_build_opts $dpkg_buildpackage_opts
...
"all" was not a valid value. Searched for list of valid values but did not find.

#21 Updated by Charles Atkinson 8 months ago

  • Status changed from In Progress to Closed

Issue closed because .deb creation now well established for bung 3.x.x under #11861#note-21 and earlier

Also available in: Atom PDF