Commit graph

460 commits

Author SHA1 Message Date
Piotr Wójcik 588e8cdb7b common: retry ftp download during CI 2020-02-11 23:05:31 +01:00
q66 9fb3ffe131 common: make XBPS{_TARGET}_NEED_ATOMIC less misleading 2020-02-07 21:41:39 +01:00
Jürgen Buchmüller 5bed028584 common.sh: introduce XBPS{,_TARGET}_NEED_ATOMIC 2020-02-07 21:29:22 +01:00
Zach Dykstra b7389f26af Add hg.sr.ht to update_check.sh
hg.sr.ht project pages are laid out quite closely to GitHubs format. We
can basically clone the check for GitHub to a new base URL of hg.sr.ht .
2020-02-05 09:05:08 +01:00
q66 cd2e132ea1 xbps-src: per-architecture handling of repo confs
The new behavior is basically this:

1) Local repos are installed same as before.
2) Multilib configs are now handled only if they exist. They follow
   a naming system repos-{local,remote}-ARCH-multilib.conf.
3) Remote repo files follow naming repos-remote-ARCH.conf. If that
   does not exist, repos-remote.conf or repos-remote-musl.conf is
   used instead (fallback).
4) Cross follows the same behavior, just without multilib.

[ci skip]
2020-01-30 15:49:01 +01:00
q66 f0e852c511 xbps-src: implement per-architecture repocache
That means instead of having a single hostdir/repocache, there
is now hostdir/repocache-$XBPS_TARGET_MACHINE.

This solves multiple problems:

1) When cross-compiling for different architectures and the repos
   for host and target differ, you will not get signature/checksum
   conflicts for noarch packages.
2) The clean-repocache command will not delete noarch packages that
   don't belong to that architecture.
3) Clean mixing of glibc and musl masterdirs with the same hostdir
   is now possible; even when building natively, the musl masterdir
   will use its repocache-foo-musl, while the glibc masterdir will
   use repocache-foo.

[ci skip]
2020-01-28 18:25:39 +01:00
q66 4098c62ebc xbps-src: prevent builddir/destdir from always being removed
The problem here is that newly introduced behavior in commit
b2b0409be4 resulted in builddir and
destdir *always* being removed when starting a build, as a part of
masterdir auto-update. We don't want this as we may want to resume
a previous build (e.g. by running stages individually or by resuming
a failed build from where it stopped).

Therefore, explicitly override the removal to restore previous behavior.
2020-01-28 01:09:04 +01:00
Juan RP d94a15be8f
xbps-src: use XBPS_INSTALL_ARGS; allow custom xbps.d settings.
If `etc/xbps.d/custom` exists (directory) and contains
xbps.d(5) .conf files, those will be copied verbatim
to `masterdir/etc/xbps.d`.

This allows you to set your custom xbps.d(5) settings
without having to modify any other file.

This is also useful for testing xbps.d(5) noextract feature.
2020-01-26 10:41:12 +01:00
Duncaen 0ad3adfb0b
Revert "xbps-src/build.sh: always start with a clean statedir."
This reverts commit 2a959e32ee.
2020-01-13 23:34:00 +01:00
Jürgen Buchmüller 1111601672 common.sh: if $XBPS_SKIP_REMOTEREPOS rm *remote*
There may be /etc/xbps.d/*remote* config files dangling from a previous
run w/o XBPS_SKIP_REMOTEREPOS being set. Make sure to remove them
in case $XBPS_SKIP_REMOTEREPOS is set for this run.
2020-01-08 16:54:32 +01:00
Juan RP 2a959e32ee
xbps-src/build.sh: always start with a clean statedir.
Let's see if this fixes buildbot...
2020-01-05 13:02:43 +01:00
Juan RP 0ad1351c30
xbps-src/chroot.sh: repair "binary-bootstrap" for 32bit on x86_64. 2020-01-05 12:07:30 +01:00
Juan RP 12f676e6f2
xbps-src: install_pkg_deps: restore previous behaviour.
Don't msg_error if a dependency failed to build, maybe
due to broken or nocross.
2020-01-05 10:48:13 +01:00
Juan RP b2b0409be4 xbps-src/chroot.sh: simplify update_base_chroot(). 2020-01-05 10:44:56 +01:00
Juan RP b164856abd xbps-src/chroot.sh: silence grep output 2020-01-05 10:44:56 +01:00
Juan RP 5285cb9212 xbps-src: fix for x86_64 local multilib repos.
Found and fixed by @pullmoll
2020-01-05 10:44:56 +01:00
Juan RP 80b76c221a xbps-src: update_base_chroot(): remove autodeps after sysup. 2020-01-05 10:44:56 +01:00
Juan RP 817d3a8b7d chroot_init: return 0 at the end to not inherit last cmd errval. 2020-01-05 10:44:56 +01:00
Juan RP 75eca1b03e xbps-src: base-chroot-v2 (reduce artifacts/deps)
This patchset contains multiple changes to xbps-src and
its required package "base-chroot" for building packages
via chroot.

- moved xbps.d(5) conf files to `etc/xbps.d`.
- renamed xbps.d(5) repository files to `etc/xbps.d/repos-{local,remote}*`.
- do not set `--repository` to any xbps command that supports it,
  xbps-src now simply populates `rootdir/etc/xbps.d` with correct
  settings (taking care of CHROOT_READY/IN_CHROOT).
- Unless `-C` is set (to preserve builddir/destdir/autodeps), when
  entering to the chroot (if CHROOT_READY is set), xbps-src will
  clean up the masterdir and then perform a system update to always
  use a constant set of packages for that exact date.
- Improved some normal/error msgs.
- Includes support for `xbps>=0.58`.
- common/hooks: switch to bsdtar.
- base-chroot:
  - base-chroot-musl is gone, now unified for glibc/musl.
  - deps removed: gettext, mpfr, readline, texinfo, which, xz.
  - deps changed: tar -> bsdtar.

Effectively this reduces dependencies in `base-chroot`, makes
it unified for musl and glibc, switches xbps-src to use `bsdtar`
rather than GNU `tar` and `xz`, gets rid of useless host dependencies
like GNU gettext, texinfo, etc.

I've been testing these changes for 1 month or so already,
I was able to build from scratch `base-system` for both native
and multiple targets, i.e `./xbps-src -a target -Nt pkg base-system`
2020-01-05 10:44:56 +01:00
Piotr Wójcik 85f5e06fda xbps-src: don't install checkdepends for cross build 2020-01-02 23:15:41 +01:00
Juan RP 8c9aca6426
xbps-src: "build-options" obj is a string, no newlines accepted. 2019-12-29 20:19:54 +01:00
q66 7febf540a9 shutils/purge_distfiles.sh: fix purging of multiple distfiles
Previously, xbps-src would try to find checksums in template files
and not purge those found but it actually ignored everything that
had the checksums specified on multiple lines.

The reason for this is that the bash 'read' builtin reads until it
finds a newline, and if you specify multiple checksums on multiple
lines rather than on one line it would ignore all but the first.

Fix this by replacing newlines with spaces, so that it can properly
read all checksums into an array.
2019-12-23 00:53:29 +01:00
Juan RP f7412cdf8d
xbps-src: don't set -i to xbps-install(1).
As reported by @pullmoll via https://github.com/void-linux/xbps/issues/190
the culprit is this assignment, rather than in the xbps code.

Tested with local and remote repos, no regressions found.
2019-12-15 18:53:04 +01:00
q66 3a363d2611 xbps-src: account for nonfree repo with -E
This makes sure nonfree packages don't get rebuilt when -E is
passed to xbps-src.

[ci skip]
2019-12-14 11:26:36 +01:00
Stefano Ragni 3c8ad720a3 shutils/bulk.sh: use XBPS_DISTDIR when running xbps-src 2019-12-01 12:14:39 +01:00
q66 021b7cc11e xbps-src: revert to previous bulk_sortdeps code
This should restore sort-dependencies behavior back to its former
behavior of not including all of the build dependencies not in
the input list in its listing.
2019-11-19 02:04:25 +01:00
Duncaen 2a59e976c1
xbps-src/chroot.sh: don't set empty PROXY environment variables 2019-11-08 18:48:25 +01:00
John c1acedacaa xbps-src/shutils/cross.sh: error out if cross toolchain can't be compiled
$rval was the return code of pkg_available and not of build.sh, since
pkg_available returns 0. because of this install_cross_pkg() would
always return 0 for success.
2019-11-03 18:42:17 +01:00
Johannes 175b65f006 common: change wording in error message
build helper and build styles are now different concepts
2019-11-01 00:45:49 +01:00
Juan RP 23f75a842d
xbps-src/chroot.sh: fix previous correctly 2019-10-27 12:15:49 +01:00
Juan RP b469f1d590
xbps-src/chroot.sh: initialize XBPS_CONFIG_FILE.
Fixes #15208
2019-10-27 12:14:24 +01:00
John 46ac0aeb23 common/build_dependencies.sh: Disallow subpackages in makedepends 2019-10-26 00:44:22 +02:00
q66 0a38409da6 xbps-src: work around autodeps removal needing multiple passes 2019-10-25 23:53:38 +02:00
Duncaen 581ee5b7b2 xbps-src: add back support for 32bit dependencies 2019-10-21 18:37:53 +02:00
Andrew Benson 7acadabc6b xbps-src/chroot: pass proxy variables 2019-10-21 18:27:09 +02:00
John 1c16711b3f common/build_dependencies.sh: don't ignore subpkgs in host while cross
compiling, this falls (obviously) apart if it is missing.
See e.g. https://build.voidlinux.org/builders/aarch64-musl_builder/builds/22678/steps/shell_3/logs/stdio
2019-10-15 21:34:20 +03:00
John 188cb5c6d4 xbps-src/cross.sh: raise mimimum cross-vpkg-dummy version to 0.33_1 2019-09-28 11:30:13 +02:00
Piotr Wójcik 18a8a3c690 xbps-src: Actually unset subpkg functions 2019-08-27 19:26:31 +02:00
Jürgen Buchmüller f1a7dfbfef
common: fix XBPS_{,TARGET_}{ENDIAN,LIBC,WORDSIZE}
My first attempt was not working at all.

Signed-off-by: Jürgen Buchmüller <pullmoll@t-online.de>
2019-08-03 23:09:49 +02:00
Jürgen Buchmüller 6a1b977e0c Revert "profiles: introduce XBPS_{,TARGET_}{ENDIAN,LIBC}"
This reverts commit 9b3203401d.

It doesn't actually work. Use the method suggested by @q66 instead.
2019-08-03 23:09:36 +02:00
Jürgen Buchmüller 9b3203401d
profiles: introduce XBPS_{,TARGET_}{ENDIAN,LIBC}
Provide new environment variables for simplifaction of template checks:
...ENDIAN defines the (target) machine's endianness: "le" or "be".
...LIBC defines the (target) machine's C library: "glibc" or "musl".

Closes: #13356
Signed-off-by: Jürgen Buchmüller <pullmoll@t-online.de>
2019-08-02 17:34:05 +02:00
Jürgen Buchmüller ed4cdadcfd common/profiles: introduce XBPS_{,TARGET_}WORDSIZE
The host and target word size can be used to determine whether
cross compiling certain packages is possible.
2019-07-26 15:18:11 +02:00
Juan RP b79077c930
xbps-src: install cross pkg after checking dependencies.
... this should help buildbot a bit more.
2019-07-10 21:07:16 +02:00
Juan RP e4984d01ea xbps-src: multiple performance improvements
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the show-build-deps target is now much faster.
- the update-bulk/show-repo-updates targets are now much faster.
- the update-sys/show-sys-updates targets are now much faster.
- the bootstrap target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on #12433

Close #12433
Close #11282
2019-07-10 20:28:46 +02:00
John 506c2faf14 xbps-src/chroot.sh: reuse $PATH from outsude
This partially reverts commit e0e48d6f6a

This commit broke xbps-src when the xbps tools weren't in /usr/bin
2019-07-07 07:28:02 +00:00
Jürgen Buchmüller 17ba00b095 Revert "xbps-src: multiple performance improvements."
This reverts commit 2a4e178e35.
2019-07-05 12:08:45 +02:00
Jürgen Buchmüller 95beffd0f5 Revert "xbps-src: fix XBPS_CHECKVERS_XCMD."
This reverts commit 1fc6aea821.
2019-07-05 12:08:43 +02:00
Jürgen Buchmüller 7da591e1e4 Revert "xbps-src: improve bulk_sortdeps()."
This reverts commit 3b73edbaa5.
2019-07-05 12:08:42 +02:00
Jürgen Buchmüller 0eb55debee Revert "xbps-src: improve bulk code a bit more."
This reverts commit 4262450928.
2019-07-05 12:08:41 +02:00
Jürgen Buchmüller f5b502d3c5 Revert "xbps-src: bulk_build() use existing shell funcs."
This reverts commit 642cfa74c4.
2019-07-05 12:08:35 +02:00