Commit graph

447 commits

Author SHA1 Message Date
Juan RP
b2b0409be4 xbps-src/chroot.sh: simplify update_base_chroot(). 2020-01-05 10:44:56 +01:00
Juan RP
b164856abd xbps-src/chroot.sh: silence grep output 2020-01-05 10:44:56 +01:00
Juan RP
5285cb9212 xbps-src: fix for x86_64 local multilib repos.
Found and fixed by @pullmoll
2020-01-05 10:44:56 +01:00
Juan RP
80b76c221a xbps-src: update_base_chroot(): remove autodeps after sysup. 2020-01-05 10:44:56 +01:00
Juan RP
817d3a8b7d chroot_init: return 0 at the end to not inherit last cmd errval. 2020-01-05 10:44:56 +01:00
Juan RP
75eca1b03e xbps-src: base-chroot-v2 (reduce artifacts/deps)
This patchset contains multiple changes to xbps-src and
its required package "base-chroot" for building packages
via chroot.

- moved xbps.d(5) conf files to `etc/xbps.d`.
- renamed xbps.d(5) repository files to `etc/xbps.d/repos-{local,remote}*`.
- do not set `--repository` to any xbps command that supports it,
  xbps-src now simply populates `rootdir/etc/xbps.d` with correct
  settings (taking care of CHROOT_READY/IN_CHROOT).
- Unless `-C` is set (to preserve builddir/destdir/autodeps), when
  entering to the chroot (if CHROOT_READY is set), xbps-src will
  clean up the masterdir and then perform a system update to always
  use a constant set of packages for that exact date.
- Improved some normal/error msgs.
- Includes support for `xbps>=0.58`.
- common/hooks: switch to bsdtar.
- base-chroot:
  - base-chroot-musl is gone, now unified for glibc/musl.
  - deps removed: gettext, mpfr, readline, texinfo, which, xz.
  - deps changed: tar -> bsdtar.

Effectively this reduces dependencies in `base-chroot`, makes
it unified for musl and glibc, switches xbps-src to use `bsdtar`
rather than GNU `tar` and `xz`, gets rid of useless host dependencies
like GNU gettext, texinfo, etc.

I've been testing these changes for 1 month or so already,
I was able to build from scratch `base-system` for both native
and multiple targets, i.e `./xbps-src -a target -Nt pkg base-system`
2020-01-05 10:44:56 +01:00
Piotr Wójcik
85f5e06fda xbps-src: don't install checkdepends for cross build 2020-01-02 23:15:41 +01:00
Juan RP
8c9aca6426
xbps-src: "build-options" obj is a string, no newlines accepted. 2019-12-29 20:19:54 +01:00
q66
7febf540a9 shutils/purge_distfiles.sh: fix purging of multiple distfiles
Previously, xbps-src would try to find checksums in template files
and not purge those found but it actually ignored everything that
had the checksums specified on multiple lines.

The reason for this is that the bash 'read' builtin reads until it
finds a newline, and if you specify multiple checksums on multiple
lines rather than on one line it would ignore all but the first.

Fix this by replacing newlines with spaces, so that it can properly
read all checksums into an array.
2019-12-23 00:53:29 +01:00
Juan RP
f7412cdf8d
xbps-src: don't set -i to xbps-install(1).
As reported by @pullmoll via https://github.com/void-linux/xbps/issues/190
the culprit is this assignment, rather than in the xbps code.

Tested with local and remote repos, no regressions found.
2019-12-15 18:53:04 +01:00
q66
3a363d2611 xbps-src: account for nonfree repo with -E
This makes sure nonfree packages don't get rebuilt when -E is
passed to xbps-src.

[ci skip]
2019-12-14 11:26:36 +01:00
Stefano Ragni
3c8ad720a3 shutils/bulk.sh: use XBPS_DISTDIR when running xbps-src 2019-12-01 12:14:39 +01:00
q66
021b7cc11e xbps-src: revert to previous bulk_sortdeps code
This should restore sort-dependencies behavior back to its former
behavior of not including all of the build dependencies not in
the input list in its listing.
2019-11-19 02:04:25 +01:00
Duncaen
2a59e976c1
xbps-src/chroot.sh: don't set empty PROXY environment variables 2019-11-08 18:48:25 +01:00
John
c1acedacaa xbps-src/shutils/cross.sh: error out if cross toolchain can't be compiled
$rval was the return code of pkg_available and not of build.sh, since
pkg_available returns 0. because of this install_cross_pkg() would
always return 0 for success.
2019-11-03 18:42:17 +01:00
Johannes
175b65f006 common: change wording in error message
build helper and build styles are now different concepts
2019-11-01 00:45:49 +01:00
Juan RP
23f75a842d
xbps-src/chroot.sh: fix previous correctly 2019-10-27 12:15:49 +01:00
Juan RP
b469f1d590
xbps-src/chroot.sh: initialize XBPS_CONFIG_FILE.
Fixes #15208
2019-10-27 12:14:24 +01:00
John
46ac0aeb23 common/build_dependencies.sh: Disallow subpackages in makedepends 2019-10-26 00:44:22 +02:00
q66
0a38409da6 xbps-src: work around autodeps removal needing multiple passes 2019-10-25 23:53:38 +02:00
Duncaen
581ee5b7b2 xbps-src: add back support for 32bit dependencies 2019-10-21 18:37:53 +02:00
Andrew Benson
7acadabc6b xbps-src/chroot: pass proxy variables 2019-10-21 18:27:09 +02:00
John
1c16711b3f common/build_dependencies.sh: don't ignore subpkgs in host while cross
compiling, this falls (obviously) apart if it is missing.
See e.g. https://build.voidlinux.org/builders/aarch64-musl_builder/builds/22678/steps/shell_3/logs/stdio
2019-10-15 21:34:20 +03:00
John
188cb5c6d4 xbps-src/cross.sh: raise mimimum cross-vpkg-dummy version to 0.33_1 2019-09-28 11:30:13 +02:00
Piotr Wójcik
18a8a3c690 xbps-src: Actually unset subpkg functions 2019-08-27 19:26:31 +02:00
Jürgen Buchmüller
f1a7dfbfef
common: fix XBPS_{,TARGET_}{ENDIAN,LIBC,WORDSIZE}
My first attempt was not working at all.

Signed-off-by: Jürgen Buchmüller <pullmoll@t-online.de>
2019-08-03 23:09:49 +02:00
Jürgen Buchmüller
6a1b977e0c Revert "profiles: introduce XBPS_{,TARGET_}{ENDIAN,LIBC}"
This reverts commit 9b3203401d.

It doesn't actually work. Use the method suggested by @q66 instead.
2019-08-03 23:09:36 +02:00
Jürgen Buchmüller
9b3203401d
profiles: introduce XBPS_{,TARGET_}{ENDIAN,LIBC}
Provide new environment variables for simplifaction of template checks:
...ENDIAN defines the (target) machine's endianness: "le" or "be".
...LIBC defines the (target) machine's C library: "glibc" or "musl".

Closes: #13356
Signed-off-by: Jürgen Buchmüller <pullmoll@t-online.de>
2019-08-02 17:34:05 +02:00
Jürgen Buchmüller
ed4cdadcfd common/profiles: introduce XBPS_{,TARGET_}WORDSIZE
The host and target word size can be used to determine whether
cross compiling certain packages is possible.
2019-07-26 15:18:11 +02:00
Juan RP
b79077c930
xbps-src: install cross pkg after checking dependencies.
... this should help buildbot a bit more.
2019-07-10 21:07:16 +02:00
Juan RP
e4984d01ea xbps-src: multiple performance improvements
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the show-build-deps target is now much faster.
- the update-bulk/show-repo-updates targets are now much faster.
- the update-sys/show-sys-updates targets are now much faster.
- the bootstrap target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on #12433

Close #12433
Close #11282
2019-07-10 20:28:46 +02:00
John
506c2faf14 xbps-src/chroot.sh: reuse $PATH from outsude
This partially reverts commit e0e48d6f6a

This commit broke xbps-src when the xbps tools weren't in /usr/bin
2019-07-07 07:28:02 +00:00
Jürgen Buchmüller
17ba00b095 Revert "xbps-src: multiple performance improvements."
This reverts commit 2a4e178e35.
2019-07-05 12:08:45 +02:00
Jürgen Buchmüller
95beffd0f5 Revert "xbps-src: fix XBPS_CHECKVERS_XCMD."
This reverts commit 1fc6aea821.
2019-07-05 12:08:43 +02:00
Jürgen Buchmüller
7da591e1e4 Revert "xbps-src: improve bulk_sortdeps()."
This reverts commit 3b73edbaa5.
2019-07-05 12:08:42 +02:00
Jürgen Buchmüller
0eb55debee Revert "xbps-src: improve bulk code a bit more."
This reverts commit 4262450928.
2019-07-05 12:08:41 +02:00
Jürgen Buchmüller
f5b502d3c5 Revert "xbps-src: bulk_build() use existing shell funcs."
This reverts commit 642cfa74c4.
2019-07-05 12:08:35 +02:00
Juan RP
642cfa74c4 xbps-src: bulk_build() use existing shell funcs.
... rather than forking xbps-src again.

Closes: #12797 [via git-merge-pr]
2019-07-05 08:53:52 +02:00
Juan RP
4262450928 xbps-src: improve bulk code a bit more.
- use `show-avail` to discard broken pkgs.
- the parallel logic was incomplete.
- avoid unnecessary sorting.
2019-07-05 08:53:52 +02:00
Juan RP
3b73edbaa5 xbps-src: improve bulk_sortdeps().
./xbps-src sort-dependencies $(./xbps-src show-build-deps xbps)

0.202s vs 0.514s
2019-07-05 08:53:52 +02:00
Juan RP
1fc6aea821 xbps-src: fix XBPS_CHECKVERS_XCMD.
Needs to set -r $XBPS_CROSS_BASE to use correct dbdir.
2019-07-05 08:53:51 +02:00
Juan RP
2a4e178e35 xbps-src: multiple performance improvements.
- use xbps-checkvers(1) to resolve dependencies.
- all dependencies are installed at once for the host and target.
- the `show-build-deps` target is now much faster.
- the `update-bulk/show-repo-updates` targets are now much faster.
- the `update-sys/show-sys-updates` targets are now much faster.
- the `bootstrap` target now works on musl hosts.
- simplified some loops.
- use cut(1) rather than awk(1) where applicable.
- multiple random changes to improve performance.

Based on work started by @Duncaen on https://github.com/void-linux/void-packages/pull/12433

Close https://github.com/void-linux/void-packages/pull/12433
Close https://github.com/void-linux/void-packages/pull/11282
2019-07-05 08:53:51 +02:00
Piotr Wójcik
c3cfeb56e8 update-check: find versions with more components 2019-07-03 00:36:00 +02:00
Juan RP
e0e48d6f6a
xbps-src/chroot.sh: sanitize PATH. 2019-07-02 19:14:56 +02:00
Juan RP
b52aae3114
xbps-src: do not ignore EEXIST rval from xbps-install(1).
Due to -I it cannot return EEXIST, and this error value shall
be treated as a real error.

Does not change the current behaviour but will do if we drop -I.
2019-06-26 12:52:59 +02:00
Juan RP
e2f098b972
xbps-src: print detected file conflicts. 2019-06-21 18:46:14 +02:00
Juan RP
2127bd2e00
xbps-src: require xbps>=0.55.
Build dependencies are now installed with -I, --ignore-file-conflicts.
A new option introduced in xbps-0.55 that just prints
conflicting files but does not error out.

Switch to xbps-fetch and xbps-digest.
2019-06-21 17:21:38 +02:00
Juan RP
d778cbdc70
Revert "xbps-src: require xbps>=0.54."
This reverts commit 198acb8e39.

I'm reverting this until I fix the issue properly.
2019-06-21 13:04:43 +02:00
Juan RP
198acb8e39
xbps-src: require xbps>=0.54.
if xbps-install(1) returns EEXIST, treat it as an error,
do not ignore it.

xbps-install(1) now returns only EEXIST for file conflicts
in transaction, so that it should not be ignored.
2019-06-21 11:47:15 +02:00
Duncaen
0639ff03da
xbps-src: simplify set_build_options 2019-06-19 11:33:54 +02:00
Duncaen
dd9c7a967e
xbps-src: less forking in get_subpkgs and unset_package_funcs 2019-06-19 11:30:22 +02:00
Duncaen
f1ec7fac9f
xbps-src: open $XBPS_MASTERDIR/etc/xbps/xbps-src.conf only once 2019-06-19 11:15:26 +02:00
Duncaen
6d3c739c9a
xbps-src: use substitutions instead of sed for xbps-shell 2019-06-19 11:12:29 +02:00
Duncaen
723170541d
xbps-src: don't check if base-chroot is installed for every single sourced template 2019-06-19 11:08:39 +02:00
Juan RP
7b7d7c13a1
xbps-src: exit properly if mktemp(1) fails, not from its subshell.
Thanks @huglovefan
2019-06-17 17:38:07 +02:00
Juan RP
f59b295903 xbps-src: cleaned up usage of mktemp(1).
This one was on my TODO for a long while... fixes unportable
arguments (checked with the NetBSD's manual page).
2019-06-17 15:23:20 +02:00
Doan Tran Cong Danh
37ed8cb67b common: add BUILD_{CXX,CPP,FC,LD} flags 2019-06-15 16:31:53 +02:00
Juan RP
d18cd13b16 xbps-src/cross.sh: require cross-vpkg-dummy>=0.30.
Signed-off-by: Juan RP <xtraeme@gmail.com>
2019-06-13 19:35:05 +02:00
Piotr Wójcik
b8039ef838 update-check: download each url once 2019-06-12 18:06:27 +02:00
Piotr Wójcik
95e3a60feb update-check: recognize all gitlab instances 2019-06-01 14:23:37 +02:00
Piotr Wójcik
e3e08a087a update-check: skip version directories checking when specialized check exists 2019-05-14 22:48:46 +02:00
Piotr Wójcik
7ae9417947 update-check: allow to disable checking version directories 2019-05-14 22:48:46 +02:00
Piotr Wójcik
906f339d6a update-check: generalize checking version directories 2019-05-14 22:48:46 +02:00
Piotr Wójcik
79273c05d9 update-check: only check newer directories 2019-05-14 22:48:46 +02:00
Piotr Wójcik
bb0b85060c update-check: never omit specified site 2019-05-14 22:48:46 +02:00
Piotr Wójcik
795902b028 update-check: add crates.io 2019-05-14 07:48:39 +02:00
John
edad3f9f9f
xbps-src/build-dependencies.sh: check for instead of 2019-04-17 16:44:19 +02:00
John
9ae512bbbb
xbps-src: only detect host build loops when not cross compiling 2019-04-17 16:39:23 +02:00
maxice8
8fff9cee19 update_hash_cache.sh: avoid useless use of cat 2019-04-15 12:18:00 -03:00
Juan RP
35a8e8c6b6 xbps-src: detect two types of build loops and abort()
- depends on itself
- dep depends on targetpkg
2019-04-15 10:54:04 -03:00
maxice8
9c39291c28
consistency_check.sh: fix dirname 2019-04-14 09:36:15 -03:00
maxice8
372ae10639 consistency_check.sh: remove dirname usage 2019-04-14 06:22:09 -03:00
Juan RP
1ee6a3174f xbps-src: reduce basename(1) usage.
This saves a few exec()s.
2019-04-14 01:47:16 -03:00
q66
3c47efa808 common: expose build profile in cross env
[ci skip]
2019-04-09 19:26:59 -03:00
maxice8
7416477a64 purge_distfiles.sh: forcefully remove files 2019-04-09 19:19:50 -03:00
yopito
8944a9130e
xbps-src: update_check.sh: check distfile existence for kde
* KDE distfile storage is based on versionned "folder".
An existing folder_version does not mean that the corresponding
distifile exists, so let's check its existence.

* wider scope for KDE urls that will use that rule.
Fine for plenty other packages, except kdb that don't follow
this hierarchy (yet)
2019-04-09 17:45:44 +02:00
John
01d11a4938 build-style/cmake: set CMAKE_CROSSCOMPILING_EMULATOR 2019-04-05 23:04:14 +02:00
maxice8
fadca94b98 common: add chroot-style 'ethereal' 2019-04-05 05:49:34 -03:00
maxice8
b95a49b243
meta: remove last bits. 2019-03-14 19:25:00 -03:00
maxice8
a5bc81adf7 meta: remove supporting code for noarch 2019-03-14 19:22:06 -03:00
maxice8
166b70f35b meta: also run do-$step_name hooks when run_step is called. 2019-03-03 13:08:23 -03:00
maxice8
79172bd8dc
meta: also run actions in chroot when calling for patch phase. 2019-03-03 12:16:41 -03:00
maxice8
f683b0aa08
meta: fix build-class -> build-helper 2019-02-26 17:28:12 -03:00
Piotr Wójcik
2a0c3434fb xbps-src: update-check: switch sourceforge to https 2019-02-18 14:16:52 -03:00
maxice8
c8129e7d38 shutils/pkgtarget.sh: fix check for noarch 2019-02-15 12:14:53 -02:00
Enno Boland
38c457e1aa add compatibility code for noarch/only_for_archs 2019-02-15 13:19:44 +01:00
Enno Boland
6eb37e35b2 common: merge only_for_archs and noarch=yes into one.
* noarch=yes is replaced with archs=noarch
* only_for_archs= is renamed to archs=
* archs= allows the use of wildcards and negations; first matching rule applies:
  * archs="*-musl" will build the pkg only for musl-libcs
  * archs="~*-musl" will build the pkg only on non-musl-libc
  * archs="x86_64-musl ~*-musl" will build for x86_64-musl and any non-musl
    arch.
* archs= defaults to "*"
2019-02-15 13:19:44 +01:00
Johannes
026ebc069e common: unset rust-cross specific variables for native builds 2019-02-08 19:16:39 +01:00
maxice8
5850b188b4 shutils/build_dependencies.sh: also show build_helper that are set. 2019-01-27 09:15:17 +01:00
Cameron Nemo
2b569f1ef8 xbps-src: add show host/makedepends subcommand 2019-01-26 21:41:10 -02:00
maxice8
24efa877cd
shutils/common.sh: Remove variables that are now in build-helper/rust.sh 2019-01-25 20:44:42 -02:00
maxice8
a4f786276a
shutils/show.sh: add support for showing build_helper 2019-01-25 20:44:41 -02:00
maxice8
e30a9c8908
shutils/common.sh: add support for source build-helper files 2019-01-25 20:44:41 -02:00
maxice8
027cdb5bb8 shutils/common.sh: export CARGO_BUILD_TARGET 2019-01-20 19:41:18 -02:00
maxice8
54040e4b1b shutils/common.sh: define CARGO_BUILD_JOBS and
CARGO_TARGE_${RUST_TAGRGET}_LINKER.
2019-01-20 15:33:10 -02:00
maxice8
1fcdcdd959
shutils/common.sh: fix typo 2019-01-19 08:03:10 -02:00
maxice8
57ce5bc8e5 consistency_check.sh: use return instead of continue when appropiate.
Those are not meant to be continue as they are not in a loop they are in
a subshell.
2019-01-18 20:00:54 -02:00
maxice8
9964ac65d3 common.sh: don't export XBPS_GCC_* 2019-01-18 05:42:18 -02:00
Piotr Wójcik
9c8ef3d6a9 update-check: handle python3-only modules from pypi 2019-01-07 18:28:36 -02:00
Piotr Wójcik
1bbbd55871 update-check: add gitlab.freedesktop.org 2018-12-08 19:54:15 +01:00