Alexandre Oliva
2018-06-26 05:38:32 UTC
1. address the previously-mentioned fragility in the patch I posted, to
catch all cases of postbootstrap targets and their deps on
non-postbootstrap targets.
This turned out to just require some thinking to convince myself itcatch all cases of postbootstrap targets and their deps on
non-postbootstrap targets.
wouldn't come up.
There was a major problem in the earlier patch, however: @if/@endif
gcc-no-bootstrap wasn't quite what we needed to enclose the preexisting
deps, because that works for cases in which gcc is built but not
bootstrapped, but not cases in which gcc is not built. I had to
introduce @unless/@endunless to express the desired semantics.
Here's the patch I'll install if nobody objects in the next few days.
Tested on x86_64-linux-gnu with a gcc bootstrap tree, a gcc
non-bootstrap tree, and a binutils+gdb tree.
In the patch below, I've omitted hunks with only whitespace changes to
Makefile.in, so that people can more easily identify how rules are
changing.
Introduce @unless/@endunless and postbootstrap Makefile targets
From: Alexandre Oliva <***@adacore.com>
This patch turns dependencies of non-bootstrap targets on bootstrap
targets for bootstrap builds into dependencies on stage_last. This
arrangement gets stage1-bubble to run from stage_last if we haven't
started a bootstrap yet, and to use the current stage otherwise. This
was already the case of target libs, just not of non-bootstrapped host
modules.
In order to retain preexisting dependencies in non-bootstrap builds,
or in gcc-less builds, this introduces support for @unless/@endunless
pairs in Makefile.in.
There is a remaining possibility of problem if activating, in a tree
configured for bootstrap, a parallel build of two or more modules, at
least one bootstrapped and one not. In this case, make might decide
to build stage_current and stage_last in parallel, the latter will
start a submake to build stage1 while the initial make, having
satisfied stage_current, proceeds to build the bootstrapped module in
non-bootstrapped configurations. The two builds will overlap and will
likely conflict. This situation does NOT arise in normal settings,
however: a post-bootstrap build of all-host all-target will indeed
activate such targets concurrently, but only after building all
bootstrapped modules successfully, and it will have both stage_last
and stage_current targets already satisfied, so the potential race
between builds will not arise.
Another remaining problem, that is slightly expanded with this patch,
is that of an interrupted build in a tree configured for bootstrap,
continued with a non-bootstrapped target. Target modules that were
not bootstrapped would already fail to complete the current stage when
activated explicitly in the command line for a retry; host modules,
however, would attempt to build their bootstrapped dependencies, which
is what led to the problem of concurrent builds addressed with this
patch. An interrupted or failed build might still recover correctly,
if the non-bootstrapped target is activated in both builds, because
then make will remove stage_last when its build command is
interrupted, so that it will attempt to recreate it with stage1-bubble
in the second try. A bootstrap build, however, will not be attempting
to build stage_last, so the file will remain and the retry won't go
through stage1-bubble. We have lived with that for target modules, so
we can probably live with that for host modules too.
Another undesirable consequence of this change is that non-boostrapped
host modules, in a tree configured for bootstrap, when activated as
make all-<module>, will build all of stage1 instead of only the
module's usual dependencies. This is intentional and necessary to fix
the parallel-build problem. If it's not desirable, disabling the
unnecessary bootstrap configuration will suffice to restore the
original set of dependencies.
for ChangeLog
* configure.ac: Introduce support for @unless/@endunless.
* Makefile.tpl (dep-kind): Rewrite with cond; return
postbootstrap in some cases.
(make-postboot-dep, postboot-targets): New.
(dependencies): Do not output postbootstrap dependencies at
first. Output non-target ones changed for configure to depend
on stage_last @if gcc-bootstrap, and the original deps @unless
gcc-bootstrap.
* configure.in, Makefile.in: Rebuilt.
---
Makefile.in | 181 +++++++++++++++++-----------------------------------------
Makefile.tpl | 78 +++++++++++++++++--------
configure | 20 +++++-
configure.ac | 20 +++++-
4 files changed, 134 insertions(+), 165 deletions(-)
diff --git a/Makefile.in b/Makefile.in
index 32a92a6bcd17..e0dfad337a6c 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -56060,9 +56018,7 @@ all-stagefeedback-fixincludes: maybe-all-stagefeedback-libiberty
all-stageautoprofile-fixincludes: maybe-all-stageautoprofile-libiberty
all-stageautofeedback-fixincludes: maybe-all-stageautofeedback-libiberty
all-gnattools: maybe-all-target-libada
-all-gnattools: maybe-all-target-libstdc++-v3
all-lto-plugin: maybe-all-libiberty
-
all-stage1-lto-plugin: maybe-all-stage1-libiberty
all-stage2-lto-plugin: maybe-all-stage2-libiberty
all-stage3-lto-plugin: maybe-all-stage3-libiberty
@@ -56083,12 +56038,8 @@ all-stagetrain-lto-plugin: maybe-all-stagetrain-libiberty-linker-plugin
all-stagefeedback-lto-plugin: maybe-all-stagefeedback-libiberty-linker-plugin
all-stageautoprofile-lto-plugin: maybe-all-stageautoprofile-libiberty-linker-plugin
all-stageautofeedback-lto-plugin: maybe-all-stageautofeedback-libiberty-linker-plugin
-configure-libcc1: maybe-configure-gcc
-all-libcc1: maybe-all-gcc
all-gotools: maybe-all-target-libgo
-all-utils: maybe-all-libiberty
configure-intl: maybe-all-libiconv
-
configure-stage1-intl: maybe-all-stage1-libiconv
configure-stage2-intl: maybe-all-stage2-libiconv
configure-stage3-intl: maybe-all-stage3-libiconv
@@ -56142,17 +56089,10 @@ all-stagetrain-intl: maybe-all-stagetrain-libiconv
all-stagefeedback-intl: maybe-all-stagefeedback-libiconv
all-stageautoprofile-intl: maybe-all-stageautoprofile-libiconv
all-stageautofeedback-intl: maybe-all-stageautofeedback-libiconv
-configure-gdb: maybe-all-intl
configure-gdb: maybe-configure-sim
-configure-gdb: maybe-all-bfd
-configure-gdb: maybe-all-libiconv
-all-gdb: maybe-all-libiberty
-all-gdb: maybe-all-libiconv
-all-gdb: maybe-all-opcodes
all-gdb: maybe-all-readline
all-gdb: maybe-all-build-bison
all-gdb: maybe-all-sim
-all-gdb: maybe-all-libdecnumber
all-gdb: maybe-all-libtermcap
configure-libgui: maybe-configure-tcl
configure-libgui: maybe-configure-tk
@@ -56383,14 +56303,7 @@ all-stagetrain-gas: maybe-all-stagetrain-intl
all-stagefeedback-gas: maybe-all-stagefeedback-intl
all-stageautoprofile-gas: maybe-all-stageautoprofile-intl
all-stageautofeedback-gas: maybe-all-stageautofeedback-intl
-configure-gprof: maybe-configure-intl
-all-gprof: maybe-all-libiberty
-all-gprof: maybe-all-bfd
-all-gprof: maybe-all-opcodes
-all-gprof: maybe-all-intl
-all-gprof: maybe-all-gas
configure-ld: maybe-configure-intl
-
configure-stage1-ld: maybe-configure-stage1-intl
configure-stage2-ld: maybe-configure-stage2-intl
configure-stage3-ld: maybe-configure-stage3-intl
@@ -56637,42 +56530,27 @@ install-itcl: maybe-install-tcl
install-strip-itcl: maybe-install-strip-tcl
configure-tk: maybe-configure-tcl
all-tk: maybe-all-tcl
-all-sid: maybe-all-libiberty
-all-sid: maybe-all-bfd
-all-sid: maybe-all-opcodes
all-sid: maybe-all-tcl
all-sid: maybe-all-tk
install-sid: maybe-install-tcl
install-strip-sid: maybe-install-strip-tcl
install-sid: maybe-install-tk
install-strip-sid: maybe-install-strip-tk
-configure-sim: maybe-configure-intl
-all-sim: maybe-all-intl
-all-sim: maybe-all-libiberty
-all-sim: maybe-all-bfd
-all-sim: maybe-all-opcodes
all-sim: maybe-all-readline
all-sim: maybe-configure-gdb
-all-fastjar: maybe-all-zlib
all-fastjar: maybe-all-build-texinfo
-all-fastjar: maybe-all-libiberty
-all-bison: maybe-all-intl
all-bison: maybe-all-build-texinfo
all-flex: maybe-all-build-bison
-all-flex: maybe-all-intl
all-flex: maybe-all-m4
all-flex: maybe-all-build-texinfo
-all-m4: maybe-all-intl
all-m4: maybe-all-build-texinfo
configure-target-fastjar: maybe-configure-target-zlib
all-target-fastjar: maybe-all-target-zlib
configure-target-libgo: maybe-configure-target-libffi
-configure-target-libgo: maybe-all-target-libstdc++-v3
all-target-libgo: maybe-all-target-libbacktrace
all-target-libgo: maybe-all-target-libffi
all-target-libgo: maybe-all-target-libatomic
configure-target-libstdc++-v3: maybe-configure-target-libgomp
-
configure-stage1-target-libstdc++-v3: maybe-configure-stage1-target-libgomp
configure-stage2-target-libstdc++-v3: maybe-configure-stage2-target-libgomp
configure-stage3-target-libstdc++-v3: maybe-configure-stage3-target-libgomp
@@ -56682,9 +56560,7 @@ configure-stagetrain-target-libstdc++-v3: maybe-configure-stagetrain-target-libg
configure-stagefeedback-target-libstdc++-v3: maybe-configure-stagefeedback-target-libgomp
configure-stageautoprofile-target-libstdc++-v3: maybe-configure-stageautoprofile-target-libgomp
configure-stageautofeedback-target-libstdc++-v3: maybe-configure-stageautofeedback-target-libgomp
-configure-target-liboffloadmic: maybe-configure-target-libgomp
configure-target-libsanitizer: maybe-all-target-libstdc++-v3
-
configure-stage1-target-libsanitizer: maybe-all-stage1-target-libstdc++-v3
configure-stage2-target-libsanitizer: maybe-all-stage2-target-libstdc++-v3
configure-stage3-target-libsanitizer: maybe-all-stage3-target-libstdc++-v3
@@ -56716,7 +56590,6 @@ all-stagetrain-target-libstdc++-v3: maybe-configure-stagetrain-target-libgomp
all-stagefeedback-target-libstdc++-v3: maybe-configure-stagefeedback-target-libgomp
all-stageautoprofile-target-libstdc++-v3: maybe-configure-stageautoprofile-target-libgomp
all-stageautofeedback-target-libstdc++-v3: maybe-configure-stageautofeedback-target-libgomp
-all-target-liboffloadmic: maybe-all-target-libgomp
install-target-libgo: maybe-install-target-libatomic
install-target-libgfortran: maybe-install-target-libquadmath
install-target-libgfortran: maybe-install-target-libgcc
@@ -56731,12 +56604,62 @@ install-target-libobjc: maybe-install-target-libgcc
install-target-libstdc++-v3: maybe-install-target-libgcc
all-target-libgloss: maybe-all-target-newlib
all-target-winsup: maybe-all-target-libtermcap
-configure-target-newlib: maybe-all-binutils
-configure-target-newlib: maybe-all-ld
configure-target-libgfortran: maybe-all-target-libquadmath
configure-target-libgfortran: maybe-all-target-libbacktrace
+@if gcc-bootstrap
+configure-gnattools: stage_last
+configure-libcc1: stage_last
+configure-utils: stage_last
+configure-gdb: stage_last
+configure-gprof: stage_last
+configure-sid: stage_last
+configure-sim: stage_last
+configure-fastjar: stage_last
+configure-bison: stage_last
+configure-flex: stage_last
+configure-m4: stage_last
+@endif gcc-bootstrap
+
+@unless gcc-bootstrap
+all-gnattools: maybe-all-target-libstdc++-v3
+configure-libcc1: maybe-configure-gcc
+all-libcc1: maybe-all-gcc
+all-utils: maybe-all-libiberty
+configure-gdb: maybe-all-intl
+configure-gdb: maybe-all-bfd
+configure-gdb: maybe-all-libiconv
+all-gdb: maybe-all-libiberty
+all-gdb: maybe-all-libiconv
+all-gdb: maybe-all-opcodes
+all-gdb: maybe-all-libdecnumber
+configure-gprof: maybe-configure-intl
+all-gprof: maybe-all-libiberty
+all-gprof: maybe-all-bfd
+all-gprof: maybe-all-opcodes
+all-gprof: maybe-all-intl
+all-gprof: maybe-all-gas
+all-sid: maybe-all-libiberty
+all-sid: maybe-all-bfd
+all-sid: maybe-all-opcodes
+configure-sim: maybe-configure-intl
+all-sim: maybe-all-intl
+all-sim: maybe-all-libiberty
+all-sim: maybe-all-bfd
+all-sim: maybe-all-opcodes
+all-fastjar: maybe-all-zlib
+all-fastjar: maybe-all-libiberty
+all-bison: maybe-all-intl
+all-flex: maybe-all-intl
+all-m4: maybe-all-intl
+configure-target-libgo: maybe-all-target-libstdc++-v3
+configure-target-liboffloadmic: maybe-configure-target-libgomp
+all-target-liboffloadmic: maybe-all-target-libgomp
+configure-target-newlib: maybe-all-binutils
+configure-target-newlib: maybe-all-ld
+@endunless gcc-bootstrap
+
# Dependencies for target modules on other target modules are
# described by lang_env_dependencies; the defaults apply to anything
# not mentioned there.
diff --git a/Makefile.tpl b/Makefile.tpl
index b6077c9f125c..447d324595fe 100644
--- a/Makefile.tpl
+++ b/Makefile.tpl
@@ -1828,25 +1828,46 @@ configure-target-[+module+]: maybe-all-gcc[+
(define dep-maybe (lambda ()
(if (exist? "hard") "" "maybe-")))
- ;; dep-kind returns "normal" if the dependency is on an "install" target,
- ;; or if either module is not bootstrapped. It returns "bootstrap" for
- ;; configure or build dependencies between bootstrapped modules; it returns
- ;; "prebootstrap" for configure or build dependencies of bootstrapped
- ;; modules on a build module (e.g. all-gcc on all-build-bison). All this
- ;; is only necessary for host modules.
+ ;; dep-kind returns returns "prebootstrap" for configure or build
+ ;; dependencies of bootstrapped modules on a build module
+ ;; (e.g. all-gcc on all-build-bison); "normal" if the dependency is
+ ;; on an "install" target, or if the dependence module is not
+ ;; bootstrapped; otherwise, it returns "bootstrap" or
+ ;; "postbootstrap" depending on whether the dependent module is
+ ;; bootstrapped. All this is only necessary for host and target
+ ;; modules. It might seem like, in order to avoid build races, we
+ ;; might need more elaborate detection between prebootstrap and
+ ;; postbootstrap modules, but there are no host prebootstrap
+ ;; modules. If there were any non-bootstrap host modules that
+ ;; bootstrap modules depended on, we'd get unsatisfied per-stage
+ ;; dependencies on them, which would be immediately noticed.
(define dep-kind (lambda ()
- (if (and (hash-ref boot-modules (dep-module "module"))
- (=* (dep-module "on") "build-"))
- "prebootstrap"
+ (cond
+ ((and (hash-ref boot-modules (dep-module "module"))
+ (=* (dep-module "on") "build-"))
+ "prebootstrap")
- (if (or (= (dep-subtarget "on") "install-")
- (not (hash-ref boot-modules (dep-module "module")))
- (not (hash-ref boot-modules (dep-module "on"))))
- "normal"
- "bootstrap"))))
+ ((or (= (dep-subtarget "on") "install-")
+ (not (hash-ref boot-modules (dep-module "on"))))
+ "normal")
+
+ ((hash-ref boot-modules (dep-module "module"))
+ "bootstrap")
+
+ (1 "postbootstrap"))))
+
+ (define make-postboot-dep (lambda ()
+ (let ((target (dep-module "module")) (dep "stage_last"))
+ (unless (= (hash-ref postboot-targets target) dep)
+ (hash-create-handle! postboot-targets target dep)
+ ;; All non-bootstrap modules' configure target already
+ ;; depend on dep.
+ (unless (=* target "target-")
+ (string-append "configure-" target ": " dep "\n"))))))
;; We now build the hash table that is used by dep-kind.
(define boot-modules (make-hash-table 113))
+ (define postboot-targets (make-hash-table 113))
+]
[+ FOR host_modules +][+
@@ -1863,18 +1884,23 @@ configure-target-[+module+]: maybe-all-gcc[+
# to check for bootstrap/prebootstrap dependencies. To resolve
# prebootstrap dependencies, prebootstrap modules are gathered in
# a hash table.
-[+ FOR dependencies +][+ (make-dep "" "") +]
-[+ CASE (dep-kind) +]
-[+ == "prebootstrap"
- +][+ FOR bootstrap_stage +]
-[+ (make-dep (dep-stage) "") +][+
- ENDFOR bootstrap_stage +]
-[+ == "bootstrap"
- +][+ FOR bootstrap_stage +]
-[+ (make-dep (dep-stage) (dep-stage)) +][+
- ENDFOR bootstrap_stage +]
-[+ ESAC +][+
-ENDFOR dependencies +]
+[+ FOR dependencies +][+ CASE (dep-kind) +]
+[+ == "prebootstrap" +][+ (make-dep "" "") +][+ FOR bootstrap_stage +]
+[+ (make-dep (dep-stage) "") +][+ ENDFOR bootstrap_stage +]
+[+ == "bootstrap" +][+ (make-dep "" "") +][+ FOR bootstrap_stage +]
+[+ (make-dep (dep-stage) (dep-stage)) +][+ ENDFOR bootstrap_stage +]
+[+ == "normal" +][+ (make-dep "" "") +]
+[+ ESAC +][+ ENDFOR dependencies +]
+
+@if gcc-bootstrap
+[+ FOR dependencies +][+ CASE (dep-kind) +]
+[+ == "postbootstrap" +][+ (make-postboot-dep) +][+ ESAC +][+
+ENDFOR dependencies +]@endif gcc-bootstrap
+
+@unless gcc-bootstrap
+[+ FOR dependencies +][+ CASE (dep-kind) +]
+[+ == "postbootstrap" +][+ (make-dep "" "") +]
+[+ ESAC +][+ ENDFOR dependencies +]@endunless gcc-bootstrap
# Dependencies for target modules on other target modules are
# described by lang_env_dependencies; the defaults apply to anything
diff --git a/configure b/configure
index fe8e0f63410c..dd9fbe4d3229 100755
--- a/configure
+++ b/configure
@@ -7130,7 +7130,9 @@ INSTALL_GDB_TK=`echo ${GDB_TK} | sed s/-all-/-install-/g`
# extrasub-{build,host,target} not because there is any reason to split
# the substitutions up that way, but only to remain below the limit of
# 99 commands in a script, for HP-UX sed.
-# Do not nest @if/@endif pairs, because configure will not warn you at all.
+
+# Do not nest @if/@endif or @unless/@endunless pairs, because
+# configure will not warn you at all.
case "$enable_bootstrap:$ENABLE_GOLD: $configdirs :,$stage1_languages," in
yes:yes:*\ gold\ *:*,c++,*) ;;
@@ -7149,8 +7151,10 @@ for module in ${build_configdirs} ; do
extrasub_build="$extrasub_build
/^@if build-$module\$/d
/^@endif build-$module\$/d
+/^@unless build-$module\$/,/^@endunless build-$module\$/d
/^@if build-$module-$bootstrap_suffix\$/d
-/^@endif build-$module-$bootstrap_suffix\$/d"
+/^@endif build-$module-$bootstrap_suffix\$/d
+/^@unless build-$module-$bootstrap_suffix\$/,/^@endunless build-$module-$bootstrap_suffix\$/d"
done
extrasub_host=
for module in ${configdirs} ; do
@@ -7169,8 +7173,10 @@ for module in ${configdirs} ; do
extrasub_host="$extrasub_host
/^@if $module\$/d
/^@endif $module\$/d
+/^@unless $module\$/,/^@endunless $module\$/d
/^@if $module-$host_bootstrap_suffix\$/d
-/^@endif $module-$host_bootstrap_suffix\$/d"
+/^@endif $module-$host_bootstrap_suffix\$/d
+/^@unless $module-$host_bootstrap_suffix\$/,/^@endunless $module-$host_bootstrap_suffix\$/d"
done
extrasub_target=
for module in ${target_configdirs} ; do
@@ -7189,13 +7195,17 @@ for module in ${target_configdirs} ; do
extrasub_target="$extrasub_target
/^@if target-$module\$/d
/^@endif target-$module\$/d
+/^@unless target-$module\$/,/^@endunless target-$module\$/d
/^@if target-$module-$target_bootstrap_suffix\$/d
-/^@endif target-$module-$target_bootstrap_suffix\$/d"
+/^@endif target-$module-$target_bootstrap_suffix\$/d
+/^@unless target-$module-$target_bootstrap_suffix\$/,/^@endunless target-$module-$target_bootstrap_suffix\$/d"
done
# Do the final fixup along with target modules.
extrasub_target="$extrasub_target
-/^@if /,/^@endif /d"
+/^@if /,/^@endif /d
+/^@unless /d
+/^@endunless /d"
# Create the serialization dependencies. This uses a temporary file.
diff --git a/configure.ac b/configure.ac
index 28155a0e5938..a0b0917dd556 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2719,7 +2719,9 @@ INSTALL_GDB_TK=`echo ${GDB_TK} | sed s/-all-/-install-/g`
# extrasub-{build,host,target} not because there is any reason to split
# the substitutions up that way, but only to remain below the limit of
# 99 commands in a script, for HP-UX sed.
-# Do not nest @if/@endif pairs, because configure will not warn you at all.
+
+# Do not nest @if/@endif or @unless/@endunless pairs, because
+# configure will not warn you at all.
case "$enable_bootstrap:$ENABLE_GOLD: $configdirs :,$stage1_languages," in
yes:yes:*\ gold\ *:*,c++,*) ;;
@@ -2738,8 +2740,10 @@ for module in ${build_configdirs} ; do
extrasub_build="$extrasub_build
/^@if build-$module\$/d
/^@endif build-$module\$/d
+/^@unless build-$module\$/,/^@endunless build-$module\$/d
/^@if build-$module-$bootstrap_suffix\$/d
-/^@endif build-$module-$bootstrap_suffix\$/d"
+/^@endif build-$module-$bootstrap_suffix\$/d
+/^@unless build-$module-$bootstrap_suffix\$/,/^@endunless build-$module-$bootstrap_suffix\$/d"
done
extrasub_host=
for module in ${configdirs} ; do
@@ -2758,8 +2762,10 @@ for module in ${configdirs} ; do
extrasub_host="$extrasub_host
/^@if $module\$/d
/^@endif $module\$/d
+/^@unless $module\$/,/^@endunless $module\$/d
/^@if $module-$host_bootstrap_suffix\$/d
-/^@endif $module-$host_bootstrap_suffix\$/d"
+/^@endif $module-$host_bootstrap_suffix\$/d
+/^@unless $module-$host_bootstrap_suffix\$/,/^@endunless $module-$host_bootstrap_suffix\$/d"
done
extrasub_target=
for module in ${target_configdirs} ; do
@@ -2778,13 +2784,17 @@ for module in ${target_configdirs} ; do
extrasub_target="$extrasub_target
/^@if target-$module\$/d
/^@endif target-$module\$/d
+/^@unless target-$module\$/,/^@endunless target-$module\$/d
/^@if target-$module-$target_bootstrap_suffix\$/d
-/^@endif target-$module-$target_bootstrap_suffix\$/d"
+/^@endif target-$module-$target_bootstrap_suffix\$/d
+/^@unless target-$module-$target_bootstrap_suffix\$/,/^@endunless target-$module-$target_bootstrap_suffix\$/d"
done
# Do the final fixup along with target modules.
extrasub_target="$extrasub_target
-/^@if /,/^@endif /d"
+/^@if /,/^@endif /d
+/^@unless /d
+/^@endunless /d"
# Create the serialization dependencies. This uses a temporary file.
--
Alexandre Oliva, freedom fighter https://FSFLA.org/blogs/lxo
Be the change, be Free! FSF Latin America board member
GNU Toolchain Engineer Free Software Evangelist
Alexandre Oliva, freedom fighter https://FSFLA.org/blogs/lxo
Be the change, be Free! FSF Latin America board member
GNU Toolchain Engineer Free Software Evangelist