Commit graph

260 commits

Author SHA1 Message Date
Dmitry Goncharov
a9e3eb1eec [SV 65739] Add warning circular-dep.
Add a warning to control circular dependency detection.  Use "warn" as
the default action to be backward-compatible.

* src/warning.h (enum warning_type): Add warning type wt_circular_dep.
* src/warning.c (warn_init): Set default wt_circular_dep to w_warn.
* src/remake.c (update_file_1): Consult the circular-dep warning to
handle circular dependencies.
* tests/scripts/options/warn: Test --warn circular-dep flag.
* tests/scripts/variables/WARNINGS: Test .WARNINGS circular-dep flag.
* doc/make.texi: Document circular-dep warning.
* doc/make.1: Ditto.
2024-08-04 15:21:59 -04:00
Paul Smith
479c54f6ed * NEWS: Clean up UTF-8 issues 2024-04-14 15:27:12 -04:00
Dmitry Goncharov
07187db947 [SV 64822, SV 36486] Fix appending to a pattern specific variable
Appending to a pattern specific variable produces an incorrect value
in the presence of a command line definition or an env override of
the variable.  Also, fix pattern/target-specific appending to a
variable with origin override.

* At parse time record_target_var sets the value of a pattern
  specific variable to the value defined on command line or to the
  value of the env override.
* Later, at build time, recursively_expand_for_file appends this
  value of the variable (set in record_target_var) to the command
  line value again, regardless of the origin of the variable.

This patch modifies recursively_expand_for_file to avoid appending,
unless the origin of the variable beats or equals the origin of one
of the parent definitions of this variable.

Reported by Rob <robw9739@gmail.com>,
Brian Vandenberg <phantall@gmail.com>,
Markus Oberhumer <markus@oberhumer.com>.

* NEWS: Note the change.
* src/variable.c (do_variable_definition): Avoid merging a
pattern-specific variable with the parent definition when a command
line or env override is present.
* src/expand.c (recursively_expand_for_file): Avoid appending to a
pattern-specific variable, unless the origin of this pattern-specific
variable beats or equals the origin of one of the parent definitions
of this variable.
* doc/make.texi (Override Directive): Add missing cross-reference.
* tests/scripts/variables/append: Add tests.
2024-02-04 18:26:21 -05:00
Paul Smith
a493d9ab6c * NEWS: Clean up 2024-02-04 11:34:49 -05:00
Dmitry Goncharov
a382ac6cd1 [SV 64803] Set origin for unmodified variables after -e
Ensure the origin of all variables inherited from the environment is
"environment override" if -e is given.  Previously only variables
that were set in the makefile had this origin.

PDS: Most of these changes are from Dmitry but I slightly modified
the algorithm: instead of rearranging the way in which MAKEFLAGS is
parsed we reset the env_override value to the default before we
re-parse MAKEFLAGS, then we set the origin of all env vars to the
correct value based on the new setting.

* NEWS: Mention the change for backward-compatibility.
* src/main.c (main): Ensure MAKEFLAGS is always marked special.
(reset_makeflags): Set env_overrides back to default before parsing
MAKEFLAGS.
(decode_switches): Call reset_env_override() to check for changes.
* src/variable.h (reset_env_override): Declare a new function.
* src/variable.c (reset_env_override): Go through all env variables
and ensure the origin is correct based on env_overrides.
(set_env_override): Helper function for the hash.
* tests/scripts/functions/foreach: Fix tests.
* tests/scripts/functions/let: Ditto.
* tests/scripts/functions/origin: Ditto.
* tests/scripts/options/dash-e: Add tests.
2024-02-04 11:34:49 -05:00
Paul Smith
1eff20f6f6 Support conditional modifiers on all assignment operators
Rework the single "?=" operator to instead allow a "?" modifier to be
prepended to ANY assignment operator.  If "?" is given then the
variable is assigned (using whatever operator comes next) if and only
if the variable is not already defined.  If it is defined then no
action is taken (the right-hand side is not expanded, etc.)

* NEWS: Announce this new feature.
* doc/make.texi: Modify the documentation around assignment operators.
* src/variable.h: Remove the f_conditional variable flavor.
(do_variable_definition): Add an argument specifying conditional.
* src/variable.c (parse_variable_definition): Use the existing flag
"conditional" to remember if we saw "?" rather than the flavor.
When we see "?" skip it and continue trying to parse an assignment.
(try_variable_definition): Pass condition to do_variable_definition().
(initialize_file_variables): Ditto.
(do_variable_definition): Check for conditional up-front: quit if set.
Remove handling of obsolete f_conditional flavor.
* src/read.c (eval_makefile): MAKEFILE_LIST is not conditional.
(do_define): Unset conditional for define with no operator.  Pass the
conditional flag to do_variable_definition().
(construct_include_path): .INCLUDE_DIRS is not conditional.
* src/load.c (load_file): .LOADED is not conditional.
* tests/scripts/variables/conditional: Add new tests.
2024-01-28 14:20:47 -05:00
Paul Smith
31036e648f [SV 64571] Add --print-targets option
Add an option to print a list of targets defined in the makefiles.
Don't print targets of implicit rules, or special targets.  To
support this remember which files are deemed suffix rule targets.

Add a missing warning for single-suffix targets with prerequisites.

Suggested by many.  Sample implementation by Tim <tdhutt@gmail.com>.

* NEWS: Announce the new option and single-suffix warning.
* doc/make.1: Add --print-targets to the man page.
* doc/make.texi: Add --print-targets to the documentation.  Clean up
the text around the definition of suffix rules.
* src/main.c (print_targets_flag): New variable for --print-targets.
(switches): Add a new long option --print-targets.
(main): If the option was provided call print_targets() and exit.
* src/filedef.h (struct file): Add a "suffix" boolean value.  Remove
print_prereqs() since it's static.  Add new print_targets().
* src/file.c (rehash_file): Merge the new suffix value.
(print_prereqs): Used only locally: change to static.
(print_target): Print targets which are not suffix rule targets and
are not special targets.
(print_targets): Call print_target() on each file.
* src/rule.c (convert_to_pattern): Make maxsuffix local; it doesn't
need to be static.  Emit ignoring prerequisites for single-suffix
rules as well as double-suffix rules.  Remember which files are
actually suffix rules.
* tests/scripts/features/suffixrules: Test single-suffix behavior.
* tests/scripts/options/print-targets: Add tests for --print-targets.
2024-01-08 23:31:58 -05:00
Paul Smith
a80670ad41 * (all): Update Copyright year to 2024 2024-01-06 18:06:09 -05:00
Paul Smith
d86448fe5f [SV 64402] Correct locating "," in ifeq/ifneq conditionals
Ensure that we correctly skip the entirety of a macro or function
reference when searching for the "," separator in an ifeq/ifneq
conditional, including using "$," and also "${foo,bar}".  Note that
this change means that parenthesis OTHER than those used for variable
expansion are not considered special, any longer.

* NEWS: Announce the change.
* src/read.c (conditional_line): Skip variable references when looking
  for "," and ensure that we match closing parens/braces properly.
* tests/scripts/features/conditionals: Add tests for this behavior.
2024-01-06 17:42:40 -05:00
Paul Smith
1a03888afa * NEWS: Set a prerelease version 2023-06-19 10:05:19 -04:00
Paul Smith
07fcee35f0 [SV 64815] Recipe lines cannot contain conditional statements
* NEWS: Mention this change.
* src/read.c (eval): Check for ignoring for any line even if not
in a rule context.
* tests/scripts/features/conditionals: Write new tests.
2023-05-22 23:36:13 -04:00
Paul Eggert
c85b71a396 make -p now uses consistent timestamp format
* NEWS: mention this.
* src/main.c (safer_ctime, time_now): Remove.
(print_data_base): Use file_timestamp_sprintf to format timestamps.
2023-05-14 18:26:35 -04:00
Paul Smith
1748e66414 [SV 63219] Support an "unload" function for loaded objects
If a loaded object defines a symbol <object>_gmk_unload, assume it's
a function and invoke it whenever the loaded object is unloaded.
Original implementation by Dmitry Goncharov <dgoncharov@users.sf.net>

* NEWS: Announce the change.
* doc/make.texi: Describe the behavior.
* src/gnumake.h: Add information to the comments.
* src/makeint.h (unload_all): Declare a new function.
* src/main.c (die): Invoke unload_all().
* src/load.c (unload_func_t): Declare a new type for unload.
(struct load_list): Remember the unload symbol if it exists.
(load_object): Move the parsing of the object name from load_file.
Check for the _gmk_unload symbol and if found, remember it.
(load_file): Allow load_object to do object filename parsing.
(unload_file): Remove the load_list entry when unloading the object.
(unload_all): Unload all the loaded objects.
* tests/scripts/features/loadapi: Test the unload function.
2023-05-14 18:26:35 -04:00
Paul Smith
8e0e6c678f Remove the "preview" status from the loaded object feature
Add an ABI version both to the header file and passed to the setup
function.  Unfortunately this itself is an ABI break and I couldn't
find a good way to avoid it.

* NEWS: Announce the ABI is not a preview and the incompatibility.
* doc/make.texi: Remove the preview warnings for object loading.
Document the new ABI version argument.
* src/gnumake.h (GMK_ABI_VERSION): Set the ABI version to 1.
Add comments documenting the format of the setup function.
* src/load.c (setup_func_t): Rename from load_func_t.
(load_file): Pass the ABI version to the setup function.
* tests/scripts/features/load: Rework the setup function.
* tests/scripts/features/loadapi: Ditto.
2023-05-07 16:51:06 -04:00
Paul Smith
24a84f99bb * tests/test_driver.pl: Show info about failed tests
Remove working directories for skipped tests.
We use different directories per test so don't keep rmfiles.
2023-05-07 16:51:06 -04:00
Paul Smith
fec72ea308 * NEWS: Note the change to the tests suite 2023-04-23 11:35:02 -04:00
Paul Smith
5111087e12 * NEWS: Mention the new .WARNINGS variable 2023-04-02 10:29:19 -04:00
Paul Smith
03ecd94488 Add new warnings invalid-var and invalid-ref
The "invalid-var" warning triggers if the makefile attempts to assign
a value to an invalid variable name (a name containing whitespace).
The "invalid-ref" warning triggers if the makefile attempts to
reference an invalid variable name.  Both new warnings have a default
action of "warn".

* NEWS: Add these new warnings.
* doc/make.1: Document them in the man page.
* doc/make.texi (Warnings): Document them in the user's manual.
* src/warning.h: Add enum values for the new warning types.
* src/main.c (initialize_warnings): Initialize the new warnings.
* src/variable.h (undefine_variable_in_set, undefine_variable_global):
Ask callers to provide a struct floc specifying where the variable
is undefined.
* src/read.c (do_undefine): Pass floc when undefining.
* src/variable.c (check_valid_name): If invalid-var is enabled, check
the variable name.
(define_variable_in_set): Call it.
(undefine_variable_in_set): Ditto.
(check_variable_reference): If invalid-ref is enabled, check the
variable reference.
(lookup_variable): Call it.
(lookup_variable_in_set): Ditto.
* tests/scripts/options/warn: Add tests for the new warning types.
2023-04-01 11:13:12 -04:00
Paul Smith
2611e1991f Introduce a --warn command line option
Replace the singleton --warn-undefined-variables with infrastructure
to manage multiple warnings: the --warn option can take an action
"ignore", "warn", or "error" (which will apply to all warnings), or
a specific warning type and an action for that type.  Multiple
options can be provided and are consolidated.

* NEWS: Announce the new option.
* doc/make.1: Document in the man page.
* doc/make.texi (Warnings): Document in the user's manual.
* Makefile.am: Add new header warning.h.
* src/warning.h: Define enum for actions and warning types, and
macros to test whether they are set.  Keep the default settings
separate so that we can correctly reconstruct MAKEFLAGS.
* src/makeint.h: Remove deprecated warn_undefined_variables_flag.
* src/main.c: Create global variables to hold warning settings.
(switches): Add a new switch for --warn.
(initialize_warnings): Set the default warning actions.
(main): Call initialize_warnings().
(encode_warning_state, decode_warning_state): Convert warning states
between strings and enums.
(encode_warning_name, decode_warning_name): Convert warning names
between strings and enums.
(decode_warn_flags): Convert a --warn option into enum values.  If
deprecated warn_undefined_variables_flag is set convert it to --warn.
(decode_switches): Don't remove duplicates of --warn since order
matters.  Call decode_warn_flags() to handle --warn.
(define_makeflags): Special-case handling of --warn options written
to MAKEFLAGS: write out the current settings.
* src/read.c (tilde_expand): Use new warning control macros.
* src/variable.c (warn_undefined): Ditto.
* src/job.c (construct_command_argv): Ditto.
* tests/scripts/options/warn: Rename from warn-undefined-variables
and add tests for --warn.
* tests/scripts/variables/MAKEFLAGS: Expect the new behavior.
2023-04-01 11:13:12 -04:00
Paul Smith
d4692df20d Remove support for AmigaOS
There is a lot of specialized code for supporting AmigaOS and it has
not been maintained for a number of years.  It's highly unlikely that
the latest versions even compile properly on AmigaOS anymore.  After
requesting that someone step forward to own the maintenance of the
port in the GNU Make 4.4 release and getting no takers, I removed it.

* NEWS: Announce the removal.
* README.in: Remove README.Amiga reference.
* README.Amiga: Remove unused file.
* SCOPTIONS: Ditto.
* src/amiga.c: Ditto.
* src/amiga.h: Ditto.
* src/config.ami: Ditto.
* mk/Amiga.mk: Ditto.
* Makefile.am: Remove references to deleted files.
* Basic.mk.template: Ditto.
* maintMakefile: Ditto.
* src/commands.c: Remove ifdef'd Amiga code.
* src/default.c: Ditto.
* src/dir.c: Ditto.
* src/file.c: Ditto.
* src/function.c: Ditto.
* src/job.c: Ditto.
* src/job.h: Ditto.
* src/main.c: Ditto.
* src/os.h: Ditto.
* src/read.c: Ditto.
* src/remake.c: Ditto.
2023-01-15 17:32:15 -05:00
Paul Smith
fcefae5ec9 * NEWS: Update for the next release
* configure.ac: Set the version for the next release.
2023-02-26 15:40:18 -05:00
Paul Smith
d66a65ad5a GNU Make release 4.4.1
* NEWS: Set the release number and date.
* configure.ac: Update the version.
* README.git: Remind to update doc/make.1.
* doc/make.1: Update the date.  Fix some "GNU Make" references.
* doc/make.texi: Update the edition.
2023-02-26 13:46:22 -05:00
Paul Smith
37e0010743 Prerelease GNU Make 4.4.0.91
* NEWS: Update the date.
* configure.ac: Change the release.
2023-02-19 09:53:26 -05:00
Paul Smith
db351fe85b Prerelease GNU Make 4.4.0.90
* NEWS: Update the date.
* README.git: Fix a typo.
* maintMakefile (check-alt-config): Report success.
(upload-*): Use the gnulib gnupload script for uploads.
2023-01-14 17:07:04 -05:00
Paul Smith
b99b6cdf3c Update support for OS/2
Patches provided by KO Myung-Hun <komh78@gmail.com>

* NEWS: Add a note.
* AUTHORS: Add a new author.
* README.OS2: Updates to build instructions.
* src/dir.c (dir_contents_file_exists_p): Use a stack copy when
modifying a const string.
* src/job.c (construct_command_argv_internal): Ditto.
Reuse variables rather than re-defining them.
(exec_command): Cast a const string (we don't change it anyway).
* src/getopt.c (_getopt_initialize): Reference unused variables.
(_getopt_internal): Add block braces to quiet the compiler.
* src/main.c (main): Cast argument to child_execute_job().
* src/posixos.c (set_blocking): Reference unused variables.
* src/remake.c (f_mtime): Delete useless code.
2023-01-08 18:44:25 -05:00
Paul Smith
11444fb001 [SV 62654] Support GNU Make on z/OS
Original patches provided by Igor Todorovski <itodorov@ca.ibm.com>
Reworked by Paul Smith <psmith@gnu.org>.
Thanks to IBM for providing a test system.

* NEWS: Announce support.
* AUTHORS: Ditto.
* README.zOS: Provide details on building GNU Make on z/OS.
* build.sh (get_mk_var): z/OS sh has a strange bug which causes it to
generate extra lines of output: rework the function to print output
as we compute it instead of collecting it into a variable, which
works around this bug.
* src/makeint.h: Declare MK_OS_ZOS if we're building for z/OS.
* src/arscan.c: Don't include <ar.h> on z/OS.
* src/job.c: We can't change environ in ASCII mode on z/OS.
* src/main.c: Ditto.  Also we can't use pselect() on z/OS.
* src/posixos.c: pselect() seems to hang on z/OS: don't use it.
* tests/run_make_tests.pl: Handle different exit codes on z/OS.
* tests/test_driver.pl: Preserve some special z/OS env.vars.
Add special checks to output comparisons when on z/OS.
* tests/scripts/features/archives: Don't validate names.  Don't
try to compile empty files as IBM compilers complain.
* tests/scripts/features/shell_assignment: Fix octal value of #.
* tests/scripts/features/temp_stdin: Don't print "term".
* tests/scripts/functions/shell: Handle shell exit codes.
* tests/scripts/targets/ONESHELL: Ditto.
* tests/scripts/targets/POSIX: sh -x prints differently.
* tests/scripts/variables/SHELL: Ditto.
2023-01-08 10:45:38 -05:00
Paul Smith
1ceeb8c64b [SV 14927] Allow parallel builds for archives
Compare the timestamp of the object file (if it exists) with the
archived object and if the object file is newer, ensure it's updated
in the archive.

* NEWS: Announce the new capability.
* doc/make.texi (Dangers When Using Archives): Explain how to enable
parallel builds with archives.
* src/remake.c (f_mtime): For archive element files check the mod
time of the object file (if it exists) against the archive object
(if it exists).
* tests/scripts/features/archives: Add tests for this capability.
2023-01-03 01:57:35 -05:00
Paul Smith
8791d2b38e [SV 61463] Don't export inherited private variables
If a parent target has an exported variable that is private, don't
export it in child targets.

* NEWS: Mention this change.
* src/variable.c (target_environment): Ignore private inherited
variables.
* tests/thelp.pl: Add a new "env" operation to show env.var. values.
* tests/scripts/variables/private: Verify this new behavior.
2023-01-02 23:02:33 -05:00
Paul Smith
e7ce3a655e Convert references from "GNU make" to "GNU Make" 2023-01-01 17:05:12 -05:00
Paul Smith
56d2978141 Update the copyright year on all files 2023-01-01 10:06:01 -05:00
Paul Smith
8e9c7db1e1 * NEWS: Add a section for the next release
Add a note for the MAKEFLAGS enhancements.
2022-12-24 15:48:48 -05:00
Paul Smith
ed493f6c91 Release GNU Make 4.4
* NEWS: Update the version and date.
* configure.ac: Update the version.
* doc/make.texi: Update the EDITION.
2022-10-31 02:23:04 -04:00
Paul Smith
e4b3bf7f97 Release GNU Make 4.3.92
* configure.ac: Modify the release version.
* NEWS: Update the version and date.
2022-10-24 02:14:42 -04:00
Paul Smith
7c71df5695 * NEWS: Release GNU make 4.3.91 2022-10-18 14:37:47 -04:00
Paul Smith
4ec74e568d * NEWS: Use GNU Make instead of GNU make
* README.git: Ditto.
* README.Amiga: Ditto.
* README.DOS: Ditto.
* README.OS2: Ditto.
* README.VMS: Ditto.
* README.W32: Ditto.
* README.customs: Ditto.
* make-gdb.py: Ditto.
* tests/run_make_tests.pl: Ditto.
2022-10-18 14:37:47 -04:00
Paul Smith
d9fc1ac5d2 Update URLs to use https rather than http
* (all): Change http:// to https://
* README.W32: Remove invalid link to mingw.org.
2022-10-18 14:37:47 -04:00
Paul Smith
dee6af912a * NEWS: Deprecate Xenix 2022-10-03 15:11:57 -04:00
Paul Smith
7296991d6c [SV 63098] Temporarily revert the change to pattern rule behavior
The fix for SV 12078 caused a backward-compatibility issue with some
makefiles.  In order to allow users to resolve this issue, revert
that change for this release cycle: it will be reinstated in the
next release cycle.  Introduce a warning if we detect that the recipe
of a multi-target pattern rule doesn't create all the targets.

* NEWS: Announce the future backward-incompatibility.
* doc/make.texi (Pattern Intro): Describe the behavior and that it
will change in the future.
* src/remake.c (check_also_make): Check for also_make targets that
were not created and generate a warning.
(update_goal_chain): Call the new function.
(check_dep): Ditto.
(update_file_1): Defer implicit rule detection until after we check
all the also_make files (as it used to be).
* tests/scripts/features/patternrules: Add tests of the new warning.
Skip the tests for SV 12078.
2022-10-02 10:18:21 -04:00
Paul Smith
26b9f30714 * NEWS: Announce deprecation of OS/2 and AmigaOS ports 2022-10-02 08:55:29 -04:00
Paul Smith
d4cb7ae6d4 * configure.ac: Update to GNU make 4.3.91
* NEWS: Update the version and clarify some backward-compat issues.
2022-09-25 13:37:12 -04:00
Paul Smith
990d6f9b9a Release GNU make 4.3.90
* NEWS: Update for the prerelease.
* bootstrap.conf: Choose the latest stable gnulib.
* README.git: Updates.
2022-09-20 13:23:44 -04:00
Paul Smith
88d126d6c4 * doc/make.texi (Parallel Disable): Add clarifications.
* NEWS: Suggest how to change makefiles that use $(MAKEFLAGS).
2022-09-13 18:22:29 -04:00
Paul Smith
f6ea899d83 [SV 13862] Implement the .WAIT special target
The next version of the POSIX standard defines parallel execution
and requires the .WAIT special target as is implemented in some other
versions of make.

This implementation behaves similarly to others in that it does not
create a relationship between targets in the dependency graph, so
that the same two targets may be run in parallel if they appear as
prerequisites elsewhere without .WAIT between them.

Now that we support .WAIT it's trivial to also support prerequisites
of the .NOTPARALLEL special target, which forces the prerequisites of
those targets to be run serially (as if .WAIT was specified between
each one).

* NEWS: Announce the new .WAIT and .NOTPARALLEL support.
* doc/make.texi (Parallel Disable): A new section to discuss ways in
which parallel execution can be controlled.  Modify cross-refs to
refer to this section.
* src/dep.h (struct dep): Add a new wait_here boolean.
(parse_file_seq): Add PARSEFS_WAIT to check for .WAIT dependencies.
* src/file.c (split_prereqs): Use PARSEFS_WAIT.
(snap_deps): If .NOTPARALLEL has prerequisites, set .WAIT between
each of _their_ prerequisites.
(print_prereqs): Add back in .WAIT when printing prerequisites.
* src/implicit.c (struct patdeps): Preserve wait_here.
(pattern_search): Ditto.  Use PARSEFS_WAIT when parsing prereqs for
pattern rule expansion.
* src/read.c (check_specials): Don't give up early: remembering to
update these options is not worth the rare speedup.
(check_special_file): If .WAIT is given as a target show an error--
once--if it has prereqs or commands.
(record_files): Call check_special_file on each target.
(parse_file_seq): If PARSEFS_WAIT is given, look for .WAIT prereqs.
If we see one assume that we are building a struct dep chain and set
the wait_here option while not putting it into the list.
* src/remake.c (update_file_1): If wait_here is set and we are still
running, then stop trying to build this target's prerequisites.
* src/rule.c (get_rule_defn): Add .WAIT to the prerequisite list.
* src/shuffle.c (shuffle_deps): Don't shuffle the prerequisite list
if .WAIT appears anywhere in it.
* tests/scripts/targets/WAIT: Add a test suite for this feature.
2022-09-12 18:35:29 -04:00
Paul Smith
86fc644dcc Support the MAKE_TMPDIR environment variable
Allow build systems to choose an alternative location for make to
store its temporary files.

* NEWS: Announce the new environment variable.
* doc/make.texi (Temporary Files): Provide documentation.
* src/misc.c (get_tmpdir): Split into a new function.  Compute the
temporary directory and store it in a static location.
* tests/scripts/features/jobserver: Add a test of MAKE_TMPDIR.
2022-09-11 10:56:08 -04:00
Paul Smith
70ba0357a0 [SV 63040] shell: Fall back to the callers environment
If we detect a recursive variable reference when constructing the
environment for the shell function, return the original value from the
caller's environment.  Other options such as failing, returning the
empty string, or returning the unexpanded make variable value have
been shown to not behave well in real-world environments.  If the
variable doesn't exist in the caller's environment, return the empty
string.

Found by Sergei Trofimovich <slyich@gmail.com> when testing older
versions of autoconf.

* NEWS: Clarify this behavior.
* doc/make.texi (Shell Function): Ditto.  Also add info about !=.
* src/expand.c (recursively_expand_for_file): Search the caller's
environment if we detect a recursive variable expansion.
* tests/scripts/functions/shell: Add tests for this behavior.
2022-09-10 16:27:47 -04:00
Paul Smith
4da2055a10 Rework output sync to lock a temp file on POSIX
Some POSIX systems do not allow locks to be taken on non-files, such
as pipes.  This is a problem since very often make is invoked with
its stdout redirected to a pipe.  Also, if stdout is redirected to a
file that already has a lock on it for some other reason (perhaps a
shared file such as /dev/null) it can cause a hang.

This means our previous method of locking stdout, although it had some
nice advantages, is not portable enough.  Instead, use a temporary
file and take the lock on that.  We pass the name of the file to child
make processes.  On Windows we continue to use a shared mutex for
output sync.

Remove POSIX emulation functions like fcntl from Windows; instead
follow the lead of the jobserver and create an interface in os.h for
output sync, and move the OS-specific content to posixos.c and
w32os.c.

* NEWS: Add a note.
* src/makeint.h (ALL_SET): Check that all bits are set.
* src/os.h: Add bits for checking the state of stdin/stdout/stderr.
Add prototypes for OS-specific output sync methods.
* src/posixos.c (check_io_state): Determine the status of stdin,
stdout, stderr an return a suite of bits describing them.
(osync_enabled): If the global variable holding the FD of the lock
file (osync_handle) is valid return true.
(osync_setup): Create a temporary file and remember its name in a
global variable (osync_tmpfile), and set osync_handle.
(osync_get_mutex): If output sync is enabled, return the filename
of the lock file prefixed with "fnm:" to denote a filename.
(osync_parse_mutex): If the provided filename has the wrong format
disable output sync.  Else open the lock file and set osync_handle.
(osync_clear): Close osync_handle.  If we're the parent make, then
also unlink the temporary file.
(osync_acquire): Take a lock on the osync_handle descriptor.
(osync_release): Release the lock on the osync_handle descriptor.
(fd_set_append): Add APPEND mode to a file descriptor.
* src/w32/w32os.c: Perform the same actions as posixos.c, copying
the details from src/w32/compat/posixfcn.c.  Use a mutex rather
than locking a temporary file.
* src/output.h: Remove all the OS-specific content.
* src/output.c: Remove all the OS-specific content.
(set_append_mode): Remove and replace with fd_set_append().
(sync_init): Remove and replace with check_io_state().
(acquire_semaphore): Remove and replace with osync_acquire().
(release_semaphore): Remove and replace with osync_release().
(setup_tmpfile): If the IO state is not obtained, get it.  If stdout
and/or stderr are valid, set up a tempfile to capture them.
(output_init): Set io_state if not set already, and check it when
deciding whether to close stdout on exit.
* src/main.c (main): If we're syncing, set up the mutex using the
new osync_setup() / osync_parse_mutex() methods.
(prepare_mutex_handl_string): Replace with osync_parse_mutex().
(die): Call osync_clear().
* src/w32/compat/posixfcn.c: Remove implementations of fcntl(),
record_sync_mutex(), create_mutex(), and same_stream().
2022-08-30 15:44:43 -04:00
Paul Smith
7ad2593b2d Support implementing the jobserver using named pipes
Using anonymous pipes for jobserver support has some advantages:
for example there is nothing on disk that needs to be cleaned up.
However it has many obscure problems, related to the fact that in
order for it to work we need to ensure these resources are properly
passed through to child processes that want to use the jobserver.
At the same time we don't want to pass the pipe to process which
DON'T know about the jobserver.

Other processes can open file descriptors which we then think are
our jobserver, but aren't.  And, we open the pipe file descriptors
in blocking mode which doesn't work for all users.

See issues such as SV 57178, SV 57242, and SV 62397

To avoid these issues, use named pipes (on systems where they are
available) instead of anonoymous pipes.  This simplifies many things:
we never need to pass open file descriptors to our children; they
can open the jobserver named pipe.  We don't need to worry about
recursive vs. non-recursive children.  Users don't have to "pass
through" the resources if they are invoking sub-makes.  Each child
can open its own file descriptor and set blocking as needed.

The downside is the named pipe exists on disk and so must be cleaned
up when the "top-level" make instance exits.

In order to allow make to continue to be used in build systems where
older versions of GNU make, or other tools that want to use the
jobserver, but don't understand named pipes, introduce a new option
--jobserver-style that allows the user to choose anonymous pipes.

* NEWS: Announce the change and the --jobserver-style option.
* doc/make.1: Add --jobserver-style documentation.
* doc/make.texi (Special Variables): Add missing items to .FEATURES.
(Options Summary): Add --jobserver-style.
(POSIX Jobserver): Named pipes, changes to --jobserver-auth, and the
--jobserver-style option.
(Windows Jobserver): Document --jobserver-style for Windows.
* configure.ac: Check for mkfifo.
* src/config.h-vms.template: Undefined HAVE_MKFIFO.
* src/config.h.W32.template: Ditto.
* src/main.c: Add jobserver-style as a new command line option.
(main): Add jobserver-fifo to .FEATURES if supported.  Pass the style
option to jobserver_setup().
* src/os.h (jobserver_setup): Accept a style string option.
* src/posixos.c (enum js_type): Enumeration of the jobserver style.
(js_type): Which style we are currently using.
(fifo_name): The path to the named pipe (if in use).
(jobserver_setup): If no style is given, or "fifo" is given, set up a
named pipe: get a temporary file and use mkfifo() on it, then open it
for reading and writing.  If something fails fall back to anonymous
pipes.
(jobserver_parse_auth): Parse jobserver-auth to determine the style.
If we are using a named pipe, open it.  If we're using anonymous pipes
ensure they're valid as before.
(jobserver_get_invalid_auth): Don't invalidate the jobserver when
using named pipes.
(jobserver_clear): Clean up memory used for named pipes.
(jobserver_acquire_all): Unlink the named pipe when done.
* src/w32/w32os.c (jobserver_setup): Check the style argument.
* tests/scripts/features/jobserver: Use --jobserver-style to test
the anonymous pipe behavior, and also test named pipe/semaphore
behavior.  Check invalid jobserver-style options.
* tests/scripts/functions/shell: Use --jobserver-style to test the
anonymous pipe behavior, and also test named pipe/semaphore
behavior.
2022-08-02 23:36:35 -04:00
Dmitry Goncharov
07eea3aa49 [SV 62706] Only second-expand targets that might be built
Second-expand only the prerequisites of the targets being built.
Defer second-expanding the prerequisites of targets until we need
to decide if they should be built.

* NEWS: Mention the change in behavior.
* doc/make.texi (Secondary Expansion): Document the new behavior.
* src/filedef.h (struct file): Add flag snapped.
(expand_deps): Declare a function to second expand the
prerequisites of a target.
* src/file.c (rehash_file): Merge flag snapped.
(expand_deps): Remove qualifier static. Check flag snapped.
(snap_deps): Remove the loop which performed second expansion for all
targets.
* src/remake.c (update_file_1): Second expand the prerequisites of
the considered target.
* tests/scripts/features/se_explicit: Add tests.
* tests/scripts/features/se_implicit: Ditto.
* tests/scripts/features/se_statpat: Ditto.
2022-07-30 18:40:28 -04:00
Paul Smith
16e14b4114 Disable the jobserver in non-recursive children
Savannah issues such as SV 57242 and SV 62397 show how passing
references to closed file descriptors via the --jobserver-auth option
in MAKEFLAGS can lead to problematic outcomes.

When computing the child environment for a non-recursive shell, add
an extra option to MAKEFLAGS to disable the file descriptors for the
jobserver.

Unfortunately this doesn't modify the value of the make variable
MAKEFLAGS, it only modifies the value of the sub-shell environment
variable MAKEFLAGS.  This can lead to confusion if the user is not
considering the distinction.

* src/makeint.h: Publish the jobserver-auth value.  Add a global
definition of the name of the command line option.
* src/os.h (jobserver_get_invalid_auth): New function to return a
string invalidating the jobserver-auth option.
* src/w32/w32os.c (jobserver_get_invaid_auth): Implement it.  On
Windows we use a semaphore so there's no need to invalidate.
* src/posixos.c (jobserver_parse_auth): If we parse the invalid
auth string, don't set up the jobserver.
(jobserver_get_invalid_auth): Return an invalid option.
* src/variable.h (target_environment): Specify if the target
environment is for a recursive shell or non-recursive shell.
* src/variable.c (target_environment): Move checking for MAKELEVEL
into the loop rather than doing it at the end.
Along with this, check for MAKEFLAGS and MFLAGS, and update them
based on whether we're invoking a recursive or non-recursive child,
and also on whether it's necessary to invalidate the jobserver.
* src/function.c (func_shell_base): Shell functions can never be
recursive to pass 0 to target_environment().
* src/job.c (start_job_command): Specify whether the child is
recursive when calling target_environment().
* src/main.c: Export jobserver_auth.  sync_mutex doesn't need to
be exported.  Use the global definition for the option name.
* tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
2022-07-30 18:40:28 -04:00
Paul Smith
98da874c43 [SV 10593] Export variables to $(shell ...) commands
Export all variables, including exported makefile variables, when
invoking a shell for the $(shell ...) function.  If we detect a
recursive variable expansion, silently ignore that variable and do
not export it.  We do print a debug message.

* NEWS: Announce the potential backward-incompatibility.
* doc/make.texi (Shell Function): Document the export behavior.
* src/main.c (main): Add "shell-export" to .FEATURES.
* src/job.h: New function to free struct childbase.
* src/job.c (free_childbase): Implement it; call from free_child.
* src/function.c (func_shell_base): Use target_environment() to
obtain the proper environment for the shell function.
Use free_childbase() to free memory.
(windows32_openpipe): Don't reset the environment: the caller
already provided a proper PATH variable in envp.
* src/variable.c (target_environment): If we detect a recursive
expansion and we're called from func_shell, ignore the variable.
(sync_Path_environment): Simplify and reduce memory allocation.
* tests/scripts/functions/shell: Add tests for this.
2022-07-09 10:44:21 -04:00