util.startup: Fix exiting on pidfile trouble prosody.shutdown() relies on prosody.main_thread, which has not been set yet at this point. Doing a clean shutdown might actually be harmful in case it tears down things set up by the conflicting Prosody, such as the very pidfile we were looking at. Thanks again SigmaTel71 for noticing

Merge 0.12->trunk

prosodyctl check: Warn about invalid domain names in the config file This ensures that domain names of virtual hosts and components are valid in XMPP, and that they are encoded correctly.

util.startup: Abort before initialization of logging when started as root Prevents creation of log files owned by the root user which could be inaccessible once started correctly.

util.startup: Don't use not yet existent shutdown procedure when started as root (thanks SigmaTel71)

util.startup: Check root after detecting platform and reading config (thanks SigmaTel71) Ensures that startup.detect_platform() runs so know whether to use the POSIX method of checking the current user or something else. Also after reading the config so we know whether the root override setting is set.

mod_posix: Move everything to util.startup This allows greater control over the order of events. Notably, the internal ordering between daemonization, initialization of libunbound and setup of signal handling is sensitive. libunbound starts a separate thread for processing DNS requests. If this thread is started before signal handling has been set up, it will not inherit the signal handlers and instead behave as it would have before signal handlers were set up, i.e. cause the whole process to immediately exit. libunbound is usually initialized on the first DNS request, usually triggered by an outgoing s2s connection attempt. If daemonization happens before signals have been set up, signals may not be processed at all.

mod_bosh: Set base_type on session This fixes a traceback with mod_saslauth. Ideally we move this to util.session at some point, though.

util.startup: Back out 598df17b8ebb Broke signal handling again, such that an early s2s connection results in libunbound catching signals and getting Prosody killed on e.g. SIGHUP This returns to the situation where prosody --daemonize does not respond to signals.

util.startup: Hook signals after daemonization signalfds stop working with epoll after forking hooking signals later should not affect anything

mod_pubsub: Ignore shadowed variable [luacheck]

mod_pubsub: Add shell commands to create and list nodes

core.features: Advertise that events are fired for SIGUSR1/2 Moved here from mod_posix since these events no longer originate there

util.startup: Fix firing of USR1/2 events

net.server: Restore epoll signalfd handling Reverts 4a9a69659727

mod_posix: Move POSIX signal handling into util.startup to avoid race When libunbound is initialized, it spawns a thread to work in. In case a module initializes libunbound, e.g. by triggering a s2s connection, Prosody would not handle signals, instead immediately quit on e.g. the reload (SIGHUP) signal. Likely because the libunbound thread would not have inherited the signal mask from the main Prosody thread. Thanks Menel, riau and franck-x for reporting and help narrowing down

net.server: Disable epoll signalfd handling by default until problems resolved

net.server_epoll: Log creation of signalfd handles at noise level To aid in tracking down signalfd-related problems

util.bit53: Add bnot() method

util.signal: Fail signalfd() if unable to change signal mask By aborting early, the failure should be brought to the attention somehow.

net.server_epoll: Log failure to hook signals To make any such failures noticeable

Merge 0.12->trunk

net.http.files: Validate argument to setup function Fixes error in #1765 by throwing an error earlier

mod_s2s: Comment on why we avoid hostnames in stanza bounce messages

mod_cron: Fix log format to account for float that was integer before

mod_cron: Sync Teal source with 92301fa7a673 Yeah, it's weird to have two versions. Needing more build dependencies is also something we want to avoid, so here we are.

util.signal: Wrap signalfd in an userdatum for gc handling etc

net.server_epoll: Support hooking signals via signalfd Handling signal events the same way as all other events makes sense and seems safer than the signal handling just jumping around in C and messing with Lua states.

util.signal: Add support for signalfd(2) on Linux signalfd allows handling signal events using the same method as sockets, via file descriptors. Thus all signal dispatch can go through the same main event loop as everything else, removing need for thread-scary signal handling where execution would just jump to the signal handler regardless of the state of Lua, and needing to keep track of Lua states/threads.

features: Add mod_server_info

mod_server_contact_info: Update to publish fields via new mod_server_info

mod_server_info: New module to manage the serverinfo disco extension form This allows multiple modules to populate the form dynamically. Currently the form is "owned" by mod_server_contact_info, which prevents other modules from contributing to it. A further commit will port mod_server_contact_info to use this module.

util.strbitop: Add common_prefix_bits() to Teal interface description

util.hashes: Add missing entries to Teal interface description Also sorted to match C source

util.crypto: Update Teal interface description to match C sources Was missing some entries. Rearranged to match order of entries in the C source Reg table.

tools/test_mutants.sh: Load loader helper when running busted

util.ip: Remove ip.bits and related code, switch to more efficient strbitop 100,000 iterations of match() on my laptop from 3.5s -> 0.5s.

util.strbitop: Remove unused import in tests

util.strbitop: Add common_prefix_bits() method This returns the number of bits that two strings have in common. It is significantly more efficient than similar calculations in Lua.

util.ip: Add another test case for match() and commonPrefixLength()

util.strbitop: Rename spec file to correct name so tests actually run

util.rfc6724: Remove, unused since introduction of Happy Eyeballs It was mainly used to determine whether to try IPv6 or IPv4 first, following the rules for this in the RFC. Now we always try IPv6 and IPv4 at roughly the same time, thus there no need to carry these rules.

features: Add module-ready (for commit e20949a10118)

util.startup: Expose core.features.available as prosody.features for convenience.

mod_s2s_auth_certs: Handle potential string error conn:ssl_peerverification() can now return a single error in case the connection has been closed for whatever reason

net.server_epoll: Prevent traceback when checking TLS after connection gone Unclear why this would be done, but an error is not great.

mod_cron: Allow configuring various "internal" delay parameters Notably, it is now possible to add a randomized spread factor to the check interval.

mod_c2s: Fix error on role change on Components (thanks Menel)

mod_smacks: Adjust buckets for resumption age statistic Given that there are recommendations floating around recommending 24 hours session lifetime, having buckets up to 10 minutes wouldn't be useful in that case. Would be nice if we had some way to automatically assign suitable number series for buckets, scaled to what the configuration might be.

mod_storage_internal: Fix off-by-one when searching archive for Fixes a test case provided by MattJ where the very first item matched by a 'start' timestamp was not returned.

mod_s2s_auth_dane_in: Try single TLSA lookup per draft-ietf-dance-client-auth Moves some complexity from the implementation into DNS operations.

mod_s2s_auth_dane_in: Simplify result processing Fewer loops

MUC: Record reason for affiliation changes and return in list (fixes #1227)

MUC: Test that <subject/> + <thread/> is not handled as subject change Ref #667 Ref #1838

mod_invites: Fix argument handling Not sure what the next() was supposed to do. Reject unknown --options perhaps?

mod_invites: Show short help instead of traceback on missing hostname

mod_invites: Show help if --help passed instead of hostname Because I couldn't guess the right way to get the help message without reading the source twice.

mod_invites: Allow specifying invite ttl on command line Was missing a way to pass TTL via command or shell.

Merge 0.12->trunk

mod_disco: Advertise disco#info and #items on bare JIDs to fix #1664 Having to add these in *there* places seems less than ideal. I would also think that advertising disco#info is a bit redundant, since it is a requirement for everything in XMPP and if it was missing you would get an error back.

scansion: Use new style for accessing Lua globals

scansion: Use new prosody namespace in import

scansion: Use captures or wildcards instead of mocking time > Mockery is one of the things I hold dear! And he's making a mockery of it!! -- Belkar Bitterleaf

util.xtemplate: Test the each template function It iterates over childtags, so a template like {foo|each{...}} would be equivalent to root:childtags("foo"), while a deeper query needs the bit that becomes arguments to :childtags as an argument to each, e.g. {foo/bar|each(baz)} would behave like root:get_child("foo"):get_child("bar"):childtags("baz")

tools: Fix selection of container engine Seems command -v in sh only checks and returns one argument, unlike bash.

mod_storage_internal, tests: Fix before/after combined with the 'reverse' flag

util.http: Silence strict luacheck warning in tests

core.moduleapi: Silence strict luacheck warnings in tests

util.throttle: Silence some strict luacheck warnings

util.prosodyctl.shell: Fix lint [luacheck]

CHANGES: Mention new prosodyctl shell method behavior

util.prosodyctl.shell: Add :method syntax to make e.g. MUC commands easier e.g. prosodyctl shell muc room room@muc.example.com :set_name "This Room"

util.xtemplate: Add some initial tests Strict typing does not magically make code correct

util.xtemplate: Adopt {-path-} syntax to strip preceding and/or trailing whitespace Seen in some other template languages

mod_http_errors: Simplify CSS via built-in dark mode

CHANGES: Document some of the recent changes and features in trunk

configmanager: Fix linter issues

configmanager: Support for appending to existing config options ...and some other useful operations

configmanager: Make _G accessible via `Lua` variable, deprecate direct access

configmanager: Allow referencing previously-set options in the config file

mod_user_account_management: Clear pending deletion if account re-enabled

mod_saslauth: Fire event per SASL step This matches the behaviour of the newer mod_sasl2 implementation. It allows plugins to observe (and potentially, with caution, modify) the SASL exchange.

util.jsonschema: Return basic structured validation response

mod_c2s: Make c2s_timeout timer reachable to allow access from other modules E.g. the timeout could be extended under certain conditions.

tools: Add a tool for comparing DOAP to the latest XEP versions Needs wget, awk, sed and xml2

mod_s2s: Close connection on smacks timeout This merges the mod_s2s_smacks_timeout behavior from prosody-modules This event is fired by mod_smacks when the connection has not responded to an ack-request for a period of time defaulting to 30 seconds, indicating that the connection has become stuck or non-responsive. Closing it prevents routing further messages via this connection and frees resources. A stuck connection may otherwise remain until for a time determined by the OS TCP subsystem, which can be quite long.

mod_saslauth: Fire event at start of authentication attempt As extension point for rate limiting and similar checks, so they can hook a single event instead of <{sasl1}auth> or stream features, which might not be fired in case of SASL2 or e.g. HTTP based login.

net.http.server: Fix whitespace-ignoring syntax

Merge 0.12->trunk

net.http.parser: Reject overlarge header section earlier This case would eventually be rejected by the buffer size limit.

lua-format: Let simple things be one line This doesn't really handle nesting all that nicely tho.

lua-format: Further tweaks Keeping things a single line makes very deeply nested things "pyramids" a single line, which makes them hard to read.

lua-format: Add new settings

lua-format: Tweaks attempting to fit our code style

lua-format: Check in defaults By starting with the built-in defaults, we get a nice history of differences from those as we figure out what settings suit us Sorted make comparisons easier.

mod_user_account_management: Fire events with a fake (not destroyed) session Previously these events fired after the session had been destroyed, which removes many of the useful properties. The ones I chose to preserve here are the ones used by the community module mod_audit, which seems like a good baseline.

mod_cron: Rebuild with new LuaFormatter settings (tabs!)

util.datamanager: Fix missing pack format when reading first index entry Thanks MattJ

mod_user_account_management: Add support for soft-deletion of accounts via IBR When registration_delete_grace_period is set, accounts will be disabled for the specified grace period before they are fully deleted. During the grace period, accounts can be restored with the user:restore() shell command. The primary purpose is to prevent accidental or malicious deletion of a user's account, which is traditionally very easy for any XMPP client to do with a single stanza.

moduleapi: Log error message when ambiguous period spec is found in config

util.human.io: Don't accept ambiguous durations by default The new method parse_duration_lax() exports the old behaviour, mainly for compatibility purposes.

mod_cron: Update Teal source and rebuild

mod_cron: Rename variable to fix shadowing (#luacheck)

mod_cron: Add shell command to list registered cron tasks with status

mod_saslauth: Allow plugins to override return SASL condition/text

luacheckrc, mod_http_file_share: Update for module API change (once->on_ready)

modulemanager: Allow modules to expose module.ready - to be called after init This is a shortcut for module:on_ready() which exposes the functionality in an idiomatic way consistent with module.load, module.unload, etc. module.ready runs when the module is loaded and the server has finished starting up.

moduleapi: Rename :once() to :on_ready() for clarity 'Once' is ambiguous - once per what? on_ready() executes its parameter when the module is loaded *and* the server has finished starting.

usermanager, mod_auth_internal_hashed: Support metadata when disabling a user This allows us to store a time, actor, comment and/or reason why an account was disabled, which seems a generally useful thing to support.

util.datamanager: Load first item into index earlier Should get rid of fseek() call

mod_mam: Use for loop in metadata query Some storage drivers will perform cleanup after the last iteration, but if only one step is taken this might be delayed until the garbage collector gets to it.

mod_tokenauth: Ignore invalid grants in storage that have no id

mod_invites: Fix linter issues

mod_admin_shell: Fix linter issues

mod_invites: Use new shell-command API

mod_admin_shell: Remove verbose logging

mod_admin_shell: Remove timer:info() (it's been debug:timers() for some time)

mod_admin_shell: Support for 'shell-command' items (global and per-host) This should simplify adding shell commands from other modules, which will reduce the growth of mod_admin_shell and make it easier for community modules to expose commands too.

mod_admin_shell: Refactor help to data structures for extensibility This makes it easier for commands added by other modules to add to the help output, for example.

migrator: Add an escape hatch to allow arbitrary config options Previously only SQL settings and the 'path' for internal storage could be set, and only for SQL and internal storage. input { type = "whatever"; config = { whatever_foobar = "something" } }