net.server_epoll: Add support for systemd socket activation Allows creating listening sockets and accepting client connections before Prosody starts. This is unlike normal Prosody dynamic resource management, where ports may added and removed at any time, and the ports defined by the config. Weird things happen if these are closed (e.g. due to reload) so here we prevent closing and ensure sockets are reused when opened again.

Merge 0.12->trunk

mod_admin_shell: Allow matching on host or bare JID in c2s:show Only supporting exact match on full JID isn't helpful if you want to list sessions per host or user. Backport of 430333198e4c Fixes #1857

mod_blocklist: Drop blocked messages without error, option to restore compliant behavior From XEP-0191: > For message stanzas, the server SHOULD return an error, which SHOULD > be <service-unavailable/>. Following this may leak to a blocked JID that they have been blocked, which seems contrary to the goal of pretending to be perpetually offline.

mod_pep: Implement 'roster' (group) access_model Allows e.g. restricting your vcard4 to only family or similar. Notes: This does not include roster groups in the configuration form, so the client will have to get them from the actual roster.

mod_announce: Suppress luacheck warnings

mod_announce: Add shell commands and APIs for sending to all/online/roles

prosodyctl shell: Fix invocation with 3+ command arguments The code correctly inserted the ',' when there was already a "%q" in the format string, but then the next argument would fail to match because it inserted ", %q" instead of "%q". The code now matches both, ensuring the generated code will not produce a syntax error with multiple arguments.

mod_blocklist: Fix fix signal for letting stanzas pass Returning nothing/nil lets stanzas pass, returning anything else blocks

mod_blocklist: Check JID of mediated MUC invite sender against blocklist This ensures that someone on your blocklist is unable to invite you to MUC rooms.

mod_saslauth: Log when tls-exporter is NOT supported, as well as when it is

net.unbound: Show canonical name in textual format (e.g. in shell) libunbound does not tell us the whole chain of CNAMEs, only the final canonical name. This is to aid in debugging since it will only be shown in the shell.

mod_http_file_share: Fix expiry disabled check for new config API Similar to 26c30844cac6

util.startup: Fix notifying config-reload to systemd Does this event name seem backwards to anyone else?

mod_version: Fix uname result style (thanks riau) `result[, err]`, not `ok, err|result`, must have confused it with pcall

mod_server_contact_info: Sort form fields to please scansion The unstable hash table order caused the tests to fail and I don't know how to tell scansion to ignore the order.

mod_version: Handle access denied from uname() Discovered while experimenting with a stricter SystemCallFilter setting See man:systemd.exec(5)

mod_admin_shell: Add connection created time This adds an output format option to show the time that the connection was created. Ref #1852

Merge 0.12->trunk

util.startup: Support systemd Type=notify service type This lets Prosody report its lifecycle status to systemd, so it knows when Prosody has completed its startup, when it's reloading and shutting down. Both Type=notify and Type=notify-reload is supported Example systemd .service configuration snippet: [Service] Type=notify

mod_invites_adhoc: Fix result form type (thanks betarays)

MUC: Fix legacy hats (thanks nicoco) Why do we not have tests for this?

MUC: Switch to official XEP-0317 namespace for Hats (including compat) (thanks nicoco)

util.startup: Fix exiting on pidfile trouble prosody.shutdown() relies on prosody.main_thread, which has not been set yet at this point. Doing a clean shutdown might actually be harmful in case it tears down things set up by the conflicting Prosody, such as the very pidfile we were looking at. Thanks again SigmaTel71 for noticing

Merge 0.12->trunk

prosodyctl check: Warn about invalid domain names in the config file This ensures that domain names of virtual hosts and components are valid in XMPP, and that they are encoded correctly.

util.startup: Abort before initialization of logging when started as root Prevents creation of log files owned by the root user which could be inaccessible once started correctly.

util.startup: Don't use not yet existent shutdown procedure when started as root (thanks SigmaTel71)

util.startup: Check root after detecting platform and reading config (thanks SigmaTel71) Ensures that startup.detect_platform() runs so know whether to use the POSIX method of checking the current user or something else. Also after reading the config so we know whether the root override setting is set.

mod_posix: Move everything to util.startup This allows greater control over the order of events. Notably, the internal ordering between daemonization, initialization of libunbound and setup of signal handling is sensitive. libunbound starts a separate thread for processing DNS requests. If this thread is started before signal handling has been set up, it will not inherit the signal handlers and instead behave as it would have before signal handlers were set up, i.e. cause the whole process to immediately exit. libunbound is usually initialized on the first DNS request, usually triggered by an outgoing s2s connection attempt. If daemonization happens before signals have been set up, signals may not be processed at all.

mod_bosh: Set base_type on session This fixes a traceback with mod_saslauth. Ideally we move this to util.session at some point, though.

util.startup: Back out 598df17b8ebb Broke signal handling again, such that an early s2s connection results in libunbound catching signals and getting Prosody killed on e.g. SIGHUP This returns to the situation where prosody --daemonize does not respond to signals.

util.startup: Hook signals after daemonization signalfds stop working with epoll after forking hooking signals later should not affect anything

mod_pubsub: Ignore shadowed variable [luacheck]

mod_pubsub: Add shell commands to create and list nodes

core.features: Advertise that events are fired for SIGUSR1/2 Moved here from mod_posix since these events no longer originate there

util.startup: Fix firing of USR1/2 events

net.server: Restore epoll signalfd handling Reverts 4a9a69659727

mod_posix: Move POSIX signal handling into util.startup to avoid race When libunbound is initialized, it spawns a thread to work in. In case a module initializes libunbound, e.g. by triggering a s2s connection, Prosody would not handle signals, instead immediately quit on e.g. the reload (SIGHUP) signal. Likely because the libunbound thread would not have inherited the signal mask from the main Prosody thread. Thanks Menel, riau and franck-x for reporting and help narrowing down

net.server: Disable epoll signalfd handling by default until problems resolved

net.server_epoll: Log creation of signalfd handles at noise level To aid in tracking down signalfd-related problems

util.bit53: Add bnot() method

util.signal: Fail signalfd() if unable to change signal mask By aborting early, the failure should be brought to the attention somehow.

net.server_epoll: Log failure to hook signals To make any such failures noticeable

Merge 0.12->trunk

net.http.files: Validate argument to setup function Fixes error in #1765 by throwing an error earlier

mod_s2s: Comment on why we avoid hostnames in stanza bounce messages

mod_cron: Fix log format to account for float that was integer before

mod_cron: Sync Teal source with 92301fa7a673 Yeah, it's weird to have two versions. Needing more build dependencies is also something we want to avoid, so here we are.

util.signal: Wrap signalfd in an userdatum for gc handling etc

net.server_epoll: Support hooking signals via signalfd Handling signal events the same way as all other events makes sense and seems safer than the signal handling just jumping around in C and messing with Lua states.

util.signal: Add support for signalfd(2) on Linux signalfd allows handling signal events using the same method as sockets, via file descriptors. Thus all signal dispatch can go through the same main event loop as everything else, removing need for thread-scary signal handling where execution would just jump to the signal handler regardless of the state of Lua, and needing to keep track of Lua states/threads.

features: Add mod_server_info

mod_server_contact_info: Update to publish fields via new mod_server_info

mod_server_info: New module to manage the serverinfo disco extension form This allows multiple modules to populate the form dynamically. Currently the form is "owned" by mod_server_contact_info, which prevents other modules from contributing to it. A further commit will port mod_server_contact_info to use this module.

util.strbitop: Add common_prefix_bits() to Teal interface description

util.hashes: Add missing entries to Teal interface description Also sorted to match C source

util.crypto: Update Teal interface description to match C sources Was missing some entries. Rearranged to match order of entries in the C source Reg table.

tools/test_mutants.sh: Load loader helper when running busted

util.ip: Remove ip.bits and related code, switch to more efficient strbitop 100,000 iterations of match() on my laptop from 3.5s -> 0.5s.

util.strbitop: Remove unused import in tests

util.strbitop: Add common_prefix_bits() method This returns the number of bits that two strings have in common. It is significantly more efficient than similar calculations in Lua.

util.ip: Add another test case for match() and commonPrefixLength()

util.strbitop: Rename spec file to correct name so tests actually run

util.rfc6724: Remove, unused since introduction of Happy Eyeballs It was mainly used to determine whether to try IPv6 or IPv4 first, following the rules for this in the RFC. Now we always try IPv6 and IPv4 at roughly the same time, thus there no need to carry these rules.

features: Add module-ready (for commit e20949a10118)

util.startup: Expose core.features.available as prosody.features for convenience.

mod_s2s_auth_certs: Handle potential string error conn:ssl_peerverification() can now return a single error in case the connection has been closed for whatever reason

net.server_epoll: Prevent traceback when checking TLS after connection gone Unclear why this would be done, but an error is not great.

mod_cron: Allow configuring various "internal" delay parameters Notably, it is now possible to add a randomized spread factor to the check interval.

mod_c2s: Fix error on role change on Components (thanks Menel)

mod_smacks: Adjust buckets for resumption age statistic Given that there are recommendations floating around recommending 24 hours session lifetime, having buckets up to 10 minutes wouldn't be useful in that case. Would be nice if we had some way to automatically assign suitable number series for buckets, scaled to what the configuration might be.

mod_storage_internal: Fix off-by-one when searching archive for Fixes a test case provided by MattJ where the very first item matched by a 'start' timestamp was not returned.

mod_s2s_auth_dane_in: Try single TLSA lookup per draft-ietf-dance-client-auth Moves some complexity from the implementation into DNS operations.

mod_s2s_auth_dane_in: Simplify result processing Fewer loops

MUC: Record reason for affiliation changes and return in list (fixes #1227)

MUC: Test that <subject/> + <thread/> is not handled as subject change Ref #667 Ref #1838

mod_invites: Fix argument handling Not sure what the next() was supposed to do. Reject unknown --options perhaps?

mod_invites: Show short help instead of traceback on missing hostname

mod_invites: Show help if --help passed instead of hostname Because I couldn't guess the right way to get the help message without reading the source twice.

mod_invites: Allow specifying invite ttl on command line Was missing a way to pass TTL via command or shell.

Merge 0.12->trunk

mod_disco: Advertise disco#info and #items on bare JIDs to fix #1664 Having to add these in *there* places seems less than ideal. I would also think that advertising disco#info is a bit redundant, since it is a requirement for everything in XMPP and if it was missing you would get an error back.

scansion: Use new style for accessing Lua globals

scansion: Use new prosody namespace in import

scansion: Use captures or wildcards instead of mocking time > Mockery is one of the things I hold dear! And he's making a mockery of it!! -- Belkar Bitterleaf

util.xtemplate: Test the each template function It iterates over childtags, so a template like {foo|each{...}} would be equivalent to root:childtags("foo"), while a deeper query needs the bit that becomes arguments to :childtags as an argument to each, e.g. {foo/bar|each(baz)} would behave like root:get_child("foo"):get_child("bar"):childtags("baz")

tools: Fix selection of container engine Seems command -v in sh only checks and returns one argument, unlike bash.

mod_storage_internal, tests: Fix before/after combined with the 'reverse' flag

util.http: Silence strict luacheck warning in tests

core.moduleapi: Silence strict luacheck warnings in tests

util.throttle: Silence some strict luacheck warnings

util.prosodyctl.shell: Fix lint [luacheck]

CHANGES: Mention new prosodyctl shell method behavior

util.prosodyctl.shell: Add :method syntax to make e.g. MUC commands easier e.g. prosodyctl shell muc room room@muc.example.com :set_name "This Room"

util.xtemplate: Add some initial tests Strict typing does not magically make code correct

util.xtemplate: Adopt {-path-} syntax to strip preceding and/or trailing whitespace Seen in some other template languages

mod_http_errors: Simplify CSS via built-in dark mode

CHANGES: Document some of the recent changes and features in trunk

configmanager: Fix linter issues

configmanager: Support for appending to existing config options ...and some other useful operations

configmanager: Make _G accessible via `Lua` variable, deprecate direct access

configmanager: Allow referencing previously-set options in the config file

mod_user_account_management: Clear pending deletion if account re-enabled

mod_saslauth: Fire event per SASL step This matches the behaviour of the newer mod_sasl2 implementation. It allows plugins to observe (and potentially, with caution, modify) the SASL exchange.

util.jsonschema: Return basic structured validation response

mod_c2s: Make c2s_timeout timer reachable to allow access from other modules E.g. the timeout could be extended under certain conditions.

tools: Add a tool for comparing DOAP to the latest XEP versions Needs wget, awk, sed and xml2

mod_s2s: Close connection on smacks timeout This merges the mod_s2s_smacks_timeout behavior from prosody-modules This event is fired by mod_smacks when the connection has not responded to an ack-request for a period of time defaulting to 30 seconds, indicating that the connection has become stuck or non-responsive. Closing it prevents routing further messages via this connection and frees resources. A stuck connection may otherwise remain until for a time determined by the OS TCP subsystem, which can be quite long.

mod_saslauth: Fire event at start of authentication attempt As extension point for rate limiting and similar checks, so they can hook a single event instead of <{sasl1}auth> or stream features, which might not be fired in case of SASL2 or e.g. HTTP based login.

net.http.server: Fix whitespace-ignoring syntax

Merge 0.12->trunk

net.http.parser: Reject overlarge header section earlier This case would eventually be rejected by the buffer size limit.

lua-format: Let simple things be one line This doesn't really handle nesting all that nicely tho.

lua-format: Further tweaks Keeping things a single line makes very deeply nested things "pyramids" a single line, which makes them hard to read.

lua-format: Add new settings

lua-format: Tweaks attempting to fit our code style

lua-format: Check in defaults By starting with the built-in defaults, we get a nice history of differences from those as we figure out what settings suit us Sorted make comparisons easier.

mod_user_account_management: Fire events with a fake (not destroyed) session Previously these events fired after the session had been destroyed, which removes many of the useful properties. The ones I chose to preserve here are the ones used by the community module mod_audit, which seems like a good baseline.

mod_cron: Rebuild with new LuaFormatter settings (tabs!)