# HG changeset patch

# Parent 7a77fde8f7eea66ad121440646e1ffb63ad82749

[work-in-progress] Unified command option parsing

  * complete change of commands.h interface

    - uses different symbols to make compatibility layer possible

  * completion system is broken now

  * usable commands:

    - roster

    - color

    - status/status_to

    - add/del

    - group

    - say/msay/say_to

    - buffer

    - clear

    - info

  * it does compile, but have not tested at all

  * privatized say_cmd()

  * dropped cmd_setstatus()

  * process_line() still expects line to be in local encoding,

    while cmd_execute() expects utf8 and rw

  * process_line() can return error values, different from 255

  ** PREVIOUS ACHIEVEMENTS **

  * function interface changes:

    * buddy_search() now expects argument in utf8

    * cmd_setstatus() now expects separate status and message arguments

    * say_cmd()'s second argument is now of new type msgtype_t

    * scr_multi* now store multiline in utf8

  * /del:

    * allows specifying jid, as /add does

    * -n(--dryrun) switch for debugging purposes

  * /rename:

    * -r(--reset) instead of '-'

    * -j(--jid), -g(--group), -n(--name)

  * /move:

    * -j(--jid), -n(--name)

  * misc:

    * fix help for /buffer date

diff -r 7a77fde8f7ee mcabber/doc/HOWTO_commands.mdwn

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000

+++ b/mcabber/doc/HOWTO_commands.mdwn	Thu Aug 22 00:44:41 2013 +0300

@@ -0,0 +1,977 @@

+

+**New commands interface for MCabber**

+

+[[!toc levels=2]]

+

+# Overview

+

+New command interface was designed with next goals in mind:

+

+ * Share as much argument checking code as possible.

+ * Remove cumbersome parsing code from commands.

+ * Encourage adding options and switches to commands.

+ * Use the same rules, when handling arguments everywhere.

+ * Integrate and improve completion system.

+ * Add common generic stuff, like '--help'.

+ * Try to still be lightweight.

+ * Try to still be readable.

+

+It is built around static structure, "command description".  User can add or

+remove these structures to list of commands.  *FIXME* more

+

+## Command description struct, 'cmdopts_t'

+

+[[!format c """// --------------------------------------------------------------

+

+typedef struct cmdopts_struct cmdopts_t;

+

+typedef enum {

+    cmd_default = 0x0000,

+    cmd_safe    = 0x0001,

+} cmd_flags_t;

+

+struct cmdopts_struct {

+    const char          *name;

+    const cmd_flags_t   flags;

+    const cmd_checker_t check;

+    const cmd_handler_t handle;

+    const cmdopt_t      *opts;

+    const cmdarg_t      *args;

+    const cmdopts_t     *cmds;

+    const gpointer      userdata;

+    size_t              valno;

+};

+

+// --------------------------------------------------------------     """     ]]

+

+This struct describes command as a whole and links to argument descriptions.

+This struct is also used to describe individual subcommands, as they are quite

+similar to normal command, because they can have their own options, arguments

+and subcommands.  The fields of this struct:

+

+ * 'name' - name of the command or subcommand.

+

+ * 'flags' - currently there's only one flag:

+

+    + 'cmd_safe' -  command is safe to execute in main mcabberrc during

+      initialization.  Have no meaning for subcommands.

+    + 'cmd_default' - default value of no flags enabled.

+ 

+ * 'check' - execution environment checker for command.  This callback is used

+   to do general checks before even parsing command line.  You can write your

+   own checker or use standard ones, for example - 'cmd_check_online', that

+   checks, if you are currently online.

+

+ * 'handle' - command function.  It is executed only if command line was

+   successfully parsed and argument values passed the type checking.  Unused in

+   subcommands.

+

+ * 'opts' - pointer to the array of 'cmdopt_t' structs, describing command-line

+   options ("-f bar") and switches ("-x"), that this command accepts.  Array

+   must end with option, that have 0 as short option character.

+ 

+ * 'args' - similarly, pointer to the array of 'cmdarg_t' structs, that describe

+   command-line positional arguments (in order).  Array must end with argument,

+   that have NULL name.

+ 

+ * 'cmds' - pointer to the array of subcommands of this command (or subcommand).

+   How parser switches to subcommands we will describe later.  Array must end

+   with subcommand, that have NULL name.

+ 

+ * 'userdata' - arbitrary pointer, where you can put some data, that should

+   accompany this command or subcommand.  Unused by parser.

+

+ * 'valno' - this is internal value, that is initialized at command definition

+   time, you should not modify it.  Currently unused in subcommands.

+

+## Command function, 'cmd_handler_t'

+

+[[!format c """// --------------------------------------------------------------

+

+typedef gchar *(*cmd_handler_t) (const cmdopts_t *command,

+                                 cmdarg_value_t  *values);

+

+// --------------------------------------------------------------     """     ]]

+

+Command function is passed it's command definition struct (mainly to give it

+access to userdata) and dynamically allocated array of parsed argument values.

+

+It should return NULL in case all went smoothly, or dynamically allocated error

+string, that will be g_free()'d after displaying.

+

+So, as you can see, command definition should give parser info on what can

+appear in command line and how to map all these options and arguments to array

+of values.

+

+## Option description struct, 'cmdopt_t'

+

+[[!format c """// --------------------------------------------------------------

+

+typedef struct {

+    const char stortopt;

+    const char *longopt;

+    cmdarg_t   arg;

+} cmdopt_t;

+

+// --------------------------------------------------------------     """     ]]

+

+This struct just adds short option character and long option name to generic

+argument struct, that we'll look at right now.

+

+## Argument description struct, 'cmdarg_t'

+

+[[!format c """// --------------------------------------------------------------

+

+typedef struct cmdarg_struct cmdarg_t;

+

+typedef enum {

+    cmdarg_default  = 0x0000,

+    cmdarg_catchall = 0x0001,

+    cmdarg_plain    = 0x0002,

+    cmdarg_check    = 0x0004,

+    cmdarg_required = 0x0008,

+    cmdarg_subcmd   = 0x0010,

+    cmdarg_switch   = 0x0020,

+    cmdarg_eol      = 0x0003, // catchall + plain

+    cmdarg_chreq    = 0x000C, // check + required

+    cmdarg_special  = 0x0030, // subcmd + switch

+    cmdarg_trigger  = 0x0024, // switch + check

+} cmdarg_flags_t;

+

+struct cmdarg_struct {

+    const char           *name;

+    const guint          pos;

+    const cmdarg_flags_t flags;

+    const char           *defval;

+    const cmdarg_type    *type;

+    gconstpointer        chkdata;

+    gconstpointer        userdata;

+};

+

+// --------------------------------------------------------------     """     ]]

+

+This struct stores information about mapping between command-line entity

+(switch, option, argument, subcommand) and element in 'values' array.  First,

+let's briefly describe fields, and then walk through their use in different

+entities.

+

+Fields:

+

+ * 'name' - argument name, mainly used for help (not implemented yet) and error

+   messages.

+ 

+ * 'pos' - this is the index in 'values' array, where argument value should be

+   stored.

+ 

+ * 'flags' - various tweaks to parsing process:

+

+    + 'catchall' - argument value will catch everything, remaining on command

+      line, even if unescaped spaces will appear in it.

+    + 'plain' - do not treat backslashes ('\') and quotes ('"') as escaping

+      characters.

+    + 'check' - call argument value checker (defined in 'type' field), even if

+      argument was not assigned the value during the parsing process.

+    + 'required' - if mentioned checker will return error, with this flag set,

+      this error will be considered fatal, and command function will not be

+      called.

+    + 'subcmd' - this argument is subcommand.

+    + 'switch' - this argument is part of option definition, and this option

+      have no argument (but it still needs to store switch state somewhere).

+    + 'eol' - shortcut for "rest of command line without modification".

+    + 'chreq' - shortcut for "argument must have a valid value".

+    + 'special' - this is special type of argument - subcommand or switch.

+ 

+ * 'defval' - default value for argument in "unchecked" form - i.e., in the form

+   of string, as it appears on command line (we'll discuss type checkers and

+   what can they do to value later).  Before parsing command line, parser will

+   assign this value to corresponding value structure.

+

+ * 'type' - pointer to structure, describing argument type.

+

+ * 'chkdata' - if type needs some additional info, it is a place to supply it.

+

+ * 'userdata' - place for arbitrary info, that should accompany this argument or

+   option.  Unused by parser.

+

+Now, let's discuss general case - option with argument or positional argument.

+

+When parser encounters such argument, it checks 'catchall' and 'plain' flags and

+crops argument value from command line accordingly, then it assigns it to the

+'value.arg' field of 'cmdarg_value_t' struct in array 'values' with index, given

+in 'pos'.  Also it marks such value as 'visited' and puts a link to argument

+description into value's 'src' field.  More about effect of these actions later,

+in 'cmdarg_value_t' section.

+

+However, if argument struct is a part of option description struct, and have

+flag 'switch' set, parser will not try to parse value.  Instead it will increase

+the count in 'value.swc' in corresponding 'cmdarg_value_t' struct.  Argument

+name for switches is essentially ignored, but for safety sake it is recommended

+to duplicate switch long option name here.  Flags 'catchall' and 'plain'

+obviously have no effect.  Flag 'check' makes switch a trigger, that flips

+between 'on' and 'off' for every occurence of flag on command line.  Flags

+'required' and 'subcmd' also have no effect for obvious reasons.  Default value

+is ignored for switches, they always start with count 0 (off).  Since switch is

+internal type, argument type field is ignored as well.  Flags and source fields

+of value are updated as usual.

+

+If flag 'subcmd' is set for positional argument, parser crops argument value

+according to flags as usual, but instead of assigning it, walks through list of

+subcommands in command description struct and compares obtained value to

+subcommand names.  If it finds corresponding subcommand, it assigns pointer to

+subcommand description struct to 'value.cmd' field of corresponding

+'cmdarg_value_t' struct and updates it's source and flags.  Then, instead of

+proceeding with parsing process, it recursively calls parser on remaining part

+of command line, passing subparser subcommand description.  Note, that if

+subcommand parser will end parsing before hitting end of command line, parser

+will proceed with parsing arguments for main command.  Default value and

+argument type fields are ignored for subcommands.

+

+Now let's take a look at value structure, that you'll be dealing with the most

+in the actual code.

+

+## Argument value structure, 'cmdarg_value_t'

+

+[[!format c """// --------------------------------------------------------------

+

+typedef struct cmdarg_value_struct cmdarg_value_t;

+

+typedef enum {

+    cmdval_default = 0x0000,

+    cmdval_visited = 0x0100,

+    cmdval_freeme  = 0x0200,

+} cmdval_flags_t;

+

+struct cmdarg_value_struct {

+    const cmdarg_t *src;

+    cmdval_flags_t flags;

+    union {

+        guint           uint;

+        gint            sint;

+        guint           swc;

+        const gchar     *roarg;

+        gchar           *arg;

+        const cmdopts_t *cmd;

+        struct {

+            gpointer    bud;

+            gchar       *resource;

+        } rjid;

+        gpointer        ptr;

+    } value;

+};

+

+// --------------------------------------------------------------     """     ]]

+

+Command may happen to be called recursively - i.e., something in command may

+cause the event, that will cause another call to the same command, thus we

+cannot store actual values in command/argument definition struct, and have to

+allocate dynamic memory for them.  This struct is designed exactly for this.

+Let's take a look at it's fields:

+

+ * 'src' - this points to the argument description, from which this struct is

+   holding value right now (note, that value can be initialized several times

+   during parsing process from different arguments).

+ 

+ * 'flags' - to hold parser and typechecker marks:

+

+    + 'visited' - parser uses this to track values, initialized from command

+      line as opposed to values, holding default value.

+    + 'freeme' - used by argument type checker to tell parser, that it needs to

+      call value destructor callback on value.

+ 

+ * 'value' - union, that contains various possible forms of value:

+

+    + 'uint'  - generic unsigned integer.

+    + 'sint'  - generic signed integer.

+    + 'swc'   - switch occurence count.

+    + 'roarg' - XXX read-only string - default value or like.

+    + 'arg'   - generic string value (read-write).

+    + 'cmd'   - pointer to subcommand description struct.

+    + 'rjid'  - roster jid value - pair of roster buddy and resource string.

+    + 'ptr'   - used for anything, that does not fits into these types.

+

+To better understand, how these fields are used, let's walk through parsing

+process.

+

+## Parsing

+

+Parser starts by allocating memory for values array, and then initializing it by

+walking through command description, looking at options and arguments and

+assigning default values to corresponding entries in array.  It also puts

+pointer to the argument description into value's 'src' field.  Thus, all used in

+command description values will have this field initialized, even if they were

+not specified on command line.  This comes handly later, when checking for

+reqired value presence.  For switches parser just sets the counter to zero.

+Note, that parser does not descend into subcommands at this stage.  It does the

+same procedure for subcommand, but later, when it already knows which subcommand

+is selected.  Also note, that if several arguments have the same value index,

+parser will use latest encountered one to initialize the value.  This is used

+for default value in "argument clustering", that I'll show you later.

+

+Then parser calls command environment checker callback (if present), and if it

+returns error - terminates the process right now.  Note, that subcommands can

+also have checkers.

+

+Next parser does its job of parsing command line.  Each time it extracts

+argument value, it into 'value.arg' field of corresponding value entry and

+pointer to argument description struct into 'src' field.  Also it sets 'visited'

+flag on value.  At this stage value is still just unchecked string, except for

+special argument types.  For switch occurence count in 'value.swc' gets

+increased each time argument was specified.  Note however, that if several

+switches use the same value index ("clustered switches"), counter gets reset,

+when switches change one another in sequence - i.e. "-e -s -s -s" will count as

+three "-s", but "-s -s -e -s" will count as only one "-s".  For subcommands

+parser checks for corresponding subcommand in 'cmds' list, assigns it to

+'value.cmd' and recursively passes the end of command line to be parsed with

+subcommand description.  Note, that for subcommands parser does "checking on the

+spot" - if parsed argument value does not match any subcommand, and argument

+have 'required' flag set, it raises error immediately (if flag is not set, it

+merely assigns NULL and proceeds parsing according to current command

+description).

+

+Then parser walks through array of values and performs value checking.  Note,

+that all values, that need checking at this point should have 'src' field

+initialized - either set at default value assignment step, or during parsing,

+so, parser knows properties of value's argument.  Parser only pays attention to

+the values, that either have 'visited' flag set (i.e. provided by user) or that

+have 'check' flag in argument description (useful for mandatory arguments or

+default values, that need convesion).  If value corresponds to a switch, and

+argument have 'check' flag set, switch occurence count is replaced by remainder

+of division it by 2 (this way switch behaves like trigger).  If it is a

+subcommand, and it have 'required' flag set, parser checks, if it have non-NULL

+value.  If it is usual argument (option or positional), and it does have 'type',

+that have 'check' callback set, parser calls this checker, passing it value

+structure (again, value structure contains pointer to argument description, so,

+checker can access 'chkdata' field, supplied by user).  If checker returns error

+string and argument have 'required' flag set, parser raises error.  If flag is

+not set, parser just prints warning and proceeds with checking.

+

+If checking was successful, parser calls command function, providing it with

+command description and values array.  This function can also return error, but

+at this stage it does not change process, only causes error message to be

+printed.

+

+And finally parser frees allocated resources - walks through values array,

+calling argument type 'free' callback on values, that have 'freeme' flag set and

+then frees the array itself.

+

+## Argument type, 'cmdarg_type_t'

+

+[[!format c """// --------------------------------------------------------------

+

+typedef struct cmdarg_type_struct cmdarg_type_t;

+

+typedef gchar *(*cmdarg_checker_t) (cmdarg_value_t *value);

+typedef void (*cmdarg_destructor_t) (cmdarg_value_t *value);

+// FIXME: this one is still not designed

+typedef void (*cmdarg_completor_t) (void);

+

+struct cmdarg_type_struct {

+    cmdarg_checker_t    check;

+    cmdarg_destructor_t free;

+    cmdarg_completor_t  complete;

+};

+

+// --------------------------------------------------------------     """     ]]

+

+As you can see, argument type is nothing more than a set of callbacks:

+

+ * 'check' - check parsed argument value for conformance to type rules, possibly

+   replace with something different, like corresponding integer value or roster

+   buddy pointer.

+

+ * 'free' - if type checker may need to free some data afterwards, this callback

+   should be set to corresponding function, and each time checker really needs

+   value to be freed, it should set flag 'freeme' on value.

+

+ * 'complete' - *FIXME* not yet designed callback, that will return list of

+   possible completions according to given prefix.

+

+After parsing command line parser performs argument value checking, that's where

+it calls 'check' callbacks.   Checker is given pointer to value structure, that

+it needs to check.  Checker can modify value string (except when it is default

+value, but you have to supply your default values so, that they do not need

+modifying) or completely replace it with another string or even non-string

+object.  If checker uses some resources (eg.  allocates memory for replacement

+value), it can set the flag 'freeme' on value to request call to value

+destructor, when values array will be freed.  If checker needs some additional

+data (eg. it is some generic checker, that needs list of valid values or other

+parameters), these data can be supplied in 'chkdata' field.  Checker function

+should return NULL on success or error string, that will be g_free()'d by

+parser.  Take note, that if argument does not have 'reqired' flag set, parser

+will ignore checker error, so, it is recommended to nullify invalid value before

+returning error (but it is not required).

+

+# Examples

+

+When writing description for a command, first thing, you have to do - is to

+determine, which values your command can get from user.  You don't have to be

+shy - new interface is designed to encourage commands to be as flexible and

+option-rich as possible.  

+

+Second - consider, which ones are to be specified as positional arguments, and

+which should become options or switches.

+

+Next you will want to decide, which checks and restrictions should you put on

+values.  Essentially, determine value type.