Parse variable number of commands with python argparse

Question

I'm developing a command line tool with Python whose functionality is broken down into a number of sub-commands, and basically each one takes as arguments input and output files. The tricky part is that each command requires different number of parameters (some require no output file, some require several input files, etc).

Ideally, the interface would be called as:

./test.py ncinfo inputfile

Then, the parser would realise that the ncinfo command requires a single argument (if this does not fit the input command, it complains), and then it calls the function:

ncinfo(inputfile)

that does the actual job.

When the command requires more options, for instance

./test.py timmean inputfile outputfile

the parser would realise it, check that indeed the two arguments are given, and then then it calls:

timmean(inputfile, outputfile)

This scheme is ideally generalised for an arbitrary list of 1-argument commands, 2-argument commands, and so on.

However I'm struggling to get this behaviour with Python argparse. This is what I have so far:

#! /home/navarro/SOFTWARE/anadonda3/bin/python
import argparse

# create the top-level parser
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()

# create the parser for the "ncinfo" command
parser_1 = subparsers.add_parser('ncinfo', help='prints out basic netCDF strcuture')
parser_1.add_argument('filein', help='the input file')

# create the parser for the "timmean" command
parser_2 = subparsers.add_parser('timmean', help='calculates temporal mean and stores it in output file')
parser_2.add_argument('filein', help='the input file')
parser_2.add_argument('fileout', help='the output file')


# parse the argument lists
parser.parse_args()

print(parser.filein)
print(parser.fileout)

But this doesn't work as expected. First, when I call the script without arguments, I get no error message telling me which options I have. Second, when I try to run the program to use ncinfo, I get an error

./test.py ncinfo testfile
Traceback (most recent call last):
  File "./test.py", line 21, in <module>
    print(parser.filein)
AttributeError: 'ArgumentParser' object has no attribute 'filein'

What am I doing wrong that precludes me achieving the desired behaviour? Is the use of subparsers sensible in this context?

Bonus point: is there a way to generalise the definition of the commands, so that I do not need to add manually every single command? For instance, grouping all 1-argument commands into a list, and then define the parser within a loop. This sounds reasonable, but I don't know if it is possible. Otherwise, as the number of tools grows, the parser itself is going to become hard to maintain.

You can do something like for name in one_arg_commands: and then create each of the subparsers for that, then for name in two_arg_commands:, and so on. — Barmar
– Barmar, Commented Jan 10, 2019 at 9:10
Maybe this can help python argparse different parameters with different number of arguments — SBylemans
– SBylemans, Commented Jan 10, 2019 at 9:21
Normally you get help from argparse with -h or --help. There's no default behavior when given no arguments. During debugging I like to print(args) to have a clear idea of what attributes it has set. You might also want to add a dest to add_subparsers to see which subparser was being used. And set args=parser.parse_args(). Parsing creates a args namesapce; it doesn't modify the parser. — hpaulj
– hpaulj, Commented Jan 10, 2019 at 16:02
I'm a noob, so I can believe that there is a good reason for that default... but I don't see it. It seems a bad idea to leave the program running when the command line does not fulfil any expected configuration. As a matter of fact, the example in the response of @simleo crashes (raises an exception) when no arguments are given. I'd expect the parser to behave as if the -h flag had been used. Actually, this IS the behaviour when no sub-commands are used. I'm strongly tempted to believe this is a bug... — Pythonist
– Pythonist, Commented Jan 11, 2019 at 8:03

simleo · Accepted Answer · 2019-01-12 16:14:38Z

0

import argparse
import sys

SUB_COMMANDS = [
    "ncinfo",
    "timmean"
]


def ncinfo(args):
    print("executing: ncinfo")
    print("  inputfile: %s" % args.inputfile)


def timmean(args):
    print("executing: timmean")
    print("  inputfile: %s" % args.inputfile)
    print("  outputfile: %s" % args.outputfile)


def add_parser(subcmd, subparsers):
    if subcmd == "ncinfo":
        parser = subparsers.add_parser("ncinfo")
        parser.add_argument("inputfile", metavar="INPUT")
        parser.set_defaults(func=ncinfo)
    elif subcmd == "timmean":
        parser = subparsers.add_parser("timmean")
        parser.add_argument("inputfile", metavar="INPUT")
        parser.add_argument("outputfile", metavar="OUTPUT")
        parser.set_defaults(func=timmean)


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('-o', '--common-option', action='store_true')
    subparsers = parser.add_subparsers(help="sub-commands")
    for cmd in SUB_COMMANDS:
        add_parser(cmd, subparsers)
    args = parser.parse_args(sys.argv[1:])
    if args.common_option:
        print("common option is active")
    try:
        args.func(args)
    except AttributeError:
        parser.error("too few arguments")

Some usage examples:

$ python test.py --help
usage: test.py [-h] [-o] {ncinfo,timmean} ...

positional arguments:
  {ncinfo,timmean}     sub-commands

optional arguments:
  -h, --help           show this help message and exit
  -o, --common-option
$ python test.py ncinfo --help
usage: test.py ncinfo [-h] INPUT

positional arguments:
  INPUT

optional arguments:
  -h, --help  show this help message and exit
$ python test.py timmean --help
usage: test.py timmean [-h] INPUT OUTPUT

positional arguments:
  INPUT
  OUTPUT

optional arguments:
  -h, --help  show this help message and exit
$ python test.py -o ncinfo foo
common option is active
executing: ncinfo
  inputfile: foo
$ python test.py -o timmean foo bar
common option is active
executing: timmean
  inputfile: foo
  outputfile: bar

edited Jan 12, 2019 at 16:14

answered Jan 10, 2019 at 9:36

simleo

2,99525 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Pythonist Over a year ago

Thanks a lot for your direct and clear answer. This indeed solves the question. The only caveat I still see is that this program crashes (raises an AttributeError exception) when no arguments are given. I'd expect that in this case, the parser behaves as if the -h flag was used. This seems to be indeed the behaviour when simpler parsers without subparsers are used. Isn't this behaviour inconsistent? Is it possible to change the behaviour to stop the script when no arguments are given?

simleo Over a year ago

Yes, that's inconsitent and it looks like it's been introduced in Python 3. If you run the script with no arguments in Python2, it exits cleanly with "error: too few arguments". I have updated the above code to get the same behavior in Python 3

Collectives™ on Stack Overflow

Parse variable number of commands with python argparse

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related