Python: why str.join(iterable) instead of str.join(*strings)

Question

I'm constantly wrapping my str.join() arguments in a list, e.g.

'.'.join([str_one, str_two])

The extra list wrapper always seems superfluous to me. I'd like to do...

'.'.join(str_one, str_two, str_three, ...)

... or if I have a list ...

'.'.join(*list_of_strings)

Yes I'm a minimalist, yes I'm picky, but mostly I'm just curious about the history here, or whether I'm missing something. Maybe there was a time before splats?

Edit:

I'd just like to note that max() handles both versions:

max(iterable[, key]) max(arg1, arg2, *args[, key])

Would you rather have extra "splats", or extra lists? Besides the fact that "splats" may not have been around forever. ** is fairly new. — pydsigner
– pydsigner, Commented Nov 9, 2012 at 21:19
this is a duplicate,I can't search for the original right now.the answer is that the function exists in that form because it supports joining any type of collection(and not just lists). the version you want is basically pointless as the arguments are still received by the function as a collection — vikki
– vikki, Commented Nov 9, 2012 at 21:29
The iterable use case is far more common - How often do you really have a defined number of unique variables that you want to join symmetrically? Either they're related, and they should be in a list, or they're not, and joining them is peculiar. — Eric
– Eric, Commented Nov 9, 2012 at 21:30
@user1062565: For two strings, what's wrong with str_one + '.' + str_two? That's fewer characters and it's clearer! — Eric
– Eric, Commented Nov 9, 2012 at 21:37

lqc · Accepted Answer · 2012-11-10 22:07:12Z

4

For short lists this won't matter and it costs you exactly 2 characters to type. But the most common use-case (I think) for str.join() is following:

''.join(process(x) for x in some_input) 
# or
result = []
for x in some_input:
    result.append(process(x))
''.join(result)

where input_data can have thousand of entries and you just want to generate the output string efficiently.

If join accepted variable arguments instead of an iterable, this would have to be spelled as:

''.join(*(process(x) for x in some_input))
# or
''.join(*result)

which would create a (possibly long) tuple, just to pass it as *args.

So that's 2 characters in a short case vs. being wasteful in large data case.

History note

(Second Edit: based on HISTORY file which contains missing release from all releases. Thanks Don.)

The *args in function definitions were added in Python long time ago:

==> Release 0.9.8 (9 Jan 1993) <==

Case (a) was needed to accommodate variable-length argument lists; there is now an explicit "varargs" feature (precede the last argument with a '*'). Case (b) was needed for compatibility with old class definitions: up to release 0.9.4 a method with more than one argument had to be declared as "def meth(self, (arg1, arg2, ...)): ...".

A proper way to pass a list to such functions was using a built-in function apply(callable, sequence). (Note, this doesn't mention **kwargs which can be first seen in docs for version 1.4).

The ability to call a function with * syntax is first mentioned in release notes for 1.6:

There's now special syntax that you can use instead of the apply() function. f(*args, **kwds) is equivalent to apply(f, args, kwds). You can also use variations f(a1, a2, *args, **kwds) and you can leave one or the other out: f(args), f(*kwds).

But it's missing from grammar docs until version 2.2.

Before 2.0 str.join() did not even exists and you had to do from string import join.

edited Nov 10, 2012 at 22:07

answered Nov 9, 2012 at 21:35

lqc

7,4081 gold badge28 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Alex Buchanan Over a year ago

I hadn't though of the list comprehension case. Thanks!

Don Question Over a year ago

seems like *args aka varargs was added in 0.9.8 see: svn.python.org/projects/python/trunk/Misc/HISTORY and search vor vararg

Gareth Rees Over a year ago

You write, "If join accepted variable arguments instead of an iterable, this ... would create a (possibly long) tuple." But if you pass an iterable to join, Python creates a (possibly long) list anyway, so it makes no difference. See PyUnicode_Join, which calls PySequence_Fast, which converts an iterable to a list.

Mark Byers · Accepted Answer · 2012-11-09 21:32:32Z

2

You'd have to write your own function to do that.

>>> def my_join(separator, *args):
        return separator.join(args)

>>> my_join('.', '1', '2', '3')
'1.2.3'

Note that this doesn't avoid the creation of an extra object, it just hides that an extra object is being created. If you inspect the type of args, you'll see that it's a tuple.

If you don't want to create a function and you have a fixed list of strings then it would be possible to use format instead of join:

'{}.{}.{}.{}'.format(str_one, str_two, str_three, str_four)

It's better to just stick with '.'.join((a, b, c)).

edited Nov 9, 2012 at 21:32

answered Nov 9, 2012 at 21:21

Mark Byers

843k202 gold badges1.6k silver badges1.5k bronze badges

5 Comments

Don Question Over a year ago

i was about to explain, but hit enter to switch to the next line, which istn so clever in comments ;-) and i SURELY DIDNT downvote - i seldom do - and never without explanation

Alex Buchanan Over a year ago

Thanks Mark! My question isn't about how, so much as why. Introducing a custom join is overkill, and format() for joining two strings always seems overkill too. Again, I'm purposefully being overly picky about style because Python sets the bar so high in terms of style.

Don Question Over a year ago

i was searching for a reference to the introduction of the splash-operator, because i had the impression it's not been in python from the beginnning and i would reason that the method signature S.join(iterable) -> string which doesn't incorporate the splash-operator should remain mostly untouched because of compatibility reasons, like you don't want something originally build with cpython break under another implementation on such basic matters

Mark Byers Over a year ago

@user1062565: Why is something only the person who wrote str.join can answer. Probably Guido...

Don Question Over a year ago

I just don't find any reference to the introduction of the splash operator. The earliest reference i found was for python 1.5, so i can't tell if the "late" introduction (which would be a possible 'why' if it's not a figment of my imagination) may have to do with it - sorry.

Kos · Accepted Answer · 2012-11-09 21:43:52Z

2

Argh, now this is a hard question! Try arguing which style is more minimalist... Hard to give a good answer without being too subjective, since it's all about convention.

The problem is: We have a function that accepts an ordered collection; should it accept it as a single argument or as a variable-length argument list?

Python usually answers: Single argument; VLAL if you really have a reason to. Let's see how Python libs reflect this:

The standard library has a couple examples for VLAL, most notably:

when the function can be called with an arbitrary number of separate sequences - like zip or map or itertools.chain,
when there's one sequence to pass, but you don't really expect the caller to have the whole of it as a single variable. This seems to fit str.format.

And the common case for using a single argument:

When you want to do some generic data processing on a single sequence. This fits the functional trio (map*, reduce, filter), and specialized spawns of thereof, like sum or str.join. Also stateful transforms like enumerate.
- The pattern is "consume an interable, give another iterable" or "consume an iterable, give a result".

Hope this answers your question.

Note: map is technically var-arg, but the common use case is just map(func, sequence) -> sequence which falls into one bucket with reduce and filter.

*The obscure case, map(func, *sequences) is conceptually like map(func, izip_longest(sequences)) - and the reason for zips to follow the var-arg convention was explained before.

I Hope you follow my thinking here; after all it's all a matter of programming style, I'm just pointing at some patterns in Python's library functions.

answered Nov 9, 2012 at 21:43

Kos

72.7k28 gold badges174 silver badges239 bronze badges

2 Comments

mgilson Over a year ago

I think map(func,*sequences) is more like starmap(func,izip_longest(sequences)) -i.e. the tuple gets unpacked when sent to func instead of being passed as a tuple which is just a minor nitpick.

Kos Over a year ago

True, that's why I said "conceptually" (~= "in terms of what data get passed, not how it gets passed"). I didn't think of starmap though, it's a good example here too!

Collectives™ on Stack Overflow

Python: why str.join(iterable) instead of str.join(*strings)

3 Answers 3

History note

3 Comments

5 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

History note

3 Comments

5 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related