Convert a list of string sentences to words

Question

I'm trying to essentially take a list of strings containg sentences such as:

sentence = ['Here is an example of what I am working with', 'But I need to change the format', 'to something more useable']

and convert it into the following:

word_list = ['Here', 'is', 'an', 'example', 'of', 'what', 'I', 'am',
'working', 'with', 'But', 'I', 'need', 'to', 'change', 'the format',
'to', 'something', 'more', 'useable']

I tried using this:

for item in sentence:
    for word in item:
        word_list.append(word)

I thought it would take each string and append each item of that string to word_list, however the output is something along the lines of:

word_list = ['H', 'e', 'r', 'e', ' ', 'i', 's' .....etc]

I know I am making a stupid mistake but I can't figure out why, can anyone help?

Marden

Answer 1

You need str.split() to split each string into words:

word_list = [word for line in sentence for word in line.split()]

Kure

Thanks again, I knew I was missing something easy like that, much appreciated! – Marden

This should be [word for line in sentence for word in line.split()]. – Newmann

Upvoted, though keep in mine more than 2 clauses of iteration is generally frowned upon in list comprehensions. – Accusatorial

I know this has been sometime but can I get explanation on the code? I understand [line for line in sentence] but I dont understand the second half for word in line.split(). How is it different from [line.split() for line in sentence] ? – Ephram

One important syntax to learn here is in single list notation with multiple for loops, thanks .. – Zumwalt

Answer 2

Just .split and .join:

word_list = ' '.join(sentence).split(' ')

Wilmerwilmette

Answer 3

You haven't told it how to distinguish a word. By default, iterating through a string simply iterates through the characters.

You can use .split(' ') to split a string by spaces. So this would work:

for item in sentence:
    for word in item.split(' '):
        word_list.append(word)

Hawkshaw

Answer 4

for item in sentence:
    for word in item.split():
        word_list.append(word)

Lacagnia

Answer 5

Split sentence into words:

print(sentence.rsplit())

Irrelevant