list management python

Question

I have extracted some url list and want to manipulate this list. Following is extracted list sample:

http://help.naver.com/service/svc_index.jsp?selected_nodeId=NODE0000000235
http://www.naver.com/rules/service.html
http://news.naver.com/main/principle.nhn
http://www.naver.com/rules/privacy.html
http://www.naver.com/rules/disclaimer.html
http://help.naver.com/claim_main.asp
http://news.naver.com/main/ombudsman/guidecenter.nhn?mid=omb
http://www.nhncorp.com/
http://www.nhncorp.com/

I want to extract only URLs that start with 'http://www.naver.com', so finally what I want list is following

http://www.naver.com/rules/privacy.html
http://www.naver.com/rules/disclaimer.html
http://www.naver.com/rules/service.html

How can I only extract what I want?

Answer 1

If your old list is contains all urls as strings you can use a list comprehension to filter them.

new = [url for url in old if url.startswith('http://www.naver.com')]

You could write it as a explicit loop, but it adds nothing but lines of code:

new = []
for url in old:
   if url.startswith('http://www.naver.com'):
       new.append( url )

If you planned on removing items from the original list while looping over it: Don't ever do that, it won't work. You can modify the original list instead with the same LC:

old[:] = [url for url in old if url.startswith('http://www.naver.com')]

Answer 2

You can do this with a List Comprehension. These are a very powerful way to work with lists with Python.

By adding add an if to the list comprehension you can filter the list.

Assuming your URLs are stored in the variable myurls:

filteredurls = [url for url in myurls if url.startswith('http://www.naver.com')]

Answer 3

urlList = [ ... ] # your list of urls
extractedList = [url for url in urlList if url.startswith('http://www.naver.com')]

Answer 4

result = []
for url in myListOfUrls:
    if 'http://www.naver.com' in url:
        result.append(url)

Answer 5

Someone suggested this alternative answer based on filter() but deleted it, I'll post it here again for completeness:

newList = filter(lambda url: url.startswith('http://www.naver.com'), oldList)

The list comprehension method seems faster though (and in my opinion, more readable):

$ python -m timeit -c "filter(lambda url: url.startswith('1'), map(str, range(100)))"
10000 loops, best of 3: 143 usec per loop

$ python -m timeit -c "[ url for url in map(str, range(100)) if url.startswith('1') ]"
10000 loops, best of 3: 117 usec per loop