当前位置 博文首页 > python使用BeautifulSoup分析网页信息的方法

    python使用BeautifulSoup分析网页信息的方法

    作者:令狐不聪 时间:2021-07-29 18:52

    本文实例讲述了python使用BeautifulSoup分析网页信息的方法。分享给大家供大家参考。具体如下:

    这段python代码查找网页上的所有链接,分析所有的span标签,并查找class包含titletext的span的内容

    复制代码 代码如下:
    #import the library used to query a website
    import urllib2

    #specify the url you want to query
    url = "http://www.python.org"

    #Query the website and return the html to the variable 'page'
    page = urllib2.urlopen(url)

    #import the Beautiful soup functions to parse the data returned from the website
    from BeautifulSoup import BeautifulSoup

    #Parse the html in the 'page' variable, and store it in Beautiful Soup format
    soup = BeautifulSoup(page)

    #to print the soup.head is the head tag and soup.head.title is the title tag
    print soup.head
    print soup.head.title

    #to print the length of the page, use the len function
    print len(page)

    #create a new variable to store the data you want to find.
    tags = soup.findAll('a')

    #to print all the links
    print tags

    #to get all titles and print the contents of each title
    titles = soup.findAll('span', attrs = { 'class' : 'titletext' })
    for title in allTitles:
    print title.contents

    希望本文所述对大家的Python程序设计有所帮助。

    jsjbwy