标签：python nltk text-mining

我一直在修补NLTK,目的是从一些新闻文章中提取实体,但我不断收到错误,ValueError：块结构必须包含标记的标记或树.

谁能帮我？

import lxml.html
import nltk, re, pprint 



def ie_preprocess(document):
    """This function takes raw text and chops and then connects the process to break     
       it down into sentences, then words and then complete part-of-speech tagging"""
    sentences = nltk.sent_tokenize(document)
    sentences = [nltk.word_tokenize(sent) for sent in sentences]
    sentences = [nltk.pos_tag(sent) for sent in sentences]
    return sentences


    #import story
    base_url = "http://www.thisisstaffordshire.co.uk/Yobs-pelt-999-crews-bottles-fireworks-Shelton/story-17256383-detail/story.html"
    page = lxml.html.parse(base_url)
    story = page.xpath('//*[@id="story"]/div[2]/div[1]')
    raw_text = story[0].text_content()
    #tokenize
    output = ie_preprocess(raw_text)
    print output
    #chunk
    grammar = r'''
       NP: 
       {<DT><NN.*><.*>*<NN.*>} 
       '''
    cp = nltk.RegexpParser(grammar)

    chunked = cp.parse(output)

    print chunked

更新：这是完整的错误消息.

Traceback (most recent call last):
  File "geo_locator.py", line 30, in <module>
    chunked = cp.parse(output)
  File "/Users/davidelks/pythontests/venv/lib/python2.7/site-packages/nltk/chunk/regexp.py", line 1183, in parse
    chunk_struct = parser.parse(chunk_struct, trace=trace)
  File "/Users/davidelks/pythontests/venv/lib/python2.7/site-packages/nltk/chunk/regexp.py", line 999, in parse
     chunkstr = ChunkString(chunk_struct)
  File "/Users/davidelks/pythontests/venv/lib/python2.7/site-packages/nltk/chunk/regexp.py", line 93, in __init__
tags = [self._tag(tok) for tok in self._pieces]
  File "/Users/davidelks/pythontests/venv/lib/python2.7/site-packages/nltk/chunk/regexp.py", line 103, in _tag
    raise ValueError('chunk structures must contain tagged '
ValueError: chunk structures must contain tagged tokens or trees

解决方法:

parse()函数一次只能处理一个句子.

这有效：

chunked = []
for s in output:
    chunked.append(cp.parse(s))

结果：

[Tree('S', [(u'POLICE', 'NN'), (u'are', 'VBP'), (u'hunting', 'VBG'), ... 

标签：python,nltk,text-mining
来源： https://codeday.me/bug/20190613/1231236.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。