Azure HDInsight -资源U‘记号器/punkt/english.泡菜’找不到
Azure HDInsight -资源U‘记号器/punkt/english.泡菜’找不到
提问于 2016-04-19 04:37:19
我进口了nltk包裹。我需要使用nltk.sent_tokenize和nltk.word_tokenize，当我使用时，无论发生什么，我都会得到以下错误：
An error occurred while calling z:org.apache.spark.api.python.PythonRDD.runJob.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, 10.0.0.4): org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "/usr/hdp/current/spark-client/python/lib/pyspark.zip/pyspark/worker.py", line 111, in main
process()
File "/usr/hdp/current/spark-client/python/lib/pyspark.zip/pyspark/worker.py", line 106, in process
serializer.dump_stream(func(split_index, iterator), outfile)
File "/usr/hdp/current/spark-client/python/lib/pyspark.zip/pyspark/serializers.py", line 263, in dump_stream
vs = list(itertools.islice(iterator, batch))
File "<stdin>", line 2, in <lambda>
File "/usr/bin/anaconda/lib/python2.7/site-packages/nltk/tokenize/__init__.py", line 85, in sent_tokenize
tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
File "/usr/bin/anaconda/lib/python2.7/site-packages/nltk/data.py", line 781, in load
opened_resource = _open(resource_url)
File "/usr/bin/anaconda/lib/python2.7/site-packages/nltk/data.py", line 895, in _open
return find(path_, path + ['']).open()
File "/usr/bin/anaconda/lib/python2.7/site-packages/nltk/data.py", line 624, in find
raise LookupError(resource_not_found)
LookupError:
**********************************************************************
Resource u'tokenizers/punkt/english.pickle' not found. Please
use the NLTK Downloader to obtain the resource: >>>
nltk.download()
Searched in:
- '/home/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- u''
**********************************************************************我已经参考过许多关于这个主题的帖子，并且尝试了nltk.download(all')，-d，以及排列子文件夹。
Azure ML的python不附带nltk库。在这个平台上应该有其他方法来使用nltk。救命啊！！谢谢!
回答 1
Stack Overflow用户
发布于 2016-04-20 09:24:58
根据错误信息，我认为问题是由@alvas说的以下两个原因引起的。
- 资源不存在。
- 资源路径不追加路径列表
nltk.data.path。
我试图按照@alvas发布的评论来执行以下步骤。
# 1. Installation for package `nltk` using `pip`
~ $ pip install nltk
# 2. Download resource in Python REPL.
~ $ python
>>> import nltk
>>> nltk.download()
NLTK Downloader
---------------------------------------------------------------------------
d) Download l) List u) Update c) Config h) Help q) Quit
---------------------------------------------------------------------------
Downloader> d
Download which package (l=list; x=cancel)?
Identifier> punkt
Downloading package punkt to /home/<username>/nltk_data...
Unzipping tokenizers/punkt.zip.
---------------------------------------------------------------------------
d) Download l) List u) Update c) Config h) Help q) Quit
---------------------------------------------------------------------------
Downloader> q
True
>>>
# 3. Check the resource path listed in the path list `nltk.data.path`
>>> nltk.data.path
['/home/<username>/nltk_data', '/usr/share/nltk_data', '/usr/local/share/nltk_data', '/usr/lib/nltk_data', '/usr/local/lib/nltk_data']
~ $ pwd && ls nltk_data
/home/<username>
tokenizers
# 4. If the dir `nltk_data` not in the path list `nltk.data.path`,
# you can try to move the dir to the specified path listed in the path list,
~ $ mv nltk_data /usr/local/share/
# or append the path of the dir `nltk_data` into the list variable `nltk.data.path`.
>>> nltk.data.path.append('/home/<username>/nltk_data/')希望能帮上忙。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/36708664
复制相关文章
相似问题
腾讯云开发者
Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有
深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 
粤公网安备44030502008569号
腾讯云计算（北京）有限责任公司 京ICP证150476号 | 京ICP备11018762号