Python - 语料库访问

  • 简述

    Corpora 是一个提供多个文本文档集合的组。单个集合称为语料库。其中一个著名的语料库是古腾堡语料库,其中包含大约 25,000 本免费电子书,托管在 http://www.gutenberg.org/。在下面的示例中,我们仅访问语料库中那些文件名以 .txt 结尾的纯文本文件的名称。
    
    from nltk.corpus import gutenberg
    fields = gutenberg.fileids()
    print(fields)
    
    当我们运行上述程序时,我们得到以下输出 -
    
    [austen-emma.txt', austen-persuasion.txt', austen-sense.txt', bible-kjv.txt', 
    blake-poems.txt', bryant-stories.txt', burgess-busterbrown.txt',
    carroll-alice.txt', chesterton-ball.txt', chesterton-brown.txt', 
    chesterton-thursday.txt', edgeworth-parents.txt', melville-moby_dick.txt',
    milton-paradise.txt', shakespeare-caesar.txt', shakespeare-hamlet.txt',
    shakespeare-macbeth.txt', whitman-leaves.txt']
    
  • 访问原始文本

    我们可以使用在 nltk 中也可用的 sent_tokenize 函数从这些文件中访问原始文本。在下面的示例中,我们检索 blake poen 文本的前两段。
    
    from nltk.tokenize import sent_tokenize
    from nltk.corpus import gutenberg
    sample = gutenberg.raw("blake-poems.txt")
    token = sent_tokenize(sample)
    for para in range(2):
        print(token[para])
    
    当我们运行上述程序时,我们得到以下输出 -
    
    [Poems by William Blake 1789]
     
    SONGS OF INNOCENCE AND OF EXPERIENCE
    and THE BOOK of THEL
     SONGS OF INNOCENCE
     
     
     INTRODUCTION
     
     Piping down the valleys wild,
       Piping songs of pleasant glee,
     On a cloud I saw a child,
       And he laughing said to me:
     
     "Pipe a song about a Lamb!"
    So I piped with merry cheer.