Automatic Keyword extraction using Topica in Python

• Why to extract keywords
• Extract keyword using Topica in Python
• What is happening at background?

• AUTOMATIC KEYWORD EXTRACTION USING RAKE IN PYTHON
• AUTOMATIC KEYWORD EXTRACTION USING TEXTRANK IN PYTHON

• You can judge a comment or sentence within a second just by looking at keyword of a sentence.

• You can make decision whether the comment or sentence is worth reading or not.

• Further you can categorize the sentence to any category. For example whether a certain comment is about mobile or hotel etc.

• You can also use keywords or entities as a feature for your supervised model to train.

from topia.termextract import extract
from topia.termextract import tag

# Setup Term Extractor
extractor = extract.TermExtractor()

# Some sample text
text ='''
  ... Police shut Palestinian theatre in Jerusalem.
  ...
  ... Israeli police have shut down a Palestinian theatre in East Jerusalem.
  ...
  ... The action, on Thursday, prevented the closing event of an international
  ... literature festival from taking place.
  ...
  ... Police said they were acting on a court order, issued after intelligence
  ... indicated that the Palestinian Authority was involved in the event.
  ...
  ... Israel has occupied East Jerusalem since 1967 and has annexed the
  ... area. This is not recognised by the international community.
  ...
  ... The British consul-general in Jerusalem , Richard Makepeace, was
  ... attending the event.
  ...
  ... "I think all lovers of literature would regard this as a very
  ... regrettable moment and regrettable decision," he added.
  ...
  ... Mr Makepeace said the festival's closing event would be reorganised to
  ... take place at the British Council in Jerusalem.
  ...
  ... The Israeli authorities often take action against events in East
  ... Jerusalem they see as connected to the Palestinian Authority.
  ...
  ... Saturday's opening event at the same theatre was also shut down.
  ...
  ... A police notice said the closure was on the orders of Israel's internal
  ... security minister on the grounds of a breach of interim peace accords
  ... from the 1990s.
  ...
  ... These laid the framework for talks on establishing a Palestinian state
  ... alongside Israel, but left the status of Jerusalem to be determined by
  ... further negotiation.
  ...
  ... Israel has annexed East Jerusalem and declares it part of its eternal
  ... capital.
  ...
  ... Palestinians hope to establish their capital in the area.
  ... '''

# Extract Keywords
keywords_topica = extractor(text)
print(keywords_topica)

Output:

[('East Jerusalem', 3, 2), ('Palestinian Authority', 2, 2), ('Palestinian', 6, 1), ('peace accords', 1, 2), ('East', 4, 1), ('event', 6, 1), ('literature festival', 1, 2), ('Israel', 4, 1), ('security minister', 1, 2), ('Israeli police', 1, 2), ('Palestinian state', 1, 2), ('police notice', 1, 2), ('Richard Makepeace', 1, 2), ('theatre', 3, 1), ('Palestinians hope', 1, 2), ('Palestinian theatre', 2, 2), ('British consul-general', 1, 2), ('British Council', 1, 2), ('court order', 1, 2), ('Mr Makepeace', 1, 2), ('Israeli authorities', 1, 2), ('opening event', 1, 2), ('Jerusalem', 8, 1)]

Step 1 (POS tagging of text):

from topia.termextract import tag

# Setup POS tagger of topica
tagger = tag.Tagger()
tagger.initialize()

# POS tagging using topica
print(tagger(text))

 [‘…’, ‘:’, ‘…’]]

[‘hope’, ‘NN’, ‘hope’],

1. First number is nothing but number of time that keyword appears in the document as Noun.
2. Second Number is indicating how many word to make that keyword phrase.

• AUTOMATIC KEYWORD EXTRACTION USING RAKE IN PYTHON
• AUTOMATIC KEYWORD EXTRACTION USING TEXTRANK IN PYTHON

Conclusion:

• What is keyword?
• Why to extract keyword?
• Setting up topica for python
• Extract keywords automatically using Topica in python
• What is happening at background of Topica/ How Topica works?