Movie review sentiment analyzer robot

This robot is included in our downloadable example robots. You can also find the code at the example robots repository.

There was an attempt to compliment. Or was there...?

> I love what you are trying to do with your hair...
> Hey, thanks! ๐Ÿ˜Š
> Wait a second...
> What do you mean, _trying_? ๐Ÿค”
> You are at the top of the bell curve!
> Thank you so much! ๐Ÿ˜„
> Being at the top is a good thing, right?!
> Wait...
> Did you just tell me I'm not special in any way, in fact, totally average? ๐Ÿคจ

Analyzing human language. How hard can it be?

As some of you might have noticed from the above examples, interpreting the actual meaning of a phrase is not always as easy as one might think. The rest of you can return to the top of the bell curve, thanks, that would be great!

An insult disguised as a compliment? A compliment wrapped in a thick layer of sarcasm? A human language is a powerful tool for expressing feelings. One short phrase can contain many well or not so well hidden meanings.

What has all this to do with our robot? Where's my coffee? How much wood would a woodchuck chuck if a woodchuck could chuck wood? Let's find out! At least for the robot part...

A study of a failure

The assistant to the regional manager of the Acme Movie Studios had asked the Department of Very Important Tasks to collect some positive reviews for their new movies. These positive and praising reviews could be used to attract more audience. Everyone wants to see a great movie, right? Right.

The team at the Department of Very Important Tasks was super busy, though. Coffee needed brewing, gossip required spreading. Jane's neighbor had a brand new and expensive Chihuahua. How did she afford it?!

An automated solution for collecting positive reviews and discarding the negative reviews was called for. Soon. As in "yesterday". And could it have a bit more yellow, and could you use Comic Sans for... everything? Thanks, dear!

The task was assigned to the one and only person in the company doing actual real work... I mean... Ehm. Anyway. Eric, the engineer, was given the task to automate-all-the-things. There was no budget nor planning, nobody to ask requirements from. The typical project at Acme Movie Studios. Eric started hacking away.

Eric hacked away with no budget nor specifications to automate the collection of reviews from the movie review site (click on the Movie Search link on the site to find the movie search section).

Movie review challenge: Popular movies

The following is what Eric came up with. If you want to, you can try and fail as miserably as he did. Don't worry; it does not hurt that much. Follow along!

The robot:

  • Uses either Free Sentiment Analyzer or Amazon Comprehend
  • Searches popular movies
  • Starts and completes the review classification (positive vs. negative)
  • Submits the classification and captures the final results

Run this robot locally in Robocorp Lab

You can run this robot on your local machine using Robocorp Lab:

  1. Set up your development environment.
  2. Download the example robots.
  3. Open the movie-review-sentiment-analyzer example.
  4. Complete the Setup section below to create a vault for the credentials!
  5. Open the tasks.robot file and run it.

Setup

You need AWS credentials to use Amazon Comprehend. You can create a IAM user with limited access to the Amazon Comprehend API. It is not recommended to use your AWS root user credentials.

After setting up the AWS credentials, the robot is almost ready to run, but still needs the vault for the AWS credentials. See how to use a vault for secrets on how to set up the vault.

Paste this content in the vault file:

{
  "aws": {
    "AWS_KEY": "aws-key-here",
    "AWS_KEY_ID": "aws-key-id-here"
  }
}

Also setup the vault in Robocorp Cloud, if you want to run the robot there.

Robot script

*** Settings ***
Documentation     A movie review sentiment analyzer robot. Tries to classify
...               written free-text reviews either as positive or negative.
...               How hard can it be? What could possibly go wrong? ๐Ÿ˜…
Library           RPA.core.notebook
Library           RPA.Browser
Library           RPA.Cloud.AWS    robocloud_vault_name=aws

*** Variables ***
${AWS_REGION}=    us-east-2
${MOVIE_BROWSER_INDEX}=    2
${MOVIE_REVIEWS_WEBSITE_URL}=    http://www.rpachallenge.com
${SENTIMENT_ANALYSIS_WEBSITE_URL}=    https://www.danielsoper.com/sentimentanalysis/default.aspx
${SENTIMENT_BROWSER_INDEX}=    1
${USE_COMPREHEND}=    False

*** Keywords ***
Open movie reviews website
    Open Available Browser    ${MOVIE_REVIEWS_WEBSITE_URL}
    Maximize Browser Window
    Click Element When Visible    css:a[href="/movieSearch"]

*** Keywords ***
Initialize sentiment services
    Run Keyword Unless    ${USE_COMPREHEND}    Open sentiment analysis website
    Open movie reviews website
    Run Keyword If    ${USE_COMPREHEND}    Initialize Comprehend client

*** Keywords ***
Open sentiment analysis website
    Open Available Browser    ${SENTIMENT_ANALYSIS_WEBSITE_URL}

*** Keywords ***
Initialize Comprehend client
    Init Comprehend Client    use_robocloud_vault=True    region=${AWS_REGION}

*** Keywords ***
Switch to movie reviews website
    Run Keyword Unless    ${USE_COMPREHEND}    Switch Browser    ${MOVIE_BROWSER_INDEX}
    Run Keyword Unless    ${USE_COMPREHEND}    Wait Until Page Contains    RPA Challenge

*** Keywords ***
Search popular movies
    Switch to movie reviews website
    Click Button    Get Popular Movies
    ${movie_links_locator}=    Set Variable    css:span.linkPointer
    Wait Until Element Is Visible    ${movie_links_locator}

*** Keywords ***
Start challenge. This will be easy!
    Wait Until Page Does Not Contain Element    css:button.disabled
    Click Button    Start Timer

*** Keywords ***
Get movie links
    Switch to movie reviews website
    ${movie_links_locator}=    Set Variable    css:span.linkPointer
    Wait Until Element Is Visible    ${movie_links_locator}
    @{movie_links}=    Get WebElements    ${movie_links_locator}
    [Return]    @{movie_links}

*** Keywords ***
Get reviews
    ${reviews_locator}=    Set Variable    css:.reviewsText .card
    Wait Until Element Is Visible    ${reviews_locator}
    @{reviews}=    Get WebElements    ${reviews_locator}
    [Return]    @{reviews}

*** Keywords ***
Classify reviews as positive or negative. Not even breaking a sweat...
    @{movie_links}=    Get movie links
    FOR    ${movie_link}    IN    @{movie_links}
        Open movie modal    ${movie_link}
        @{reviews}=    Get reviews
        Classify reviews    @{reviews}
        Close movie modal
    END

*** Keywords ***
Open movie modal
    [Arguments]    ${movie_link}
    Click Element When Visible    ${movie_link}

*** Keywords ***
Comprehend sentiment
    [Arguments]    ${text}
    ${sentiment}=    Detect Sentiment    ${text}
    Notebook Json    ${sentiment}
    ${sentiment_score}=    Set Variable If    "${sentiment["Sentiment"]}" == "NEGATIVE"    -1    1
    [Return]    ${sentiment_score}

*** Keywords ***
Classify reviews
    [Arguments]    @{reviews}
    FOR    ${review}    IN    @{reviews}
        ${review_text}=    Get Text    ${review.find_element_by_class_name("card-content")}
        Notebook Print    REVIEW: ${review_text}
        ${sentiment}=    Run Keyword If    ${USE_COMPREHEND}    Comprehend sentiment    ${review_text}
        ...    ELSE    Get sentiment    ${review_text}
        Switch to movie reviews website
        ${actions}=    Get WebElement    ${review.find_element_by_class_name("card-action")}
        ${positive_link}=    Get WebElement    ${actions.find_element_by_css_selector("a:first-child")}
        ${negative_link}=    Get WebElement    ${actions.find_element_by_css_selector("a:last-child")}
        Run Keyword If    ${sentiment} >= 0    Click Link    ${positive_link}
        Run Keyword If    ${sentiment} < 0    Click Link    ${negative_link}
    END

*** Keywords ***
Switch to sentiment analysis website
    Run Keyword Unless    ${USE_COMPREHEND}    Switch Browser    ${SENTIMENT_BROWSER_INDEX}
    Run Keyword Unless    ${USE_COMPREHEND}    Wait Until Page Contains    Free Sentiment Analyzer

*** Keywords ***
Get sentiment
    [Arguments]    ${text}
    Switch to sentiment analysis website
    Input Text    accordionPaneSentimentAnalysis_content_txtText    ${text}
    Click Button    accordionPaneSentimentAnalysis_content_btnAnalyzeText
    Wait Until Element Is Visible    css:#accordionPaneSentimentAnalysis_content_lblInterpretation span:nth-of-type(2)
    ${sentiment_score_text}=    Get Text    css:#accordionPaneSentimentAnalysis_content_lblInterpretation span:nth-of-type(2)
    ${sentiment_score}=    Convert To Number    ${sentiment_score_text}
    Notebook Print    SENTIMENT: ${sentiment_score}
    [Return]    ${sentiment_score}

*** Keywords ***
Close movie modal
    Switch to movie reviews website
    Click Element When Visible    css:.modal-close
    Wait Until Page Does Not Contain Element    css:.modal-overlay.velocity-animating

*** Keywords ***
Submit challenge. See? Easy!
    Click Button    Submit

*** Keywords ***
Admire my accomplishment!
    Wait Until Element Is Visible    css=.congratulations
    Capture Element Screenshot    css=.congratulations    challenge-results.png

*** Tasks ***
Analyze movie reviews, like a boss.
    Initialize sentiment services
    Search popular movies
    Start challenge. This will be easy!
    Classify reviews as positive or negative. Not even breaking a sweat...
    Submit challenge. See? Easy!
    Admire my accomplishment!
    [Teardown]    Close All Browsers

Modify AWS_REGION variable to control AWS region. Change USE_COMPREHEND to True when you want to use Amazon Comprehend for sentiment analysis.

Task failed successfully

I would have given this LESS than a 1 rating if it were possible. The entire film should have been left on the cutting-room floor. What a pathetic waste of time, money and effort! Let's see...assemble the prettiest cast you can find (which of course is in direct proportion to the amount of talent they lack)...throw together the thinnest plot you can dig up...and viola! An abominable piece of trash that the director and/or producer should be ashamed to put their name on. How much WORSE can the horror genre get? And don't use low budget as an excuse...I have seen many GREAT low-budget films....in fact some of the best horror classics of all time were low-budget. If you don't have the talent and ingenuity to make a GOOD horror film, then for God's sake don't make one at all!!!

Interpretation: This text has a sentiment score of 100.0.
This means that the overall sentiment or tone of this text
is very positive / enthusiastic.

To put it short. Eric was fired. The assistant to the regional manager was promoted. A new developer-to-blame was hired, straight from the elementary school. You know the story!

What did we learn?

Well. The robot did stuff, which is great.

The robot used two browsers, switching between the movie review website and the sentiment analysis website. This enables automating processes that use multiple separate systems, jumping between the systems as required.

The robot was able to switch between sentiment analysis services using conditional keywords and variables.

We learned that Robot Framework keywords could be expressive and use rich language, if needed (see the main task).

The robot outsourced the hard part of analyzing human language to an external service. Building a machine-learning-based solution inhouse was out of the question due to OutOfBudgetException. Outsourcing a part of the automation process makes sense in these scenarios.

However, we also learned that not all services are an automatic fit for all problems. One needs to carefully select the services based on the business needs, and do proper testing, to see if the solution works.

As for the possible "proper" solution for language analysis: Many of the large cloud service providers offer paid cloud-based services. It might be a good idea to test those offerings for your real-world needs.

Good luck on your journey in your own Acme Movie Studio projects! Hang in there!