A large toy maker has been growing very quickly and expanding their product line. After each new toy release, the company wants to understand how consumers are enjoying and using their products. Additionally, the company wants to ensure that their consumers are having a good experience with their products. As the toy ecosystem grows, the company wants to ensure that their products are still relevant to their customers and that they can plan future roadmaps items based on customer feedback. The company wants to capture the following from social media:
- Understand how consumers are using their products
- Ensure customer satisfaction
- Plan for future roadmaps
Capturing the data from various social networks is relatively easy but the challenge is building the intelligence programmatically. After the data is ingested, the company wants to be able to analyze and classify the data in a cost-effective and programmatic way. To do this, the following architecture can be used:
- The first thing to do is decide on which social media sites to listen to. Then create an application that poll those sites via their corresponding APIs and run it on Amazon EC2.
- Next, an Amazon Kinesis stream is created, because we might have multiple data sources: Twitter, Tumblr, and so on. This way, a new stream can be created each time for each new data source added and you can use the existing application code and architecture. Additionally, in this example, a new Amazon Kinesis stream is created to copy the raw data to Amazon S3.
- For archival, long term analysis, and historical reference, raw data is stored into Amazon S3. Additional Amazon ML batch models can be run from data located in Amazon S3 to do predictive analysis and track consumer buying trends.
- As noted in the architecture diagram, Lambda is used for processing and normalizing the data and requesting predictions from Amazon ML. After theÂ Amazon ML prediction is returned, the Lambda function can take action based on the prediction â€“ for example, to route a social media post to the customer service team for further review.
- Amazon ML is used to make predictions on the input data. For example, an ML model can be built to analyze a social media comment to determine whether the customer expressed negative sentiment about a product. To get accurate predictions with Amazon ML, start with training data and ensure that your ML models are working properly. If you are creating ML models for the first time, see Tutorial: Using Amazon ML to Predict Responses to a Marketing Offer. 68 As mentioned earlier, if multiple social network data sources are used, then a different ML model for each one is suggested to ensure prediction accuracy.
- Finally, actionable data is sent to Amazon SNS using Lambda, and delivered to the proper resources via text or email for further investigation.
- As part of the sentiment analysis, creating an Amazon ML model that is updated regularly is imperative for accurate results. Additional metrics about a specific model can be graphically displayed via the console, such as: accuracy, false positive rate, precision, and recall.
By using a combination of Amazon Kinesis Streams, Lambda, Amazon ML, and Amazon SES, we have create a scalable and easily customizable social listening platform. It is important to note that this picture does not depict the action of creating a ML model. This act would be done at least one time, but usually done on a regular basis to keep the model up to date. The frequency of creating a new model depends on the workload and is really only done to make the model more accurate when things change.
Reference : https://d0.awsstatic.com/whitepapers/Big_Data_Analytics_Options_on_AWS.pdf