Skip navigation

Digital business demands dictate an ever increasing need for the extraction of information and useful deductions from live/ real time data on the web. Be it manufacturing or sales or any other department, they all require to base their decisions on real time data. Building visualizations from static data however, is very simple. But this does not provide a solution, for example, to a sales management executive who wants to visualize real time sales by product category. This is where there arises a need for a method for extraction and visualization of live data. While SAP Lumira helps best with the visualization, we need a method for extracting this live data into SAP Lumira.

 

Import.io seems to be the best option to fit the requirement. It is free, easy to use and powerful with lot of features that we can use for extracting the data into SAP Lumira.  Read on to find out how to scrape a website and bring in live data for creating visualizations.

 

Note

I had come across this great blog post: http://scn.sap.com/community/lumira/blog/2014/06/08/using-importio-to-extract-any-data-from-the-web-into-lumira-shuffle-up-and-deal-edition by Ronald Konijnenburg about using import.io in Lumira to extract data from the web where you can follow to setup and expose Data as an API inside import.io

We would need to have live data. Every time we need Lumira to fetch the updated data from the website and so the CSV method would not work. The DA extensions of SAP Lumira would help us connect to the Data source. So let’s build a DA extension for SAP Lumira. For our simplicity of understanding, we are taking the same WSOP data from http://www.wsop.com/players/index.asp?pagenum=1


Process

The process of DA extensions is simple enough to fetch the API and bring the data into Lumira. The data is converted into CSV and is then consumed by Lumira for Visualization. The extension is then installed into Lumira.

Figure 1: Process flow

 

The extension is built with Python. Requests module is used to send the request to the import.io API URL and the response is then used as data for SAP Lumira for the Visualization.  Import.io parses pagination in website and returns data across as pages, so we can mention how many pages of data that we need. EasyGUI is used to capture that input from the user.

Figure 2: Parameters

 

Finally the Data is printed into Lumira that appears in dimensions and measures that can be used to build the Visualization.

Figure 3: Print data

 

You can then build the extension into binary file and use it in Lumira.

 

Note

You could check out the following Lumira - Open Source Data Access Extensions to learn about DA extensions and developing DA extensions for SAP Lumira.


Installing and Executing the Extension


To find out how you can download and install this extension Please take a read @ Website to Visuals – Scraping Website int... | SCN

Trend analysis plays a vital role in understanding context and relevance of business data. Hence, visualizing Time Series has to be quick, easy, and intuitive. This is particularly true, with tools such as SAP Lumira which enable the business user to self-service his or her analytics. In this situation, the right kind of visualization and easy customization play critical roles in the ability to make quick decisions.

 

This blog is based on a presentation I gave at  SAP Analytics Infoday 2015, where we had built a special type of chart using Lumira SDK to analyse and visualize Time Series data in SAP Lumira. This chart type is one of a kind for SAP Lumira. It is a smart combination of categorical analysis which drills down to a Time Series analysis within a single component.

 

The custom chart consists of a bar chart for categorical comparison, which acts as  filter for a calendar heatmap which then facilitates the day-after-day analysis. This calendar heatmap can accommodate the entire year within a scroll-less canvas. In this blog, I will explain the chart properties while providing a glimpse into the SAP Analytics Infoday 2015 session  conducted cooperatively by SAP and Visual BI Solutions.

As an example, I have day-wise sales data for different products of a store for three years and I would like to analyze the sales patterns. The following visualizations make an attempt in accommodating all of this data into a single viz.

 

What it was before

 

SAP Lumira visualization

 

SAP Lumira visualizations

 

Looks murky doesn’t it?  It looks that way because there are so many data points to be captured. But not so anymore! Because we now have our BewilderBeast, which can accommodate all of this data in a slick manner. Take a look at it below:

 

What it is now


To know the lean and easy way to represent the above. Take a look @ A Novel Way to Visualize Time Series in SAP Lumira | SCN

SharePoint is a collaboration and document management platform from Microsoft tailored for Enterprises and Small Business. It is a powerful document store where organization typically use it as a repository to hold their data files. In this post, we will see how we can connect to SharePoint from SAP Lumira and consume data to build storyboard.

 

Use Case

Many of the enterprises have Intranet and repository for various department in SharePoint. These hold files like their finance statement, excel files and documents shared to the entire department or to group of people. In this scenario you would need to build storyboard from the files updated there and also share your local files to SharePoint.

The traditional approach is to download the files and build storyboard and upload it back to SharePoint, but any data that changes in the data source will not be reflected and the process has to be repeated.

The below method eliminates the needs and also acts as easy method to retrieve data from SharePoint and might be useful for enterprise which runs on SharePoint as a document repository. It also works with Office 365 SharePoint sites.

 

Mounting SharePoint as local drive

SharePoint document store can be considered as a folder or placeholder where all the files are being put. It also gives an option to mount a particular directory as local drive to access the files from explorer.

 

Navigate to the SharePoint site where the document repository exist. You should be seeing the document view with list of document.

 

Expand Library menu and you would get list of menu options. Select the option Open with Explorer.

Note: The Open with Explorer options works only with Internet Explorer and the SharePoint site should be added to trusted site

The complete documents would be opened in the Windows Explorer. Select the path of the explorer. The path would usually be the site URL followed by the documents path

 

 

Navigate to My Computer and Click on Map Network drive. Select a mount point and give the above path in the folder. You would have mounted the drive. Now you can access it from within Explorer.

Read this complete blog @ Connect SAP Lumira with Microsoft SharePoint to... | SCN

We will be using a Custom Data Access Connector that connects to Couchbase and consumes the data into SAP Lumira. Here, we will be using the game simulation sample, which comes by default as a sample data along with the Couchbase installation.You can download the data from https://github.com/couchbase/couchbase-examples

 

The gamesim example

The gamesim example contains the results of a game simulation with scores and details about each gamer. It is loaded as “gamesim-sample” bucket as part of Couchbase setup. Typical data from that bucket would look like this.

 

 

Finding the needle in a haystack

Couchbase is built on top of JSON as its data store. Large volume of unstructured data can be stored in Couchbase very effectively. In order to visualize this unstructured data, we need to find and bring only the data that is needed to be used inside SAP Lumira. We will be using process called MapReduce, which maps our data and reduces the result down to meaningful data that is needed.

We will be creating views in Couchbase, which helps us execute Mapreduce on original dataset and return the results. We can create different views for different type of analysis,and also perform aggregation and calculation, using Javascript.

Here is a basic view that returns the necessary details about each player.

 

After publishing the view to production post testing, we can execute to see the result on the complete dataset.

Now that we have the required data, let’s go ahead and use it inside SAP Lumira.

 

Read on to find out how you can consume the above data and create cool storyboard here

Health and Safety is very important in any workplace, especially places like manufacturing plant, construction areas and mining fields fall under high Hazards workplace. Workforce are the main backbone to any industry. Special personals are appointed and regulation bodies take necessary steps to prevent and reduce Hazards in industries.

The Health and Safety Authority (HSA) said there were 6,956 nonfatal injuries and 54 workplace fatalities in 2011. These injuries were based on the characteristics of the accident victim, the nature of the incident and the working environment.

 

Fatal facts

Based on the information from US Department of Labour and OSHA, the following fatal facts are inferred.

  • Out of 4,101 worker fatalities in private industry in the year 2013, 828 or 20.2% were in construction.
  • This caused more than half of construction worker death in 2013
  • Eliminating this would save more than 478 workers life in America
  • There have been more than 12 fatalities in the US, each day in 2013
  • A major proportion of these fatalities happened in the construction industry
  • OSHA statistics across major industries for the years 2005 till 2011 shows that construction industry had the highest TCR (Total Case  Rate ) followed by mining and manufacturing industries
  • California, Texas, Florida and Illinois are the top ranking states in terms of workplace injuries in construction
  • Construction industry alone accounts for more casualties than mining and manufacturing together

  • Falls and highway crashes account for almost 60% of the all the causes of workplace injuries.
  • Heads and Legs gets injured a lot during the Hazard
  • Head gets the highest injury which leads to fatal accidents

 

Beyond Columns and Pies


Read along to find out how the above can be easily visualized in a better intuitive way here