The Visual Crossing Weather Widget for MicroStrategy was designed for the latest MicroStrategy Visualization API and provides two main capabilities. First is the ability to query into the Visual Crossing Weather Data Server and retrieve data to join against live data in your MicroStrategy Dossiers. Second is the ability to be an analytical widget for both historical weather analysis as well as calendar-based weather forecast tool. In this guide we will walk you through the process of using the widget in your Dossiers.
In order to use the Weather Widget you must have the data necessary to join with the weather data. There are only three main requirements: Location, Time and your business metrics. Location will be in the form of Location Name/ID, Latitude and Longitude and depending upon your configuration you can live geocode addresses using the widget. The geocoding process effectively takes address information and converts it to a lat/long internally so that it can match weather data. Once the system has the location data it can send it to the weather server which will find nearby weather towers and retrieve the correct weather data.
NOTE: The process to get the correct weather data for your locations is very complex and require geographic distance interpolation and cleansing across multiple weather data sets to ensure highly accurate weather data for your location and time. Please see our blog entry and retrieving weather data for more information on this process. Just know that the data being retrieved for your locations is the most accurate data available.
The second item of data that is required will be Time. This can be in the form of date or date/time. We highly recommend that you use date/time as the data retrieval process is done at the hourly level. This will ensure greater accuracy and remove the need to work with daily averages. Also, the weather widget allows you to analyze by your custom business hours, so having the data in hour format will allow you to understand weather during business hours vs. times when your business is closed.
The third and final item is the business data. These are any business metrics that you have that you feel might be affected by weather. Footfall to store, Revenue, Employee Attendance, Late Delivery Times, Count of Claims are all examples that could be key to your analysis. The system doesn't care about the magnitude of these values and the system will use the range of each metric, normalize the data and create a scoring scale for you that can match to the weather variable it is being compared against.
Here is an example dataset that we will use for this guide:
Notice that we have Date, Store Name to identify the location and lat/long columns to give to the server and metrics that we wish to compare against weather. Now it is time to convert this grid to a weather widget.
Adding the Weather Widget to your Dossier
Once properly installed, the weather widget icon will appear in your Dossier as you see below:
By clicking on the Weather Widget icon you will now add it to your dossier. You can also use the replace function on the default grid widget so that you won't have to re-add your data. If you are adding a new widget please make sure that you re-select the correct Date/Time, Location and Metric Columns.
NOTE: If you don't have an appropriate location, time and metric column the widget will pop up a warning telling you that the necessary data is not available. This most often happens when the user adds the widget to an empty dataset where the solution is simply to drag over the correct data to the Rows and Metrics drop zones. You may be required to re-add the widget after doing this operation.
Once the widget is added and it has deemed the data in the drop zones adequate, it will ask you if you wish to do a historical or forecast analysis.
Choosing History will put the widget into the History mode and start analyzing weather vs your metric performance. The UI for History mode is a series of graphs that show correlations between your data and weather. Forecast will fetch the forecast for these locations and attempt to predict your performance based upon what it learned from the historical analysis. The UI is a calendar list of your locations for the week ahead. Most of the time it is best to do a historical weather analysis first. This will create a 'Score' for each of your locations for all of the dates and metrics provided. You can go directly to Forecast mode but you will be required to create your own score manually that can be applied. For this tutorial purpose we will choose History first.
Once you choose History it will want to know what datasource to you. This is a convenience feature that would allow you to import external data sets.
Click on Source datasource and choose Container Datasources
In the mode that we are using the widget our Container is MicroStrategy as the widget is embedded in MicroStrategy and wants to use the data in the MicroStrategy dataset. This widget will ask you to map out the columns found in the dataset to known fields that it requires.
Once all green checkmarks are achieved, we can continue by selecting the checkmark icon in the upper right. The results of the datasource scan can now be seen and we can select continue.
We can now confirm the date format and ensure that it is being parsed correctly. In case the format can be changed here following the Java Date Format definition.
If all looks ok you can simply continue with no action. The one option you need to review is if you only want high quality data to be joined and ignore low quality data. We strongly encourage users to ignore low quality data and let the averages of high quality data be the basis for your analysis.
Clicking continue you will be asked to choose the business hours you wish to analyze. This is where you would ask the widget to only analyze those hours that fit into your business hours.
By leaving the default, the widget will analyze all business hours for the day. We can now click continue and the widget will ask if we want to use a predefined score for this analysis.
If this is your first time through this dataset you will want to simply continue and let the widget create a custom score for you based upon the analysis.
You have now completed the setup for your first Weather Widget and can move onto analysis.
Now that you have chosen History, you will notice a delay while the system fetches clean and accurate data for every single date and location on the grid. It will create and manage a joined dataset within the widget. Once it completes the download it will then run the correlation analysis on every location and show you the results of the analysis as you see below:
Metric Chooser - Every weather score is done per metric, per location, per weather variable... this chooser allows you to determine which metric the page will analyze.
Location Chooser - This control allows the user to focus on individual locations or see the total score for all locations. It is not recommended to rely upon 'All Locations' as various locations may counteract each other. Your Alaska location may be negatively affected while you California location is positively impacted giving the impression that weather is not affecting your business.
Weather Variable Score List - This is the list of all weather variables that were correlated against your metrics. Clicking on any one of them will bring the score into the highlight window.
Data Volume Visualizer - The graphic below the correlated data graph shows how much data was used in creating the statistical analysis. Certain segments of the graph may have higher accuracy than others. In fact, the graph may itself turn gray or not show data where low volumes of sampling exist. The thicker gray areas have a higher sampling and therefore are more reliable.
Highlighted Weather Score - This score window highlights and adds more detail to a scoring window. It will show correlation values, data volume visualization and ideal weather values contributing to the score.
Full Score - The Full Score window shows the correlation to the chosen metric for all weather variables that correlated well enough to be part of the official score for the selected location. Note that not all weather variables will contribute. When individual locations are shown the contributing variables will be listed in this window to let the user know what weather variables are being scored.
Score Chooser - This window opens the metadata file browser to allow the user to choose a specific saved scoring definition rather than using the one calculated. Scores can be stored anywhere in the metadata and can be mixed with other metadata objects so be certain to choose the correct score when using this option.
Settings Menu - The settings menu contains several sub menus:
- Settings - A list representation of all options chosen by the History wizard to set up the analysis.
- Reset - Resets the entire analysis
- Refresh - Refreshes the weather data behind the dashboard and reanalyzes as necessary
- Score - Allows the user to analyze individual scores and modify weighted distributions, NOTE: the user must choose a location and metric to see the weather variables contributing to the score.
- Save - Saves the score in the metadata for future use
- Properties - This editor allows the user to modify some visualization settings
- History Data - Shows the dataset of joined weather and business data. It also allows for export of the table in CSV format which can be reimported for analysis in other forms.
- Unit Groups - The list of available units of measure can be chosen. The currently selected unit will be highlighted with a '*'
The analytical value from the Weather Widget comes from the Scoring System. As mentioned earlier, the History mode automatically calculates the correlation between a weather variable and a business metric for every location. Once a strong correlation is determined it creates a score for it. For instance, perhaps a bike shop does its best sales between 70-80 degrees. As it gets hotter or colder the sales drop off in a correlated manner. The scoring system would give strong weight when it encounters a future forecast that is between 70-80 degrees. Likewise the value of the score drops as it moves away from the ideal temp. The Weather Widget creates this distribution for you. It is also modifiable for specific uses or a user can create a custom score. Lets look at the score for a specific location created by the system. Take the following steps as seen in this image:
Step 1: Choose a Location
Step 2: Choose a Weather Score Component
Step 3: View/Edit the Score Components for this Location
Notice that the range or weather values from high to low are distributed and scored based on their correlation to the metric.
What if we don't want to score our future forecasts on history? A classic example of this is when to pour concrete. Imagine managing hundreds of construction sites and you want to see which days I can pour concrete on. The score for this would be binary: Pour - Don't Pour. This may be done on a single weather variable like Temperature. Below 32 or above 95 = Don't Pour. This is how we would create the score:
Step 1: Choose the Score menu in Settings
Step 2: Add a Criteria to your new Score
Step 3: Add value bins by entering in your values and clicking 'Add' for a new bin entry
Notice we give a score of 100 for a good pouring temperature and a 0 for all others. This would allow the forecast window to color code the days/hours of every site to when they can pour.
IMPORTANT: One last critical point... Saving your Score! When you create a score through the history widget or manually you must click on the save menu item found in settings menu and name your score and save it. This would allow you in the future to load this score into the history or forecast modes.
First Click on the Settings icon and choose the 'Weather Score' option:
Next you will click the 'Save' icon for your default score, choose a folder, name your score and click on the submit check mark to complete your save.
We now have a save Score set up for future analysis. This score is intelligently based upon past performances of your metrics based on weather.
Forecast mode has a different set of requirements. We don't need dates because we know that we are analyzing the next 7-10 days. We don't need metrics because the score is already created. We simply need locations and of course we will apply or create a score for those locations. Let's get started with a simpler grid than the one we have from earlier:
We have Location, Latitude, Longitude and an unused Metric that is required by the system but not necessary for our forecast analysis. At this point we can simply apply the Weather Widget to the Dossier visualization. When prompted, choose 'Forecast'.
As with History Mode it will ask us a series of questions about which source to use and how to align the data. We can answer these questions in the same fashion using a Container Datasource option:
IMPORTANT: Before you can see the Forecast calendar, you will need to load in a Score. The Score tells the system how to rate the weather relative to the metrics on your dataset. If you don't have a score set up the system will show the following to let you know that one is required.
To add a score use the Settings icon and choose the score that you created in the Historical section above. Note that you can create new scores and add simple scoring criteria as mentioned above. For our case we will use the correlated score created by our History widget.
Open the 'Weather Score' editor, open the metadata folder browseer, navigate to your folder, choose your saved weather score, and complete your action by clicking on the check mark.
Now we have our forecast:
Locations List - This is the list of your locations provided in the container datasource, in this case your Dossier Grid.
Metric Chooser - The forecast window is based upon analyzing how weather at each location will perform by a specific metric. Use this popdown control to choose what metric you will analyze by. In some instances you may not have any metrics at all such as in our Pour/No-Pour Scoring example above.
Day/Hour Toggle - Remember that weather data is done hourly and can be aggregated for daily use. This toggle will show you hour by hour how weather is affecting your business.
Forecast Alert Calendar - For every location you can see 7-10 days ahead and every cell of the calendar will show the expected score for that day based upon your historical score. It will highlight the icons for the weather variables that affect the score significantly. For instance if your see a red cell with a temperature icon, that means the temperature at that location, on that day will negatively affect your metric.
Alerts Heatmap - The heatmap at the right is a condensed, quick-viewing visualization of the entire calendar. The user can quickly see at a glance where the problem areas will be for every location in a single glance.
Magnified View - This window on the heatmap acts as a window selector and magnifier for the heatmap. As the user drags the magnifier over an area, those locations will come into view on the main Calendar.
Settings Menu - Please refer to the settings menu in the History Mode section for a description of all menu items.
We recommend having separate pages for History Mode and Forecast Mode. Also, saving of Dossier's with particular Score analysis is very helpful to users so they don't have to make choices dynamically. As they run Dossier's the state of either Historical or Forecast modes will be the default view ready for analysis. The data to populate this historical and forecast widgets is dynamically update on each run.