Wednesday, November 4. 2009
Discover JMP with New Book in Documentation Set
Wednesday, October 21. 2009
Oil and Gas Industry Sees Value in JMP
Monday, October 12. 2009
Answering Your Demand for Design of Experiments
Recently, JMP has been deluged with requests for information about Design of Experiments (DOE or DOX). Was it due to atmospheric disruption when NASA hit the lunar south pole last week? Or shall we just chalk it up to a growing desire to work smart and make better use of resources in the workplace?
Never fear, JMP is responding. On October 28, author and Arizona State University professor Douglas Montgomery and JMP’s Brad Jones are offering a free seminar on DOE in Phoenix. At the seminar we will give away some copies of several books, including Montgomery’s latest edition of Design and Analysis of Experiments and a new SAS Press book by W.L. Gore employees and JMP users José and Brenda Ramírez, Analyzing and Interpreting Continuous Data Using JMP: A Step-by-Step Guide.
Join Doug, Brad, Susan Glick, John Guerrero and the southwest US JMP team on Thursday, October 28. Seats are filling fast so register today.
We also just added a December 3 live DOE Webcast hosted by JMP’s Tom Donnelly. It is part of our Master Your JMP Skills series. Sit back at your desk, put your feet up and join this DOE event. Registration is required.
Never fear, JMP is responding. On October 28, author and Arizona State University professor Douglas Montgomery and JMP’s Brad Jones are offering a free seminar on DOE in Phoenix. At the seminar we will give away some copies of several books, including Montgomery’s latest edition of Design and Analysis of Experiments and a new SAS Press book by W.L. Gore employees and JMP users José and Brenda Ramírez, Analyzing and Interpreting Continuous Data Using JMP: A Step-by-Step Guide.
Join Doug, Brad, Susan Glick, John Guerrero and the southwest US JMP team on Thursday, October 28. Seats are filling fast so register today.
|
|
Tuesday, September 8. 2009
Solar Array Data, January Through August
In my Solar Array Surprises post about the SAS solar farm, one of the surprises was a midday dip in the power output, for which commenters supplied several possible explanations.
That data was from April, and we could only speculate what the summer data would look like. But now summer is over (by some accounts), and we can look at eight months of solar output data.
Looking at the summer data brings a new surprise: It's very noisy.
I know we had a fair amount of rain this summer, but I remember some dry spells, too. I guess the clouds were always around. At least July 14 looks clear and sunny, which is better than I can say for any day in June or August.
Getting back to the midday dip, here's a Graph Builder plot using the Data Filter to show only the most sunny day in each month (hand-picked).
A few observations from the plot:
Points 1 and 2 are obvious. The third point supports the idea that the horizontal axis of the panel rotation is causing the dip, since it is more pronounced when the sun is lower in the sky. The fourth point could just be the weather, but it might be exacerbated by the power output being near capacity for the array.
Ambient temperature was also mentioned as a possible factor in the power output, but I haven't analyzed it. I was hoping to find at least one cool sunny day and one warm sunny day from each season for comparison. It's still possible the temperature accounts for the slightly higher output in the morning than the afternoon since solar cells are more efficient at lower temperatures.
UPDATE (09-09-09): The solar array data is now available from the JMP File Exchange. Scroll down to the bottom of my author page to find the file titled "Solar Array Data Jan - Aug 2009."
That data was from April, and we could only speculate what the summer data would look like. But now summer is over (by some accounts), and we can look at eight months of solar output data.
Looking at the summer data brings a new surprise: It's very noisy.
I know we had a fair amount of rain this summer, but I remember some dry spells, too. I guess the clouds were always around. At least July 14 looks clear and sunny, which is better than I can say for any day in June or August.
Getting back to the midday dip, here's a Graph Builder plot using the Data Filter to show only the most sunny day in each month (hand-picked).
A few observations from the plot:
- Power output is greater in the summer (height of the curve).
- Daylight lasts longer in the summer (width of the curve).
- The midday dip is more pronounced in the winter.
- Summer output is noisier.
Points 1 and 2 are obvious. The third point supports the idea that the horizontal axis of the panel rotation is causing the dip, since it is more pronounced when the sun is lower in the sky. The fourth point could just be the weather, but it might be exacerbated by the power output being near capacity for the array.
Ambient temperature was also mentioned as a possible factor in the power output, but I haven't analyzed it. I was hoping to find at least one cool sunny day and one warm sunny day from each season for comparison. It's still possible the temperature accounts for the slightly higher output in the morning than the afternoon since solar cells are more efficient at lower temperatures.
UPDATE (09-09-09): The solar array data is now available from the JMP File Exchange. Scroll down to the bottom of my author page to find the file titled "Solar Array Data Jan - Aug 2009."
Monday, July 27. 2009
Using JMP to Count Cars
I'm a rising senior at the North Carolina School of Science and Mathematics. During this summer, I worked as a technical summer student at SAS.
While JMP 8 doesn’t have image manipulation support, it does allow the user to create a custom DLL that can be invoked from JSL. So I created a DLL that allowed some functions from ImageMagick, an open source image library, to be called from JSL.
Using this DLL, I tried to count cars in photos from some of the North Carolina Department of Transportation's Webcams. Here's an image from the one near Exit 289 off Interstate 40, in the Raleigh area:
Unfortunately, the more distant parts of the road are hard to see because everything blurs together. So I didn't look at that part. I cropped it out.
To make the later calculations easier, I also converted the images to grayscale, so I only had to work with a single intensity for each pixel, rather than three color channels.
The image then looked like this:
In order to count the cars, I needed to know what parts of the pictures were cars. Since the Webcams update every 3 minutes, I had a lot of other pictures to which to compare each one. So for each picture, I analyzed, I looked at the 10 pictures before it and the 10 pictures after it and averaged the 20 of them together. The result was pretty close to a picture of the road with no cars, as you can see:
This picture told me what parts of the image I was looking at were not cars. By subtracting it from the image I was analyzing, I got a nice picture of what parts were cars:
At this point, it is pretty easy, even from a script, to tell which parts of the picture are cars. The gray regions are cars; the dark regions are not. There were some regions that were dark gray, where it wasn't clear whether they should be considered cars or not. I found that what seemed to work best for them was to consider the areas where the color intensity was less than 60 to be background, and the areas where the intensity was 60 or more to be cars. With this sort of filter applied, the picture looks like this:
I then had a collection of (x, y) coordinates of pixels that composed several cars. Since the pixels of a single car should be closer to each other than to those of other cars, I tried using JMP's cluster analysis tool to divide these points into clusters. Ideally, each cluster would represent one car. Here's that same picture with each cluster given its own color:
As you can see, it seems to have done a pretty good job on this picture. But since JMP's cluster analysis tool needs a number of clusters before it can do the analysis, it isn't the best tool for automated counting. That's why so many individual pixels seem to be their own clusters; JMP is splitting them up into the default number of clusters, 20, when there are only 8 cars in the picture.
Because JMP's cluster analysis tool isn't the best tool for this job, I ended up using a different one. Once you account for the fact that more distant cars look smaller, most cars are of similar sizes. Therefore, back at the black-and-white step, I could count white pixels (making sure to account for the distances of the cars from the camera) to get an approximation of the number of cars in the picture.
Around this time I also decided to switch Webcams. The problem with this one was that it moved around a lot, which prevented the averaging from working. Here are four consecutive pictures from it:
These four are a bit more extreme than most, but you get the idea. After some looking, I found a different Webcam that didn't move around. Here's a picture from it:
Since the size of a car varies with the y-coordinate in a predictable, linear way, it is possible to adjust for it using relatively simple calculations. Unfortunately, doing this requires identifying four points (two on each side of the road), and communicating those four points to a script is rather tricky.
There is an easier way. JMP has a modeling platform. If you count cars in some of the pictures by hand and give that data to JMP, it can create a model that will predict the number of cars in the other pictures, given the location and distribution of pixels in them.
I divided each image into 5-pixel-tall horizontal stripes, as is approximately shown in the (cropped) copy below of the image above.
After subtracting the average picture from it, I counted the number of pixels with intensity higher than 60 in each stripe in each pixel. I could have fit a model right then; but since I was trying to explain the number of cars based on the number of pixels in each stripe, there were roughly 30 explanatory variables -- one for each stripe. I would have needed a lot of images to create the model, which wouldn't have left many to use it on.
To fix this, I extracted principal components. Principal components are linear combinations of the original 30 explanatory variables computed in such a way that the first principal component explains the most variation in the data and the 30th the least variation. It turns out that the top 11 principal components explained more than 95% of the variation. So by modeling based on the principal components instead of the raw data, I reduced the number of explanatory variables to roughly a third of what it was, while still retaining 95% of the information.
I hand-counted cars in a randomly chosen selection of 120 pictures (I had more than 600, so this was only a small portion of them) and fed those numbers, along with the pixel counts, to JMP's Fit Model platform. JMP came up with a model that I used to create a table of calculated number of cars by time. I then graphed that data for Thursday afternoon and Friday. With a spline fit to it, the graph looks like this:
This graph does indeed show the traffic patterns I expected. There are significant rises in traffic around morning and evening rush hours as well as lunchtime.
While JMP 8 doesn’t have image manipulation support, it does allow the user to create a custom DLL that can be invoked from JSL. So I created a DLL that allowed some functions from ImageMagick, an open source image library, to be called from JSL.
Using this DLL, I tried to count cars in photos from some of the North Carolina Department of Transportation's Webcams. Here's an image from the one near Exit 289 off Interstate 40, in the Raleigh area:
Unfortunately, the more distant parts of the road are hard to see because everything blurs together. So I didn't look at that part. I cropped it out.
To make the later calculations easier, I also converted the images to grayscale, so I only had to work with a single intensity for each pixel, rather than three color channels.
The image then looked like this:
In order to count the cars, I needed to know what parts of the pictures were cars. Since the Webcams update every 3 minutes, I had a lot of other pictures to which to compare each one. So for each picture, I analyzed, I looked at the 10 pictures before it and the 10 pictures after it and averaged the 20 of them together. The result was pretty close to a picture of the road with no cars, as you can see:
This picture told me what parts of the image I was looking at were not cars. By subtracting it from the image I was analyzing, I got a nice picture of what parts were cars:
At this point, it is pretty easy, even from a script, to tell which parts of the picture are cars. The gray regions are cars; the dark regions are not. There were some regions that were dark gray, where it wasn't clear whether they should be considered cars or not. I found that what seemed to work best for them was to consider the areas where the color intensity was less than 60 to be background, and the areas where the intensity was 60 or more to be cars. With this sort of filter applied, the picture looks like this:
I then had a collection of (x, y) coordinates of pixels that composed several cars. Since the pixels of a single car should be closer to each other than to those of other cars, I tried using JMP's cluster analysis tool to divide these points into clusters. Ideally, each cluster would represent one car. Here's that same picture with each cluster given its own color:
As you can see, it seems to have done a pretty good job on this picture. But since JMP's cluster analysis tool needs a number of clusters before it can do the analysis, it isn't the best tool for automated counting. That's why so many individual pixels seem to be their own clusters; JMP is splitting them up into the default number of clusters, 20, when there are only 8 cars in the picture.
Because JMP's cluster analysis tool isn't the best tool for this job, I ended up using a different one. Once you account for the fact that more distant cars look smaller, most cars are of similar sizes. Therefore, back at the black-and-white step, I could count white pixels (making sure to account for the distances of the cars from the camera) to get an approximation of the number of cars in the picture.
Around this time I also decided to switch Webcams. The problem with this one was that it moved around a lot, which prevented the averaging from working. Here are four consecutive pictures from it:
These four are a bit more extreme than most, but you get the idea. After some looking, I found a different Webcam that didn't move around. Here's a picture from it:
Since the size of a car varies with the y-coordinate in a predictable, linear way, it is possible to adjust for it using relatively simple calculations. Unfortunately, doing this requires identifying four points (two on each side of the road), and communicating those four points to a script is rather tricky.
There is an easier way. JMP has a modeling platform. If you count cars in some of the pictures by hand and give that data to JMP, it can create a model that will predict the number of cars in the other pictures, given the location and distribution of pixels in them.
I divided each image into 5-pixel-tall horizontal stripes, as is approximately shown in the (cropped) copy below of the image above.
After subtracting the average picture from it, I counted the number of pixels with intensity higher than 60 in each stripe in each pixel. I could have fit a model right then; but since I was trying to explain the number of cars based on the number of pixels in each stripe, there were roughly 30 explanatory variables -- one for each stripe. I would have needed a lot of images to create the model, which wouldn't have left many to use it on.
To fix this, I extracted principal components. Principal components are linear combinations of the original 30 explanatory variables computed in such a way that the first principal component explains the most variation in the data and the 30th the least variation. It turns out that the top 11 principal components explained more than 95% of the variation. So by modeling based on the principal components instead of the raw data, I reduced the number of explanatory variables to roughly a third of what it was, while still retaining 95% of the information.
I hand-counted cars in a randomly chosen selection of 120 pictures (I had more than 600, so this was only a small portion of them) and fed those numbers, along with the pixel counts, to JMP's Fit Model platform. JMP came up with a model that I used to create a table of calculated number of cars by time. I then graphed that data for Thursday afternoon and Friday. With a spline fit to it, the graph looks like this:
This graph does indeed show the traffic patterns I expected. There are significant rises in traffic around morning and evening rush hours as well as lunchtime.
Thursday, July 16. 2009
Easy Way to Embed Flash into PowerPoint Using Freeware
Tuesday, June 30. 2009
JMP Workshops for Professors
Tuesday, June 9. 2009
Vector Plots in JMP
Vector plots show arrows on a two-dimensional plot and allow one to see four dimensions of data: x position, y position, arrow angle, and arrow length. Equivalently, the four dimensions can be x start position, y start position, x end position, and y end position. The latter form is most convenient for JMP. Though JMP doesn't have a menu command to create vector plots, arrows can be added to almost any plot without much trouble.
While lots of four-variable data sets can be adapted for vector display, I'll demonstrate with one of the most natural applications: wind data. This data set records wind speed and direction for various weather stations in the Chicago area. I used formulas to convert the speed and direction into end positions for the arrows and then added the following graphics script to a bivariate scatterplot using Customize from the graph's right-click menu.
Pen Size( 2 );
For Each Row( Arrow( {lon1, lat1}, {lon2, lat2} ) );
Here's a screen capture showing a little bit of the data table, graph, and customize dialog:
The next steps were to delete the rows with no wind and to turn off the points in the graph.
To provide context, I added the map of weather stations provided by the weather service. Placing a image in a graph is a bit tricky (and could be the subject of another post). There are two options: Draw the picture in another custom graphics script, or drag and drop an existing image into the graph. I chose the latter since I had an image file. First, I sized the graph to be the same size as the image, and then I dragged the image file into the center of the graph. After a few tries, the alignment was pretty good. (BTW, we're working on making this better in JMP 9.)
The plot shows relative wind readings taken at 6 p.m. on May 31, 2009. With the help of the background image, we can see how wind patterns vary in relation to Lake Michigan.
UPDATE 10/6/09: The data file, with scripts, is now available in the JMP File Exchange. Look for the file called "ChicagoWind.jmp" in the Data Visualization section.
Source of data and image: the National Severe Storms Laboratory Historical Weather Data Archives, Norman, Oklahoma.
Wednesday, June 3. 2009
Stephen Few's New Book Is a Must-Read
“Now You See It” is data visualization expert Stephen Few’s new book, explaining how to use simple visual techniques for quantitative analysis. In this textbook-sized offering, Stephen explores one of the more overlooked aspects of analysis: the graphic representation of information.
Stephen lays the foundation for good visual analysis in Part I by defining how we perceive information. He states, "…there are ways to visually display data that are effective because they correspond naturally to the working of vision and cognition, and there are ways that break the rules and consequently don’t work. If we wish to display information in a way that will enable us and others to make sense of it, we must understand and follow the rules.”
He goes on to explain that we "…perceive several basic attributes of visual images pre-attentively, that is, prior to and without the need for conscious perception."
Below is a list of those pre-attentive attributes that are quantitatively perceived in and of themselves, without having values arbitrarily assigned to them:
Length – longer =greater
2-D Position – offset higher/lower, or left/right=greater
Width – Wider=greater
Size – Bigger=greater
Intensity – Darker=greater
Blur – Clearer=greater
These pre-attentive attributes help us all consume and digest information and make sense of their meaning. However, Stephen points out that "pre-attentive symbols become less distinct as the variety of distracters increases. It is easy to spot a single hawk in a sky full of pigeons, but if the sky contains a greater variety of birds, the hawk will be more difficult to see."
Good visualizations not only take advantage of the pre-attentive attributes mentioned above, but they also use them appropriately while considering the limitations of our visual memory.
Stephen explains that we have both "working memory and long-term memory. Working memory stores information only briefly. Working memory is where information resides when we are thinking about it. If we think about it long enough, it will end up in long-term memory."
And visual memory (which is part of working memory) is very limited. Visual memory processes information in “chunks.” How much is a chunk of information? Well, it depends on how it is conveyed. Information chunks have to be relatively small when they consist of text or numbers. However, they can be larger when served up graphically.
Part II, which is the meat of the book, takes the reader through several different analyses and shows examples of good visual techniques to best convey the information used in each one.
Part III covers promising new trends. Here, Stephen discusses “Illuminating Predictive Models,” and this is where JMP is prominently featured.
Noting that his book largely focuses on analysis of existing information, or descriptive statistics, here he highlights the benefits of predictive statistics. "If we understand the past well enough to describe it clearly and accurately, we can often build a model that we can use to predict what will likely happen as a result of particular conditions, events, or decisions in the future," Stephen writes.
He explains that "the goal of predictive analysis is not to produce certainty about the future, but to reduce uncertainty to a degree that enables us to make better decisions.“
He then describes what he refers to as “transparent predictive models.” And it is here where Stephen uses JMP to explain how transparent predictive models help us make better decisions with less risk. He says, “This level of involvement in the analytical process [using transparent predictive models] takes advantage of our brains in a way that throws open the windows to insights that we might never otherwise experience.”
Many of the visualizations discussed in Stephen’s book are available in JMP. But even more importantly, JMP’s visualizations also incorporate world-class analytics. This is why, when Stephen turns his attention to more advanced analytics, like predictive modeling, JMP is prominently featured.
“Now You See It” is a must-read for anyone who needs to explore and understand data that guides his or her decision-making process. Following Stephen’s advice will help readers explore their data better, discover trends and patterns more quickly, and make decisions with confidence.
Stephen lays the foundation for good visual analysis in Part I by defining how we perceive information. He states, "…there are ways to visually display data that are effective because they correspond naturally to the working of vision and cognition, and there are ways that break the rules and consequently don’t work. If we wish to display information in a way that will enable us and others to make sense of it, we must understand and follow the rules.”
He goes on to explain that we "…perceive several basic attributes of visual images pre-attentively, that is, prior to and without the need for conscious perception."
Below is a list of those pre-attentive attributes that are quantitatively perceived in and of themselves, without having values arbitrarily assigned to them:
Length – longer =greater
2-D Position – offset higher/lower, or left/right=greater
Width – Wider=greater
Size – Bigger=greater
Intensity – Darker=greater
Blur – Clearer=greater
These pre-attentive attributes help us all consume and digest information and make sense of their meaning. However, Stephen points out that "pre-attentive symbols become less distinct as the variety of distracters increases. It is easy to spot a single hawk in a sky full of pigeons, but if the sky contains a greater variety of birds, the hawk will be more difficult to see."
Good visualizations not only take advantage of the pre-attentive attributes mentioned above, but they also use them appropriately while considering the limitations of our visual memory.
Stephen explains that we have both "working memory and long-term memory. Working memory stores information only briefly. Working memory is where information resides when we are thinking about it. If we think about it long enough, it will end up in long-term memory."
And visual memory (which is part of working memory) is very limited. Visual memory processes information in “chunks.” How much is a chunk of information? Well, it depends on how it is conveyed. Information chunks have to be relatively small when they consist of text or numbers. However, they can be larger when served up graphically.
Part II, which is the meat of the book, takes the reader through several different analyses and shows examples of good visual techniques to best convey the information used in each one.
Part III covers promising new trends. Here, Stephen discusses “Illuminating Predictive Models,” and this is where JMP is prominently featured.
Noting that his book largely focuses on analysis of existing information, or descriptive statistics, here he highlights the benefits of predictive statistics. "If we understand the past well enough to describe it clearly and accurately, we can often build a model that we can use to predict what will likely happen as a result of particular conditions, events, or decisions in the future," Stephen writes.
He explains that "the goal of predictive analysis is not to produce certainty about the future, but to reduce uncertainty to a degree that enables us to make better decisions.“
He then describes what he refers to as “transparent predictive models.” And it is here where Stephen uses JMP to explain how transparent predictive models help us make better decisions with less risk. He says, “This level of involvement in the analytical process [using transparent predictive models] takes advantage of our brains in a way that throws open the windows to insights that we might never otherwise experience.”
Many of the visualizations discussed in Stephen’s book are available in JMP. But even more importantly, JMP’s visualizations also incorporate world-class analytics. This is why, when Stephen turns his attention to more advanced analytics, like predictive modeling, JMP is prominently featured.
“Now You See It” is a must-read for anyone who needs to explore and understand data that guides his or her decision-making process. Following Stephen’s advice will help readers explore their data better, discover trends and patterns more quickly, and make decisions with confidence.
Friday, May 29. 2009
JMP Tree Map in Data Visualization Report
In case you missed my Twitter update about it last week, a JMP tree map created by Daniel Arneman of UNC Energy Services was featured in a recent data visualization report by Intelligent Enterprise titled "Seeing Connections: Visualization Makes Sense of Data." The report, by Seth Grimes, is available as a free download, with registration, and it's definitely worth a read.
Arneman's tree map visualizes greenhouse gas emissions by buildings on the campus of the University of North Carolina at Chapel Hill. It is also featured in the "Visualization Image Gallery" that accompanies the report (registration isn't required to view this), along with an explanation of the value of this type of data visualization:
"Cell Size Indicates Variable Contribution: The University of North Carolina at Chapel Hill developed this treemap visualization using SAS JMP software to show that certain buildings and classes of buildings are responsible for more greenhouse gas emissions than others. Treemaps are not designed to convey numerical quantities; the intent is to show relative rankings...."
You can read more about how Arneman uses JMP to assess, monitor and help reduce greenhouse gas emissions at UNC in our story about his work. You'll see the same tree map in our image gallery (though, using different colors), and you'll also see lots of other cool visualizations by Arneman, including a series of Bubble Plots and the Profiler.
Wednesday, May 20. 2009
Scientific Computing Review: 'Stunning' Graphics in JMP 8
Tuesday, May 19. 2009
3-D Pie Reply
Monday, May 18. 2009
Soccer Analytics Using JMP
NOTE: This entry comes to the JMP Blog from our colleague Jerome Bryssinck of SAS Belgium. Jerome had seen Jeff Perkinson's examples of basketball analytics using JMP and created his own example using football (or soccer) data. In response to comments from readers, Jerome updated his model on May 26, and this blog post now reflects those changes.
THE QUESTION: Has the game been decided yet? HTGBD
This is the question that most people constantly ask themselves when they are watching a football game. This question can take different forms depending on the circumstances. If you're lucky to support the winning team, you might ask yourself: "How secure is the lead?" And for the less fortunate of us: "Is there still a chance for my team to win?"
THE ANSWER: Analytics
Graph1: Probability of the game having been decided in function of the elapsed time and the number of goals difference.
Graph1 shows the probablility of the game having been decided in function of the elapsed time and the number of goals difference. It is possible to change the elapsed time and the number of goal difference on the graph by clicking on a different value.
Some interpretation examples:
If Time=45 and Goal Difference=0: The game has been going on for 45 minutes, and the number of goal difference is 0. There is a 23% probability that the outcome of the game won't change. Here, as the teams are even (0 goal difference), this would mean that there is a 23% probability the game will end in a tie.
If Time=45 and Goal Difference=1: The game has been going on for 45 minutes, and one of the teams is leading by 1 goal difference, then we have a 60% probability that the outcome of the game won't change. Here, this would mean that the leading team has a 60% probability to win.
More Details about the Answer
The model used above has been built using data from the UK Premier League from 2002 to 2006. The type of model used is a regression model.
The following representations are useful to understand the underlying data.
Graph2: Has the Game Been Decided vs. Time
Graph2 shows the percentage of the games that have been decided in function of the Elapsed Time. I must say that I wasn't surprised by this graph, which basically states that the Elapsed Time and the HTGBD (Has The Game Been Decided) are directly proportional.
Graph3: Has the Game Been Decided vs. Time By Goal Difference
Graph3 shows the percentage of the games that have been decided in function of the Elapsed Time by the number of goal difference. According to this graph, the number of goal difference is an excellent predictor for the HTGBD.
Additional readings:
Similar models are available for basketball. Check out Bill James and Jeff Perkinson if you want to learn more.
This entry was first published in Jerome Bryssinck's blog, Brisink. It is republished here with his permission.
THE QUESTION: Has the game been decided yet? HTGBD
This is the question that most people constantly ask themselves when they are watching a football game. This question can take different forms depending on the circumstances. If you're lucky to support the winning team, you might ask yourself: "How secure is the lead?" And for the less fortunate of us: "Is there still a chance for my team to win?"
THE ANSWER: Analytics
Graph1: Probability of the game having been decided in function of the elapsed time and the number of goals difference.
Graph1 shows the probablility of the game having been decided in function of the elapsed time and the number of goals difference. It is possible to change the elapsed time and the number of goal difference on the graph by clicking on a different value.
Some interpretation examples:
If Time=45 and Goal Difference=0: The game has been going on for 45 minutes, and the number of goal difference is 0. There is a 23% probability that the outcome of the game won't change. Here, as the teams are even (0 goal difference), this would mean that there is a 23% probability the game will end in a tie.
If Time=45 and Goal Difference=1: The game has been going on for 45 minutes, and one of the teams is leading by 1 goal difference, then we have a 60% probability that the outcome of the game won't change. Here, this would mean that the leading team has a 60% probability to win.
More Details about the Answer
The model used above has been built using data from the UK Premier League from 2002 to 2006. The type of model used is a regression model.
The following representations are useful to understand the underlying data.
Graph2: Has the Game Been Decided vs. Time
Graph2 shows the percentage of the games that have been decided in function of the Elapsed Time. I must say that I wasn't surprised by this graph, which basically states that the Elapsed Time and the HTGBD (Has The Game Been Decided) are directly proportional.
Graph3: Has the Game Been Decided vs. Time By Goal Difference
Graph3 shows the percentage of the games that have been decided in function of the Elapsed Time by the number of goal difference. According to this graph, the number of goal difference is an excellent predictor for the HTGBD.
Additional readings:
Similar models are available for basketball. Check out Bill James and Jeff Perkinson if you want to learn more.
This entry was first published in Jerome Bryssinck's blog, Brisink. It is republished here with his permission.
Tuesday, May 5. 2009
Solar Array Surprises
As you may have heard, SAS has a 1 megawatt solar farm in operation.
Naturally, I wanted to see what kind of data was available, and it turns out the power output is recorded every 15 minutes, even during the night (who knows -- maybe we'll get some power from the sunlight reflected by a full moon or from a passing comet). The output is measured separately for two halves of the farm called Array A and Array B. You can see the arrays, along with the sheep that maintain the grass at the solar farm, in the photo below.
Here's a graph of both arrays for the few sunny days in April we had here in Cary, North Carolina. I used the Data Filter to include only the sunny days and modified the axes to show only the daylight hours.
The flatness of the power output curve is a testament to the sun-tracking rotation of the panels. They're on a horizontal axis that runs north-south, and so can turn toward east or west.
Two interesting features stand out:
I think I know why these things are happening. But what do you think the reasons are?
In case you want to explore the solar farm data from April, it's available in the JMP File Exchange (scroll down to the bottom of the page to find the solar farm file).
SAS Solar Farm photo is by Dave Horne.
Naturally, I wanted to see what kind of data was available, and it turns out the power output is recorded every 15 minutes, even during the night (who knows -- maybe we'll get some power from the sunlight reflected by a full moon or from a passing comet). The output is measured separately for two halves of the farm called Array A and Array B. You can see the arrays, along with the sheep that maintain the grass at the solar farm, in the photo below.
Here's a graph of both arrays for the few sunny days in April we had here in Cary, North Carolina. I used the Data Filter to include only the sunny days and modified the axes to show only the daylight hours.
The flatness of the power output curve is a testament to the sun-tracking rotation of the panels. They're on a horizontal axis that runs north-south, and so can turn toward east or west.
Two interesting features stand out:
- There's a dip in the outputs around midday.
- Array A lags Array B in the early morning.
I think I know why these things are happening. But what do you think the reasons are?
In case you want to explore the solar farm data from April, it's available in the JMP File Exchange (scroll down to the bottom of the page to find the solar farm file).
SAS Solar Farm photo is by Dave Horne.
Tuesday, April 7. 2009
Analyzing UNC Tar Heel JMP Shots in NCAA Championship Game
ABOUT THIS BLOG
JMP Statistical Discovery Software from SAS
is proud to bring you this blog on all things related to
data visualization, visual Six Sigma, design of experiments
and other statistical topics.
The blog content appearing on this site does not necessarily represent the opinions of SAS. Your use of this blog is governed by the Terms of Use.
Categories
Quicksearch
Syndicate This Blog
Calendar
|
November '09 |
|
||||
| Mon | Tue | Wed | Thu | Fri | Sat | Sun |
| 1 | ||||||
| 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| 9 | 10 | 11 | 12 | 13 | 14 | 15 |
| 16 | 17 | 18 | 19 | 20 | 21 | 22 |
| 23 | 24 | 25 | 26 | 27 | 28 | 29 |
| 30 | ||||||