Live blog of Randall Munroe keynote

Simple comic drawing of Randall Munroe

Randall Munroe explains "Complicated Stuff in Simple Words" at Discovery Summit 2016.

Randall Munroe of the popular webcomic xkcd takes the stage at Discovery Summit today. He will discuss "Complicated Stuff in Simple Words" in his keynote.

The author of the science question-and-answer blog What If, Munroe was born in Easton, Pennsylvania, and grew up outside Richmond, Virginia. After studying physics at Christopher Newport University, he got a job building robots at NASA Langley Research Center. In 2006, he left NASA to draw comics on the Internet full time, supporting himself through the sale of xkcd t-shirts, prints, posters and books

View the live blog of this speech.

See photos and tweets from the conference at jmp.com/live

Post a Comment

Live blog of Chris Nachtsheim keynote

Chris Nachtsheim

Chris Nachtsheim gives a keynote speech "DOE: Is the Future Optimal" at Discovery Summit 2016 on Sept. 22.

Christopher Nachtsheim is the Frank A. Donaldson Chair of Operations Management in the Carlson School of Management at the University of Minnesota. Nachtsheim's teaching and research interests center on the optimal design of industrial experiments, regression and predictive analytics and quality management.

He has co-written several related books, most notably Applied Linear Statistical Models and Applied Linear Regression Models. Nachtsheim has also published over 70 articles in the statistics literature and currently serves as Associate Editor of the Journal of Quality Technology.

His keynote speech at Discovery Summit 2016 is titled "DOE: Is the Future Optimal?"

View the live blog of this speech.

See photos and tweets from the conference at jmp.com/live.

Post a Comment

Live blog of Tom Lange keynote

Tom Lange

Tom Lange delivers a keynote speech at JMP Discovery Summit on Sept. 21.

Tom Lange is a 37-year veteran of Procter & Gamble, where he founded and directed the modeling and simulation (M&S) group. His M&S team led efforts in consumer modeling, computational chemistry and biology, computer-aided engineering, and production system throughput and reliability.

Over the course of his career, Lange contributed to such projects as the improvement of peanut butter production systems, the development and expansion of a chocolate chip cookie brand, and the quality and reliability of baby diapers.

He now spends his professional time consulting with small and medium enterprises on ways to improve their competitive edge with the latest computer-based modeling and simulation tools.

View the live blog of this speech.

See photos and tweets from the conference at jmp.com/live.

Post a Comment

Follow JMP Discovery Summit 2016 live coverage

Discovery Summit 2016 is about to begin, but if you are not able to join us in person, you can still get a sense of the conference.

You can see photos, read live blogs, follow along on Twitter and watch the live stream of the keynote speech by John Sall, SAS co-founder and Executive Vice President who is also the chief architect of JMP. His speech marks the launch of JMP 13, the newest version of JMP.

Here's what you need to know:

Post a Comment

10 things to tell your friends about JMP 13

JMP 13 reports on a monitor

There's a lot in JMP 13 to tell your friends about, starting with one-click re-running of analysis scripts and a quick way to join lots of data tables.

JMP 13 is coming this week, and I am quite excited about this release. I’ve been using JMP 13 during the entire development cycle (about 18 months now), and I am impressed how this version has really changed the way I use JMP. For me, after using JMP 13, there is no going back. There are just too many things I would miss.

So when you get your copy of JMP 13, try out some of these new features and enhancements, and let me know what your favorites are.

  1. Play Button in Data Table. We’ve reduced the number of clicks required to re-run an analysis script to a one. This also makes running scripts a bit less scary for novice JMP users. No longer do you need to click the red triangle in a data table script and then be faced with options such as Run Script, Edit or Delete (although these still are available with a right-click). Simply click the big green play button and your script runs. As a bonus, when you save scripts to the data table, they are named with a bit more detail than in previous versions of JMP. If you’ve ever had a data table with script names such as Distribution 1 – Distribution 24, you’ll appreciate this improvement as well.
  2. JMP Query Builder. If you’ve experienced the challenge of joining more than two tables using Tables > Join, I have your new favorite feature in JMP 13. The Query Builder for JMP tables is the latest addition to the Tables menu and the Query Builder family. It brings all of the features from the Query Builders for databases and SAS tables, but to JMP tables. You can join up to 64 JMP tables, set up prompting filters, make summaries, groups and sorting options and run post-query scripts. It’s also a really useful tool for prototyping SQL, can be a bridge between disparate data sources and is much easier than specifying all the detail in the Join command largely due to the fact that you get an informative preview of the results before running the command. Just try it!
  3. Virtual Join. While the Query Builder is quite useful for joining tables, there is another feature in JMP 13 that lets you bring all the power of a join without having to take up the memory footprint – the Virtual Join. Basically, you set up a Link ID (a unique primary key of sorts) in a dimension table, and then in a Fact table set up a Link Reference to that table with the Link ID (this is way easier in practice than it seems here. Your Fact table with the Link Reference gets access to all the columns in the dimensions table(s) without the need for making the physical join, which takes up valuable memory. This opens up JMP to looking at much larger data problems. There are other important advantages to using a Virtual Join as well – you can update data tables asynchronously, for example. And when you have a join with measures taken at different time scales (say a sensor measurement every five minutes and a summary value of a different sensor taken daily), you can save a lot of redundant rows by using the Virtual Join – 287 a day in each column in this example.
  4. Dashboard Builder. Dashboards provide quick access to key performance information that scientists and engineers need to make effective decisions or to communicate findings with others. In JMP 13, they are very easy to build with the new Dashboard Builder. With this you can go from data to dashboard in minutes with no coding required. And we’ve limited the choices to just a few so that even basic JMP users can build dashboards of their results without feeling overwhelmed. Just pick from a template, drag and drop reports to the canvas, pick a logo, give your dashboard a title, and you are ready to go. The templates also give you single-click access to graph “selection” filters, which lets you use graphs as filters as opposed to a traditional list-based Data Filter.
  5. Interactive HTML with Graph Builder. JMP has turned into a really good interactive HTML report generator since the feature (File -> Save As -> Interactive HTML with data) was first released in JMP 11. And those generated reports are a great way to share results with those who don’t have JMP. JMP 13 adds the Graph Builder “done” state (you are pressing done before saving your Graph Builder graphs to reports and presentations, right?). You’ll get interactive versions of the most popular elements of Graph Builder. And you’ll also be able to use this feature with your dashboards, saving interactive versions of your arranged graphs to share with others on the web.
  6. Create Web Report. This new option in the View menu of JMP lets you pick any number of JMP graphs, reports and dashboards have it automatically generate a summary index page, which will link to individual interactive reports. This is useful for putting context around your analysis – the index includes a notes section, a thumbnail image and a time stamp. The collection of documents can be placed on your webpage to share with others. Try this feature out – the combination of the Dashboard Builder, interactive HTML versions of Graph Builder graphs and this web report feature are a trifecta of sharing goodness!
  7. Formula Depot. This is about the best example of “I can’t go back” that I can think of in JMP 13. The Formula Depot in JMP Pro is one of those game-changers. The depot collects all the JMP Pro models you build (simply Save Columns -> Publish Prediction Formula from modeling platforms) without burdening your data table with extra formula columns. This amounts to major data table size savings for sure. And the depot lets you easily investigate the formula JSL scripts and compare models using the Profiler or Model Comparison platform, and it drastically simplifies the creation of score code to a single click. When you want to turn your JMP Pro model into code that other systems can use, you have options to automatically generate SAS, C, JavaScript, SQL and Python. No need to manually convert JSL formulas to code in these languages. It’s done automatically by JMP.
  8. Text Data. Prior to JMP 13, there was very little you could do with free text data in customer surveys, engineering reports, comment fields and audit reports. This all changes with the Text Explorer. This interactive platform allows you to analyze, visualize and model a major new type of data. The Text Explorer helps you with feature creation, visualization and analysis. And the useful thing is that the Text Explorer turns unstructured text data into a set of columns that can be used in any of your other favorite JMP Pro modeling platforms to give you even more insight into the problems you are trying to solve. (Note: Many of the exploration and visualization features of the Text Explorer are in JMP, and JMP Pro adds the ability to use text in your predictive models, as well as a diverse set of analysis options.)
  9. Fit Model Fit and Finish. Fit Model: Effect Screening is the bread and butter of the statistical modeling features in JMP. For JMP 13, there are a number of improvements that make it easier to find significant effects, apply transforms and look at outliers. Overall, it simplifies the experience so you can stay in the analysis flow with fewer distractions. Plus, the styling of the graphs has really been improved – check it out, and let us know what you think. Should we prioritize similar polishing on other standard platforms in JMP in the future?
  10. Formula Editor. There is a new Formula Editor in JMP 13. Formulas are very popular in JMP, and in JMP 13, we’ve made it easier to create and manage them. You’ll notice a huge improvement to the layout, and you’ll find that looking at large formulas is much more manageable.

These are my favorite things about JMP 13. Now it’s your turn to find out what your favorite features or improvements will be. Enjoy JMP 13!

P.S. One more: Right-click on an x-axis of Graph Builder, and then Order By > Ascending or Descending. It’s very cool. I could go on, but “10 things to tell your friends about JMP 13” is much catchier than “136 things I love about JMP”!

Post a Comment

JMP 13 Preview: Compare Designs for picking the best design for an experiment

The design of experiments (DOE) capabilities of JMP are world-class. You can choose from many designs, such as custom, definitive screening, classical, space-filling, choice and covering arrays. But how do you decide which design to use?

A Fraction of Design Space plot comparing three designs

A Fraction of Design Space plot comparing three designs

It used to be a time-consuming process to compare two designs for an experiment, for example, a definitive screening design compared to a traditional fractional-factorial screening design. Or a one-factor-at-a-time experiment compared to an optimal design produced by Custom Design. But not anymore, with the new Compare Designs platform in JMP 13.

“Now it’s easy to compare two or three designs. The platform does side-by-side comparisons, and you can see which design is better for which diagnostic,” says Ryan Lekivetz, Sr. Research Statistician Developer who worked on the new platform.

Many of the diagnostics in Compare Designs include a color dashboard that helps you quickly determine which design is better. In addition to comparing two or three different designs, you can also see the impact of varying the number of runs on the same type of design – allowing you to perform a trade-off analysis in terms of experimental budget and design diagnostics.

Color indications in power analysis in Compare Designs

The Power Analysis diagnostic in Compare Designs uses color to help you see which design is better

Like many new capabilities in JMP 13, Compare Designs was driven by customer requests. And the feedback has been very positive.

Christine Anderson-Cook, a research scientist and JMP user at Los Alamos National Laboratory (LANL), demonstrates the new platform in a recent Analytically Speaking episode and has high praise for it: “JMP has long been leading the way with its experiment design construction and assessment tools, but the new Compare Designs platform takes choosing the right design for the goals of your specific experiment to a new level.” Christine has also written an article on the new platform for the upcoming JMP Foreword magazine.

Ryan notes that Compare Designs could be useful as a tool to understand and show the value of modern experiment design. “Statisticians often get the results of an experiment that they didn’t design. With this platform, they can compare the design that was used to another design, and they can see how much was lost by not using the better design,” he explains.

Working on DOE in JMP is fun for Ryan. “You can apply DOE techniques to almost anything, like cooking hard-boiled eggs, making iced tea and dyeing toy cars,” he says, referring to a few of the experiments he has blogged about.

And he loves the challenge of helping customers with designing experiments. “I like hearing about the tough problems where people don’t think they can create a design and then showing them how they can do it, thanks to the flexibility of custom designs,” Ryan says.

You can talk to Ryan about your own experiment design problems at Discovery Summit 2016 in a few weeks. He’ll be around at “Meet the Developers” and will present a breakout session on the Simulate Responses option in JMP 13.

To learn more about what’s coming in JMP 13, stop by the preview page on our website. There, you can sign up to watch a live stream of JMP chief architect John Sall’s tour of JMP 13 on Sept. 21, as well as watch short videos about JMP 13 and JMP Pro 13.

Post a Comment

JMP 13 Preview: More enhancements to generalized regression

Clay Barker has been busy extending the usefulness of the Generalized Regression platform in JMP Pro, adding many new models and enhancing ease of use. Generalized Regression (or GenReg for short) debuted in JMP Pro 11 as the place to do a trio of popular penalized regression techniques: Lasso, Elastic Net and Ridge. These penalized techniques are attractive because they lead to simpler models that are less prone to overfitting. After adding more modeling and selection techniques in JMP Pro 12 and now JMP 13, GenReg has become a place to do variable selection quickly and easily for a wide variety of problems.

Variable selection is where much of the “art” is in model building, and even more so with ever-wider data.

“Customers have been asking for variable selection for time-to-event data for some time, and now GenReg will be able to do that. There are not a lot of easy options for doing variable selection across so many different scenarios” says Clay, a Senior Research Statistician Developer at JMP.

Screen Shot 2016-08-29 at 4.01.45 PM

Now you can use Generalized Regression on time-to-event data.

One of the biggest enhancements in the Generalized Regression platform for JMP Pro 13 is the ability to handle censored data to do parametric survival analysis and proportional hazards models. Fellow developer Peng Liu even added a link to the Generalized Regression platform from within the Survival Analysis platform (in JMP Pro).

Variable selection is an active area of research in statistics. To do variable selection well and build really useful models, you need a breadth of tools and even hybrid approaches using automated selection techniques like the Lasso and Elastic Net. But the interactive nature of GenReg makes it easy to adopt a hybrid strategy where you can easily explore alternative models supported by the automated selection technique. Now JMP Pro provides some powerful new variable selection methods, including a modified two-stage forward selection method — first on the main effects and then on the higher-order effects involving the main effects, making Generalized Regression a premier tool for analyzing designed experiments.

Also new in JMP Pro 13 is the Double Lasso, a two-stage modeling technique where a first pass of the Lasso screens for variables to select and then a second pass of the Lasso is done on the variables selected in the first pass. Doing two passes of the lasso effectively separates the selection and shrinkage process of the Lasso, which can lead to better predictions. Another highlight is the addition of the Extended Regularized Information Criterion (ERIC). ERIC is similar in spirit to the Bayesian Information Criterion, but it was derived specifically for the Adaptive Lasso.

You can find out more about what’s coming in JMP Pro 13 by visiting the preview page on our website. There, you can sign up to watch a live stream of JMP chief architect John Sall’s tour of JMP 13 on Sept. 21, as well as watch short videos about JMP 13 and JMP Pro 13.

Post a Comment

JMP 13 Preview: Making definitive screening designs more accessible

The purpose of screening in designed experiments is “to separate the vital few factors that have a substantial effect on the response from the trivial many that have negligible effects….The definitive screening design can reliably accomplish the task of screening even if there are a couple of second-order effects,” wrote Bradley Jones, JMP Research Fellow, in a blog post introducing definitive screening designs (DSDs) in 2012.

After DSDs became available in JMP 11, the response from users was swift and uniform. “I’m amazed by how quickly these designs have been accepted for routine use by the community of practitioners,” says Bradley, co-inventor of DSDs.

Why do people like DSDs? “They have a lot of properties that are desirable compared to other screening designs. Typically, you start with a screening design and then fit a higher-order model. With DSDs, you can sometimes skip that second step,” he explains.

Fit definitive screening design (left) with follow-up reduced model (right) run from this new platform.

Fit Definitive Screening design (left) with follow-up reduced model (right) run from this new platform in JMP 13.

Now in JMP 13, released later this month, users have a more robust way to statistically model DSDs, with an option to Fit Definitive Screening available in the DOE menu.

What does this mean? JMP now includes a complete end-to-end workflow for building and fitting DSDs, making it easier for scientists and engineers to try them out for their own work. The table generated by the definitive screening design automatically includes a script to fit the design with the new platform.

This platforms aids in finding second-order effects and quadratic effects, if they exist, with a fitting procedure that takes advantage of the structure of a DSD to provide more clear-cut results than generic model selection tools can provide. Ultimately, this new option in DOE means that people who have limited statistics expertise can be successful in finding real causal effects.

“I believe DOE is the most beneficial thing statistics has provided to people in science and engineering because practically any system, product or process can be improved by employing DOE. That’s the reason I’ve dedicated my professional life to working on DOE. I want to lower the barrier of accessibility for engineers and scientists to use it, so they don’t have to be an expert or have a professional statistician available at their elbow,” says Bradley.

While he is hopeful that more JMP users will try definitive screening designs in JMP 13, he cautions that the design is not for every situation they may encounter. Refer to his blog post about “Proper and improper uses of definitive screening designs” for details.

To learn more about what’s coming in JMP 13, stop by the preview page on our website. There, you can sign up to watch a live stream of JMP chief architect John Sall’s tour of JMP 13 on Sept. 21, as well as watch short videos about JMP 13 and JMP Pro 13.

Post a Comment

JMP 13 Preview: New text analytics in JMP Pro

Building on the new features in JMP 13 for exploring unstructured text data, JMP Pro 13 enables you to do more with text data, like cluster terms and phrases and use text in predictive models. You’ll be able to answer more questions, scale to larger data and stay in flow. Because organizations collect so much text data, JMP now provides visual, interactive, and easy-to-use capabilities to analyze all that text data and make valuable use of it.

As Chris Gotwalt, Director of Statistical R&D at JMP, explains, some of the capabilities required for text analysis are analogous to those required for tabular data so that adding text analytics made sense for the product. “Text analytics is like general multivariate analysis. Topic analysis is like factor analysis. Singular value decomposition (SVD) is like principal component analysis, but these algorithms need to be fast enough to be useful on text data,” Chris says.

While SVDs are a standard means for dealing with the high dimensionality that is typical of text data in most text mining software, the challenge is to do it quickly so the user’s analysis “flow” is not disrupted. Chris has implemented not only a super-fast sparse Lanczos SVD, but also designed it so that it handles messy data well and yields more meaningful factors. This SVD implementation also supports topic analysis.

Screen Shot 2016-08-23 at 11.47.20 AM

The "Show Text" option allows you to see text associated with a single data point or the text in common for several selected data points. Here it's surfaced in the SVD plot.

In addition, JMP Pro 13 also includes latent class analysis (LCA), useful for another kind of topic analysis (as multinomial mixtures) as well as for clustering text data. This LCA clustering approach customized for applications within Text Explorer allows for overlapping cluster membership probabilities for each document and takes advantage of sparse data to calculate fast summaries to show where high factor loadings are, which is important when dealing with ultra-wide data so typical of text data.

Screen Shot 2016-08-29 at 1.18.28 PM

Latent Class Analysis clustering allows for overlapping cluster membership probabilities for each document.

JMP Pro provides text scoring with SVD scores, but also saves the formula to calculate SVD scores for all analyses (any variable, scoring matrices, parses all tokens). You can also save the document-term-matrix, SVD and LCA scores as inputs to other analyses, such as predictive models.

And of course, these implementations are integrated and interactive as you would expect them to be in JMP, with new graphics to visualize and further explore findings.

Heath Rushing, co-founder of Adsurgo, is a fan. "I have used many text mining tools. In terms of ease of use, Text Explorer is the best of the breed. You can efficiently clean unstructured data, visualize relationships, find major themes and group documents. Brilliant!"  Heath says.

Whenever Chris has shown these text analytics additions, he runs out of time because the audience asks so many questions like, “Does it do this or that?” or “Can I use this to analyze my survey, maintenance log, web data, etc.?”

“There is a lot of excitement when people see the platform. Upon first sight, they start thinking of all the new things they could do with the text data that they have always had lying around but could never take advantage of before,” Chris says.

John Sall, chief architect of JMP, also worked on the new text analytics features and enjoyed it. “It’s been fun to work on a new area, supporting one more form of data from which users can derive value,” John says.

To learn more about the new Text Explorer platform, watch the Analytically Speaking interview with Adsurgo co-founder, Heath Rushing. Heath was very influential in the development of Text Explorer, including naming the new platform.

Heath also prepared two great demos for a Technically Speaking webcast. You can also check out the talk he and co-presenter James Wisnowski are giving at Discovery Summit, “Mind the Gap: JMP on the Text Explorer Express.” Others will be presenting more about text exploration at Discovery Summit, including a tutorial by Chris Gotwalt, “The U-to-the-V: A Hitchhiker’s Guide to JMP 13 Text Explorer.”

Post a Comment

JMP 13 Preview: Improvements to the Analyze menu for a better user experience

From time to time, the addition of new features requires a review of how capabilities are organized and presented in JMP. Are they located where it makes the most sense and where users would expect to find them? For example, in JMP 12 there was enough new material combined with existing functionality to warrant a Consumer Research submenu in the Analyze menu.

In JMP 13, users will also see some changes to the Analyze menu:

  • The old Modeling submenu has been replaced by two new submenus: Predictive Modeling and Specialized Modeling.
  • A Clustering submenu has been added so that you can quickly find your favorite clustering technique.
You can easily find the Fit Curve platform under Specialized Modeling in JMP 13.

You can easily find the Fit Curve platform under Specialized Modeling in JMP 13.

The Predictive Modeling submenu is the new home for a variety of modeling platforms that emphasize prediction, such as Neural, Partition, Random Forest, and K-Nearest Neighbors. The Specialized Modeling menu is where you will find platforms like Fit Curve, Nonlinear, and Time Series.

Senior Research Statistician Developer Clay Barker, who wrote the Fit Curve and Normal Mixture platforms, was happy to see these platforms find a home in the reorganized menus. “They used to be buried inside other platforms (Fit Curve inside Nonlinear and Normal Mixtures inside k-Means), which made them harder to find. Now they will get more exposure, and hopefully more people will start using them,” Clay says.

Screen Shot 2016-08-30 at 1.20.11 PM

The new Clustering submenu includes Normal Mixtures and Cluster Variables among other platforms.

The new Clustering submenu also contains a new platform: Latent Class Analysis or LCA for short. The addition of the generalized LCA for categorical clustering didn’t fit with the continuous response clustering methods that already existed. This was also the case with the new specialized implementation of LCA for text analytics in JMP Pro. Some of the output for the new LCA clustering includes Multidimensional Scaling (MDS) plots and “share” charts.

Screen Shot 2016-08-30 at 11.48.01 AM

Multi-Dimensional Scaling plot allows you to see distance between clusters.

While JMP includes a number of good clustering methods, they were hard to find especially if you didn’t know where to look, so many users didn’t know about them. Some, like Normal Mixture clustering or Variable Clustering, were “platform personalities” or in a list of red-triangle menus. The new Clustering submenu groups new and existing clustering methods together.

Screen Shot 2016-08-30 at 11.56.47 AM

The new share charts in the LCA report show the conditional probabilities given cluster membership for each cluster and each Y, plotted as a horizontal stacked bar chart.

The addition of LCA will enable users to do more with their text data. Chris Gotwalt, director of statistical R&D at JMP, enjoyed working on the sparse-matrix LCA, as he had never done anything like applying sparse methods to a clustering algorithm.

“The response so far has been positive on the new functionality as well as the new visuals. Having the other clustering methods more prominently featured in the submenu will hopefully lead to greater use,” Chris says.

Both Clay and Chris will be leading tutorials at Discovery Summit 2016, and it's not too late to sign up for these special sessions. In addition, they'll be presenting a paper titled "Visually Exploring Design of Experiments Models with the Generalized Regression Platform" at conference.

To learn more about what's coming in JMP 13, visit the preview page at our website.

Post a Comment