Everything But the Kitchen Sink Feature Generation


This blog is a part of a series on the Data Science Pilot Action Set. In my first blog we introduced the action set and the actions for building data understanding. In this blog we dive into feature generation and selection. 

The Data Science Pilot Action Set is included with SAS Visual Data Mining and Machine Learning (VDMML) and consists of actions that implement a policy-based, configurable, and scalable approach to automating data science workflows. The two actions we will examine in this blog are the featureMachine action and the selectFeatures action.

Data Science Pilot Actions

The featureMachine action

The featureMachine action not only generates new features, but it also explores the data and screens variables. That means that this action can also take on data exploration. Not only is this action taking on double duty, it can save you time by running task in parallel. When it comes to new features, this action creates everything but the kitchen sink. The list of transformations includes missing indicators, several types of imputation, several types of binning, and much more. Unfortunately, the featureMachine action doesn't include any subject matter expertise. This action won't replace domain knowledge in feature generation, but it has everything else covered.

The featureMachine action includes the explorationPolicy and screenPolicy, and the transformationPolicy. The explorationPolicy specifies how the data is grouped together, the screenPolicy controls how much data messiness is acceptable, and the transformationPolicy defines which types of features to create. The resulting output of this action set includes information about the generated features, the features generated using the input data, and an analytic store file for generating the features with new data.

/* Create new features using featureMachine Action */
proc cas;
	loadactionset "dataSciencePilot";
		/	table 			= "hmeq"
			target 			= "BAD"
			copyVars 		= "BAD"
			explorationPolicy      	= {cardinality = {lowMediumCutoff = 40}}
		    	screenPolicy           	= {missingPercentThreshold=35}
            		transformationPolicy   	= {entropy = True, iqv = True,  kurtosis = True, Outlier = True}
            		transformationOut      	= {name = "TRANSFORMATION_OUT", replace = True}
           		featureOut             	= {name = "FEATURE_OUT", replace = True}
            		casOut                 	= {name = "CAS_OUT", replace = True}
            		saveState              	= {name = "ASTORE_OUT", replace = True}

The selectFeatures action

The selectFeatures Action will filter features based on a specified measure. Using the selectionPolicy, you can specify the measure you want to filter on and how many features you want. If no measure is specified, the Mutual Information criterion is used as a default.  In continuation of my coding example, I fed the data generated from the featureMachine action into the selectFeatures action to select the best ten features.

/* Select features using selectFeatures Action */ 
proc cas;
	loadactionset "dataSciencePilot";
		/ 	table 		= "CAS_OUT"
			casOut 		= {name = "SELECT_FEATURES_OUT", replace = True}
			target 		= "BAD"
			selectionPolicy = {topk=10}


In this blog, we took control over more aspects of the data science workflow. Using the featureMachine action, we were able to create many new features and using the selectFeatures action, we narrowed our features down into a usable number. However, if you are looking for one piece of code that will do it all (or almost it all), stayed tuned for my next blog! In the upcoming blog, we will introduce the dsAutoMl action.


About Author

Sophia Rowland

Product Manager | SAS Model Manager

Sophia Rowland is a Senior Product Manager focusing on ModelOps and MLOps at SAS. In her previous role as a data scientist, Sophia worked with dozens of organizations to solve a variety of problems using analytics. As an active speaker and writer, Sophia has spoken at events like All Things Open, SAS Explore, and SAS Innovate as well as written dozens of blogs and articles. As a staunch North Carolinian, Sophia holds degrees from both UNC-Chapel Hill and Duke including bachelor’s degrees in computer science and psychology and a Master of Science in Quantitative Management: Business Analytics from the Fuqua School of Business. Outside of work, Sophia enjoys reading an eclectic assortment of books, hiking throughout North Carolina, and staying upright while ice skating.

Related Posts

Comments are closed.

Back to Top