Everything But the Kitchen Sink Feature Generation


This blog is a part of a series on the Data Science Pilot Action Set. In my first blog we introduced the action set and the actions for building data understanding. In this blog we dive into feature generation and selection. 

The Data Science Pilot Action Set is included with SAS Visual Data Mining and Machine Learning (VDMML) and consists of actions that implement a policy-based, configurable, and scalable approach to automating data science workflows. The two actions we will examine in this blog are the featureMachine action and the selectFeatures action.

Data Science Pilot Actions

The featureMachine action

The featureMachine action not only generates new features, but it also explores the data and screens variables. That means that this action can also take on data exploration. Not only is this action taking on double duty, it can save you time by running task in parallel. When it comes to new features, this action creates everything but the kitchen sink. The list of transformations includes missing indicators, several types of imputation, several types of binning, and much more. Unfortunately, the featureMachine action doesn't include any subject matter expertise. This action won't replace domain knowledge in feature generation, but it has everything else covered.

The featureMachine action includes the explorationPolicy and screenPolicy, and the transformationPolicy. The explorationPolicy specifies how the data is grouped together, the screenPolicy controls how much data messiness is acceptable, and the transformationPolicy defines which types of features to create. The resulting output of this action set includes information about the generated features, the features generated using the input data, and an analytic store file for generating the features with new data.

/* Create new features using featureMachine Action */
proc cas;
	loadactionset "dataSciencePilot";
		/	table 			= "hmeq"
			target 			= "BAD"
			copyVars 		= "BAD"
			explorationPolicy      	= {cardinality = {lowMediumCutoff = 40}}
		    	screenPolicy           	= {missingPercentThreshold=35}
            		transformationPolicy   	= {entropy = True, iqv = True,  kurtosis = True, Outlier = True}
            		transformationOut      	= {name = "TRANSFORMATION_OUT", replace = True}
           		featureOut             	= {name = "FEATURE_OUT", replace = True}
            		casOut                 	= {name = "CAS_OUT", replace = True}
            		saveState              	= {name = "ASTORE_OUT", replace = True}

The selectFeatures action

The selectFeatures Action will filter features based on a specified measure. Using the selectionPolicy, you can specify the measure you want to filter on and how many features you want. If no measure is specified, the Mutual Information criterion is used as a default.  In continuation of my coding example, I fed the data generated from the featureMachine action into the selectFeatures action to select the best ten features.

/* Select features using selectFeatures Action */ 
proc cas;
	loadactionset "dataSciencePilot";
		/ 	table 		= "CAS_OUT"
			casOut 		= {name = "SELECT_FEATURES_OUT", replace = True}
			target 		= "BAD"
			selectionPolicy = {topk=10}


In this blog, we took control over more aspects of the data science workflow. Using the featureMachine action, we were able to create many new features and using the selectFeatures action, we narrowed our features down into a usable number. However, if you are looking for one piece of code that will do it all (or almost it all), stayed tuned for my next blog! In the upcoming blog, we will introduce the dsAutoMl action.


About Author

Sophia Rowland

Product Manager | SAS Model Manager

Sophia Rowland is a Senior Product Manager focusing on ModelOps and MLOps at SAS. Previously, Sophia was a Systems Engineer on a team that focuses on Data Science and ModelOps applications for the Financial Services industry. Sophia is an active speaker and author in the field of ModelOps. She has spoken at All Things Open and SAS Explore and has written dozens of blogs and articles for Open Data Science, SAS Communities, and the SAS Data Science Blog. Sophia is an alumnus of both UNC-Chapel Hill and Duke. At UNC-Chapel Hill, Sophia double majored in Computer Science and Psychology. At Duke, Sophia attended the Fuqua School of Business and completed a Master of Science in Quantitative Management: Business Analytics. After work, Sophia can be found reading, hiking, and baking.

Related Posts

Comments are closed.

Back to Top