Two sayings I’ve heard countless times throughout my life are “Work smarter, not harder,” and “Use the best tool for the job.” If you need to drive a nail, you pick up a hammer, not a wrench or a screwdriver. In the programming world, this could mean using an existing function library instead of writing your own or using an entirely different language because it’s more applicable to your problem. While that sounds good in practice, in the workplace you don’t always have that freedom.
So, what do you do when you’re given a hammer and told to fasten a screw? Or, like in the title of this article’s case, what do you do when you have Python functions you want to use in SAS?
Recently I was tasked with documenting an exciting new feature for SAS — the ability to call Python functions from within SAS. In this article I will highlight everything I’ve learned along the way to bring you up to speed on this powerful new tool.
PROC FCMP Python Objects
Starting with May 2019 release of SAS 9.4M6, the PROC FCMP procedure added support for submitting and executing functions written in Python from within a SAS session using the new Python object. If you’re unfamiliar with PROC FCMP, I’d suggest reading the documentation. In short, FCMP, or the SAS Function Compiler, enables users to write their own functions and subroutines that can then be called from just about anywhere a SAS function can be used in SAS. Users are not restricted to using Python only inside a PROC FCMP statement. You can create an FCMP function that calls Python code, and then call that FCMP function from the DATA step. You can also use one of the products or solutions that support Python objects including SAS High Performance Risk and SAS Model Implementation Platform.
The Why and How
So, what made SAS want to include this feature in our product? The scenario in mind we imagined when creating this feature was a customer who already had resources invested in Python modeling libraries but now wanted to integrate those libraries into their SAS environment. As much fun as it sounds to convert and validate thousands of lines of Python code into SAS code, wouldn’t it be nice if you could simply call Python functions from SAS? Whether you’re in the scenario above with massive amounts of Python code, or you’re simply more comfortable coding in Python, PROC FCMP is here to help you. Your Python code is submitted to a Python interpreter of your choice. Results are packaged into a Python tuple and brought back inside SAS for you to continue programming.
Programming in Two Languages at Once
So how do you program in SAS and Python at the same time? Depending on your installation of SAS, you may be ready to start, or there could be some additional environment setup you need to complete first. In either case, I recommend pulling up the Using PROC FCMP Python Objects documentation before we continue. The documentation outlines the addition of an output string that must be made to your Python code before it can be submitted from SAS. When you call a Python function from SAS, the return value(s) is stored in a SAS dictionary. If you’re unfamiliar with SAS dictionaries, you can read more about them here Dictionaries: Referencing a New PROC FCMP Data Type.
Getting Started
There are multiple methods to load your Python code into the Python object. In the code example below, I’ll use the SUBMIT INTO statement to create an embedded Python block and show you the basic framework needed to execute Python functions in SAS.
/* A basic example of using PROC FCMP to execute a Python function */ proc fcmp; /* Declare Python object */ declare object py(python); /* Create an embedded Python block to write your Python function */ submit into py; def MyPythonFunction(arg1, arg2): "Output: ResultKey" Python_Out = arg1 * arg2 return Python_Out endsubmit; /* Publish the code to the Python interpreter */ rc = py.publish(); /* Call the Python function from SAS */ rc = py.call("MyPythonFunction", 5, 10); /* Store the result in a SAS variable and examine the value */ SAS_Out = py.results["ResultKey"]; put SAS_Out=; run; |
You can gather from this example that there are essentially five parts to using PROC FCMP Python objects in SAS:
- Declaring your Python object.
- Loading your Python code.
- Publishing your Python code to the interpreter.
- Executing your Python Code.
- Retrieving your results in SAS.
From the SAS side, those are all the pieces you need to get started importing your Python code. Now what about more complicated functions? What if you have working models made using thousands of lines and a variety of Python packages? You still use the same program structure as before. This time I’ll be using the INFILE method to import my Python function library by specifying the file path to the library. You can follow along in by copying my Python code into a .py file. The file, blackscholes.py, contains this code:
def internal_black_scholes_call(stockPrice, strikePrice, timeRemaining, volatility, rate): import numpy from scipy import stats import math if ((strikePrice != 0) and (volatility != 0)): d1 = (math.log(stockPrice/strikePrice) + (rate + (volatility**2)\ / 2) * timeRemaining) / (volatility*math.sqrt(timeRemaining)) d2 = d1 - (volatility * math.sqrt(timeRemaining)) callPrice = (stockPrice * stats.norm.cdf(d1)) - \ (strikePrice * math.exp( (-rate) * timeRemaining) * stats.norm.cdf(d2)) else: callPrice=0 return (callPrice) def black_scholes_call(stockPrice, strikePrice, timeRemaining, volatility, rate): "Output: optprice" import numpy from scipy import stats import math optPrice = internal_black_scholes_call(stockPrice, strikePrice,\ timeRemaining, volatility, rate) callPrice = float(optPrice) return (callPrice,) |
My example isn’t quite 1000 lines, but you can see the potential of having complex functions all callable inside SAS. In the next figure, I’ll call these Python functions from SAS.
(/*Using PROC FCMP to execute Python functions from a file */ proc fcmp; /* Declare Python object */ declare object py(python); /* Use the INFILE method to import Python code from a file */ rc = py.infile("C:\Users\PythonFiles\blackscholes.py"); /* Publish the code to the Python interpreter */ rc = py.publish(); /* Call the Python function from SAS */ rc = py.call("black_scholes_call", 132.58, 137, 0.041095, .2882, .0222); /* Store the result in a SAS variable and examine the value */ SAS_Out = py.results["optprice"]; put SAS_Out=; run; |
Calling Python Functions from the DATA step
You can take this a step further and make it useable in the DATA step-outside of a PROC FCMP statement. We can use our program from the previous example as a starting point. From there, we just need to wrap the inner Python function call in an outer FCMP function. This function within a function design may be giving you flashbacks of Inception, but I promise you this exercise won’t leave you confused and questioning reality. Even if you’ve never used FCMP before, creating the outer function is straightforward.
/* Creating a PROC FCMP function that calls a Python function */ proc fcmp outlib=work.myfuncs.pyfuncs; /* Create the outer FCMP function */ /* These arguments are passed to the inner Python function */ function FCMP_blackscholescall(stockprice, strikeprice, timeremaining, volatility, rate); /* Create the inner Python function call */ /* Declare Python object */ declare object py(python); /* Use the INFILE method to import Python code from a file */ rc = py.infile("C:\Users\PythonFiles\blackscholes.py"); /* Publish the code to the Python interpreter */ rc = py.publish(); /* Call the Python function from SAS */ /* Since this the inner function, instead of values in the call */ /* you will pass the outer FCMP function arguments to the Python function */ rc = py.call("black_scholes_call", stockprice, strikeprice, timeremaining, volatility, rate); /* Store the inner function Python output in a SAS variable */ FCMP_out = py.results["optprice"]; /* Return the Python output as the output for outer FCMP function */ return(FCMP_out); /* End the FCMP function */ endsub; run; /* Specify the function library you want to call from */ options cmplib=work.myfuncs; /*Use the DATA step to call your FCMP function and examine the result */ data _null_; result = FCMP_blackscholescall(132.58, 137, 0.041095, .2882, .0222); put result=; run; |
With your Python function neatly tucked away inside your FCMP function, you can call it from the DATA step. You also effectively reduced the statements needed for future calls to the Python function from five to one by having an FCMP function ready to call.
Looking Forward
So now that you can use Python functions in SAS just like SAS functions, how are you going to explore using these two languages together? The PROC FCMP Python object expands the capabilities of SAS and by result improves you too as a SAS user. Depending on your experience level, completing a task in Python might be easier for you than completing that same task in SAS. Or you could be in the scenario I mentioned before where you have a major investment in Python and converting to SAS is non-trivial. In either case, PROC FCMP now has the capability to help you bridge that gap.
12 Comments
Hi Mike,
We have a python code pre written at our firm to use. how to install (like pip) that package and use (import) in SAS? Please help.
Thank you in advance!
Hi , I am trying to use proc fcmp for executing python code and I am using python libraries like pdfminer and beautifulsoup. I am getting an error stating that the library cannot be found. I understand that when this error occurs , we need to install the library. How and where do I install these libraries ?
Thanks and Regards.
Hi Mike,
thanks for the tips.
Juste one additional question : Is it planned to reproduce this kind of functiunality in a R Software context ?
Regards,
Gaetan
Yeah bookmaking this wasn't a risky decision outstanding post!
Pretty useful article, great work!
Very interesting ! I will try it out !
Hi Mike, great article, very well written. Would you consider a follow-up article with more complex return types such as lists and objects? I guess for the latter that we would need some wrapper function to de-objectify those data, could be interesting to see your take on that.
Do you need advance SQL experience to learn SAS? I am new to SAS/SQL and wondered if you have any options for a free course on SQL/SAS so that I can advance in my career pursuits?
Hi Gwen,
SQL experience is not required to learn/use SAS. However, if you have SQL experience, there are ways you can use it inside SAS.
As far as learning resources, SAS has a bunch of tutorials and e-learnings for beginners and experienced programmers on our learn page https://support.sas.com/en/software/base-sas-support.html#get-started
Hi Mike, thanks for your explanation.
Some questions:
1) how does SAS know where is located the Python interpreter?
2) the M6 release had been released in November 2018. Is this a new feature introduced later?
thanks in advance
Hi Andrea,
To your first question, SAS uses the Python interpreter specified by the environment variable "MAS_PYPATH". Information about the environment variables needed to use this feature can be found here https://go.documentation.sas.com/?docsetId=biasag&docsetTarget=n1mquxnfmfu83en1if8icqmx8cdf.htm&docsetVersion=9.4&locale=en
To your second question, yes the features described in this post use a newer version of M6 that was introduced in May 2019.
Hope this helps!
Hi Andrea,
In addition you can specify in MAS_PYPATH an interpreter which is in a virtual environment. This allow you tu use the librairies included in this virtual environnment