Jenine Milum is the Vice President and Analytics Manager at Wells Fargo Bank. About 10 years ago, she learned a valuable, but little known solution to cutting the CPU processing time when dealing with large data sets.
"We were processing log activity for our website on a daily basis," says Milum, who's been a SAS user for more than 20 years. "As the website became more popular, the volumes of data coming in for us to analyze grew at a mammoth rate. We were constantly looking for ways to speed up our daily processes and make them more efficient.
According t Milum, the resource drain was not the incoming data; it was matching the data to other files. "Every time we did it, we had to sort and then merge that massive file. That meant reading it twice, and we needed a way to avoid doing so."
Her innovative solution involves a matching method utilizing PROC Format, replacing the CPU-heavy sort/sort/merge. Read more in this Wells Fargo technical case study. Download Milum's paper, Proc Format, a Speedy Alternative to Sort/Sort/Merge (Best Contributed paper in Coder's Corner at SAS Global Forum 2012).