Move beyond spreadsheets to data mining, forecasting, optimization – and more
“어떤 알고리즘을 사용해야 할까요?” 수많은 종류의 머신러닝 알고리즘을 맞닥뜨린 초급자 분들이 가장 많이 물어보는 전형적인 질문인데요. 사실 이 질문에 대한 답변은 하단 내용을 비롯한 수많은 요인에 따라 달라집니다. 데이터의 크기, 품질, 특성 가용 연산(계산) 시간 작업의 긴급성 데이터를 이용해 하고 싶은 것 그렇기에 숙련된 데이터 과학자(Data scientist)조차도 여러 알고리즘을 직접
In this blog post I am going to cover the example of importing data into SAS Viya using Cloud Analytic Services (CAS) actions via REST API. For example, you may want to import data into a CASLib via REST API. This means you can perform an import of data outside
Analytics-driven forecasting means more than measuring trend and seasonality. It includes all categories of methods (e.g. exponential smoothing, dynamic regression, ARIMA, ARIMA(X), unobserved component models, and more), including artificial intelligence, but not necessarily deep learning algorithms. That said, deep learning algorithms like neural networks can also be used for demand forecasting,
With all the recent talk about some people wanting to move from the US to Canada, I got to wondering how cold, and how far north Canada is. And after a few Google searches, I was surprised to learn that 27 US states are actually farther north than the southernmost point
In this post I describe the important tasks of data preparation, exploration and binning.These three steps enable you to know your data well and build accurate predictive models. First you need to clean your data. Cleaning includes eliminating variables which have uneven spread across the target variable. I give an
This is the fifth post in my series of machine learning best practices. Hyperparameters are the algorithm options one "turns and tunes" when building a learning model. Hyperparameters cannot be learned using that algorithm. So, these parameters need to be assigned before training of the model. A lot of manual
1980년대 후반에만 해도 머신러닝(machine learning)이나 데이터 과학자와 같은 개념은 없었습니다. 대신 통계, 분석, 데이터 마이닝, 데이터 모델링과 같은 단어가 사용됐는데요. 이후 글로벌 기업들은 30년 이상 머신러닝 모델을 연구해 왔으며, 페이스북의 이미지 인식 소프트웨어, 아마존의 음성 비서 알렉사, KT의 인공지능 서비스 기가 지니(GiGA Genie)까지 그 결과들이 연이어 쏟아지고 있죠! 이러한 결실
Recently, I was asked whether SAS can perform a principal component analysis (PCA) that is robust to the presence of outliers in the data. A PCA requires a data matrix, an estimate for the center of the data, and an estimate for the variance/covariance of the variables. Classically, these estimates
It's a common mantra many parents use to encourage their children to expand their food choices and try something new. Even as adults we’re often more comfortable easing into the unfamiliar, taking small bites, tiny samples, even dipping a toe in the water before diving in headfirst. Software is no
This is the fourth post in my series of 10 machine learning best practices. It’s common to build models on historical training data and then apply the model to new data to make decisions. This process is called model deployment or scoring. I often hear data scientists say, “It took
This is the third post in my series of machine learning techniques and best practices. If you missed the earlier posts, read the first one now, or review the whole machine learning best practices series. Data scientists commonly use machine learning algorithms, such as gradient boosting and decision forests, that automatically build
When it comes to economic activity, a handful of the largest metropolitan areas in the US account for lion's share. In 2013, the top 23 Metropolitan Statistical Areas (MSAs) accounted for 50% of the total US Gross Domestic Product (GDP). I recently came across a map created by Alexandr Trubetskoy
Wenn es darum geht, mit Daten zu arbeiten, dann ist der klassische SAS Datastep eines der wirksamsten Werkzeuge, das uns zur Verfügung steht. Es ist sehr einfach, mithilfe von Datenauswahl, Formeln und Bedingungen im Datastep sowie in spezifischen SAS Prozeduren zu Ergebnissen zu kommen. Hieraus sind in der Praxis umfangreiche
This is the second post in my series of machine learning best practices. If you missed it, read the first post, Machine learning best practices: the basics. As we go along, all ten tips will be archived at this machine learning best practices page. Machine learning commonly requires the use of
In the digital world where billions of customers are making trillions of visits on a multi-channel marketing environment, big data has drawn researchers’ attention all over the world. Customers leave behind a huge trail of data volumes in digital channels. It is becoming an extremely difficult task finding the right
My previous blog post focused on a graph, showing the % of women earning STEM degrees in various fields. While that graph was was designed to answer a very specific question, let's now look at the data from a broader perspective. Let's look at the total number of STEM degrees
An important problem in machine learning is the "classification problem." In this supervised learning problem, you build a statistical model that predicts a set of categorical outcomes (responses) based on a set of input features (explanatory variables). You do this by training the model on data for which the outcomes
I started my training in machine learning at the University of Tennessee in the late 1980s. Of course, we didn’t call it machine learning then, and we didn’t call ourselves data scientists yet either. We used terms like statistics, analytics, data mining and data modeling. Regardless of what you call
I recently showed how to compute a bootstrap percentile confidence interval in SAS. The percentile interval is a simple "first-order" interval that is formed from quantiles of the bootstrap distribution. However, it has two limitations. First, it does not use the estimate for the original data; it is based only
When building models, data scientists and statisticians often talk about penalty, regularization and shrinkage. What do these terms mean and why are they important? According to Wikipedia, regularization "refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. This information usually
Tomorrow is Independence Day, a federal holiday in the United States. Flags are displayed everywhere, especially in Washington, D.C., where I live. So let's have a little Fun with Flags! The current U.S. flag has 50 stars, one per state, with five rows of six stars interleaved with four rows
みなさま、こんにちは。 さて突然ですが、フーリエ変換ってご存知ですか？ おそらく物理学や経済学で波形データを分析したことのある方には馴染みがあるでしょうが、フーリエ変換は波形データを扱う手法です。 フーリエ変換では周期的な波形を、sin波やcos波の重ね合わせで説明しようというものです。 たとえば以下のような波形データは、どの時間にどのくらいの強さの波が流れているかを表現しています。 これをフーリエ変換することで、周波数と振幅で表すことができるようになります。 ↓ フーリエ変換！ ↓ 従来のSAS製品では波形データでフーリエ変換をする機能を提供していなかったのですが、SAS ViyaのSAS Forcastingという製品を使うことで、フーリエ変換を実施することができるようになりました。 SAS Viyaでできるのは短時間フーリエ変換（Short time Fourier transform）です。 今回はSAS Viyaでフーリエ変換を実施してみたいと思います。プログラミング言語はPythonを使用します。 まずは前準備として、必要なライブラリをインポートし、CAS sessionを作成します。 CAS sessionはSAS Viyaでデータ分析を行うCASというエンジンへ認証し、接続するものです。 # CAS sessionの用意 import swat host = "localhost" port = 5570 user = "user" password = "p@ssw0rd" mysession = swat.CAS(host, port, user, password) #
Let me start by posing a question: "Are you forecasting at the edge to anticipate what consumers want or need before they know it?" Not just forecasting based on past demand behavior, but using real-time information as it is streaming in from connected devices on the Internet of Things (IoT).
One way to assess the precision of a statistic (a point estimate) is to compute the standard error, which is the standard deviation of the statistic's sampling distribution. A relatively large standard error indicates that the point estimate should be viewed with skepticism, either because the sample size is small
Unser Interviewpartner Marius Gödtel ist Leiter des Competence Centers Business Intelligence IT-Consulting bei der adesso AG, einem IT-Dienstleister im BI-Umfeld. Marius Gödtel ist ehemaliger Geschäftsführer der flitcon GmbH, die seit 01. Juli 2016 Teil der adesso AG ist. Herr Gödtel, wie präsentieren Sie die adesso AG auf dem SAS Forum in
In a world that never stands still, especially in the post-Brexit whirlwind the UK will soon be entering, it’s imperative that government is agile and responsive. More importantly, how can this capability be arrived at without breaking the bank? The UK Government recently released its Transformation Strategy 2017 to 2020.
La digital transformation spazza come un vento tutti i settori industriali e promette di essere un motore per la competitività, la produttività e l’occupazione. Investire sul miglioramento delle competenze e il rafforzamento dei profili professionali è il punto di partenza per attuare la trasformazione di cui tanto si parla. Partendo
„Durchsatz ist wichtig, jaja“, Supply-Chain-Leiter Herr Aklit lehnt sich zurück, faltet seine Hände über dem üppigen Bauch und sagt zu Lenin: „Sie haben ja schon einiges in Fluss gebracht mit Ihren Projekten zur Datenanalyse im Internet of Things.“ Er atmet tief durch und schaut aus dem Fenster: „Alles fließt …“,
Citizens served by the government are increasingly the same digital savvy consumers that market disruptors in banking, retail and utilities are attracting with sophisticated, data-driven online experiences. It’s a mutually beneficial arrangement; consumers get to buy services in ways that suit them while businesses get the efficiencies they want, wrestling
고객관계관리(CRM), 여행 산업의 핵심 경쟁력으로 떠올라 요즘 ‘먹방’과 ‘쿡방’의 인기를 무섭게 따라잡고 있는 트렌드가 있습니다. 바로 ‘여행 방송’입니다! 꽃보다 할배의 열풍을 시작으로 원나잇 푸드트립, 패키지로 세계 일주 – 뭉쳐야 뜬다, 배틀 트립 등 각종 여행 방송이 쏟아지고 있는데요. 실제 방송 후 관련 여행 상품들의 매출이 증가하면서 그 인기를 증명하고 있죠!
