/* SAS program to accompany the article "Visualize Christmas songs" by Rick Wicklin, published 10DEC2018 on The DO Loop blog: https://blogs.sas.com/content/iml/2018/12/10/visualize-christmas-lyrics.html Based on the earlier article "Visualize repetition in song lyrics" https://blogs.sas.com/content/iml/2018/03/14/visualize-repetition-lyrics.html This program shows how to construct a "repetition matrix" for song lyrics or other text sources (speeches, poems, etc.) The program consists of two parts 1. Read the words into a SAS data set that contains one character variable (WORD). Strip out all punctuation and normalize the words to lowercase. 2. Use SAS/IML software to construct and visualize the repetition matrix. This idea is from Colin Morris and appeared in Significance magazine. See https://colinmorris.github.io/blog/weird-pop-songs and https://www.significancemagazine.com/570 */ %macro BreakIntoWords; length word $20; input word @@; word = lowcase( compress(word, ,'ps') ); /* remove punctuation and spaces */ %mend; data SantaClausComing; %BreakIntoWords; datalines4; You better watch out You better not cry Better not pout I'm telling you why Santa Claus is coming to town He's making a list And checking it twice; Gonna find out Who's naughty and nice Santa Claus is coming to town He sees you when you're sleeping He knows when you're awake He knows if you've been bad or good So be good for goodness sake! O! You better watch out! You better not cry Better not pout I'm telling you why Santa Claus is coming Santa Claus is coming Santa Claus is coming to town ;;;; data TwelveDays; %BreakIntoWords; datalines4; On the first day of christmas my true love sent to me A partridge in a pear tree On the second day of christmas my true love sent to me Two turtle doves, and A partridge in a pear tree On the third day of christmas my true love sent to me Three french hens Two turtle doves, and A partridge in a pear tree On the fourth day of christmas my true love sent to me Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the fifth day of christmas my true love sent to me Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the sixth day of christmas my true love sent to me Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the seventh day of christmas my true love sent to me Seven swans a-swimming Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the eighth day of christmas my true love sent to me Eight maids a-milking Seven swans a-swimming Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the nineth day of christmas my true love sent to me Nine ladies dancing Eight maids a-milking Seven swans a-swimming Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the tenth day of christmas my true love sent to me Ten lords a-leaping Nine ladies dancing Eight maids a-milking Seven swans a-swimming Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the eleventh day of christmas my true love sent to me Eleven pipers piping Ten lords a-leaping Nine ladies dancing Eight maids a-milking Seven swans a-swimming Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree On the twelth day of christmas my true love sent to me Twelve drummers drumming Eleven pipers piping Ten lords a-leaping Nine ladies dancing Eight maids a-milking Seven swans a-swimming Six geese a-laying Five gold rings Four colly birds Three french hens Two turtle doves, and A partridge in a pear tree ;;;; data SilverBells; %BreakIntoWords; datalines4; Silver Bells, Silver Bells It's Christmas time in the city Ring-a-ling, hear them ring Soon it will be Christmas Day City sidewalks Busy sidewalks Dressed in holiday style In the air there's A feeling of Christmas Children laughing People passing Meeting smile after smile And on every Street corner you'll hear Silver Bells, Silver Bells It's Christmas time in the city Ring-a-ling, hear them sing Soon it will be Christmas Day Strings of street lights Even stoplights Blink a bright red and green As the shoppers rush home With their treasures Hear the snow crunch See the kids bunch This is Santa's big scene And above all This bustle you'll hear Silver Bells, Silver Bells It's Christmas time in the city Ring-a-ling, hear them sing Soon it will be Christmas Day Silver Bells, Silver Bells It's Christmas time in the city Ring-a-ling, hear them sing Soon it will be Christmas Day Soon it will be Christmas Day Soon it will be Christmas Day Soon it will be Christmas Day Soon it will be Christmas Day ;;;; data DrummerBoy; %BreakIntoWords; datalines4; Come they told me Pa rum pum pum pum A new born King to see, Pa rum pum pum pum Our finest gifts we bring Pa rum pum pum pum To lay before the King Pa rum pum pum pum Rum pum pum pum Rum pum pum pum So to honor Him Pa rum pum pum pum, When we come. Little Baby Pa rum pum pum pum I am a poor boy too, Pa rum pum pum pum I have no gift to bring Pa rum pum pum pum That's fit to give our King Pa rum pum pum pum Rum pum pum pum Rum pum pum pum Shall I play for you! Pa rum pum pum On my drum. Mary nodded Pa rum pum pum pum The ox and lamb kept time Pa rum pum pum pum I played my drum for Him Pa rum pum pum I played my best for Him Pa rum pum pum pum Rum pum pum pum Rum pum pum pum Then He smiled at me Pa rum pum pum pum Me and my drum. ;;;; data SilentNight; %BreakIntoWords; datalines4; Silent night, holy night! All is calm, all is bright Round yon Virgin, Mother and Child Holy Infant so tender and mild Sleep in heavenly peace Sleep in heavenly peace Silent night, holy night! Shepherds quake at the sight Glories stream from heaven afar Heavenly hosts sing Alleluia! Christ the Savior is born Christ the Savior is born Silent night, holy night! Son of God, love's pure light Radiant beams from Thy holy face With the dawn of redeeming grace Jesus Lord, at Thy birth Jesus Lord, at Thy birth ;;;; data Rockin; %BreakIntoWords; datalines4; Rockin' around the Christmas tree At the Christmas party hop Mistletoe hung where you can see Every couple tries to stop Rockin' around the Christmas tree Let the Christmas spirit ring Later we'll have some pumpkin pie And we'll do some caroling You will get a sentimental feeling when you hear Voices singing, let's be jolly Deck the halls with boughs of holly Rockin' around the Christmas tree Have a happy holiday Everyone dancin' merrily In the new old-fashioned way You will get a sentimental feeling when you hear Voices singing, let's be jolly Deck the halls with boughs of holly Rockin' around the Christmas tree Have a happy holiday Everyone dancin' merrily In the new old-fashioned way ;;;; data Someday; %BreakIntoWords; datalines4; Someday at Christmas men won't be boys Playing with bombs like kids play with toys One warm December our hearts will see A world where men are free Someday at Christmas there'll be no wars When we have learned what Christmas is for When we have found what life's really worth There'll be peace on earth Someday all our dreams will come to be Someday in a world where men are free Maybe not in time for you and me But someday at Christmas time Someday at Christmas we'll see a land No hungry children and no empty hand One happy morning people will share Our world where people care Someday at Christmas there'll be no tears All men are equal and no men have fears One shining moment my heart ran away From our world today Someday all our dreams will come to be Someday in a world where men are free Maybe not in time for you and me But someday at Christmas time Someday at Christmas man will not fail Take hope because your love will prevail Someday a new world that we can start With hope in every heart Someday all our dreams will come to be Someday in a world where men are free Maybe not in time for you and me But someday at Christmas time Someday at Christmas time ;;;; data HappyXMas; %BreakIntoWords; datalines4; So this is Christmas And what have you done Another year over And a new one just begun And so this is Christmas I hope you have fun The near and the dear one The old and the young A very Merry Christmas And a happy New Year Let's hope it's a good one Without any fear And so this is Christmas (war is over) For weak and for strong (if you want it) For rich and the poor ones (war is over) The world is so wrong (now) And so happy Christmas (war is over) For black and for white (if you want it) For yellow and red ones (war is over) Let's stop all the fight (now) A very Merry Christmas And a happy New Year Let's hope it's a good one Without any fear And so this is Christmas (war is over) And what have we done (if you want it) Another year over (war is over) A new one just begun (now) And so happy Christmas (war is over) We hope you have fun (if you want it) The near and the dear one (war is over) The old and the young (now) A very Merry Christmas And a happy New Year Let's hope it's a good one Without any fear War is over, if you want it War is over now ;;;; /* Last Christmas, by Wham! */ data LastChristmas; %BreakIntoWords; datalines4; Last Christmas I gave you my heart, But the very next day you gave it away. This year To save me from tears I'll give it to someone special. Last Christmas I gave you my heart, But the very next day you gave it away. This year To save me from tears I'll give it to someone special. Once bitten and twice shy I keep my distance, But you still catch my eye. Tell me, baby, Do you recognize me? Well, it's been a year, It doesn't surprise me. Merry Christmas. I wrapped it up and sent it With a note saying "I love you". I meant it. Now I know what a fool I've been, But if you kissed me now I know you'd fool me again. Last Christmas I gave you my heart, But the very next day you gave it away. This year To save me from tears I'll give it to someone special. Last Christmas I gave you my heart, But the very next day you gave it away. This year To save me from tears I'll give it to someone special. Oh, oh, baby. A crowded room. Friends with tired eyes. I'm hiding from you And your soul of ice. My god, I thought you were someone to rely on. Me? I guess I was a shoulder to cry on. A face on a lover with a fire in his heart. A man under cover, but you tore me apart. Now I've found a real love. You'll never fool me again. Last Christmas I gave you my heart, But the very next day you gave it away. This year To save me from tears I'll give it to someone special. Last Christmas I gave you my heart, But the very next day you gave it away. This year To save me from tears I'll give it to someone special. A face on a lover with a fire in his heart. I gave you my heart. A man under cover, but you tore him apart. Maybe next year I'll give it to someone— I'll give it to someone special. Special. Someone. Someone. ;;;; /* Use the following SAS/IML function to visualize lyrics in songs */ ods graphics / width=500 height=500 NXYBINSMAX=1000000; proc iml; /* define a function that creates and visualizes the repetition matrix */ start VizLyrics(DSName, Title); use (DSName); read all var _CHAR_ into Word; close; N = nrow(Word); M = j(N,N,0); do i = 1 to N; M[,i] = element(Word, Word[i]); end; run heatmapdisc(M) title=Title colorramp={lightpink green} displayoutlines=0 showlegend=0; upperIdx = loc(col(M)>row(M)); return ( M[upperIdx][:] ); /* proportion of words that are repeated */ finish; score = VizLyrics("SantaClausComing", "Santa Claus is Coming to Town"); score = VizLyrics("TwelveDays", "The Twelve Days of Christmas"); score = VizLyrics("DrummerBoy", "The Little Drummer Boy"); score = VizLyrics("SilverBells", "Silver Bells"); score = VizLyrics("SilentNight", "Silent Night"); /*****************/ score = VizLyrics("LastChristmas", "Last Christmas (Wham!)"); score = VizLyrics("Someday", "Someday at Christmas (Stevie Wonder)"); score = VizLyrics("Rockin", "Rockin' Around the Christmas Tree (Brenda Lee)"); score = VizLyrics("HappyXMas", "Happy XMas (War Is Over) (John Lennon/Yoko Ono)"); QUIT;