Data Retrieval

Home , Before He Cheats, Bobbie Gentry, Boyz II Men, Candle in the Wind 1997, Charlie Puth, Foolish Games

Data Retrieval

ttp://www.billboard.com/charts/hot-100/2016-09-17 http://www.billboard.com/fe-ajax/spotify/preview/chart-tracks/HSI/2016-09-17/US 2 �� hot100Data.nb

getRankingxml_, maxRank_:= Cases xml,

XMLElement"div","class"→"chart-row__main-display", ___, XMLElement"div","class"→"chart-row__rank", ___, XMLElement"span","class"→"chart-row__current-week",rank_, ___ , ___, XMLElement"div","class"→"chart-row__container", ___, XMLElement"div","class"→"chart-row__title", ___, XMLElement_,"class"→"chart-row__song",{name_}, ___, XMLElement_,___,"class"→"chart-row__artist",___,artist_, ___ , ___ , ___  ⧴ <| "Rank"→ ToExpressionrank, "Score"→ maxRank+1- ToExpressionrank, "Name"→ name, "ArtistName"→ StringReplaceartist, StartOfString ~~WhitespaceCharacter... |>,

Infinity 

getRankingurl_String, maxRank_:= getRankingImportURLReadurl, CharacterEncoding→" UTF8

getRankingHot100date_DateObject:= getRankingURLBuild"http://www.billboard.com", " charts

getRankingBillboard200date_DateObject:= getRankingURLBuild"http://www.billboard.com "

hot100StartDate= DateObject[{1958, 8, 9}];

billboard200StartDate= DateObject[{1963, 8, 24}]; hot100Data.nb ��3

Hot 100

FloorToday- hot100StartDate  7 days 

3032

hot100Raw= MonitorTableWithdate= hot100StartDate+ 7 days *n, <| "Date"→ date, "Rankings"→ getRankingHot100[date]|>, n, 0, FloorToday- hot100StartDate  7 days , n;

(*DumpSave["~/Dropbox/mathematica/hot100/hot100Raw.mx",hot100Raw];*)

hot100= Dataset[hot100Raw];

(*DumpSave["~/Dropbox/mathematica/hot100/hot100.mx",hot100];*)

Billboard 200

FloorToday- billboard200StartDate  7 days 

2769

billboard200Raw= MonitorTableWithdate= billboard200StartDate+ 7 days *n, <|"Date"→ date, "Rankings"→ getRankingBillboard200[date]|>, n, 0, FloorToday- billboard200StartDate  7 days , n;

(*DumpSave["~/Dropbox/mathematica/hot100/billboard200Raw.mx",billboard200Raw];*)

billboard200= Dataset[billboard200Raw];

(*DumpSave["~/Dropbox/mathematica/hot100/billboard200.mx",billboard200];*)

Notes

The actual Hot 100 is full of bugs! Here are missing indexes with links to check them on the original site: 4 �� hot100Data.nb

hot100DeleteCases[{{}, _}], Query"Rankings"/*Complement[Range[100],#]&, All, "Rank", "Date"/*Hyperlink@URLBuild[{"http://www.billboard.com", "charts", "hot-100", DateString[#,{"Year", "-", "Month", "-", "Day"}]}]& △ {51, 64, 75} http://www.billboard.com/charts/hot-100/1958-08-09 {10, 82} http://www.billboard.com/charts/hot-100/1958-08-16 {6, 72, 82} http://www.billboard.com/charts/hot-100/1958-08-23 {9} http://www.billboard.com/charts/hot-100/1958-08-30 {29} http://www.billboard.com/charts/hot-100/1960-03-19 {35} http://www.billboard.com/charts/hot-100/1961-02-04 {29} http://www.billboard.com/charts/hot-100/1961-04-08 {19} http://www.billboard.com/charts/hot-100/1961-04-15 {69} http://www.billboard.com/charts/hot-100/1961-08-26 {59} http://www.billboard.com/charts/hot-100/1961-09-02 {64} http://www.billboard.com/charts/hot-100/1961-09-16 {45} http://www.billboard.com/charts/hot-100/1961-09-23 {40} http://www.billboard.com/charts/hot-100/1961-09-30 {38} http://www.billboard.com/charts/hot-100/1961-10-07 {25} http://www.billboard.com/charts/hot-100/1961-10-14 {46} http://www.billboard.com/charts/hot-100/1961-10-21 {38} http://www.billboard.com/charts/hot-100/1961-10-28 {38} http://www.billboard.com/charts/hot-100/1961-11-04 {36} http://www.billboard.com/charts/hot-100/1961-11-11 {37} http://www.billboard.com/charts/hot-100/1961-11-18

showing 1–20 of 99 ▽

Some entries in the Billboard 200 just don’t exist: hot100Data.nb ��5

billboard200[ListLinePlot, "Rankings"/* Length]

200

150

100

500 1000 1500 2000 2500 billboard200[Position[0], "Rankings"/* Length]

672 1115 1145 1148 1149 1150 1153 1154 1155 1158 1159 1160 1163 1164 1165 1197 1306 1316 6 �� hot100Data.nb

Analysis

Initialization

Constants

Get["~/Dropbox/mathematica/hot100/hot100.mx"]

Get["~/Dropbox/mathematica/hot100/billboard200.mx"]

hot100Entries= hot100[Map[Map[Prepend["Date"→#Date],#Rankings]&] /* Catenate];

hot100Artists= hot100Entries[GroupBy["ArtistName"]];

hot100Songs= hot100Entries[GroupBy[{#Name,#ArtistName}&]];

hot100Years= hot100[GroupBy[DateObject[{DateValue[#Date, "Year"]}]&]];

billboard200Entries= billboard200[Map[Map[Prepend["Date"→#Date],#Rankings]&] /* Catenate];

billboard200Artists= billboard200Entries[GroupBy["ArtistName"]];

billboard200Albums= billboard200Entries[GroupBy[{#Name,#ArtistName}&]];

billboard200Years= billboard200[GroupBy[DateObject[{DateValue[#Date, "Year"]}]&]];

Functions

fillDates[data_Association]:= Merge[{Association@Thread[Normal@hot100[All, "Date"]→0], data}, Last]

fillDates[data_List]:= KeyValueMap[List, fillDates@Association[Rule @@@ data]]

fillDates[data_Dataset]:= fillDates[Normal[data]]

Basic Info

Distribution of artist scores: hot100Data.nb ��7

hot100Artists[Histogram, Total, "Score"] 3000

2500

2000

1500

1000

500

0 0 1000 2000 3000 4000 5000 hot100Artists[All, Total, "Score"][Sort/* Reverse] △ Madonna 57 370 Elton John 56 042 Taylor Swift 49 685 Mariah Carey 45 172 Stevie Wonder 42 555 The Beatles 42 470 Michael Jackson 39 697 Rod Stewart 37 863 Whitney Houston 37 742 The Rolling Stones 37 095 Rihanna 36 746 Chicago 35 663 Aretha Franklin 34 880 Billy Joel 34 705 Daryl Hall John Oates 33 576 Bee Gees 32 890 Neil Diamond 32 711 Kelly Clarkson 32 707 R. Kelly 30 776 Janet Jackson 30 727

showing 1–20 of 9146 ▽ 8 �� hot100Data.nb

hot100Artists[All, Total, "Score"][WordCloud]

�� -��

Weeks with #1 by artist:

hot100[Counts/* Histogram, "Rankings", 1, "ArtistName"]

200

150

100

2 4 6 8 10 12 14 hot100Data.nb ��9

hot100[Counts/* Sort/* Reverse, "Rankings", 1, "ArtistName"] △ Mariah Carey 60 The Beatles 54 Boyz II Men 34 Madonna 32 Whitney Houston 31 Michael Jackson 30 The Black Eyed Peas 28 Bee Gees 27 Adele 24 Elton John 23 Usher 22 Janet Jackson 21 Elvis Presley With The Jordanaires 21 The Supremes 19 TLC 18 Katy Perry 18 The Rolling Stones 17 Rod Stewart 17

Olivia Newton-John 17 The 4 Seasons 16

showing 1–20 of 701 ▽

Most common words in song titles: hot100Entries[Catenate/* DeleteStopwords/* WordCloud, "Name"/* TextWords/* ToLowerCase] $Aborted

Most common words in song titles for #1 songs: 10 �� hot100Data.nb

hot100[Catenate/* DeleteStopwords/* WordCloud, "Rankings", 1, "Name"/* ToLowerCase/* TextWords] �� / ��

hot100Entries[Catenate/* DeleteStopwords/* WordCloud, "Name"/* TextWords/* ToLowerCase] Songs with the highest scores: hot100Data.nb ��11

hot100Songs[All, Total, "Score"][Sort/* Reverse] △ Radioactive Imagine

How Do I Live LeAnn

Foolish Games/You Were Meant For Me Jewel

I'm Yours Jason

Party Rock Anthem LMFAO

Counting Stars OneRepublic

Uptown Funk! Mark

Rolling In The Deep Adele

Smooth Santana

Somebody That I Used To Know Gotye

I Gotta Feeling The

Need You Now Lady

Dark Horse Katy

Before He Cheats Carrie

Truly Madly Deeply Savage

Too Close Next

Macarena(Bayside Boys Mix) Los

Thinking Out Loud Ed Sheeran

Ho Hey The

Trap Queen Fetty

▽ showing 1–20 of 27 459

#1 songs by artist: 12 �� hot100Data.nb

hot100Songs[Select[Min[#[[All, "Rank"]]] ===1&] /* Counts, 1, "ArtistName"][ Sort/* Reverse] △ The Beatles 19 Mariah Carey 15 Madonna 12 Whitney Houston 11 Michael Jackson 11 The Supremes 10 Bee Gees 9 The Rolling Stones 8 Stevie Wonder 7 Janet Jackson 7 Rihanna 6 Phil Collins 6 Katy Perry 6 Elvis Presley With The Jordanaires 6 Daryl Hall John Oates 6 Usher 5 Paula Abdul 5 KC And The Sunshine Band 5 George Michael 5 Elton John 5

showing 1–20 of 701 ▽

Artists Over Time

The Beatles hot100Data.nb ��13

hot100[DateListPlot[#, PlotRange→ All]&,{"Date", Query["Rankings", Select[#ArtistName === "The Beatles" &] /* Total, "Score"]}]

800

600

400

200

0 1960 1980 2000 hot100DateListPlot#, PlotRange →  1960 , 1975 , All&,{"Date",

Query["Rankings", Select[#ArtistName === "The Beatles" &] /* Total, "Score"]}

800

600

400

200

0 1960 1965 1970 1975

Elton John 14 �� hot100Data.nb

hot100[DateListPlot[#, PlotRange→ All]&,{"Date", Query["Rankings", Select[#ArtistName === "Elton John" &] /* Total, "Score"]}]

150

100

0 1960 1980 2000

Imagine Dragons

hot100DateListPlot#, PlotRange →  2005 , Today, All&,{"Date",

Query["Rankings", Select[#ArtistName === "Imagine Dragons" &] /* Total, "Score"]}

150

100

0 2006 2008 2010 2012 2014 2016

Taylor Swi� hot100Data.nb ��15

hot100DateListPlot#, PlotRange →  2005 , Today, All&,{"Date",

Query["Rankings", Select[#ArtistName === "Taylor Swift" &] /* Total, "Score"]}

600

500

400

300

200

100

0 2006 2008 2010 2012 2014 2016

U2 hot100DateListPlot#, PlotRange →  1980 , Today, All&,

{"Date", Query["Rankings", Select[#ArtistName === "U2" &] /* Total, "Score"]}

140

120

100

0 1980 1990 2000 2010

Coldplay 16 �� hot100Data.nb

hot100DateListPlot#, PlotRange →  2000 , Today, All&,

{"Date", Query["Rankings", Select[#ArtistName === "Coldplay" &] /* Total, "Score"]}

250

200

150

100

0 2000 2005 2010 2015

Billy Joel

hot100[DateListPlot[#, PlotRange→ All]&,{"Date", Query["Rankings", Select[#ArtistName === "Billy Joel" &] /* Total, "Score"]}]

150

100

0 1960 1980 2000 hot100Data.nb ��17

hot100Artists"The Beatles", DeleteDuplicates/*hot100Songs[#, All,{"Date", "Score"} /* Values]&/* KeyValueMap[Tooltip[#2,#1[[1]]]&] /*DateListPlot[#, PlotRange→ All]&, {"Name", "ArtistName"} /* Values/* Key

100

0 1970 1980 1990 hot100Artists"The Beatles", DeleteDuplicates/*hot100Songs[#, All,{"Date", "Score"} /* Values]&/* KeyValueMap[Tooltip[#2,#1[[1]]]&] /* DateListPlot[#, PlotRange→{{DateObject[{1963}], DateObject[{1971}]}, All}]&, {"Name", "ArtistName"} /* Values/* Key

100

0 1964 1966 1968 1970 18 �� hot100Data.nb

billboard200Artists"The Beatles", DeleteDuplicates/*billboard200Albums[#, All,{"Date", "Score"} /* Values]&/* KeyValueMap[Tooltip[#2,#1[[1]]]&] /* DateListPlot[#, PlotRange→{{DateObject[{1963}], DateObject[{1972}]}, All}]&, {"Name", "ArtistName"} /* Values/* Key

200

150

100

0 1964 1966 1968 1970 1972

hot100Artists"The Beatles", DeleteDuplicates/*hot100Songs[#, All, "Score"]&/* KeyValueMap[Tooltip[#2,#1[[1]]]&] /*ListLinePlot[#, PlotRange→ All]&, {"Name", "ArtistName"} /* Values/* Key

100

5 10 15 20 25 hot100Data.nb ��19

billboard200Artists"The Beatles", DeleteDuplicates/*billboard200Albums[#, All, "Score"]&/* KeyValueMap[Tooltip[#2,#1[[1]]]&] /*ListLinePlot[#, PlotRange→ All]&, {"Name", "ArtistName"} /* Values/* Key

200

150

100

50 100 150 200

Artist Introductions

Distribution of the rank of artists’s first song to hit top 100:

hot100Artists[Histogram, 1, "Rank"]

800

600

400

200

40 50 60 70 80 90 100

Distribution of the max rank of artists’s first song to hit top 100: 20 �� hot100Data.nb

hot100ArtistsHistogram, 1, {"Name", "ArtistName"} /*hot100Songs[Key[{#Name,#ArtistName}], Min, "Rank"]&

800

600

400

200

0 20 40 60 80 100

Song Introductions

Distribution of the ranks of songs entering:

hot100Songs[Histogram, 1, "Rank"]

5000

4000

3000

2000

1000

0 40 60 80 100

Distribution of the min (best) ranks of songs: hot100Data.nb ��21

hot100Songs[Histogram, Min, "Rank"]

2500

2000

1500

1000

500

0 0 20 40 60 80 100

Song Charting Tendencies

Scores of random songs over time:

hot100SongsRandomSample[#, 10]&/* KeyValueMap[Legended[#2, Row[{Style[#1[[2]], Bold], ": ",#1[[1]]}]]&] /* ListLinePlot, All, "Score"

100 Drafi: Marble Breaks And Iron Bends Theola Kilgore: This Is My Prayer 80 The Everly Brothers: All I Have To Do Is Dream Brian McKnight: Back At One 60 Keith Washington: Kissing You The Kinks: Better Things 40 Johnny Cash And The Tennessee Two: Aerosmith: Hole In My Soul 20 Celine Dion And Clive Griffin: When I Fall Richard Hayman And His Orchestra: Night 5 10 15 20 25 30 35

Longest charting songs: 22 �� hot100Data.nb

hot100SongsSortBy[Length] /* Reverse/* ;; 10/* KeyValueMap[ Legended[Tooltip[#2,#1[[1]]], Row[{Style[#1[[2]], Bold], ": ",#1[[1]]}]]&] /* ListLinePlot, All, "Score"

100 Imagine Dragons: Radioactive AWOLNATION: Sail 80 Jason Mraz: I'm Yours LeAnn Rimes: How Do I Live 60 LMFAO Featuring Lauren Bennett & GoonRock OneRepublic: Counting Stars 40 Adele: Rolling In The Deep Jewel: Foolish Games/You Were Meant For 20 Carrie Underwood: Before He Cheats Lifehouse: You And Me 20 40 60 80 hot100Data.nb ��23

hot100Songs[SortBy[Length] /* Reverse] △

Radioactive Imagine Dragons

Sail AWOLNATION

I'm Yours Jason Mraz

How Do I Live LeAnn Rimes ▽

Party Rock Anthem LMFAO Featuring Lauren

Counting Stars OneRepublic

Rolling In The Deep Adele

Foolish Games/You Were Meant For Me Jewel

Before He Cheats Carrie Underwood

You And Me Lifehouse

Distribution of weeks songs chart for: 24 �� hot100Data.nb

hot100Songs[Histogram, Length]

3000

2500

2000

1500

1000

500

0 0 10 20 30 40

hot100Songs[Histogram[#,{1}] &, Length]

2000

1500

1000

500

0 10 20 30 40

Distribution of weeks #1s chart for:

hot100Songs[Select[Min[#[[All, "Rank"]]] ===1&] /* Histogram, Length]

150

100

0 10 20 30 40 hot100Data.nb ��25

hot100SongsSelect[Min[#[[All, "Rank"]]] ===1&] /*Histogram[#,{1}]&, Length

0 10 20 30 40

Number of weeks before a #1 reaches #1: hot100Songs Select[Min[#[[All, "Rank"]]] ===1&] /*Histogram[UnitConvert[#, "Weeks"],{1}]&, {MaximalBy["Score"] /* First, 1} /* Query[All, "Date"] /* Apply[Subtract], {"Date", "Score"}

120

100

0 5 10 15 20 hot100Songs[Select[Min[#[[All, "Rank"]]] ===1&], {MaximalBy["Score"] /* First, 1} /* Query[All, "Date"] /* Apply[Subtract], {"Date", "Score"}]QuantityMagnitude[#, "Weeks"]&/* Values/* FindDistribution MixtureDistribution[{0.634081, 0.365919}, {PoissonDistribution[8.47627], GeometricDistribution[0.0922922]}] 26 �� hot100Data.nb

hot100SongsSelect[Min[#[[All, "Rank"]]] ===1&] /*UnitConvert[#, "Weeks"]&, {MaximalBy["Score"] /* First, 1} /* Query[All, "Date"] /* Apply[Subtract], {"Date", "Score"}[Sort/* Reverse] △ {Red Red Wine, UB40} 246 wk {Last Friday Night(T.G.I.F.), Katy Perry} 50 wk {Macarena(Bayside Boys Mix), Los Del Rio} 48 wk {Set Fire To The Rain, Adele} 47 wk {Baby, Come To Me, Patti Austin With James Ingram} 43 wk {Imma Be, The Black Eyed Peas} 39 wk {Amazed, Lonestar} 39 wk {All Of Me, John Legend} 34 wk {Someone Like You, Adele} 27 wk {With Arms Wide Open, Creed} 26 wk {Everything You Want, Vertical Horizon} 25 wk {I Am Woman, Helen Reddy} 24 wk {Party Rock Anthem, LMFAO Featuring Lauren Bennett & GoonRock} 22 wk {Just Dance, Lady GaGa Featuring Colby O'Donis} 22 wk {Cheap Thrills, Sia Featuring Sean Paul} 22 wk {Rolling In The Deep, Adele} 21 wk {Chariots Of Fire- Titles, Vangelis} 21 wk {Thrift Shop, Macklemore & Ryan Lewis Featuring Wanz} 20 wk {The Way You Move, OutKast Featuring Sleepy Brown} 20 wk {Sad Eyes, Robert John} 20 wk

showing 1–20 of 1065 ▽

Number of weeks a�er a #1 first reaches #1 that it stays charting: hot100Data.nb ��27

hot100Songs Select[Min[#[[All, "Rank"]]] ===1&] /*Histogram[UnitConvert[#, "Weeks"],{1}]&, {-1, MaximalBy["Score"] /* First} /* Query[All, "Date"] /* Apply[Subtract], {"Date", "Score"}

120

100

0 5 10 15 20 25 30 35 hot100Songs[Select[Min[#[[All, "Rank"]]] ===1&], {-1, MaximalBy["Score"] /* First} /* Query[All, "Date"] /* Apply[Subtract], {"Date", "Score"}]QuantityMagnitude[#, "Weeks"]&/* Values/* FindDistribution MixtureDistribution[{0.7258, 0.2742}, {PoissonDistribution[10.0006], LogSeriesDistribution[0.998211]}] 28 �� hot100Data.nb

hot100SongsSelect[Min[#[[All, "Rank"]]] ===1&] /*UnitConvert[#, "Weeks"]&, {-1, MaximalBy["Score"] /* First} /* Query[All, "Date"] /* Apply[Subtract], {"Date", "Score"}[Sort/* Reverse] △ {Let's Go Crazy, Prince And The Revolution} 1650 wk {Billie Jean, Michael Jackson} 1631 wk {Kiss, Prince And The Revolution} 1569 wk {How Will I Know, Whitney Houston} 1359 wk {Greatest Love Of All, Whitney Houston} 1346 wk {I Wanna Dance With Somebody(Who Loves Me), Whitney Houston} 1288 wk {I Honestly Love You, Olivia Newton-John} 1245 wk {Daydream Believer, The Monkees} 990 wk {Venus, Frankie Avalon} 890 wk {Breaking Up Is Hard To Do, Neil Sedaka} 709 wk {Monster Mash, Bobby Boris Pickett And The Crypt-Kickers} 569 wk {Ode To Billie Joe, Bobbie Gentry} 469 wk {The Twist, Chubby Checker} 80 wk {Light My Fire, The Doors} 62 wk {Uptown Funk!, Mark Ronson Featuring Bruno Mars} 59 wk {Party Rock Anthem, LMFAO Featuring Lauren Bennett & GoonRock} 53 wk {I Gotta Feeling, The Black Eyed Peas} 53 wk {Shake It Off, Taylor Swift} 49 wk {See You Again, Wiz Khalifa Featuring Charlie Puth} 47 wk {Rolling In The Deep, Adele} 47 wk

showing 1–20 of 1065 ▽

Number of weeks a�er at #1 (of songs that reach number 1): hot100Data.nb ��29

hot100Songs[DeleteCases[0] /* Histogram, Count[1], "Rank"]

350

300

250

200

150

100

5 10 15 hot100Songs[DeleteCases[0] /* Values/* FindDistribution, Count[1], "Rank"] LogSeriesDistribution[0.838522] hot100Songs[Values/* FindDistribution, Count[1], "Rank"] MixtureDistribution[{0.975349, 0.0246515}, {PoissonDistribution[0.00195204], LogSeriesDistribution[0.937824]}] 30 �� hot100Data.nb

hot100Songs[All, Count[1], "Rank"][Sort/* Reverse] △ One Sweet Day Mariah Carey & Boyz II Men

We Belong Together Mariah Carey

Uptown Funk! Mark Ronson Featuring Bruno Mars

Macarena(Bayside Boys Mix) Los Del Rio

I Will Always Love You(From "The Bodyguard") Whitney Houston

I ll Make Love To You Boyz II Men

I Gotta Feeling The Black Eyed Peas

Candle In The Wind 1997/Something About The Way You Look Tonight Elton John

End Of The Road(From "Boomerang") Boyz II Men

Yeah! Usher Featuring Lil Jon & Ludacris

The Boy Is Mine Brandy

Smooth Santana Featuring Rob Thomas

See You Again Wiz Khalifa Featuring Charlie Puth

Lose Yourself Eminem

Boom Boom Pow The Black Eyed Peas

Blurred Lines Robin Thicke Featuring T.I.+ Pharrell

Un-Break My Heart Toni Braxton

I Swear All-4-One

You Light Up My Life Debby Boone

We Found Love Rihanna Featuring Calvin Harris

▽ showing 1–20 of 27 459

Total absolute derivatives:

hot100Songs[Histogram, Differences/* Abs/* Total, "Score"]

6000

5000

4000

3000

2000

1000

0 0 50 100 150 200 250 300 hot100Data.nb ��31

Diversity of Hits

Number of distinct artists on the Hot 100 over time:

hot100[DateListPlot,{"Date", Query["Rankings", CountDistinct, "ArtistName"]}]

100

1960 1980 2000

billboard200[DateListPlot,{"Date", Query["Rankings", CountDistinct, "ArtistName"]}]

200

150

100

0 1980 2000 32 �� hot100Data.nb

hot100TimeSeries/*MovingAverage[#, 50]&/* DateListPlot, {"Date", Query["Rankings", CountDistinct, "ArtistName"]}

100

1960 1980 2000

billboard200TimeSeries/*MovingAverage[#, 50]&/* DateListPlot, {"Date", Query["Rankings", CountDistinct, "ArtistName"]}

190

180

170

160

150

140

130

1980 2000

Number of distinct artists each year on the Hot 100: hot100Data.nb ��33

hot100Years[DateListPlot, Flatten/* CountDistinct, "Rankings", All, "Name"]

800

600

400

200

0 1960 1980 2000 billboard200Years[DateListPlot, Flatten/* CountDistinct, "Rankings", All, "Name"]

1500

1000

500

0 1980 2000 34 �� hot100Data.nb

Artist Characteristics

hot100Artists[Values/* ListPlot,{Length, Query[Mean, "Score"]}]

100

20 40 60 80 100 hot100Data.nb ��35

hot100ArtistsValues/*ListPlot[#, PlotRange→ All]&, {Length, Query[Mean, "Score"]}

100

200 400 600 800 36 �� hot100Data.nb

hot100ArtistsValues/*SmoothDensityHistogram#, AspectRatio→1 GoldenRatio&, {Length, Query[Mean, "Score"]}

hot100ArtistsValues/*ListPlot[#, PlotRange→{{0, 50}, All}]&, {Query[CountDistinct, "Name"], Query[Mean, "Score"]}

100

10 20 30 40 50 hot100Data.nb ��37

hot100ArtistsValues/*ListPlot[#, PlotRange→ All]&, {Query[CountDistinct, "Name"], Query[Mean, "Score"]}

100

50 100 150

Artist Career Lengths

hot100Artists[All,{-1, 1} /* Apply[Subtract], "Date"][Histogram]

4000

3000

2000

1000

0 0 500 1000 1500 2000 2500 38 �� hot100Data.nb

hot100Artists[All,{-1, 1} /* Apply[Subtract], "Date"][Sort/* Reverse] △ Nat King Cole 20 Bobby Helms 20 Brenda Lee 20 Glen Campbell 19 Elvis Presley 16 Michael Jackson 16 David Bowie 15 Stevie Wonder 14 The Rolling Stones 14 Prince 13 The Isley Brothers 13 Aretha Franklin 13 Cher 13 Paul McCartney 13 The O'Jays 12 Queen & David Bowie 12 Van Halen 12 The Tokens 12 Bruce Springsteen 12 Fleetwood Mac 12

showing 1–20 of 9146 ▽ hot100Data.nb ��39

hot100Artists[All,{-1, 1} /* Apply[Subtract], "Date"][ QuantityMagnitude/* WordCloud] �� -� �� -��

Predicting Future Movement

Predict

pf= hot100Songs[ ;; 20 000]Catenate/*Predict[#, Method→ "LinearRegression"]&, Most[#]→ Last[#]&/@ Partition[#, 6, 1] &, "Rank"

 �� (�� )  PredictorFunction �� 40 �� hot100Data.nb

hot100Songs[20 001 ;;]Select[Length[#] > 15 &] /*567, Nest[Append[#, pf[#[[-5 ;;]]]]&, #[[ ;; 5]], 30]&, Identity/* ListPlot, "Rank"

200

150

100

5 10 15 20 25 30 35

hot100SongsSelect[Length[#] > 15 &] /*10, ListPlot, "Rank"

5 10 15 hot100Data.nb ��41

NetChain

net= NetChain[{ 10, Ramp, 15, Ramp, 1, Ramp}]

�� (�� ) � �� (�� ) � �� (�� ) NetChain � �� (�� )  � �� (�� ) � �� (�� ) �� (�� ) (��)

training= Normalhot100Songs[ ;; 20 000]Select[Length[#]≥2&] /* Values, PadLeft[Take[#, UpTo[10]], 10, 100]&, Last/* Apply[Rule], "Rank"; testing= Normalhot100Songs[20 001 ;;]Select[Length[#]≥2&] /* Values, PadLeft[Take[#, UpTo[10]], 10, 100]&, Last/* Apply[Rule], "Rank";

trained= NetTrain[net, training, ValidationSet→ testing]

�� (�� ) � �� (�� ) � �� (�� ) NetChain � �� (�� )  � �� (�� ) � �� (�� ) � �� (�� ) �� 42 �� hot100Data.nb

hot100Songs[20 001 ;;]Select[Length[#] > 15 &] /*567, Nest[Append[#, trained[#[[-10 ;;]]]]&, #[[ ;; 10]], 30]&, Identity/* ListPlot, "Rank"

100

10 20 30 40

hot100Songs[20 001 ;;]Select[Length[#] > 15 &] /*10, Nest[Append[#, trained[#[[-10 ;;]]]]&, #[[ ;; 10]], 30]&, Identity/* ListPlot, "Rank"

140

120

100

10 20 30 40

Web Applet

databin= Databin["fVrEIvZl"] hot100Data.nb ��43

BlockartistNames= Normal@Keyshot100Artists,

CloudDeploy

FormFunction "Artist"→<| "Interpreter"→"String", "Hint"→"The Beatles","Help"→"Arist name. Must be written exactly(i.e., |> , input  DatabinAdddatabin, input"Artist"; If MemberQartistNames, input"Artist", hot100

DateListPlot #, PlotRange→Select[#,#[[2]]>0&][[{1,-1},1]]+- 5yr , 5yr ,All, FrameLabel→"time","score(the total of the inverses of the ranks PlotLabel→input"Artist", ImageSize→700 &, "Date", Query"Rankings",Select#ArtistName===input"Artist"&/* , Text"Artist \""<>input"Artist"<>"\" not found." , "SVG", AppearanceRules→<| "Title"→"Music Arists Over Time", "Description"→"Type in the name of a music artist or group below and see ,

Permissions→"Public" 

CloudObject https://www.wolframcloud.com/objects/78519beb-5feb-470b-82ef-a7e20355f4cf

Ideas

How fast do artists appear? Individual songs? Do they ever make revivals (songs or artists)? Follow individual songs. Highest score when they appear for artists. 44 �� hot100Data.nb

Have the characteristic curves changed over time? Do artists have characteristic curves? Has the field become more diverse? More diﬀerent artists or has it consolidated? Total derivatives. Can you predict an artist’s future success? Average over each year. https://pudding.cool/2017/03/timeless/?utm_source=Scout+Newsletter&utm_campaign=16bf595358- EMAIL_CAMPAIGN_2017_04_25&utm_medium=email&utm_term=0_803a7dbf2b-16bf595358-43744112 9&mc_cid=16bf595358&mc_eid=ca6637a47c http://waxy.org/2008/05/the_whitburn_project/ http://www.modestinsights.com/analyzing-the-billboard-hot-100/