Is the 370 the worst bus in ?

"I go to a different grocery shop now, just so I don't have to catch the 370."

- Older lady I met at the bus stop 11 October, 2016

Questions:

» Is the 370 is the worst bus route in Sydney?

» Bus privitisation? Better or worse? Transport for NSW Open Data » Old timetables » Bus occupancy » Bus/train/light-rail/ferry patronage » Contract areas and details » usage and stats » Transport Forecasts » Population Forecasts » Fare compliance » Walking data » Aviation data » Cycling data TripView - Grofsoft https://www.grofsoft.com/ GTFS General Transit Feed Specification Published by Google Apache 2.0 License

Separate static (timetable) and realtime data https://developers.google.com/transit/gtfs/ https://developers.google.com/transit/gtfs-realtime/ GTFS [Static Timetable]

Map paths (shapes)

Stop names Bus Trip stop Agencies Trips and Routes times locations

Week days Dates this Exceptions Fare this trip pattern to this information runs applies

This is a zip of CSV files GTFS Realtime trip { stop_time_update { trip_id: "631043" stop_sequence: 47 start_time: "01:55:00" arrival { start_date: "20180111" delay: 125 schedule_relationship: SCHEDULED time: 1516714432 route_id: "2441_370" } } departure { vehicle { delay: 120 id: "42558_346913_3000_14_1" time: 1516714446 } } stop_id: "228776" schedule_relationship: SCHEDULED } stop_time_update { stop_sequence: 48 This is a giant protobuf ... }

Cloudwatch trigger (every minute)

TfNSW Open AWS Lambda Data Hub (Python) S3

Fetching and storing realtime data GTFS Static GTFS Realtime

1 year, 3 months ~1 year 29 Separate timetable feeds 1 realtime feed (combined NSW) 2818 Timetable updates 466,414 collections (every 1min) 11.4 GB 1.4TB

EC2 Linux Instance

S3 Realtime Agency Route Trip Trip date stop entry

Trip stop Stop time

My data model Map paths (shapes)

Route ID Trip ID Stop ID Stop names Bus Trip stop Agencies Trips and Routes times locations

Fare Dates this Exceptions information trip will run to this

GTFS Static Timetable (Zip of CSV files) GTFS Realtime Data trip { stop_time_update { trip_id:Trip ID "631043" stop_sequence: 47 start_time: "01:55:00" arrival { start_date: "20180111" delay: 125 schedule_relationship: SCHEDULED time: 1516714432 route_id:Route ID "2441_370" } } departure { vehicle { delay: 120 id: "42558_346913_3000_14_1" time: 1516714446 } } stop_id:Stop ID "228776" schedule_relationship: SCHEDULED } stop_time_update { stop_sequence: 48 ... } Map paths (shapes) 25:55:00 26:05:00 Route ID Trip ID ... Stop ID Stop names Bus Trip stop Agencies Trips and Routes times locations

Fare Dates this Exceptions information trip will run to this

2018-01-10 GTFS Realtime Data "" "631043_2" trip { stop_time_update { trip_id:Trip ID "631043" stop_sequence: 47 start_time: "01:55:00" arrival { start_date: "20180111" delay: 125 schedule_relationship: SCHEDULED time: 1516714432 route_id:Route ID "2441_370" } } ??? departure { vehicle { delay: 120 id: "42558_346913_3000_14_1" time: 1516714446 } } stop_id:Stop ID "228776" schedule_relationship: SCHEDULED } stop_time_update { stop_sequence: 48 ... } Route did not exist: _382 Route did not exist: _382 Route did not exist: _317 Route did not exist: 2436_993 Route did not exist: 2454_TROL Route did not exist: 2436_994 Route did not exist: 2433_RAIL Route did not exist: 2452_RAIL Route did not exist: 2433_4000 Route did not exist: 2433_RAIL Route did not exist: 2436_994 Route did not exist: 2433_RAIL Route did not exist: 2436_994 Route did not exist: 2454_JC Route did not exist: 2436_993 Route did not exist: 2436_993 Route did not exist: 2454_TROL Processing realtime data 1. Download 1 realtime dump

2. Parse realtime data protobuf

3. Match each of the ~7,000 trips with the timetable

4. Write ~20,000 realtime updates to the DB EC2 EC2

EC2 EC2

EC2

Postgres RDS

Processing the data Processing realtime data » 1 year of data in ~2 days of processing

» ~150MB/sec writes to the db

» 412,950,578 realtime entries

» 130GB db size Is the 370 the worst bus in Sydney? Early More than 2min early On time 2 min early - 5 min late (inclusive) Late More than 5min late Very late More than 20min late Results Recorded bus trips: 10,286,805

On time: 3,351,660 (32.58%)

More than 20min late: 273,880 (2.66%) Best Routes

# of trips % on time % >20min Route Route Name

1 630 97.46 0 829 to North Parramatta (Loop Service)

2 31382 97.21 0 Stkn Stockton Ferry

3 662 96.68 0 6 Woronora to Sydney Olympic Park

4 662 96.37 0 2 Glebe to Sydney Olympic Park via Gladesville

5 666 96.1 0 7 Cronulla to Sydney Olympic Park

6 4405 94.57 0.02 994 Unknown Name

7 24030 91.42 0.24 273 Fassifern to Toronto

8 656 89.79 0.46 172 Kurri Kurri Rotary Park to Deakin St

9 16604 87.42 0.1 954 Hurstville Grove to Hurstville

10 4767 86.7 0.73 453 Percival Street, Rockdale to Rockdale Station Worst Routes (by % on time)

# of trips % on time % >20min Route Route Name

1 2237 2.15 4.25 160 Cessnock to Newcastle 2 343 2.62 2.04 SJ15 Sierra Dr at Bong Bong Rd to St Joseph HS

3 329 3.04 3.65 S863 Rover Depot, Vincent St to Hunter Christian School

4 323 3.41 4.02 S003 Austinmer to Cedars Christian College

5 3512 3.62 1.91 594H Hornsby to City QVB

6 736 4.35 7.34 600 West Pennant Hills to Parramatta via Cherrybrook

7 6612 4.49 1.12 627 Castle Hill to Chatswood

8 3730 4.88 2.82 740 Macquarie Park to Plumpton via Stanhope Gardens

9 1338 5.01 2.62 801 Badgerys Creek to Liverpool

10 5073 5.14 2.37 628 Norwest to Chatswood

30 41425 9.51 20.09 370 Leichhardt Marketplace to Coogee Worst Routes (by % > 20min late)

# of trips % on time % >20min late Route Route Name

1 656 11.43 49.7 402 Singleton to Darlington & Hunterview (Loop Service)

2 1385 35.31 37.62 403 Singleton to Singleton Heights via Hunterview (Loop Service)

3 4391 26.03 24.78 277 Castle Cove to Chatswood

4 6634 18.15 23.59 281 Davidson to Chatswood

5 6813 35.07 22.84 560 Gordon to West Pymble (Loop Service)

6 337 16.91 22.55 832 New Lambton Heights to Merewether High

7 659 29.59 22 690C Springwood to Faulconbridge (Loop Service)

8 7394 15.62 21.96 283 Belrose to Chatswood

9 3285 22.74 21.46 571 to South Turramurra (Loop Service)

10 337 28.78 20.77 809 Belmont Depot to St Pauls HS

11 41425 9.51 20.09 370 Leichhardt Marketplace to Coogee Worst Agencies (by % on time)

# of trips % on time % >20min late Agency

1 902105 22.25 1.74

2 38020 22.66 3.21

3 194869 25.22 5.83

4 433830 25.78 1.21 Transit Systems

5 4465459 30.02 3.08 State Transit Sydney

6 983013 30.12 3.35 NSW

7 276447 35.77 3.13 Busabout

8 93913 36.24 3.56 Blue Mountains Transit

9 239153 36.57 3.1 Premier Illawarra

10 826889 37.08 2.15 Western Sydney Conclusions » Bus privitisation - could go either way

» The 370 is one of the worst bus routes in Sydney. (worst is probably 277 - Castle Cove to Chatswood)

Any questions?

Find me at » katiebell.net » @notsolonecoder » github.com/katharosada/bus-shaming https://transportnsw.info/travel-info/ways-to-get-around/bus/bus-operator-maps