$ /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home//bin/java - classpath /Users/datafireball/Downloads/solr-5.4.1/dist/solr-core-5.4.1.jar - Dauto=yes -Dc=gettingstarted -Ddata=web -Drecursive=3 -Ddelay=0 org.apache.solr.util.SimplePostTool http://datafireball.com/ SimplePostTool version 5.0.0 Posting web pages to Solr url http://localhost:8983/solr/gettingstarted/update/extract Entering auto mode. Indexing pages with content-types corresponding to file endings xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html ,txt,log SimplePostTool: WARNING: Never crawl an external web site faster than every 10 seconds, your IP will probably be blocked Entering recursive mode, depth=3, delay=0s Entering crawl at level 0 (1 links total, 1 new) POSTed web resource http://datafireball.com (depth: 0) Entering crawl at level 1 (52 links total, 51 new) POSTed web resource http://datafireball.com/2015/06 (depth: 1) POSTed web resource http://datafireball.com/2015/07 (depth: 1) POSTed web resource http://datafireball.com/2016/01/13/api-jersey (depth: 1) POSTed web resource http://datafireball.com/2015/08 (depth: 1) POSTed web resource http://datafireball.com/2015/09 (depth: 1) POSTed web resource http://datafireball.com/2016/02/02/hdfs (depth: 1) POSTed web resource http://datafireball.com/2015/03 (depth: 1) POSTed web resource http://datafireball.com/2015/04 (depth: 1) POSTed web resource http://datafireball.com/2015/05 (depth: 1) POSTed web resource http://datafireball.com/page/2 (depth: 1) POSTed web resource http://datafireball.com/2016/02/06/learned-from-garcia (depth: 1) POSTed web resource http://datafireball.com/2015/01 (depth: 1) POSTed web resource http://datafireball.com/2014/11/21/mac-some-notes- installing-tsoutliers/comment-page-1 (depth: 1) POSTed web resource http://datafireball.com/2016/01/17/eclipse-understand-all- the-java-versions (depth: 1) POSTed web resource http://datafireball.com/2013/12 (depth: 1) SimplePostTool: WARNING: Skipping URL with unsupported type application/rss+xml SimplePostTool: WARNING: The URL http://datafireball.com/comments/feed returned a HTTP result status of 415 POSTed web resource http://datafireball.com/2013/11 (depth: 1) POSTed web resource http://datafireball.com/2016/01/13/api-java-frameworks (depth: 1) POSTed web resource http://datafireball.com/2016/01/12/api-java-servlets-and- java-server-faces (depth: 1) POSTed web resource http://datafireball.com/category/python (depth: 1) POSTed web resource http://datafireball.com/2016/01 (depth: 1) POSTed web resource http://datafireball.com/2016/02 (depth: 1) POSTed web resource http://datafireball.com/2013/10 (depth: 1) POSTed web resource http://datafireball.com/2016/02/20/solr-simple-post-tool (depth: 1) POSTed web resource http://datafireball.com/2014/12 (depth: 1) POSTed web resource http://datafireball.com/author/datafireball (depth: 1) POSTed web resource http://datafireball.com/category/java (depth: 1) POSTed web resource http://datafireball.com/2015/12/09/cloudera-manager-tsquery (depth: 1) SimplePostTool: WARNING: Skipping URL with unsupported type application/rss+xml SimplePostTool: WARNING: The URL http://datafireball.com/feed returned a HTTP result status of 415 POSTed web resource http://datafireball.com/category/hbase (depth: 1) POSTed web resource http://datafireball.com/2014/11/09/pay-more-attention-to- communication-and-people-i/comment-page-1 (depth: 1) POSTed web resource http://datafireball.com/2014/05/18/study-source-code- episode-1/comment-page-1 (depth: 1) POSTed web resource http://datafireball.com/2016/01/10/api-node-js-loopback (depth: 1) POSTed web resource http://datafireball.com/category/uncategorized (depth: 1) POSTed web resource http://datafireball.com/category/visualization (depth: 1) POSTed web resource http://datafireball.com/2014/11 (depth: 1) POSTed web resource http://datafireball.com/2014/10 (depth: 1) POSTed web resource http://datafireball.com/2014/04 (depth: 1) POSTed web resource http://datafireball.com/2014/03 (depth: 1) POSTed web resource http://datafireball.com/2014/08 (depth: 1) POSTed web resource http://datafireball.com/2014/07 (depth: 1) POSTed web resource http://datafireball.com/2014/06 (depth: 1) POSTed web resource http://datafireball.com/2014/05 (depth: 1) POSTed web resource http://datafireball.com/2014/09 (depth: 1) POSTed web resource http://datafireball.com/category/math (depth: 1) POSTed web resource http://datafireball.com/2016/01/18/eclipse-source-auto- generate (depth: 1) POSTed web resource http://datafireball.com/2014/07/26/nutch-plugin-how-nutch- makes-http-request/comment-page-1 (depth: 1) POSTed web resource http://datafireball.com/category/r (depth: 1) POSTed web resource http://datafireball.com/2015/10 (depth: 1) POSTed web resource http://datafireball.com/2015/11 (depth: 1) POSTed web resource http://datafireball.com/2015/12 (depth: 1) Entering crawl at level 2 (266 links total, 215 new) POSTed web resource http://datafireball.com/2015/06/30/scala-sbt (depth: 2) POSTed web resource http://datafireball.com/2015/06/10/practical-business-python (depth: 2) POSTed web resource http://datafireball.com/2015/06/18/hbase-a-few-things-about- the-hbase-shell (depth: 2) POSTed web resource http://datafireball.com/2015/06/19/rosen-brock (depth: 2) POSTed web resource http://datafireball.com/2015/07/10/how-to-backup-on-premise- hadoop-data-into-cloud (depth: 2) SimplePostTool: WARNING: The URL http://datafireball.com/2016/01/13/api- jersey/?share=google-plus-1 returned a HTTP result status of 302 SimplePostTool: WARNING: The URL http://datafireball.com/2016/01/13/api- jersey/?share=twitter returned a HTTP result status of 302 The URL http://datafireball.com/2016/01/13/api-jersey/?share=facebook caused a redirect to http://www.facebook.com/sharer.php?u=http%3A%2F%2Fdatafireball.com%2F2016%2F01%2 F13%2Fapi-jersey%2F&t=API%20-%20Jersey SimplePostTool: WARNING: The URL http://datafireball.com/2016/01/13/api- jersey/?share=facebook returned a HTTP result status of 302 POSTed web resource http://datafireball.com/2015/08/25/django-in-production- using-apache-server-and-mod_wsgi (depth: 2) POSTed web resource http://datafireball.com/2015/08/06/pci-compliant (depth: 2) POSTed web resource http://datafireball.com/2015/08/23/run-centos-on-virtualbox (depth: 2) POSTed web resource http://datafireball.com/2015/08/18/a-few-functions-about- python-path (depth: 2) POSTed web resource http://datafireball.com/2015/08/23/apache-server-on- virtualbox (depth: 2) POSTed web resource http://datafireball.com/2015/08/17/django-tutorials (depth: 2) POSTed web resource http://datafireball.com/2015/09/04/aws-sdk-for-java (depth: 2) POSTed web resource http://datafireball.com/2015/09/07/openssl-public-key- endecryption-and-signature-verification (depth: 2) POSTed web resource http://datafireball.com/2015/09/05/1370 (depth: 2) POSTed web resource http://datafireball.com/2015/09/09/tricks-from-the-chef (depth: 2) POSTed web resource http://datafireball.com/2015/09/06/study-notes-cryptography- i-statistical-tests (depth: 2) POSTed web resource http://datafireball.com/2015/09/05/java-cipher- getmaxallowedkeylength (depth: 2) POSTed web resource http://datafireball.com/2015/09/05/study-notes-cryptography- i-block-ciphers (depth: 2) POSTed web resource http://datafireball.com/2015/09/05/study-notes-cryptography- i-cryptography-history (depth: 2) SimplePostTool: WARNING: The URL http://datafireball.com/2016/02/02/hdfs/?share=twitter returned a HTTP result status of 302 SimplePostTool: WARNING: The URL http://datafireball.com/2016/02/02/hdfs/?share=google-plus-1 returned a HTTP result status of 302 The URL http://datafireball.com/2016/02/02/hdfs/?share=facebook caused a redirect to http://www.facebook.com/sharer.php?u=http%3A%2F%2Fdatafireball.com%2F2016%2F02%2 F02%2Fhdfs%2F&t=HDFS SimplePostTool: WARNING: The URL http://datafireball.com/2016/02/02/hdfs/?share=facebook returned a HTTP result status of 302 POSTed web resource http://datafireball.com/2015/03/22/why-requests-is-not- working-well-as-well-as-urllib2/lancomerequests (depth: 2) POSTed web resource http://datafireball.com/2015/03/22/why-requests-is-not- working-well-as-well-as-urllib2 (depth: 2) POSTed web resource http://datafireball.com/2015/03/23/bash-script-tips (depth: 2) POSTed web resource http://datafireball.com/2015/03/21/1215 (depth: 2) POSTed web resource http://datafireball.com/2015/03/18/1212 (depth: 2) POSTed web resource http://datafireball.com/2015/04/17/running-spark-locally-on- ios (depth: 2) POSTed web resource http://datafireball.com/2015/04/20/pyspark-read-in-avro-file (depth: 2) POSTed web resource http://datafireball.com/2015/04/18/freecad-best-open-source- model-development-environment (depth: 2) POSTed web resource http://datafireball.com/2015/04/page/2 (depth: 2) POSTed web resource http://datafireball.com/2015/04/13/nifi-drag-and-drop-tool- for-data-source-management (depth: 2) POSTed web resource http://datafireball.com/2015/04/14/annual-percentage-rate- finally-solve-the-equation-thanks-to-wikipedia (depth: 2) POSTed web resource http://datafireball.com/2015/04/14/apache-uima-unstructured- information-management (depth: 2) POSTed web resource http://datafireball.com/2015/04/13/apache-zeppelin-an- interactive-analytic-platform-for-spark-development (depth: 2) POSTed web resource http://datafireball.com/2015/04/14/intel-mkl-math-kernal- library (depth: 2) POSTed web resource http://datafireball.com/2015/04/19/linux-export (depth: 2) POSTed web resource http://datafireball.com/2015/04/22/apache-spark-scala