hacker bits May 2016 new bits

Hello from Redmond!

Life as a tech professional ain’t easy… we know because y'all wrote to tell us! Among the top issues were keeping up with the onslaught of new technology and maintaining work-life balance. But have no fear, because we are here to help! :)

In this issue of Hacker Bits, we chat with John Sonmez on how tech professionals can become better at their jobs and lead rich, fulfilled lives.

Hint: The trick is learning to learn.

If you’ve gone on the job interview loop lately, chances are you’ve caught a glimpse of the industry’s fickle hiring practices. Dan Luu’s well-researched piece reveals how some companies are hiring irrationally by focusing only on particular candidates.

And as always, we have a line-up of all the essential tech articles you should be reading, including the lowdown on writing and organizing good code, and what you can expect from state of the art JavaScript in 2016.

So dig in and enjoy another meaty issue of Hacker Bits!

Happy learning!

— Maureen and Ray [email protected] content bits May 2016

Interview: How to become a better State of the art JavaScript in 2016 06 32 programmer

Four strategies for organizing Trackers 14 36 code

Why I switched to Android after 7 16 Kill your dependencies 43 years of iOS

“We only hire the best” means we On asking job candidates to code 18 only hire the trendiest 48

Writing good code: How to reduce I tried to virtually stalk Mark 26 the cognitive load of your code 52 Zuckerberg

My biggest regret as a 30 programmer

hacker bits 3 contributor bits

Francois Ward Jacques Mattheij Mike Perham Dan Luu Francois is a software Born in 1965, just in Mike is CEO of Contrib- Dan has worked on engineer and opinion- time for a front-row- uted Systems, purvey- deep learning, CPU ated JavaScript enthu- seat for the PC revo- ors of open source- verification/test/ucode/ siast. He specializes lution, playing with based software. Their RTL/etc, and other in front end tooling, technology since I was main product is Sidekiq performance critical infrastructure and is old enough to hold a Enterprise, the finest problems for Microsoft, obsessed with all things screwdriver, mechan- background job frame- Google, and Centaur. related to unit test- ical stuff, electrical, work money can buy. Check out his blog at ing. Currently, he's on electronics and finally It's artisanally-crafted in danluu.com. Hubspot on the front software. Inventor of Portland, OR. end as a service team in live-streaming-video Cambridge, MA. on the web, currently mostly active as interim CEO/CTO and doing technical due-diligence for top-flight VCs.

Christian Mackeprang Martin Sandin Henrik Joreteg Alex Kras Christian is an indepen- Martin is a software Henrik is a progressive Alex is a Software Engi- dent web developer and engineer who likes Web App developer, neer by day and Online worked for two decades programming lan- consultant, author, and Marketer by night. You in some of the largest guages, concurrency, educator. He believes can find his blog and websites in Argenti- distributed systems, the Web is the future of learn more about him na. Now he's focused and organizing stuff. mobile and IoT. at alexkras.com. on giving something He has been program- back to the community ming since he was a kid through his blog and and occasionally writes writing about software about it to make sure craftsmanship. that at least he under- stands his own idiosyn- crasies.

4 hacker bits Phil Calçado Andrew Wulf Ray Li Maureen Ker Phil is the Director of Andrew, “The Codist,” Curator Editor Software Engineering at is still delivering after DigitalOcean. Previous- 35 years as a program- Ray is a software en- Maureen is an editor, ly, he was the Director mer. He is currently gineer and data en- writer, enthusiastic of Core Engineering at working in Florida after thusiast who has been cook and prolific collec- SoundCloud, and his spending most of his blogging at rayli.net tor of useless kitchen team was responsible life in Texas. for over a decade. He gadgets. She is the for “keeping the trains loves to learn, teach author of 3 books and running” in our micro- and grow. You’ll usu- 100+ articles. Her work services architecture. ally find him wrangling has appeared in the Before that he was the data, programming and New York Daily News, Director of Product lifehacking. and various adult and Engineering at Sound- children’s publications. Cloud, and before joining SoundCloud he was a Lead Consultant for ThoughtWorks in Australia and the UK.

hacker bits 5 Programming

State of the art JavaScript in 2016 By FRANCOIS WARD

6 hacker bits o, you’re starting a brand new JavaScript front-end Sproject or overhauling an old one, and maybe you haven’t kept up with the breakneck pace of the ecosystem. Or you did, but there are too many things to choose from. React, Flux, Angular, Aurelia, Mocha, Jasmine, Babel, Type- Script, Flow, oh my! By trying to make things simpler, some fall into a trap captured by one of my favorite XKCD comics. Well, the good news is the Credit: www.xkcd.com ecosystem is starting to slow down. Projects are merging. Best practices are starting to become • The learning curve is very and Angular that promise to clear. People are building on top flat. The important APIs take care of everything, but the of existing stuff instead of build- would fit on one page. React ecosystem, while requiring ing new frameworks. a few more decisions (that’s why As a starting point, here’s • JSX is awesome. You get all you’re reading this!), is much my personal picks for most piec- the power of JavaScript and more robust. Many of these es of a modern web application. its tooling when writing your frameworks, such as Angular Some choices are likely contro- markup. 2.0, are playing catch-up with versial and I will only give basic • It is the natural match for React. reasoning behind each choices. Flux and Redux (more on Picking React isn’t a tech- Keep in mind they’re mostly that later).The React com- nology decision, it’s a business my opinion based on what I’m munity is amazing, and decision. seeing in the community and produced many best-of- Bonus points: once you start personal experiences. Your mile- breed tools such as Redux working on your mobile apps, age may vary. (also more on that later). you’ll be ready for it thanks to Writing high quality data React Native. flow is much easier in large Core library: React applications than dealing The clear winner right now is with 2-way data binding (eg: Application life cycle: React. Knockout). Redux • If you ever need to do serv- Now that we have our view and • Components all the way er-side rendering, React is component layer, we need some- down makes your applica- where it’s at. tion much easier to reason thing to manage state and the lifecycle of our application. Re- about. There’s plenty of monolithic dux is also a clear winner here. frameworks like Ember, Aurelia

The clear winner right now is React.

hacker bits 7 Alongside React, Facebook notch as Redux itself. From the features such as algebraic data presented a design pattern nearly magical devtool to the types (and you really want those for one-way data flow called amazing memoization utility if you’re going to do static Flux. Flux largely delivered reselect, the Redux community types!). It also doesn’t handle on its promise of simplifying got your back. nulls as well as Flow. state management, but it also One thing to be careful is EDIT: It was brought up that brought with it more questions, the natural instinct to try and TypeScript does have things like such as how to store that state abstract away the Redux boiler- union types which cover a lot and where to do Ajax requests. plate. There are good reasons of use cases. I only meant that To answer those questions, behind all those pieces. Make I think when it comes to type countless frameworks were sure you tried it and understand system in UX-land, you go all the built on top of the Flux pattern: the “why” before trying to blind- way or not at all. Fluxible, Reflux, Alt, Flummox, ly improve on it. Flow can be much more pow- Lux, Nuclear, Fluxxor and many, erful, catching a wider variety many more. of bugs, but it can be hard to One Flux-like implementa- Language: ES6 with set up. It’s also behind Babel in tion eventually caught the com- Babel. No types (yet) terms of language features and munity’s attention, and for good has poor Windows support. reasons: Redux. Avoid CoffeeScript. Most of its I’ll say something contro- In Redux, almost all of the better features are now in ES6, a versial: types are not nearly as moving parts are pure functions. standard. Tooling (such as Cof- critical to front-end development There is one centralized store feeLint) is very weak. Its commu- as some will have you believe

Types are not nearly as critical to front-end development as some will have you believe.

and source of truth. Reducer nity is also rapidly declining. (the whole argument will have functions are responsible for ES6 is a standard. Most of it to be in a future blog post). Wait manipulating data that makes is supported in the latest version until the type systems are more up the store. Everything is much of major browsers. Babel is an robust and stick to Babel for clearer than in vanilla Flux. amazing “pluggable” ES6 com- now, keeping an eye on Flow as More importantly, learning piler. Configure it with the right it matures. Redux is a snap. Redux’s au- presets for your target browsers thor, Dan Abramov is a fantastic and you’re good to go. teacher, and his training videos What about types? Type- Linting & style: ESLint are fantastic. Watch the videos Script and Flow both offer ways with Airbnb and, become a Redux expert. to add static typing to JavaS- I’ve seen a team of engineers go cript, enhance tooling and catch Another clear winner is ESLint. from nearly zero React expe- bugs without needing tests. With With its React plugin and awe- rience to having a production- that said, I suggest a wait and some ES6 support, one could ready application with top notch see approach for now. not ask for more of a linter. code in a few weeks. TypeScript tries too hard JSLint is dated. ESlint does what Redux’s ecosystem is as top to make JavaScript like C# or the JSHint + JSCS combo does in Java, and lacks modern system a single tool.

8 hacker bits Another clear winner is ESLint.

You do have to configure you finish coding and you start • It has features to fix broken it with your style preferences. deploying. Use a shrinkwrap file modules. I highly recommend Airbnb’s to freeze your dependencies (I • It can handle CSS. styleguide, most of which can recommend using Uber’s shrink- be enforced via the ESlint airbnb wrap to get more consistent out- • It has the most comprehen- config. If your team is the kind put). Also consider hosting your sive cache busting/hashing that will argue on code style, own private NPM server using system (if you push your just use this style guide as gos- something like Sinopia. stuff to CDNs). pel and the end-all and be-all of Babel will compile ES6 mod- • It supports hot reload out of arguments. It isn’t perfect, but ule syntax to CommonJS. You’ll the box. the value of having consistent get a future proof syntax, and • It can load almost anything. code is highly underestimated. the benefits of static code anal- Once you’re comfortable ysis, such as tree shaking when • It has an impressive list of with it, I’d suggest enabling using a build tool that supports optimizations. even more rules. The more that it (Webpack 2.0 or Rollup). can be caught while typing (with Webpack is also by far the an ESlint plugin for your favorite best for handling extremely editor), the less decision fatigue Build tool: Webpack large SPA applications with you’ll have and the more pro- Unless you fancy adding hun- built-in code splitting and lazy ductive you and your team will dreds of script tags to your loading. be. pages, you’ll need a build tool to Be warned that the learning bundle your dependencies. You curve is brutal! But once you get also need something to allow it, you’ll be rewarded with the Dependency NPM packages to work in brows- best build system available. management: It’s ers. This is where Webpack But what about Gulp or all about NPM, comes in. Grunt? Webpack is much better A year ago you had a lot of at processing assets. They can CommonJS and ES6 potential options here. Your still be useful if you need to modules environment, such as Rails’ run other kind of tasks though sprockets could do it. RequireJS, (usually you won’t). For basic This one is easy. Use NPM. Browserify and Webpack were tasks (such as running Webpack For everything. Forget about the JavaScript-based solutions. or ESlint), I recommend simply Bower. Build tools such as Now, RollupJS promises to han- using NPM scripts. Browserify and Webpack brings dle ES6 modules optimally. NPM’s power to the web. Ver- After trying them all, I highly sioning is handled easily and recommend Webpack: Testing: Mocha + Chai you get most of the Node.js eco- + Sinon (but it’s not system. Handling of CSS is still • It is more opinionated yet less than optimal though. can be configured to handle that simple) One thing you’ll want to even the craziest scenarios. There are a LOT of options for consider is how to handle build- unit testing in JavaScript, and ing on your deployment server. • All main module formats you can’t go wrong with any Unlike Ruby’s Bundler, NPM uses (AMD, CommonJS, globals) of them. If you have unit tests, wildcard versions, and packages are supported. that’s already good! can change between the time

hacker bits 9 Lodash is by far the king and contains the entire kitchen sink.

Some choices are Jasmine, tures and support. If you want lets you include only the func- Mocha, Tape and AVA and Jest. something more minimalist, tions you use (pretty important I’m sure I’m forgetting some. read this article about Tape. considering how large it has They all have something they do Edit 3/11/2016: Since I become). As of 4.x, Lodash also better than the rest. posted this, Facebook posted this natively supports an optional My criteria for a test frame- article on how they make Jest “functional” mode for the FP work are as follow: scale. Probably a bit too com- geeks among us. See how to use plex for most people, but if you it here. • It should work in the brows- have the resources and don’t If you’re into functional pro- er for ease of debugging. care about running things in the gramming, however, take a look • It should be fast. browser, you can probably make at the fantastic Ramda. If you • It should easily handle asyn- it awesome. decide to use it, you might still chronous tests. Additionally, a bunch of need to include some Lodash • It should be easy to use from people thought I dismissed AVA functions (Ramda is focused on the command line. too quickly. Don’t get me wrong, data manipulation and function- • It should let me use whatev- AVA is amazing. One of my al construct almost exclusively), er assertion and mock library criteria, however, is full browser but you’ll get a lot of the power I want. support, so people can run them of functional programming lan- straight from any browser (to guages in a JavaScript-friendly The first criteria knocks out test cross browser support) and way. AVA (even though it looks awe- debug them easily. If you don’t some) and Jest (auto-mocking care about that, you can use the isn’t nearly as nice as it sounds, fantastic iron-node for debug- Http requests: Just and is very slow anyway). ging. Coupled with the bundled use fetch! You can’t really go wrong power asserts, it would easily with Jasmine, Mocha or Tape. beat my Mocha setup. Many React applications don’t I prefer Chai asserts because need jQuery at all anymore. of all the available plugins and Unless you’re working on a Sinon’s mocks to Jasmine’s Utility library: Lodash legacy application or have 3rd built-in construct, and Mocha’s is king, but look at party libraries that depend on it, asynchronous test support is there’s no reason to include it. superior (you don’t have to deal Ramda That means you need to replace with done callbacks). JavaScript doesn’t have a strong $.ajax. Chai as Promised is amaz- core of utilities like Java or .NET I like to keep it simple and ing. I highly recommend using does, so you’ll most likely want just use fetch. It’s promise Dirty Chai to avoid some head- to include one. based, it’s built in Firefox and aches though. Webpack’s mo- Lodash is by far the king and Chrome, and it Just Works™. cha-loader lets you automatically contains the entire kitchen sink. For other browsers, you’ll need run tests as you code. It is also one of the most per- to include a polyfill. I suggest For React specific tooling, formant, with features such as isomorphic-fetch, to ensure you look at Airbnb’s Enzyme and lazy evaluation. You don’t have have all your bases covered, Teaspoon (this isn’t the Rails- to include the whole thing if you including server-side. based Teaspoon). don’t want to, either: Lodash There are other good librar- I really enjoyed Mocha’s fea- ies such as Axios, but I haven’t

10 hacker bits needed much beyond fetch. its own, or even in ADDITION Universal For more details about why to your preferred processor for promises are important, see my things such as AutoPrefixer in- (Isomorphic) post on asynchronous program- stead of importing a big library JavaScript: Make sure ming. like Bourbon. you need it. One thing worthy of atten- tion though, are CSS modules. Universal or Isomorphic JavaS- Styling: Consider CSS CSS modules prevent the “cas- cript refers to JavaScript that can modules cading” part of CSS, allowing be used on both the client and us to keep our dependencies the server. This is primarily used This is an area I feel is lagging explicit, and prevent conflict. to pre-render pages server-side behind. Sass is the current go You’ll never have to worry about for performance and SEO pur- to, and node-sass is a great way overriding classes by accident pose. Thanks to React, what was to use it in your JavaScript proj- or having to make ultra ex- once only the realm of giants ect. That said, I feel it’s missing plicit names for your classes. such as Ebay or Facebook is now a lot to be a perfect solution. It works great with React, too. within reach of most develop- Lack of reference imports (a way One drawback: css-loader with ment shops. It is still not “free” to import just variables and mix- CSS modules enabled is REALLY though, adding significant com- ins from a file, without dupli- slow, so if you plan on having plexity and limiting your options cating selectors) and native URL hundreds of kilobytes of CSS, in term of libraries and tooling. rewriting makes it harder than you may want to avoid it until it If you are building a B2C needed to keep things lean and gets better. (Business to Customer) website,

If I were to start a large project from scratch today, I’d probably just use PostCSS along with pre- compiled versions of my favorite CSS libraries.

clean in production. Node-sass If I were to start a large such as an e-commerce website, is a C library, and will have to project from scratch today, I’d you may not have a choice but be kept in sync with your Node probably just use PostCSS along to go that route. For internal version. with pre-compiled versions of web or B2B (Business to Busi- LESS does not suffer from my favorite CSS libraries. ness) applications however, that these issues, but has fallen out Regardless of what you kind of initial performance may of favor because it lacks many choose, you may want to look not be required. Discuss with of Sass’ features. at my post on CSS performance your product manager to see if PostCSS is much more with Webpack, especially if you the cost:benefit ratio is worth promising, allowing you to kind go with Sass. the trouble. of “make your own CSS proces- sor”. I’d recommend using it on

hacker bits 11 The API: There’s still I don’t think any solution out Desktop applications: there is a slam dunk, but here’s no true answer. what I think your API should Electron It seems everyone lately is have: Electron is the foundation of the asking themselves what to do great Atom editor and can be for API. Everyone is jumping in • It should be predictable. used to make your own applica- the RESTful API bandwagon, and Your endpoints should fol- tions. At its core, it is a version SOAP is a memory of the past. low consistent conventions. of Node that can open Chrome There are various specifications • It should allow fetching mul- windows to render a GUI, and such as HATEOAS, JSON API, tiple entities in one round has access to the operating HAL, GraphQL among others. trip: needing 15 queries to system’s native APIs without a GraphQL gives a large fetch everything you need browser’s typical security sand- amount of power (and respon- on page load will give poor boxing. sibility) to the client, allowing it performance. You’ll be able to package to make nearly arbitrary queries. your application and distribute Along with Relay, it handles • Make sure you have a good it like any other desktop applica- client state and caching for you. update story: many specifi- tion, complete with an installer Implementing the server-side cations only covers reads, and auto-updates. portion of GraphQL is difficult and you’ll need to update This is one of the easiest and most of the documentation stuff sometimes. ways to make an application is for Node though. • It should be easy to debug: that can run on OSX, Windows Netflix’s Falcor looks like it looking at the Chrome in- and Linux while reusing all the will eventually give us a lot of spector’s network tab should tools listed above. It is well-doc- what GraphQL/Relay offers, with easily let me see what hap- umented and has a very active simpler server requirements. pened. community. It is however only a developer You may have heard of nw.js • It should be easy to con- preview and not ready for prime (formerly node-webkit), which sume: I should be able to time. has existed longer (and does easily consume it with fetch, All the well-known specifi- almost the same thing), but or have a well-supported cations have their quirks. Some Electron is now more stable and client library (like Relay) are overly complex. Some only is easier to use. handle reads and don’t cover Take a look at this great update. Some stray significantly I haven’t found a solution that boilerplate to play around with from REST. Many people choose covers all of the above. If there Electron, React and hot reload. to make their own, but then is one, let me know. You’ll probably want to start have to solve all the design Consider looking at Swagger from scratch if you’re serious problems on their own. to document your API if you go about making your own applica- the standard RESTful path. tion so you’d understand how all the pieces work.

Everyone is jumping in the RESTful API bandwagon, and SOAP is a memory of the past.

12 hacker bits Who to follow and Getting started with Redux video state. Are you making a simple series. I can’t overstate how CRUD application? You don’t where to learn more? amazing it is at teaching Redux. need Relay. Are you learning This is a place where I’m falling The official Redux FAQ. ES6? You don’t need Async/ short, but on Twitter I follow the Dan also published his own Await or Decorators. Are you following people: list, and it’s probably better than just starting to learn React? mine. You don’t need Hot reload and

The most important thing to remember is to keep it simple and only use what you need.

• Dan Abramov, author of Mark Erikson’s collection server rendering. Are you start- Redux. of React/Redux links is an ever ing out with Webpack? You don’t • Christopher Chedeau aka growing gold mine. need code splitting and multiple Vjeux, a React developer at Read Removing user inter- chunks. Are you starting with Facebook very active in the face complexity, or why React is Redux? You don’t need Re- community. awesome to get a walkthrough dux-Form or Redux-Sagas. • Jeff Morrison, one of the of how React is designed and Keep it simple, one thing at main Flow contributors. why. a time, and you’ll wonder why • Sebastian Markbåge, another people ever complained about React developer at Facebook, JavaScript fatigue. involved with the TC39. If you don’t need it, • Pete Hunt. Originally from don’t use it Instagram and Facebook, Did I miss anything? The JavaScript ecosystem is famous for his Rethinking And there you have it, my view best practices talk, partly thriving and moving quickly, but there’s finally an end at the light of the current state of JavaScript. responsible for putting React Do you think I forgot an im- on the map. of the tunnel. Best practices are no longer changing constantly, portant category? Do you think • React, because duh. I’m objectively wrong on one of • The above are more are and it is becoming increasing- ly clear which tools are worth these choices? Do you recom- aggregated in React Influenc- mend something else? Let me ers. learning. The most important thing to know!  While there’s many more worth remember is to keep it simple noting, those people retweet and only use what you need. almost anything worth looking Is your application only 2–3 at, so they’re a good start. screens? Then you don’t need a Consider reading Pete Hunt’s router. Are you making a single Learning React. Follow the order! page? Then you don’t even need Dan Abramov published the Redux, just use React’s own

Reprinted with permission of the original author. First appeared at medium.com/javascript-and-opinions.

hacker bits 13 Interesting

Trackers By JACQUES MATTHEIJ

couple of weeks ago I Instead of walking in with me next morning my newspaper went to the local shop- through the front door, they had been cut up, with all kinds Aping centre looking for a forced open the back door and of blank windows between the thermometer. After entering one installed themselves at my table. articles. They assured me that store and upon leaving without One of them had found my all of this was entirely legal, and buying anything, a tracker was mobile phone and was going that may be so, but it left me assigned to me. I didn’t think through my list of contacts, with a weird feeling that these much of it at first, but he fol- adding the names and the tele- anonymous entities would know lowed me dutifully around the phone numbers of the people more about me than my own shopping centre, and took care- that I knew to a thing they called mother does. ful note of how I walked. a ‘social graph.’ Upset about this, I decid- Whenever I visited a store, It mattered a lot to them, ap- ed to read my newspaper in he made a note in his little black parently, and even though I took the park, where I expected at book (he kept calling it my pro- the phone away from him and least a little bit more privacy. file, and he didn’t want to show made him wipe the copy that I probably should have known me what was in it so I assume he’d made, I realized that if they better. The trackers of course it was actually his, rather than did the same to my friends (who followed me to the park and set mine). likely would not take as forceful up an impromptu auction of the Each of those stores of a stance against the trackers as space created by the holes in my course assigned trackers to me I did), they’d already have most paper. as well and soon enough I was of that information anyway and Every time I would open a followed by my own personal there was nothing I could do page the space on that page veritable posse of nondescript about it. would come up for auction, and guys with little black books mak- Every visitor to my house the tracker that won the auction ing notes. was asked a whole pile of per- would quickly glue an advertise- After doing my shopping I sonal questions, and if they ment that he’d brought with him went home. To my surprise, they didn’t answer, their photographs over the gap in the page. Mys- expected to come into the house would be used to complete the teriously quite a few of the ads with me and stay there, which I gaps in their profiles and mine. were for thermometers, even objected. During the night they’d been though I no longer had a need That didn’t stop them. up to something because the for one (having found one in

14 hacker bits the drawer in the bathroom that particular restaurant chain, an alone. Some still tried but for same evening). ad popped up overlaying my the most part it seems I was at The guy the newspaper mobile screen indicating that least a little bit safer from prying hired to do the auctions thought the restaurant had my favorite eyes. this was a-ok, and encouraged dish on sale that day. (How did Even more powerful methods the trackers to bid even higher they figure out my favorite dish of getting rid of trackers exist based on all the stuff that he’d anyway? Was it because I had and there are many variations told them about me. searched for the recipe a few to choose from, with different The information they used weeks ago?). functionality. For some victims included all the stuff in ‘my’ I resisted the temptation however, even bouncers were profile (some of which came as and decided to go anywhere but not enough to regain their much a surprise to me, for instance, there. (Even if that meant the deserved privacy. he knew roughly what I earned, trackers could still influence me. Some of the bouncers were knew the fact that I had kids and At least they’d have to add an trying to be a little bit too clever a whole raft of other details that inversion to all their little plans). for my taste: they actually took I did not consider to be any of Finally after a couple of the place of the trackers they his business, and even today I weeks of this I decided I’d had blocked. Serves me right for have no idea how he got that in- enough. The final straw was hiring tough guys, I thought, formation) as well as our current when a tracker popped up in so I’ve been more careful since location on that park bench. my bed, between me and my then and now check out care- Some of the more enter- wife when I was looking for the fully which party gets to place prising trackers took down the shortest route to my meeting themselves between me and the stuff the newspaper guy told the next morning. Really, that trackers. them about me in their own did it. Even with the tough guys little black books and soon were The advertising industry did in tow there were still ways in auctioning that information off not have the right to become which I could be tracked. But at to other trackers and to any- involved in my private life to this least that seemed to, for now, body that was willing to part degree, to track my every move, be beyond the pale for most re- with some money. It became and to keep detailed information spectable businesses employing quite a crowd (a mini WallStreet, of my daily whereabouts and in- trackers. actually) around my chosen teractions in the real world. And Some venues did not allow park bench and the atmosphere the last place I expected them to me to enter with my bouncers turned bad, to the point where track me to was in the privacy of in tow, so I stopped visiting. It’s I did not feel as if the park had my bedroom. not that I minded their adver- anything to offer any longer. I hired a couple of ‘tough tising to me, but that I mind- I figured maybe I’d go for guys’, bouncers, really. Their ed being followed, and if they some lunch at a nearby restau- assignments were simple: keep wouldn’t change their ways then rant. The trackers had long since these advertising types out of I would have to change mine. figured out my mobile phone my life, hurt them if they have All of this is really too bad, number. They combined a little to. I’ve really had enough of it. since some of the stores de- embedded code in a game that Another side benefit of this pended to a certain extent on I’d once installed (and played was that the burglars and other my patronage. But I guess there twice, although it wasn’t a very shady types that used the infor- are not enough people that good game). It kept a continu- mation gathered by the track- are concerned about privacy ous read-out of my phone and ers to target me for an entirely to make a difference. Or are all its sensors, and phoned different kind of operation were there?  home to the mother ship when- also shut out. ever it could. After the first couple of I probably shouldn’t have run-ins between the bouncers been surprised when just as I and trackers, the trackers got was passing by the door of one the message and mostly left me

Reprinted with permission of the original author. First appeared at jacquesmattheij.com.

hacker bits 15 Programming

Kill your dependencies

By MIKE PERHAM

his post talks about Ruby for those gems to pull in other cation has the potential to bloat but it’s true of every lan- gems transitively, leading to your app, to destabilize your Tguage community: Python, Rails apps which “download the app, to inject odd behavior via JavaScript, Java, etc. The scourge Internet” and have hundreds of monkeypatching or buggy native of dependencies spares no one. dependencies. code. When you are considering This is a dependency visu- When you publish a Ruby- adding a dependency to your alization of every Rails app I’ve gem, every one of your depen- Rails app, it’s a good idea to do ever used. See figure below for dencies transitively becomes a a quick sanity check, in order of a dependency visualization of dependency for any app using preference: every Rails app. your gem. This multiplies the impact of bugs in those gems. 1. Do I really need this at all? Kill it. 2. Can I implement the re- The curious case of quired minimal functional- mime-types ity myself? Own it.

The mime-types gem recently If you need a gem: optimized its memory usage and 1. Does the gem have a saved megabytes of RAM. Liter- native extension? Look for ally every Rails app in existence pure ruby alternatives. can benefit from this optimiza- 2. Does the gem transitively tion because Rails depends on pull in a lot of other gems? the mime-types gem transitively:  Rails app dependency Look for simpler alterna- visualization rails -> actionmailer -> mail -> tives. mime-types. In other words, this gem Gems with native extensions wasn’t used by your app. It can destabilize your system; wasn’t used by Rails directly. they can be the source of It wasn’t used by ActionMailer Does any of this sound familiar? mysterious bugs and crashes. directly. It was used deep in Avoid gems which pull in more the bowels of the ActionMailer • Gemfile with 100s of entries. dependencies than their value implementation and it was using • Test gems loading in pro- warrants. Example of a bad gem: far too much memory. Every duction. the gem which pulls in 39 single Rails app in existence was fog • Each Rails process takes gems, more dependencies than using 10MB too much due to 100s of megabytes of RAM. rails itself and most of which are this issue. unnecessary. The Rubygems system is Lastly, make sure you only commendable for how easy it App developers, load the gem when necessary. makes packaging up Ruby for Use Bundler’s group support others to reuse. But that very listen up! to disable test gems when not ease means it’s also quite easy Every dependency in your appli- testing:

16 hacker bits group :test do remove it all. JSON has been thread_safe, and json. gem 'rspec' in the stdlib since 1.9 and you Unfortunately there’s no gem 'timecop' don’t need to declare any depen- more low-hanging fruit. I’d love # etc dencies at all. Just require 'json' to drop Nokogiri, it’s such a end and let Ruby deal with it. huge dependency with a massive Rails did it, so can you! native extension component, but there are some non-trivial dependencies on it. Oga is a Gem developers, Why choose an HTTP nice, simpler alternative. If you listen up! client when you can ship a gem which depends on Nokogiri, consider making it op- Part of your job as a library have them all? tional and defaulting to REXML (I author is to treat your user and know, but at least it’s in stdlib) their application with respect. Every Rails app pulls in a half or Oga instead. You should make an effort to dozen different HTTP clients: minimize your own dependen- faraday, rest-client, httparty, ex- con, typhoeus, curb, etc. This is cies so they don’t load unneces- Be part of the sary code or cause issues in the because various gems use them user’s application. internally. solution You control your own code A Rubygem should never use anything but Net::HTTP internal- I can help with Rails 5.0 but I but you don’t control your de- can’t fix every gem. If you are a pendencies. Any bug in a depen- ly! Learn the Net:HTTP API, kill those dependencies and stop gem developer, audit your own dency of yours becomes a bug dependencies and remove as that causes stress for your user forcing extra HTTP client gems on your users. many as you can. If you’re an and their application. app developer, take a look in As a gem developer, for each Let’s say you want to offer an optimized version using curb: your Gemfile and see if you can of your gem dependencies do find a gem or two to remove. you: ok, but make it optional. Allow the application developer to opt Simplify, simplify, simplify. As an example, I think it’s • Know how much memory into using curb but net/http should always be the default. possible for the Stripe gem to each takes? remove both of its runtime de- • Know how long each takes to pendencies. require? • Know whether it performs Optimizing Rails 5.0 any monkeypatching outside For the last few weeks, I’ve been Rules to remember of its own module? working (in tandem with several other developers, hi @_mat- Some software engineering rules: Sidekiq, with all of its func- thewd, @applerebel!) on min- tionality, has only 3 runtime imizing gem dependencies in • No code runs faster than no dependencies: , Rails 5.0. concurrent-ruby code. connection_pool and redis. • Rails 4.2.5 required 34 • No code has fewer bugs than gems. no code. Die json, die (German • Rails 5.0b1 required 55 • No code uses less memory than no code. for “The json, the”) gems. • Rails 5.0b2 required 39 • No code is easier to under- So many gems declare a depen- gems. stand than no code. dency on json, oj, multi_json, or yajl-ruby. There are so many I expect Rails 5.0 to require Kill those dependencies. Your ossified layers of cruft around 37 gems or less. So far we’ve re- gems and apps will be better for JSON processing that only one moved Celluloid, EventMachine, it.  course of action makes sense:

Reprinted with permission of the original author and mikeperham.com.

hacker bits 17 Opinion

“We only hire the best” means we only hire the trendiest

By DAN LUU

18 hacker bits n acquaintance of mine, so we have a name, let’s call this their type. TrendCo’s median let’s call him Mike, is look- company TrendCo. It’s one of employee is a recent graduate Aing for work after getting the thousands of companies that from one of maybe ten “top” laid off from a contract role at claims to have world class en- schools with 0-2 years of expe- Microsoft, which has happened gineers, hire only the best, etc. rience. They have a few experi- to a lot of people I know. Like This is one company in particu- enced hires, but not many, and me, Mike has 11 years in the lar, but it’s representative of a most of their experienced hires industry. Unlike me, he doesn’t large class of companies and the have something trendy on their know a lot of folks at trendy responses Mike has gotten. resume, not a boring old compa- companies, so I passed his ré- Anyway, (1) is code for ny like Microsoft. sumé around to some engineers “Mike’s a .NET dev, and we don’t Whether or not you think I know at companies that are like people with Windows expe- there’s anything wrong with hav-

My engineering friends thought Mike’s resume was fine, but most recruiters rejected him in the resume screening phase.

desperately hiring. My engineer- rience.” ing a type and rejecting people ing friends thought Mike’s re- I’m familiar with TrendCo’s who aren’t your type, as Thom- sume was fine, but most recruit- tech stack, which multiple em- as Ptacek has observed, if your ers rejected him in the resume ployees have told me is “a tire type is the same type everyone screening phase. fire.” Their core systems top out else is competing for, “you are When I asked why he was under 1k QPS, which has caused competing for talent with the getting rejected, the typical re- them to go down under load. wealthiest (or most overfunded) sponse I got was: Mike has worked on systems tech companies in the market.” that can handle multiple orders If you look at new grad 1. Tech experience is in irrel- of magnitude more load, but hiring data, it looks like FB is evant tech. his experience is, apparently, offering people with zero expe- 2. “Experience is too random, irrelevant. rience greater than $100k/ sal- with payments, mobile, (2) is hard to make sense ary, $100k signing bonus, and data analytics, and UX.” of. I’ve interviewed at TrendCo $150k in RSUs, for an amortized 3. Contractors are generally and one of the selling points is total comp greater than $160k/ not the strongest techni- that it’s a startup where you get yr, including $240k in the first cally. to do a lot of different things. year. TrendCo almost exclusively Google’s package has great- This response is something hires generalists but Mike is, ap- er than $100k salary, a vari- from a recruiter that was relayed parently, too general for them. able signing bonus in the $10k to me through an engineer; the (3), combined with (1), gets range, and $187k in RSUs. That engineer was incredulous at the at what TrendCo’s real com- comes in a bit lower than FB, response from the recruiter. Just plaint with Mike is. He’s not but it’s much higher than most

hacker bits 19 It’s much easier to hire people who are underrated, especially if you’re not paying market rates.

companies that claim to only trendy backgrounds. I know a lot people who are underrated, hire the best are willing to pay of these people who have impec- especially if you’re not paying for a new grad. Keep in mind cable interviewing skills and got market rates. that compensation can go much 5-10 strong offers the last time I’ve seen this hyperfocus on higher for contested candidates, they looked for work. hiring people with trendy back- and that compensation for expe- I’ve worked with someone grounds from both sides of the rienced candidates is probably like that: he was just out of table, and it’s ridiculous from higher than you expect if you’re school, his total comp package both sides. not a hiring manager who’s seen was north of $200k/year, and he On the referring side of hir- what competitive offers look like was worth every penny. ing, I tried to get a startup I was today. But think about that for a at to hire the most interesting By going after people with minute. He had strong offers and creative programmer I’ve the most sought after qualifi- from six different companies, of ever met, who was tragically un- cations, TrendCo has narrowed which he was going to accept at deremployed for years because their options down to either most one. Including lunch and of his low GPA in college. paying out the nose for employ- phone screens, the companies We declined to hire him ees, or offering non-competitive put in an average of eight hours and I was told that his low GPA compensation packages. apiece interviewing him. meant that he couldn’t be very TrendCo has chosen the And because they wanted smart. Years later, Google took latter option, which partially to hire him so much, the com- a chance on him and he’s been explains why they have, propor- panies that were really serious killing it since then. He actually tionally, so few senior devs – the spent an average of another five convinced me to join Google, compensation delta increases hours apiece of engineer time and at Google, I tried to hire one as you get more senior, and you trying to convince him to take of the most productive program- have to make a really compelling their offer. mers I know, who was promptly pitch to someone to get them Because these companies rejected by a recruiter for not to choose TrendCo when you’re had, on average, a one-sixth being technical enough. offering $150k/yr less than the chance of hiring this person, On the candidate side of hir- competition. And as people get they have to spend at least an ing, I’ve experienced both being more experience, they’re less expected (8+5) * 6 = 78 hours of in demand and being almost likely to believe the part of the engineer time1. un-hireable. Because I did my pitch that explains how much People with great back- undergrad at Wisconsin, which the stock options are worth. grounds are, on average, pretty is one of the 25 schools that Just to be clear, I don’t have great, but they’re really hard claims to be a top 10 CS/engi- anything against people with to hire. It’s much easier to hire neering school, I had recruiters

20 hacker bits beating down my door when I strong internal referrals and the I’ve been considering looking for graduated. recruiters still didn’t want to talk work2 and I wonder how trendy But that’s silly – that I at- to me, which I found funny and I’ll be if I do. Experience in tended Wisconsin wasn’t any- my friends found frustrating. irrelevant tech? Check! Random thing about me; I just happened When I could get interviews, experience? Check! Contractor? to grow up in the state of Wis- they often went poorly. A typical Well, no. But two out of three consin. If I grew up in Utah, I rejection reason was something ain’t bad. probably would have ended up like “we process millions of My point here isn’t anything going to school at Utah. When transactions per day here and about me. It’s that here’s this I’ve compared notes with folks we really need someone with person3 who has wildly different who attended schools like Utah more relevant experience who levels of attractiveness to em- and Boise State, their education can handle these things without ployers at various times, mostly is basically the same as mine. ramping up.” due to superficial factors that Wisconsin’s rank as an And then Google took a don’t have much to do with ac- engineering school comes from chance on me and I was the tual productivity. having professors who do great second person on a project to This is a really common research which is, at best, weak- get serious about deep learning story among people who end up ly correlated to effectiveness at performance, which was a 20%- at Google. If you hired them be- actually teaching undergrads. time project until just before I fore they worked at Google, you Despite getting the same engi- joined. might have gotten a great deal! neering education you could get We built the fastest deep But no one (except Google) was at hundreds of other schools, I learning system in the world. willing to take that chance. had a very easy time getting in- From what I hear, they’re now There’s something to be said terviews and finding a great job. on the Nth generation of that for paying more to get a known I spent 7.5 years in that project, but even the first gen- quantity, but a company like great job, at Centaur. Centaur eration thing we built has better TrendCo that isn’t willing to do has a pretty strong reputation per-node performance and that cripples its hiring pipeline among hardware companies performance per dollar than any by only going after people with in Austin who’ve been around other production system I know trendy resumes. for a while, and I had an easy of today, years later (excluding I don’t mean to pick on start- time shopping for local jobs follow-ons to that project, of ups like TrendCo in particular. at hardware companies. But I course). Boring old companies have their don’t know of any software folks While I was at Google I had version of what a trendy back- who’ve heard of Centaur, and as recruiters pinging me about job ground is, too. a result I couldn’t get an inter- opportunities all the time. And A friend of mine who’s des- view at most software compa- now that I’m at boring old Micro- perate to hire can’t do anything nies. There were even a couple soft, I don’t get nearly as many with some of the resumes I pass of cases where I had really recruiters reaching out to me. his way because his group isn’t

And now that I’m at boring old Microsoft, I don’t get nearly as many recruiters reaching out to me.

hacker bits 21 allowed to hire anyone without a If you read Zach Holman, Moneyball degree. Another person I know you may recall that when he is in a similar situation because said that he was fired, someone Billy Beane and Paul Depodesta his group won’t talk to people responded with “If an employer took the Oakland A’s, a baseball who aren’t already employed. has decided to fire you, then franchise with nowhere near the Not only are these decisions you’ve not only failed at your budget of top teams, and creat- non-optimal for companies, job, you’ve failed as a human ed what was arguably the best they create a path dependence being.” A lot of people treat team in baseball by finding and in employment outcomes that employment status and creden- “hiring” players who were statis- causes individual good (or bad) tials as measures of the inherent tically underrated for their price. events to follow people around worth of individuals. But a large The thing I find really for decades. You can see similar component of these markers of amazing about this is that they effects in the literature on career success, not to mention success publically talked about doing earnings in a variety of fields.4 itself, is luck. this, and then Michael Lewis Thomas Ptacek has this wrote a book, titled Moneyball, great line about how “we inter- about them doing this. Despite view people whose only prior Solutions? the publicity, it took years for work experience is “Line of enough competitors to catch on I can understand why this that the A’s strategy stopped Business .NET Developer,” and happens. At an individual level, they end up showing us how to giving them a very large edge. we’re prone to the fundamental You can see the exact same write exploits for elliptic curve attribution error. At an organiza- partial nonce bias attacks that thing in software hiring. Thomas tional level, fast growing orga- Ptacek has been talking about involve Fourier transforms and nizations burn a large fraction BKZ lattice reduction steps that how they hired unusually effec- of their time on interviews, and tive people at Matasano for at take 6 hours to run.” the obvious way to cut down If you work at a company least half a decade, maybe more. on time spent interviewing is Google bigwigs regularly that doesn’t reject people out to only interview people with of hand for not being trendy, talk about the hiring data they “good” qualifications. Unfortu- have and what hasn’t worked. you’ll hear lots of stories like nately, that’s counterproductive this. Some of the best peo- I believe they talked about when you’re chasing after the how focusing on top schools ple I’ve worked with went to same tiny pool of people as ev- schools you’ve never heard of wasn’t effective and didn’t turn eryone else. up employees that have better and worked at companies you’ve Here are the beginnings of never heard of until they ended performance years ago, but some ideas. I’m open to better that doesn’t stop TrendCo from up at Google. Some are still at suggestions! companies you’ve never heard focusing hiring efforts on top of. schools.

A large component of these markers of success, not to mention success itself, is luck.

22 hacker bits Training/mentorship of multiple-choice questions that make a better learning environ- are offered up for compliance ment than Google, they never You see a lot of talk about reasons, not to teach anyone have a good answer. Moneyball, but for some reason anything. And you’ll be assigned people are less excited about… a mentor who, more likely than trainingball? Practiceball? What- not, won’t provide any actual Process/tools/culture ever you want to call taking mentorship. Startups tend to be I’ve worked at two companies people who aren’t “the best” and even worse! It’s not hard to do teaching them how to be “the that both effectively have infinite better than that. resources to spend on tooling. best”. Considering how much mon- This is another one where One of them, let’s call them ey companies spend on hiring ToolCo, is really serious about it’s easy to see the impact and retaining “the best,” you’d through the lens of sports, tooling and invests heavily in expect them to spend at least tools. People describe tooling because there is so much good a (non-zero) fraction on train- performance data. Since it’s there with phrases like “magi- ing. It’s also quite strange that cal,” “the best I’ve ever seen,” basketball season, if we look at companies don’t focus more on college basketball, for example, and “I can’t believe this is even

Considering how much money companies spend on hiring and retaining “the best,” you’d expect them to spend at least a (non- zero) fraction on training.

training and mentorship when possible.” And I can see why. we can identify a handful of trying to recruit. programs that regularly take un- For example, if you want to Specific things I’ve learned build a project that’s millions remarkable inputs and produce in specific roles have been tre- good outputs. And that’s against of lines of code, their build mendously valuable to me, but system will make that take a field of competitors where it’s almost always either been a every team is expected to coach somewhere between 5s and 20s happy accident, or something I (assuming you don’t enable LTO and train their players. went out of my way to do. Most When it comes to tech or anything else that can’t be companies don’t focus on this parallelized)5. In the course of companies, most of the compe- stuff. tition isn’t even trying. At the a regular day at work you’ll use Sure, recruiters will tell you multiple tools that seem magi- median large company, you get that “you’ll learn so much more a couple days of “orientation”, cal, because they’re so far ahead here than at Google, which will of what’s available in the outside which is mostly legal mumbo make you more valuable,” imply- jumbo and paperwork, and the world. ing that it’s worth the $150k/ The other company, let’s call occasional “training,” which is yr pay cut, but if you ask them usually a set of videos and a set them ProdCo pays lip service to what, specifically, they do to tooling, but doesn’t really value

hacker bits 23 You can get better results by hiring undistinguished folks and setting them up for success, and it’s a lot cheaper.

it. People describing ProdCo deficiencies that are technically Conclusion tools use phrases like “world trivial (but sometimes culturally class bad software” and “I am difficult) to fix. When I started programming, I 2x less productive than I’ve ever When I was at Google, we heard a lot about how program- been anywhere else,” and “I can’t had really bad communication mers are down to earth, not believe this is even possible.” problems between the two like those elitist folks who have ProdCo has a paper on a halves of our team that were in uniforms involving suits and new build system; their claimed different locations. My fix was ties. You can even wear t-shirts numbers for speedup from par- brain-dead simple: I started to work! allelization/caching, onboarding typing up meeting notes for all But if you think program- time, and reliability, are at least of our local meetings and dis- mers aren’t elitist, try wearing two orders of magnitude worse cussions, and taking questions a suit and tie to an interview than the equivalent at ToolCo. from the remote team about sometime. You’ll have to go And, in my experience, the actu- things that surprised them in above and beyond to prove that al numbers are worse than the our notes. you’re not a bad cultural fit. claims in the paper. That’s something anyone We like to think that we’re In the course of a day of could have done, and it was a different from all those indus- work at ProdCo, you’ll use huge productivity improvement tries that judge people based on multiple tools that are multiple for the entire team. I’ve literally appearance, but we do the same orders of magnitude worse than never found an environment thing, only instead of saying the equivalent at ToolCo in mul- where you can’t massively im- that people are a bad fit because tiple dimensions. These kinds prove productivity with some- they don’t wear ties, we say of things add up and can easily thing that trivial. Sometimes they’re a bad fit because they make a larger difference than people don’t agree (e.g., it took do, and instead of saying people “hiring only the best.” months to get the non-version- aren’t smart enough because Processes and culture also control-using-team to use ver- they don’t have the right ped- matter. I once worked on a team sion control), but that’s a topic igree… wait, that’s exactly the that didn’t use version control for another post. same.  or have a bug tracker. For every Programmers are woefully no-brainer item on the Joel test, underutilized at most compa- there are teams out there that nies. What’s the point of hiring make the wrong choice. “the best” and then crippling Although I’ve only worked them? You can get better results on one team that completely by hiring undistinguished folks failed the Joel test, every team and setting them up for success, I’ve worked on has had glaring and it’s a lot cheaper.

24 hacker bits that wrote their DNS, and is now one of to link the productivity of a city today to Footnotes the world’s experts on DDoS mitigation whether or not that city had a bishop in 1 This estimate is conservative. The for companies that don’t have infinite the year 1000. math only works out to 78 hours if you resources. That guy wasn’t even a assume that you never incorrectly reject networking person before he joined 5 During orientation, the back end of a trendy candidate and that you don’t Cloudflare. He’s a brilliant generalist the build system was down so I tried have to interview candidates that you who’s created everything from a widely building one of the starter tutorials on “correctly” fail to find good candidates. used javascript library to one of the my local machine. I gave up after an If you add in the extra time for those, coolest toy systems projects I’ve ever hour when the build was 2% complete. I the number becomes a lot larger. And seen. He probably could have picked know someone who tried to build a real, if you’re TrendCo, and you won’t give up whatever problem domain you’re large scale, production codebase on senior ICs $200k/yr, let alone new struggling with and knocked it out of the their local machine over a long weekend, grads, you probably need to multiply park. Oh, and between the blog posts he and it was nowhere near done when they that number by at least a factor of 10 to writes and the talks he gives, he’s one of got back. account for the reduced probability that Cloudflare’s most effective recruiters. someone who’s in high demand is going to take a huge pay cut to work for you. 4 I’m not going to do a literature review By the way, if you do some similar because there are just so many studies math you can see that the “no false that link career earnings to external positives” thing people talk about is shocks, but I’ll cite a result that I found bogus. The only way to reduce the to be interesting: Lisa Kahn’s 2010 risk of a false positive to zero is to not Labour Economics paper. hire anyone. If you hire anyone, you’re There have been a lot of studies that trading off the cost of firing a bad hire show, for some particular negative vs. the cost of spending engineering shock (like a recession), graduating hours interviewing. into the negative shock reduces lifetime earnings. But most of those studies 2 I consider this to generally be a good show that, over time, the effect gets practice, at least for folks like me who smaller. When Kahn looked at national are relatively early in their careers. unemployment as a proxy for the state It’s good to know what your options of the economy, she found the same are, even if you don’t exercise them. thing. But when Kahn looked at state When I was at Centaur, I did a round level unemployment, she found that the of interviews about once a year and effect actually compounded over time. those interviews made it very clear The overall evidence on what happens that I was lucky to be at Centaur. I got in the long run is equivocal. If you a lot more responsibility and a wider dig around, you’ll find studies where variety of work than I could have gotten earnings normalizes after “only” 15 elsewhere, I didn’t have to deal with as years, causing a large but effectively much nonsense, and I was pretty well one-off loss in earnings, and studies paid. I still did the occasional interview, where the effect gets worse over time. though, and you should too! If you’re The results are mostly technically not worried about wasting the time of the contradictory because they look at hiring company, when I was interviewing different causes of economic distress speculatively, I always made it very clear when people get their first job, and it’s that I was happy in my job and unlikely possible that the differences in results to change jobs, and most companies are because the different circumstances are fine with that and still wanted to go don’t generalize. But the “good” result through with interviewing. is that it takes 15 years for earnings to normalize after a single bad setback. 3 It’s really not about me in particular. Even a very optimistic reading of the At the same time I couldn’t get any literature reveals that external events company to talk to me, a friend of mine can and do have very large effects on who’s a much better programmer than people’s careers. And if you want an me spent six months looking for work estimate of the bound on the “bad” full time. He eventually got a job at case, check out, for example, the Guiso, Cloudflare, and was half of the team Sapienza, and Zingales paper that claims

Reprinted with permission of the original author. First appeared at danluu.com.

hacker bits 25 Programming

Writing good code: How to reduce the cognitive load of your code

By CHRISTIAN MACKEPRANG

26 hacker bits ow bug count, good perfor- mance, easy modification. LGood code is high-impact, and is perhaps the main reason behind the existence of the pro- verbial 10x developer. And yet, despite its importance, it eludes new developers. Literature on the subject usually amounts to  Don’t personalize your work in ways that would disconnected collections of tips. require explanations. How can a new developer just memorize all that stuff? Code Complete, the greatest expo- nent in this matter, is 960 pages long! plying neuroscience to software the result somewhere else. I believe it’s possible to con- development, when people have The approach when break- struct a simple mental frame- to digest your piece of code, ing a problem down should be work that can be used with any their “mental stack” fills up and to have each section as focused language or library and which it is hard to make progress. as possible, affecting only local will lead to good quality code by Don’t code “your way”. Just state, without mixing in irrele- default. There are five main con- follow the coding standards. vant issues, and without side-ef- cepts I will talk about here. Keep This stuff is already figured out. fects if at all possible. them in mind and writing good Make your code predictable and Programming languages and code should be a breeze. easy to read by coding the way libraries often come with their people expect. issues, and abstracting them away can help your code mind Keep your personal its own business. The Single Divide and conquer it quirks out of it Responsibility Principle is anoth- Complex code can often be er example of how focused and You read some article which clarified through modulariza- localized code leads to good blows your mind with new tricks. tion, and there are more ways to design. Now you are going to write clev- do this than just creating more TDD, besides coming with its er code and all your peers will functions. Storing the result of own benefits when done right, be impressed. long conditionals into a variable has been forcing people to apply The problem is that people or two is a great way to modu- certain principles which were just want to fix their bugs and larize without the overhead of previously not as popular. State- move on. Your clever trick is a function call. This will even less code was dismissed as slow often nothing more than a dis- allow you to compose them into and unnecessary (see: most old traction. As I talked about in Ap- larger conditionals, or to reuse C/C++ code), and now everyone is talking about pure functions. Even if you don’t do TDD, you should learn the principles that drive it. Working under new paradigms will turn you into a resilient developer.

Make it discrete and process-able Your computer and your tools  I like taking advantage of variables to compartmentalize logic. can have as much of a hard time dealing with your code as you, and there is some correlation

hacker bits 27 the main elements are located. Using MVC? Place models, views and controllers in their own folders, not three folders deep or spread across several different places. I talked about modulariza- tion. There can also be exces- sive modularization, which will  Using ServiceLocator is an example of design that leads to usually make code harder to poor integration with most IDEs. locate. Your IDE might help some, but often you will be torn between having your IDE ignore a vendor/library folder due to it having too much irrelevant code, between the number of prepro- meaning of will lead you down or having it indexed and dealing cessors and mutations you have this road, and it’s a hard road with the problem manually. It’s to apply and how convoluted the to come back from, because if a lose-lose situation. Try to use code is. your IDE doesn’t understand the fewer libraries by choosing the Let’s set aside the possi- code, any refactoring features ones that cover as many of your ble benefits of those addition- it has are going to be useless needs as possible. al build tools for a moment. when you want to move to a Libraries and tooling can Chances are that they require more static architecture. also be a barrier for new devel- you to use domain-specific lan- opers. I recently built a project guages such as custom templat- using EcmaScript 7 (babel), ing, or complex and dynamic Make it readable only to later realize that our data structures such as hash Work towards having a predict- junior dev was getting stuck tables. Your IDE generally isn’t able architecture. Your team- trying to figure out what it all going to be good at handling mates will find it easier to locate meant. Huge toll on the team’s this stuff, and locating relevant things, and this will greatly productivity. I underestimated pieces of code will become reduce the time it takes them how overwhelming that can be harder. to get something done. Once to someone just starting out. Avoid using language ex- you’ve agreed on an overall Don’t use tools that are still too tensions and libraries that do architectural structure for your hard to get a grip on. Wait for a not play well with your IDE. The project, make it obvious where better time. impact they will have on your productivity will far outweigh the small benefit of easier con- figuration or saving a few key- strokes with more terse syntax. Another way to keep the “integrated” part of your IDE relevant is to avoid magic code. Most languages will provide ways for you to write more dynamic code. Abusing these features such as by using magic strings, magic array indexes and custom template language fea-  Actual code from a makefile I wrote. Junior devs can’t tures will lead to a more discon- handle overuse of new tech. nected codebase. Generally any features which only a human will know the

28 hacker bits  Fluent interfaces have been abused often in recent times.

Make it easy to digest ended up being used in less contextual ways, such as just to If you’ve made it this far, I have add type information. good news: this is probably the Finally there is always some- most important part. Choosing thing to be said about keeping a good names is known to be one low cyclomatic complexity. What of the larger issues in software this means is to keep the num- development. Build tools are ber of conditional branches as unlikely to improve on this, and low as possible. Each additional the reason is that computers branch will not only add indenta- can’t really know the reasoning tion and hurt readability, but will behind a solution. You have to more importantly increase the document the why. Relevant and number of things you have to contextual variable and function keep track of. names are a great way to do this. Names that convey purpose Conclusion & further will even reduce the need for documentation. reading Using prefixes in names is These are five simple and over- a great way to add meaning to arching concepts, and my goal them. It’s a practice that used to here was to make your learning easier by giving you boxes into which you can put all those code organization ideas. You have to Practice focusing on these aspects while programming to solidify them. If you haven’t document the why. done so yet, I really recommend Code Complete. It comes with a large number of examples and be popular, and I think misuse will dissect almost any situation is the reason it hasn’t kept up. you might run into.  Prefix systems like hungarian notation were initially meant to add meaning, but with time they

Reprinted with permission of the original author. First appeared at chrismm.com/blog/.

hacker bits 29 Interesting My biggest regret as a programmer By ANDREW WULF

little over 20 years ago I was in the Bay Area for a little ability you have as a pro- I was at a crossroad. My year around 1995 and worked grammer, no matter how good Asecond company was at Apple for the last half. Apple you are, at making a difference petering out when our 5 years looked to be falling apart and I’d or changing things that are bro- of building Deltagraph for the left to return to Texas as I didn’t ken. I simply didn’t realize how publisher ended (they wanted want to see my favorite compa- little room you have to advance to move into the nascent Inter- ny die around me. Big mistake. as just a programmer (or even net space). At that point I had Not only did Apple begin a as an architect or the like); the 13 years’ experience as a pro- huge turnaround a year later power to change exists at a level grammer but also 9 years or so when Steve came back, but the not available to you as a mere experience running a company whole Dotcom explosion hap- delivery device. (at the same time). pened. Add to that the financial I no longer wanted to do Being both an experienced benefits, the higher likelihood both. My first company (85-87) programmer and leader who of substantial IPO participation, not only built a new kind of understood what it took to de- and all the other things you gain spreadsheet program but also liver (we did 9 major releases of access to, and being a program- published it ourselves. I led the the apps during my time, all of mer means you have to be hap- company, did all the press inter- which I built the master floppies py with the opportunity to build views, managed the investors, for, with no need for hot fixes cool things. did all the usual business stuff which were hard to do then any- Over the years the worst and also was one of the three way) I can only imagine how in places I’ve worked for or helped programmers and the UI design- demand I could have been. Once as a consultant for those 5 or so er. After we shipped the product you get to the level of one of years were almost always due to in early 87 I also wound up in those titles you can keep moving inept, incompetent or downright the hospital. Trying to be both forward and up. idiotic technology management. leader and programmer was My sister started as a pro- There isn’t enough room in this simply too much. grammer 30 years ago but blog to list them all. So at that point in 1994 I jumped into management within Take the VP of engineering could have gone either into tech- the first year and has been a VP for a bank who remarked that he nical management or continued at a big company for the past didn’t need to understand tech- as a programmer. I chose pro- 15 or so years. The huge parent nology as he managed people, grammer because it was easier. of the travel company I worked yet still made technology deci- Today I realize how wrong I was for a couple years ago had a sions. despite all the great stuff I’ve CEO who started 15 years earlier The CIO at the same place been able to work on and ship as a programmer. Of course never believed anything his over the past 20 years. Going these types of jobs can be hard employees told him but believed towards the CTO/CIO/VP Engi- and unpleasant but for that the everything vendors told him. Of neering route, which was fairly remuneration is way greater. My course we knew he was taking new back then, would have been sister has 10X the assets I have. kickbacks as we kept buying a much better plan. Over the years I’ve seen how things we had no use for and

30 hacker bits I came to a fork in the road and took the one less traveled. he kept writing articles for them the next two months trying marketing weenie (young peo- relating how wonderful their desperately to get the 3 found- ple can be pretty stupid). Yet he products were for us. Yet we ers to bring in people who could started all these companies later used almost none of it. Some actually deliver (I knew several including ESPN, worked for Ap- time after I left he was fired and people) but they were afraid to ple in various leadership roles, perp-walked out of the com- make any changes and admit was a good friend of Steve Jobs pany, yet immediately he got they had screwed up in hiring and started his own VC firm. another similar CIO position. these two guys. Eventually I gave And today I am still just a The worst job I ever had up and left. programmer. Who’s the wee- started out as what I thought A year later, after still get- nie now? I doubt I will ever be would be awesome. A post-start- ting nothing from this guy, they able to really retire. Thankfully up had a successful niche in fired both of them. They tried to I am still good at delivery (I was their industry; both they and hire some consulting firm but recruited into my present job by their arch-rivals (different niche) got nothing from them either. a former manager who needed both wanted to launch into a By this time it was too late. The what he knew I can do) but still broader public market and the rivals? They became a billion all I will be until I croak is what I market was heating up. dollar public company and I see am now. I was hired as a second their commercials on TV some- Being a programmer for programmer. The other pro- times. I always want to throw a nearly 35 years and still being grammer and manager had been shoe at the TV when I see them. able to get things done and ship hired to build a new broader We had everything but a damn is still fun and I’ve been able to online store as the existing one store and actual technology work on amazing things over was too inflexible and slow for a leadership. If I had been such the years. But I can still feel the big market. a person instead of a program- regret of not seeking the chal- The company had zero mer I would have had the track lenge of just leadership. In some technical leadership otherwise, record and clout to make it ways programming was the easy and the CEO and the other happen. But all I was was a pro- choice. Given how close I got to two founders had no technical grammer. the whole Dotcom timeframe, experience or knowledge. The I could go on and on but or even the return of Steve to programmer constantly talked the key is that you can’t make Apple, and still had recent lead- about how wonderful his back- changes in how people do ership experience, I could have end code was and the man- things in a technical sense un- been almost anything. ager supported him. I built a less you have the ability, author- So yes I regret not taking front end piece, put up demos, ity and opportunity. Once you that choice and seeing where it checked in my source every day. make that call and assuming you would have led me, yet I would When I thought it a good time to find the right places to grow, the have missed all the fun of writ- integrate I discovered the other sky is really the limit. ing code and the soul-draining programmer, after 10 months, When I was on TV (Com- jobs that often come with it had checked innothing. puter Chronicles) in early 1987 where you can’t really fix any- When I pointed this out the showing our product Trapeze, thing. manager said “he never checks the other presenter was Mike I came to a fork in the road in anything until it’s perfect”. Slade who was product manager and took the one less traveled. Yet no one called this out as of Excel. At the time young me Perhaps now I realize why.  stupid other than me. I spent thought he was some random

Reprinted with permission of the original author. First appeared at thecodist.com.

hacker bits 31 Interview

Interview: How to become a better programmer

John Sonmez is the founder of SimpleProgrammer.com, author of the bestselling book Soft Skills: The Software Developer's Life Manual and creator of over 50 profes- sional developer courses ranging from iOS, Android, Game Development, Java and more.

n the past few months, we've interesting, and growing and I believe in being transpar- gotten a number of emails developing myself and sharing ent, so I do almost everything in Ifrom readers about how to what I learned in the process public and share as I learn and be an all-around more effective with others to help them along grow myself. programmer. Learning new tech- the way. I live in sunny San Diego nologies faster, becoming more My real mission in life right with my wife, Heather, and relevant, being more productive now is to help as many software 5-year-old daughter, Sophia. and interviewing better all came developers not just get better at up as hot button topics. their job and make more money, From your coaching and con- To help us through these but to really live better, happier, sulting, you've come across topics this month, we chatted and more fulfilled lives, and to engineers looking to take with John Sonmez, who's been achieve the goals they set for things to the next level. Can helping software developers, themselves. you go into detail about the programmers and other IT I developed software for 15 challenges and help us under- professionals, become better at years before I realized that the stand the steps your clients their jobs. most important skills in life are took to get to that next level? Without further ado, here's not technical, but rather what I John Sonmez. call soft skills. It's difficult to make a general- Once I realized this, I re- ization about what is holding Can you fill in some gaps in alized that I could really help engineers back from reaching that intro, and then give us other software developers get the next level, because so many a glimpse into your personal ahead in their career and reach people have so many different life? success by taking a much more ideas of what success even holistic approach to their lives, looks like, as well as very differ- Well, I'd say the main thing instead of just focusing on the ent mental models and strug- about me is that I am always technical aspects. gles. But I'd say that one of the

32 hacker bits The first and most important skill is learning to learn.

biggest things is mindset. When success, and how to build a I could talk about how you change someone's mindset, personal brand and specialize to important this single skill is for you change his/her life. become an expert in a particular hours. It seems very basic, but so area of software development, A good way to develop this many people have extremely which can yield huge results. skill is to get in over your head limiting mindsets. They be- I've seen huge success my- and do things that you are not lieve something about how the self and with many developers ready for yet, because it forces world works and place artificial I've worked with. Multiple stories you to sink or swim and to learn limitations on themselves that of 3x income increases, new and in a limited time frame, focusing prevent them from succeeding. better jobs, successful freelanc- on what is essential. They believe they are some- ing or consulting practices, and Next, I'd say is a hybrid of how not good enough or that just learning how to sell the discipline and gumption. It's they are not "lucky," instead of value you already have better, as the ability to do what needs to realizing that there are very few well as to learn how to give free be done, whether you feel like things that can't be achieved by value to others in order to build doing it or not. Most success in a person who is determined and a brand and reputation. life comes from going after what unwilling to give up. most people would have given I've coached (personally and In general, what are the top 3 up. So many people quit too ear- through my blog, books, videos essential skills a software en- ly or get discouraged when they and podcasts) tens of thou- gineer should develop? What are just a few feet from gold. sands of developers to develop is the most efficient way to To develop this skill, I prac- the kind of mindset needed to develop these skills? tice doing one thing I dislike succeed and really reach that The first and most important every day, whether I feel like it elusive next level. skill is learning to learn. or not. Discipline comes from I've had to go through it Developing the ability to learning to produce results even myself – in fact, I still am. Every teach yourself is by far the most when motivation is lacking. single day I try and expand my important skill you can develop Finally, I'd say people skills mindset, and grow a little more in life – especially for a software are very important – essential – and help others along the way. developer who has to constant- for just about anyone. Bringing things back down ly face changing technologies Learning how to properly in- to a very practical aspect, I'd and is forced to learn at a rapid teract with people and deal with say one of the biggest things pace, just to keep up. This skill tricky social situations is a skill that software developers and IT was so important that I specif- that is always valuable, because professionals don't know how to ically did a course on it called regardless of what our job is, we do, that can greatly impact their “10 Steps to Learn Anything are almost always dealing with career, is to learn how to market Quickly.” people. themselves. I sell a course on I found so many developers To develop this skill, I rec- how to market yourself as a soft- doing this wrong or not doing ommend reading Dale Carne- ware developer. It teaches devel- this at all that I felt I had to ad- gie's famous book: "How to Win opers why marketing themselves dress it, because it's so import- Friends and Influence People." is so important and critical to ant.

hacker bits 33 How can programmers balance Let's dial in on staying up-to- Programming interviews are being productive at work and date. How can software devel- possibly the worst invention in learning new frameworks, opers stay informed of what's the software industry (sorry, tools and languages to stay truly important with every- that's Ray's opinion). However, up-to-date? thing evolving so quickly in they are a fact of life if you're Devote some time each day the industry? looking to get hired. What is to self-development. I used to the #1 thing programmers Devote time each day to read can do to improve their odds walk on the treadmill and read a blogs and news sites like Hack- technical book of some sort for of nailing that interview and er News. Not a huge amount of getting hired? at least half-an-hour each day. time, but perhaps 30 minutes This helped me get in shape and each day will help keep you up- Think of them as a filter, not keep my technical knowledge to-date. an actual test of your ability to sharp. Also realize that there is perform at the job. Another important thing to nothing truly new. Everything is Don't fight them; instead do is to not learn what you don't always a reinvention of some- realize they are an opportunity need to learn. Only learn what thing else from the past.The to artificially thin the field which

Another important thing to do is to not learn what you don't need to learn.

pendulum always swings back. will actually increase your chanc- you are actually going to use. First it's thin clients, then it's es of getting the job. It's great to read blog posts thick clients, then it's thin cli- With that in mind, become and technical news to stay ents inside of a thick client and an expert at passing these kinds up-to-date on trends and devel- back again.One you realize this, of interviews. opments in the field, but don't absorbing new information and Practice coding on a white- learn every new framework that spotting trends becomes a lot board. Practice at sites like comes along. Instead, focus on easier. Codility and Top Coder to solve knowing enough about it so the kind of algorithm and data that if you need to learn it in the What is the #1 thing holding structure problems that big future, you can. Again, learn- back programmers from tak- companies like Microsoft and ing to teach yourself effectively ing things to the next level? Google will inevitably ask. will give you the confidence I used to fight these pro- you need to know that you [See John's video response. If you enjoyed his response, sub- gramming interviews and com- can do just-in-time learning to plain about them, until I realized learn something when the time scrbe to his YouTube channel of 23K subscribers.] I just needed to get good at comes, instead of potentially them. It didn't take very long, wasting hours trying to learn but it was a skill that allowed me something ahead of time that to land several really good jobs you may or may not actually use. and to feel more confident in general.

34 hacker bits It's up to you right now – in this very moment – to decide to take action and then to follow through and to keep following through until the job is done.

This is one of those times again, but you can literally do go by living the life you are not when you have to study for the anything you want right now supposed to live. Do not say test and not for the material. with your life and no one can "someday I will do this, or some- Plus, even though program- stop you but yourself. We live in day I will do that." Do not wait to ming interviews may seem silly, a time of such incredible free- live your life. Live it now. the skills I’d developed to pass dom and opportunity, yet so Give. Don't worry about what them have actually been quite many of us let our minds and you’ll get in return. You can't helpful in my career in other attitudes defeat us before we've out-give the universe. The more areas when I had to solve a dif- even given it a shot. people you help and the more ficult problem, and I knew just If you would have told me you give, the more you will get. what algorithm and data struc- that I'd earn my living by inspir- Create value for others, encour- ture to use. ing and helping people develop age them, and help them along And since most developers themselves 5 years ago, I would the way. I guarantee you that if will not do well on these kinds have told you that you were you find a way to help 1 million of interviews, since they won't crazy. I had no idea what my people, you'll earn more than 1 prepare for them, taking the own potential was. I had no idea million dollars. time to do so will give you a I could do what I wanted to do. huge advantage. I just had to be willing to work Where can people find out for it. more about you? And where What is a parting piece of ad- Stop giving up. Stop accept- can they pick up a copy of vice to Hacker Bits readers? ing less. Stop making excuses your book? Realize that while technical for yourself. This is your life Go to simpleprogrammer.com. abilities are important, they are right now. No one is coming to There you'll find my blog posts, not the most important. In fact, fix it for you. No one is coming links to my YouTube videos, probably less than 10% of the to save you. It's not going to podcasts,products and more. success you see in your career magically get better. Your prob- You can get Soft Skills at will come from your technical lems are not going to magically simpleprogrammer.com/softs- abilities. It's really the soft skills go away. It's up to you right kills. I recorded an audio version that make the difference in your now – in this very moment – to of the book on Audible.com as career and life, so actively focus decide to take action and then well.  on developing those skills. to follow through and to keep Stop limiting yourself so following through until the job much. You hear it over and over is done. Do not let another day

Interview was edited for brevity and clarity.

hacker bits 35 Programming

Four strategies for organizing code

By MARTIN SANDIN

36 hacker bits his article outlines four coupling and cohesion is mostly For classes it is fairly well different strategies for irrelevant; it doesn’t care if you understood that this should be Torganizing code: put all your code in a single mil- done so that we create logical lion line method, sort your class- objects which exhibit good co- • by component es alphabetically, or give all your hesion and fit well in the domain • by toolbox variables single letter names. model. • by layer Code organization is not With projects — which are • by kind about communicating with the separately compiled — v we have computer but rather all about to break circular dependencies I think these four form a kind of trying to make sure that people and try to make sure that they hierarchy with regards to which can understand the code well expose reasonably logical and kind of cohesion they favor and enough that they will be able to stable interfaces to other proj- in my experience they cover maintain and evolve it with some ects. most of the real-world code I’ve degree of efficiency and confi- On the level in between worked with, pleasurable and dence. — packages in Java or name- not. There are an endless num- spaces in C# — there is a lot ber of possible strategies but “Programs should be writ- more variation and in my expe- I’ve (thankfully) never encoun- ten for people to read, and rience many developers chose a tered anyone who organizes only incidentally for ma- strategy without much consider- packages into projects by cre- chines to execute.” ation given to why that particu- ation date or classes into pack- lar strategy should be employed. ages by first letter. — Structure and Inter- The first three strategies pretation of Computer described in this article can be Programs by Abelson and used at either class, package, The whys and Sussman or project level while the last whats of organizing one — organization by kind — When a unit of code grows is more or less specific to the organizing code too large and contains too many package level. It is a funny thing that most elements, it becomes hard to of the advice you will hear and navigate, hard to get an over- read on how to develop software view of, and hard to understand: Strategy #1 — basically prescribes how you it becomes complex. by component should organize your code, a Our main weapon against topic that doesn’t matter to the this complexity is divide and Organization by component computer. As far as the machine conquer: we split the unit into minimizes complexity by em- is concerned, all this talk about smaller parts which we can un- phasizing external and internal derstand in isolation. cohesion of code units, e.g.

 Reduced total complexity doesn’t follow unless you take the step of eliminating dependencies

hacker bits 37  Eliminating dependency

packages. The former means This creates code which “No client should be forced that the package has a minimal looks easier to digest but it is to depend on methods it interface, which exposes only mainly window dressing until does not use.” concepts which that are strongly further steps are taken: the related to the service the compo- benefit of reduced total com- — The Interface Segrega- nent provides. The latter means plexity doesn’t follow unless you tion Principle that the code in the package is then take the step of eliminating strongly interrelated and thus dependencies. The next step would be to strongly related to the provided In my opinion, packages eliminate the direct dependency service. which have mutual dependencies from the graph package to the A lot can be and has been should not be considered sepa- storage package. This could for written about what constitutes rate units of code at all as none example be done by creating a a good unit of abstraction and of them can be understood in GraphPersister interface in the covering even a sliver of that isolation from the others. former and having a higher level would make this article too In the example above. it is package inject an adapter imple- long by far. Suffice to say that easy to imagine that the Graph mentation into the Graph. And the SOLID principles are a great class has a reference to a Graph- once again the primary benefit place to start learning, and that Storage in which it persists itself would be that the exact set of practice and reflection on how whenever it has changed. storage functionality the graph things are working out and why Not only does the graph_ package depends on would be- that might be is paramount. storage package depend on a lot come obvious. In this article I will only of details of the graph package cover what in my experience is domain model about which it “…packages which have the single most common rea- should be rightfully ignorant, mutual dependencies son for rampant complexity in the packages also remain mu- should not be considered code bases where people have tually dependent. The easiest separate units of code at actually tried to organize things dependency to eliminate is often all…” by divide and conquer: failure that from the new package to to isolate packages into compo- the old one (see figure above). In theory this process might nents. The most important reason sound fairly easy but it takes New units of code are often that this is an improvement is a lot of experience to learn to created by identifying a subset that when reading the storage identify suitable components of the functionality contained in code one can now rely on the and strategies for isolating one (or more) existing packages fact that the only things it need them. It is quite common to and creating a new abstraction to know about that which it is start the process only to find out from the corresponding code, storing is what is in the Storable that you didn’t quite get the ab- resulting in more but smaller interface. straction right and need to back units. out of the change.

38 hacker bits sumer and each “tool” in the box isn’t big enough to warrant its own unit even though they are technically independent. Each component in a GUI li- brary might for example deserve its own package but giving each its own project is unnecessarily onerous. Similarly each collec- tion implementation might fit in a single class and putting them  Toolbox with a facade for DiskList for the sake of external consistency all in individual packages would be unnecessary bureaucracy. At least in the latter case a single collection implementa- tion which that grows beyond a The rewards for properly regards to areas such as couple of classes should get its isolating components are great time complexity and memory own package, possibly except however: code which is easy to consumption. There might for a thin facade for the sake of understand, easy to improve, also be a unifying theme to external consistency. easy to test, and — incidentally the toolbox, such as only — easy to reuse. containing disk-based data structures. Strategy #3 — by layer Strategy #2 — • Logging libraries are not necessarily toolboxes in Organization by layer favors by toolbox their entirety but often workflow cohesion instead of Organization by toolbox focuses contain a toolbox of e.g. trying to control complexity by on external cohesion, providing log-writer implementations, minimizing cross-unit coupling. a consistent toolbox which the which target different desti- The code is split along layer consumer can choose from. This nations. boundaries defined by issues strategy is weaker than organiz- such as deployment scenarios ing by component as it drops Toolboxes arise because or areas of contributor respon- the requirement for strong they are convenient to the con- sibility. internal cohesion, e.g. that the constituents are all strongly interrelated. The parts of a toolbox are often complementary implemen- tations of the same interface(s) which can be usefully chosen from or combined, rather than sharing a lot in the way of imple- mentation.

• Collection libraries are typi- cally organized as toolboxes with a set of complementary  Component coupling across layers implementations of a set of collection interfaces with varying characteristics, with

hacker bits 39 This strategy is different pled layers, pulling them togeth- Strategy #4 — by kind from organization by toolbox er into a single — often wildly in that layers don’t present a complex — unit. Organization by kind is a strate- single, minimal, and coherent Organization by layer should gy which tries to bring order to interface to the other layers, be used cautiously as it often overly complex units of code by but instead a wide interface with increases total system complex- throwing the parts into buckets many constituents which that ity rather than help control it, based on which kind of class (or are accessed piecemeal by the but there are cases where the interface, …) it is deemed to be. corresponding constituents of benefits it provides outweigh In doing this it ignores depen- the consuming layer. this drawback. dencies and conceptual rela- The typical characteristic In those cases it’s often tionships and typically produces of organization by layer is that worth sequestering your layer packages with names such as the logical coupling is stronger dependency into a single place exceptions, interfaces, manag- within the logical components in your consumer code rather ers, helpers, or entities. that span across the layers than than having its tendrils reach Organization by kind is within the layers themselves. throughout the entire code base: different from organization by The most common failure toolbox in that it drops any mode of this strategy is that • Don’t let references to lan- pretense that the classes in a most changes require touching guage resource files infil- package are complementary, in- files across all the layers, basi- trate your entire code base, terchangeable, and/or form any cally the textbook definition of but rather map all results kind of sensible library when put tight coupling. and errors from your internal together. components to language re- Nobody that I know of is “Given two [units of code], source messages in a single advocating using this strategy A and B, they are cou- place near the presentation for organizing code into sepa- pled when B must change layer. rate classes or projects (“here’s behavior only because A the class with all the string changed.” • Don’t use the value objects members” or “here’s the project generated from your JSON in which we put all our excep- — The C2 wiki schema beyond your service tions”). layer; , translate them into I consider organizing code Under this scenario, logical proper domain objects and by kind harmful as it hides the intra-component dependencies calls at the earliest time actual problems of complex end up like ugly nails driven possible. code and thus make developers through your supposedly decou- feel that they’ve fixed it while

 Layers nailed together into a single — very complex — unit

40 hacker bits  Project organized by kind

the overall complexity remains Summary the same. “…package size isn’t the The example above looks main problem, the number Organizing code is a core skill kind of neat with everything of interdependent parts is.” for software developers and as tucked into bite-sized packages, with all skills the most effective but almost every change re- I consider organization by way to improve is to reflect on quires touching every package, kind a code smell but in my ex- your previous choices and the meaning that the packages are perience from commercial proj- fallout from them. There are a in fact tightly coupled. ects — mainly in Java and C# — it wide array of different strategies The other big problem with is quite common. I believe that for organizing code and learn- this strategy is that if it is taken this happens because it seems ing to recognize both the useful to its extreme, it requires ev- to provide an easy way to par- and the dangerous ones is very ery class to be of a clear-cut tition large packages and most important.  kind. I’ve seen this warp en- people aren’t aware that pack- tire code-bases as all kinds of age size isn’t the main problem, strange things get created and the number of interdependent designated a Manager or a Help- parts is. er just to fit into some package.

Reprinted with permission of the original author. First appeared at medium.com/@msandin.

hacker bits 41 Opinion

42 hacker bits Why I switched to Android after 7 years of iOS

By HENRIK JORETEG

ast Monday I was all excited. I don’t know about you, but 4. You could even pick from I had just gotten the green the idea of having a fully capa- a few different status bar Llight to start prototyping a ble web browser in my pocket colors! new Progressive Web App for a was a huge part of the appeal. client I've been working with. That’s never changed. I don’t know whether this I pulled out an older Android Of course I don’t know type of app was actually intend- phone that I keep around for the full backstory, but it sure ed to be the primary mechanism development. Then I also got my seemed like the original plan for for 3rd party dev to build apps sleek, new, shiny iPhone 6s out 3rd party developers on iOS was for iOS but regardless,it was way of my pocket, with its smooth to have us all just build apps ahead of its time. curves and speedy OS. But as I using the web. Safari added Unfortunately, the web plat- looked at my iPhone I was kind support for making web apps form itself wasn’t quite ready of bummed out. that could be added to your for the spotlight yet. It was sort I realized that this slick piece home screen as an icon and by of possible to build web apps of Apple hardware was less including a few magical that looked and performed like capable as a platform for web tags you could use to create native apps, and I was trying to applications than my dusty old something that could sort of be do this 6 years ago using David Android dev phone. “installed” to your home screen. Kaneda’s awesome jQTouch lib. At that point I knew I was Then, when you opened it would Hilariously, the corny little demo over iOS. run in “standalone mode”. video that I posted led to a call So, instead of opening my These were, in many ways, from David and almost got me text editor, I placed an order for the original “Progressive Web a job at extjs right as they were a Nexus 6P and signed up for Apps” and this was sometime rebranding to Sencha and start- Google fi phone service (which is around 2009! ing to build Sencha Touch. But awesome, btw). the story for offline was terrible. Just like that, after 7+ years, Think about it… But anyway, as it turned out, bye bye iOS. 1. They started in the brows- the capabilities of the web on er, but then you could kind iOS were not quite enough to of upgrade them to “home satiate ravenous developers. So What!? What’s wrong screen status”. developers were left clawing for with iOS? 2. When you ran them from the ability to build stuff that ran the home screen they’d natively on the device to give Remember the original iPhone open with a splash screen them better performance and announcement? Where Steve in- and without any visible deeper API access. troduced the amazing combina- browser UI. tion of a mobile phone, an iPod, 3. You could pick a loading and an Internet communications screen and app icon. device.

hacker bits 43 Enter the iOS SDK and However, this became simply take the user to the link they clicked within the same full- App Store problematic screen web view with no way to Apple made what turned out to Despite the popularity of na- return to the app! The only way be a really smart business deci- tive apps, the original idea of out was forcibly quitting the app sion: they released an iOS SDK these standalone installable web (hopefully the user knew how to and an App Store, and the rest is apps has continued to be sup- do that). history. ported for new versions of iOS. We were running a chat First, I was excited about But, they didn’t fit into Apple’s product at the time, so any time “apps” just like everyone else business model! The App Store someone pasted a URL into chat seemed to be. Just think, here turned into a huge business, it was essentially a trap. we had been busy building the term “app” was going main- These sorts of issues con- “applications” when we really stream, and every business sud- tinued to happen release after should’ve been building “apps” denly felt they needed to have release. Soon it became obvious all along! Who knew?! ;) their own “app,” whether they that while you can sort of build Anyway, I quickly found had any users or not. these types of apps on iOS, you myself hunting for the best As Apple’s app business can’t really depend on them not apps and switching to whatev- took off these capabilities very breaking with the next update. er bank, social networks, and clearly and quickly, and some- The message from Apple other services had the best iOS what unsurprisingly were de- seemed clear: web apps are sec- apps. I bought a book on iOS prioritized. The end result for ond-class citizens on iOS. development and built a hello those of us still trying to build world or two. My old co-worker installable web apps for iOS was Ryan Youngman made iSaber to that with nearly every new iOS What of Android? let you swing a fake lightsaber release, some key feature that I didn’t care much at the time, at your friends with your phone. we were depending on broke. but somewhere in the middle Every developer I knew was I’m talking about stuff that of all of this, Android appeared talking about iOS development QA should have caught, stuff on the scene. It promised to but at some point the fun of all that if anybody at Apple was be a more open alternative this iOS stuff dried up. actually building apps this way as a mobile platform. It was a Seeing the hoops you had to would have noticed before they collaboration between several jump through to ship an app on released. big companies, and it was their iOS didn’t seem right. One quick example that bit attempt to essentially fight How quickly developers trad- me was how they broke the abili- off the fruit-company-come- ed away the wide-open spaces of ty to link out to an external web- back-kid-turned-gorilla and its the web for a walled castle with site from within an app running Mighty Joe App Store. a monarch enforcing a 30% tax. in “standalone” mode. target=_ It started gaining traction, So, I decided to focus on blank no longer worked, neither but its web experience at the building “installable web apps” did window.open or anything else I beginning was quite sub-par. for iOS instead because surely, could think of. So now since our the web would catch up. “standalone” app didn’t have a URL bar or back button, it would

It (Android) is currently the best mobile web app platform.

44 hacker bits Fast-forward five post to, if you’re interested. the ability to create app-like So yeah, Safari runs my JS experiences on the OS with web years… faster, but guess what… most technology. of your users won’t have a shiny Very little seems to be hap- 1. People are somewhat burnt new iPhone 6s, and as I’ve said pening in that regard as far as I out on Apps. before, betting on desktop-like can tell. 2. The vast majority of de- performance on the mobile web, velopers building native or sending huge frameworks like iOS apps never made back Ember to a mobile device proba- Let's look at Apple their expenses. We knew bly isn’t a great idea. and WebRTC this in 2013. With performance, there is 3. A few games are still mak- such a thing as “good enough.” A few years ago, I built Sim- ing money, but that’s a It wouldn’t matter if Safari ran JS pleWebRTC and the first version lottery. 50x faster! The only thing that of Talky.io. 4. Meanwhile, there are over matters is whether my app runs I seem to have been one of 1.4 billion active Android fast enough. Beyond that, as a the early web geeks to get re- users. user, I don’t care. ally excited about WebRTC (the 5. Android switched to using As it turns out, it’s possible browser web technology that Chrome as the default to write web apps that run at now powers Google Hangout browser. 60fps even on older, crappier video calls). Anyway, I managed 6. Chrome, Opera, and Fire- hardware. to figure out how to build one fox have added features to But, all that aside, note this: of the first, possibly the first allow building actual app I said “better app platform” not multi-user, peer-to-peer, video experiences via the Web. faster JavaScript runtime. calling WebRTC app on the web that worked with more than 2 And here I am… switching to people and worked between Android. So why not just use Chrome and FireFox. This was my first experience Chrome for iOS?! with Apple lagging behind in So why Android? Isn’t As I started tweeting about implementing new web APIs. switching, I was surprised to Although Chrome and FireFox it just more of the realize that many people don’t were both actively implement- same? know that Chrome, Opera, and ing and excited about WebRTC, Firefox for iOS all just use Web- there was not a peep from Yes. It is. Android itself bores Kit web views under the hood. Apple. iOS still hasn’t added me, honestly. There’s nothing In fact, apps that include WebRTC support to this day. all that terribly new or exciting a different browser engine are Though, they’ve apparently been here. a violation of Apple’s terms of hiring WebRTC engineers of the Save one very important de- service. Safari team. So here’s hoping… tail… IT’S CURRENTLY THE BEST They're just different UIs on But it kinda makes sense, MOBILE WEB APP PLATFORM. the same browser engine. right? Why would they? They’d rather you use FaceTime, right? What do you mean?! They seem fine with improv- But isn’t WebKit ing the browser engine, but Doesn’t Safari run my getting better? seem very slow to do anything JS faster? that involves increasing the Yes, it seems like they're picking web’s reach in the OS. Most people when they say this up some momentum recently. Anyway, we shipped Talky.io are referring to this post by Jeff But, there’s a whole lot more as a web app that worked in Atwood (a.k.a. codinghorror), to it than just what happens which I wrote a whole response in the browser window. I want

hacker bits 45 Chrome and FireFox, and even- Web Apps on Android is the So what are tually @hjon built an iOS app for most important thing to happen it too. to the mobile web since Steve Progressive Web But the thing that blew my first introduced the iPhone. Apps anyway? mind was one day I just down- Why?! Because, for the first Unfortunately, for some reason loaded Chrome on Android, time, we have a mobile platform Google has managed to teach opened it to Talky.io and sure with a huge user base, which a generation of devs the words enough… IT JUST FRIGGIN’ lets me build a web app that is “Polymer” and “Angular”, while WORKED! treated as a first-class citizen by the vast majority of web devel- Since then I’ve been paying the platform! opers that I meet and talk to much closer attention to what’s (Note: yes, I’m aware there today have still have ZERO idea happening in mobile Chrome have been other attempts to do what ServiceWorkers or Progres- and it’s very impressive. this, but none of those had 1.4 sive Web Apps are. billion active users.) Some of this is because of These folks finally gave us a the newness of it all, and some Meanwhile on platform where web apps were of this is improving recently. first-class citizens! Android Butsheesh… I hope this changes. And to be clear, I’m not just You can think of a progres- During the last couple of years, talking about a way to put a sive web app like this: a few very bright, persistent, glorified bookmark on the home It’s an app written in HTML, and idealistic developers (many screen. CSS, and JS that can completely of them at Google) who believed I’m talking about a way for masquerade as a native app. in the web have been at work us to build web apps that are pushing for, and implementing indistinguishable from native This includes: new web standards that fill the apps. 1. Living on the home screen. gaps between native and web. The term that’s sticking for 2. Existing in the Android Incredibly cool new stuff is these types of apps is “Progres- “app switcher” as a sepa- coming, like: sive Web Apps”. rate app (not as part of the In fact, I think Progressive browser app). • WebBluetooth (yup, talking Web Apps (PWAs) actually have a 3. True offline behavior, to Bluetooth devices from JS huge leg-up on native apps be- meaning when you tap on a webpage). cause you can start using them the app icon,it will open re- • WebNFC is coming too, ap- immediately. You don't have to gardless of current Internet parently. jump to an app store and wait a status. minute or two until some huge 4. The ability to run in the These things are going to blow binary is done downloading. background and triggering the roof off IoT stuff (but that’s They’re just web apps, they have OS-level notifications, even a whole other blog post). URLs, and they can be built to when the app and browser Just type chrome://flags in the load super fast. Because… well, is closed. URL bar of Chrome for Android we’ve been optimizing load time and read through all the stuff performance on the web for a Instead of starting as a that's currently in the works. It's long time. useless web page with a “please amazing! There’s just so much less install our app” banner, these Anyway, in the past couple friction for users to start using apps start life running as a tab of years these fine folks have them. Just think what that would in your browser. Then progres- built a feature that has me more do to your conversion numbers! sively they become more in- excited than I’ve been by any Because of the improved stalled/integrated into the OS. web tech for a loooooong time: on-boarding experience I believe At first, it’s really no differ- that businesses targeting An- ServiceWorker and the concept ent than any other website you droid users should be strongly of Progressive Web Apps. visit. But then if you return to questioning whether they should that same website/app in your be building native Android apps I believe that the introduction of browser again, the browser itself at all. ServiceWorker and Progressive will subtly ask the user if they’d

46 hacker bits like to add it to their home • Jake Archibald’s Offline lets us use our JS skills to write screen. Cookbook native iOS apps. From this moment on it’s • Aditya Punjani's post on how But as I’ve been saying, I indistinguishable from a native they built FlipKart lite don’t think we should be build- app to the user. ing native apps unless we abso- Also, if you build these lutely have to. correctly there’s usually nothing So what does this all The end result of React else the user has to download or mean for us? Native is that because it exists wait for at all. This means that and because it’s largely aimed adding it to the home screen is We, as web developers, can at web developers, we now have effectively an instant app install. finally build screaming fast, web devs flocking to build native Again, imagine what that’ll do to fully offline-able, and user-pri- apps just because they can! your conversions? Eh? (No, I’m vacy-protecting apps that work I fear that this undermines not Canadian) cross-platform without the our ability to use our collective Luckily, we don’t have to en- need for any friggin’ App Store bargaining power to encourage tirely guess about the business taxes, approval processes, or Apple to implement support for impact. We actually have some doorslamming users up front Progressive Web Apps. real data from a certain $20 bil- with “please install my app to To be clear, I completely lion dollar online retailer in India use this service”. understand why it was created called FlipKart, who did launch and I have a lot of respect for a PWA and have shared some of the technical achievement it their numbers. What about iOS represents, and the developers support? behind it. Key highlights from FlipKart’s Well, the beauty of it is an iOS I just don’t want us to stop experience: user can still use your web app pushing Apple to improve web • 40% returning visitors week even if service worker support support. over week. doesn’t exist. • +63% conversions from They just don’t get the extra Home screen visits. goodies, like offline and push In summary • 3x time spent on FlipKart notifications. So, all this said, these things led Lite But you could also bundle me to finally exercising the only the app with Cordova and use voting power I have as a con- That data came from Alex the Service Worker plugin, and sumer:I took my money and left. Russel’s recent Fluent Keynote that would in theory let you I don’t see this as switching on what’s next for mobile. I use the same code to do those to Android; I’m simply switch- encourage you to watch and things but bundled up as an iOS ing to the best mobile web app share it with product managers app. platform available today. and leaders at your company. It The web is the only truly does a great job of explaining open platform we’ve got. It’s the the how/why of Progressive Web Why should I care? closest thing we have to a level Apps. React Native exists playing field. This is why I’m focusing all For related reading check out: now and solves the my efforts on building Progres- • Addy Osmani’s Getting same problem. sive Web Apps, and I hope you’ll started with Progressive Web do the same.  Apps Personally, I actually kind of • Mozilla’s service worker ex- wish tools like React Native amples at: ServiceWorke.rs didn’t exist. Stay with me, let me • FlipKart’s original technical explain. React Native is an amaz- post about their PWA ing and very impressive tool that

Reprinted with permission of the original author. First appeared at joreteg.com.

hacker bits 47 Opinion

On asking job candidates to code

BY PHIL CALÇADO

48 hacker bits We truly practiced hire for attitude, not for skill.

t DigitalOcean, we are any engineering jobs would be Skype. And have people assess making some changes to always reading books. Not just me on my coding skills. I was Athe recruitment process of the reference books everybody freaking out. back-end developers. We simpli- reads for their daily jobs, but Luckily, ThoughtWorks had a fied the job description and the texts on software architecture, process that was a bit different interview process, and added a object-oriented design, arcane from the usual standards com- step that asks the candidate to programming languages, and panies had back then. write us some code. This is an advice on how to get better at After a quick recruiter account of my experience on their profession. screening, they sent me three hiring processes and their use of I had a somewhat popular options for a code challenge code reviews. programming blog in Brazil, and (a small problem I could solve Around 2005, I had inter- I wrote a job ad as a blog post. in any programming language viewed for a management posi- I described the job the best I I would like to use). My code tion at a Globo.com, the Internet could, and asked people to send submission would then be used arm of the largest Latin Ameri- me their résumé and a list of the during follow-up interviews, in- can media conglomerate. I was last three books they had read. cluding a pairing session where coming back to product devel- The response was excellent. a ThoughtWorker and I would try opment from a couple of years The temporary email address I to extend my code adding a new as a consultant, and was super had set up was full of résumés, feature. excited about the opportunity. and I was able to confirm my I spent four years at The week before I started, hypothesis: the most interesting ThoughtWorks and saw this pro- my new boss sends me an email résumés correlate with the list cess produce consistently good with what would be my first containing books from the cate- results over and over. It was also task: hire four people to join gories above. important that it didn’t particu- the four others already on my At Globo.com, we’ve hired a larly dictate what languages or team. It was a weird situation; I large team following this pro- tools a candidate would have couldn’t even tell what kind of cess, and some of us alumni still to use. My experience with people I would need as I hadn’t use something similar in our ThoughtWorks was that I could started working yet! new organisations. either choose an interesting I began thinking about what Fast forward a couple of project or an interesting pro- kind of people I knew would be years, and I’d applied for a job gramming language (I’ve written perfect for almost any engineer- at ThoughtWorks. I had worked Internet-scale web crawlers in ing job, the people I would try for some international compa- Drupal and timesheet software to poach. I then thought about nies in the past where I would be in F#), so we truly practiced hire what particular traits or habits required to speak English when for attitude, not for skill. these people shared. interacting with other offices, After all those years it was Soon enough, I realised a but this would be the first time I time to move on. I’ve had mostly common pattern: the people I would speak English continuous- a good experience at Thought- thought were great for almost ly for more than one hour. Over Works, but my last project there

hacker bits 49 was probably my worst. As I to check out what was in vogue in a small startup like Sound- looked around for a new po- in Clojure in 2011 and how one Cloud was back then, we needed sition, I dreaded having to go can pretend to know JavaScript T-Shaped people more than any- through whiteboard coding ses- by writing Scheme with curly thing), but we would ask them sions on some useless puzzle brackets, you can see the code to use a language’s standard — the kind of stuff my friends here. library, with no third-party libs reported the big Internet compa- A couple of weeks later, I got or frameworks. nies had them go through. the gig and moved to Berlin. As To make it more feasible, we Once more I was lucky and it happens with small organisa- didn’t ask for a web app but for ended up applying for a position tions, even before my probation something all platforms would at SoundCloud. Back then it was period is over, I’d started review- offer: a socket-server interface just a handful of engineers, and ing code for new candidates. and a simple string protocol. To just like ThoughtWorks their Unfortunately, the experience drive it closer to the needs of a process started with a recruiter was quite disappointing. company that grew from 10 to screen and a coding challenge. It turned out that most peo- 100 million users in less than Different from Thought- ple would put together a Rails one year, we also included the Works, though, their code app with more lines on their requirement for clients to handle challenge wasn’t a puzzle but Gemfile than lines of actual code hundreds of clients at once. something closer to the work they wrote. Worse, way too often But we’ve also done some- one would perform there. people would just get an Adobe thing else. Something that SoundCloud wanted me to build Flash uploader from a random would improve the candidate’s an uploader, from the user Flash component website and experience by making sure that interface down to any back-end not even write a single line of their code fulfils the functional layers I thought I’d need. code. As pragmatic as a plugin requirements before they ex- There was one particularly plus minimal glue code might pose it to us. Something that challenging part: how to imple- be, this wasn’t a good use of has saved us a lot of time by not ment a real-time-ish progress the candidate’s time or mine – I having to review code that obvi- bar. As brain-dead as I was needed them to write some code ously didn’t even work. after eight hours at my project, we could read before inviting Within the problem descrip- I found myself thinking about them for more interviews. tion sent to candidates, we in- possible solutions whilst on the And then we came up with cluded a functional test suíte. It train from East Croydon back to a different challenge. This consisted of a binary that when London, and immediately head- time, we would be a bit strict- started would try to connect to ing to my favourite late-night er. We would continue to not the candidate’s server imple- coffee shop in Old Street to limit whatever programming mentation, open lots of sockets, work on it. Solving the problem language the candidate used sends lots of messages, and ver- was a lot of fun, and if you want (just like with ThoughtWorks, ify the results against what the

I dreaded having to go through whiteboard coding sessions on some useless puzzle.

50 hacker bits Flávio Brasil ... not only found a bug in our test harness, but decompiled the (Scala!) code, wrote a patch and submitted it as part of his solution to the challenge.

problem description stated. The or pairing sessions. This time, we are asking candidates not to candidate was instructed only we will be giving the candidate add any personally identifiable to send their submission once an opportunity to talk about information to the submission. it passed the functional test on the code they wrote, instead of We will let reviewers know their local box. some generic question already roughly at which level the can- The best submission ever catalogued in a best-selling didate is and how many years was definitely from Flávio Brasil, book. of experience they have, but we now at Twitter, who not only Another change is that will not disclose gender, name, found a bug in our test harness we give more emphasis to nationality, geographic location, but decompiled the (Scala!) path-to-production, or in how or which schools or companies code, wrote a patch and submit- the application is tested, built, they have been at. This required ted it as part of his solution to and executed. Irrespective of a lot of work from our incredibly the challenge. We received code how product-centric we are, awesome recruiting team, and a submissions written in every DigitalOcean is an infrastruc- lot of patience from our review- language under the sun. We had ture company and we break the ers, but preliminary results were so much qualitative data, I even fourth wall all the time during quite good. gave a talk on some particular application development. Show- I am super excited about this problems we found in code sub- ing interest in build and runtime first iteration of the new pro- missions using Node.js. tools is a good indicator of cess, and more than just finding For this iteration of culture fit. the right people to grow our our recruitment process at But the most significant team, I hope we have enough DigitalOcean, I am trying to use change is how the code is sent data to share with the industry as leverage these past expe- to review. In previous jobs, as at some point in the near fu- riences as much as possible. a reviewer, I would receive the ture.  Overall the code challenge is code and the candidate’s ré- very similar to the one we devel- sumé. Over the past few years, oped at SoundCloud, but there the industry and academia have are some stark differences. built compelling evidence that The first major change is our prejudice influences our that we are trying a simplified decisions when it comes to interviewing process, closer to deciding what good or bad code ThoughtWorks’. Previously we is. To help keep our bias at bay, had some ad-hoc whiteboarding

Reprinted with permission of the original author. First appeared at philcalcado.com.

hacker bits 51 Interesting

I tried to virtually stalk Mark Zuckerberg

By ALEX KRAS

Part 1 – A naive dream time. A lot of people love to hate on Mark, but I actually like the guy. Sure he got lucky in life, but The dream at the same time he is doing something right for In late 2015, I finished reading Automate the Bor- Facebook to continue to stay successful. ing Stuff with Python and was very inspired to try In any case, one day I had a “brilliant” idea. to automate something in my life. What if I wrote a script that would follow Mark’s At the same time, I have always been fasci- public posts and send me a text whenever he post- nated by Mark Zuckerberg – the Bill Gates of our ed something new? Then I can be the first guy to comment on his post.

52 hacker bits After a while Mark would notice my comments notification when I was driving. By now the post and would begin to wonder “Who is this guy that is already had thousands of comments… always posting meaningful responses to all of my Oh well, I thought, there is always next time. posts?” Then he would invite my wife and I to his Sure enough, within a day I had another text. This house for dinner and our kids will become friends. time it was within 1 minute from the original post. :) I quickly opened the link, only to discover that So, without further ado I got to work. Mark’s post already had close to 100 comments. I’ve briefly considered using Facebook APIs Now don’t get me wrong, I am not stupid. I to get notified on Mark’s posts. I’ve had mixed knew that Mark’s posts were popular and would experience with APIs in the past, hitting rate limits get a lot of comments. pretty quickly and other related problems. Plus, I I even tried to estimate, the rate at which wanted to use my Automate the Boring Stuff with people were posting replies. I’ve looked through Python knowledge. Mark’s older posts and saw some posts with tens So I went the other route and wrote a Selenium of thousands of comments. So if you take 10,000

What if I wrote a script that would follow Mark’s public posts and send me a text whenever he posted something new?

script (which was really easy to do using selenium comments and divide by 24 hours, then divide by module in Python) that would: 60 minutes, you get about 7 posts per minute. What I didn’t realize in my estimate is that 1. Log in to Facebook. those comments were not evenly distributed in 2. Use current timestamp as the time of last time and that I had a very small chance of being post. the first to comment. 3. Keep checking every 60 seconds if Mark has I knew that I was losing my dream and I con- posted a new post. sidered my options. :) 4. Send me a Text using Twilio API, with a link I could set my script to run more often than to the new post. every 60 seconds, to give myself an early warning. By doing so I would risk showing up on Face- I happen to own a small server, so I set the book’s radar as a spammer and it just didn’t feel script to run indefinitely in a headless browser right for me to bombard their servers. (PhantomJS) and began to wait. Another option that I considered was to try to make an automated reply, in order to be one Paradise lost of the first people to comment. This approach, It took a couple of days for Mark to post some- however, would defeat the purpose of saying thing and I began to get worried that my script something meaningful and would not help me to did not work. become friends with Mark. At some point I had to go to the post office. I’ve decided against both of these ideas and Afterwards, I drove back home, parked my car, admitted my defeat. I’ve also realized that I could checked my phone and saw a new SMS text from turn this (failed) experiment into an interesting my script. My heart started to beat really fast and Data Exploration project. I rushed to open the link. I soon realized that the post took place 5 minutes ago and I’d missed the

hacker bits 53 Part 2 –Data analysis

Scraping Having made a big error in my estimate of the rate at which people were replying to Mark, I was curious to explore when and what people were saying. In order to do that, I needed a data set of comments. Without putting much thought into it, I decid- ed to scrape one of Mark’s most recent posts at the time. 1. Start at pagination 0. My first approach was to try to modify my 2. Make a comments request. notification script to: 3. Store it in memory. 4. Increment the pagination. 1. Log in to Facebook. 5. Sleep for a random amount of time from 5 2. Go to the post that Mark has made. to 10 seconds. 3. Click on “Show More Comments” link, until 6. Repeat the loop, until no more comments all comments were loaded. were found. 4. Scrape and parse the HTML for comments. 7. Save all of the comments to a JSON file. Once again I’d underestimated the scale of the I’ve ended up with an 18Mb minimized JSON file operation. There were just too many comments containing about 20,000 comments. (over 20,000) and it was too much for a browser to handle. Both Firefox and PhantomJS continued to crash without loading all of the comments. Analyzing the data I had to find another way. I proceeded to examine how View more com- First I looked at the distribution of comments ments requests were made using the Network over time Toolbar in Chrome Developer Tools. Chrome al- As can be seen in the two plots, it looked a lot like lows you to right click on any request and to copy exponential decay, with most of the comments it as CURL via “Copy as cURL” option. being made in the first two hours. I ran the resulting CURL command in my ter- The first 1,500 comments were made within minal, and it returned some JSON. BINGO! first 10 minutes. No wonder I had a hard time At that point all I had to do was to figure out making it to the top. how pagination of comments was done. This turned out to be a simple variable in the query of Next I wanted to see what people were saying the request, which acted as a pointer to the next I created a word cloud of the most commonly set of comments to fetch. used keywords in comments using a python li- I’ve converted the CURL command to a Python brary called (surprise, surprise) – Word Cloud. request code via this online tool. Looking at the word cloud, I realized that I After that I wrote a script that would:

54 hacker bits might have picked the wrong day to do this exper- iment. Most of the people responded in kind to Mark’s wishes of Merry Christmas and Happy New Year. That was great news for Mark, but kind of boring from the data exploration stand point.

Digging deeper After I finished the word cloud, I'd spent WAY TOO MUCH TIME trying to gain a deeper understanding of the data. The data set turned out to be a bit too big for me to iterate on quickly and all of the positive comments created too much noise. I decided to narrow down the data set by  Number of comments during removing all comments with any of the following first 2 hours word stems. A word stem is simply a shortest version of the word that still makes sense. For example, by removing comments that have a word “thank” in it, I was able to remove both the com- ments with the words “thank you” as well as the comments with the word “thanks.” I’ve used nltk library to help me convert my words to stems. I organized the stems by a type of comment that they usually belonged to:

Happy New Year Wishes • new • happ • year • wish • bless  Number of comments during • congrat first 24 hours • good luck • same • best • hope • you too

Comment on Photo of the Family • photo • baby • babi • beautiful • pic • max • family • famy • cute • child • love • nice • daughter  Comment word cloud • sweet

hacker bits 55 Thanking Mark for creating Facebook intersection = set(first) & set(second) • thank • connect # Words both comments have in common • help intersectionLength = len(intersection)

After removing all of the typical comments, I end- # Total length of both comments ed up with 2887 “unusual” comments. wordCount = len(first) + len(second)

Digging even deeper # Corner case I also recently finished reading Data Smart, from if wordCount == 0: which I learned that Network Analysis can be used return 0 to identify various data points that belong togeth- else: er, also known as clusters. # Intersection score between two comments One of the examples in the book used Gephi return (intersectionLength/wordCount) – an amazing software that makes cluster analysis very easy and fun. I wanted to analyze the “un- So for our example above: usual” comments in Gephi, but first I had to find a 1. Intersection between two comments is way to represent them as a Network. ["mark", "love"] which is 2 words. 2. Total length of both comments is 5 words. In order to do that, I’ve: 3. Intersection score is 2/5 = 0.4. 1. Removed meaningless words such as “and” or “or” (also known as stop words) from ev- Note: I could have used average length of two ery comment using nltk library. comments (so 2+3/2 = 2.5) instead of total length 2. Broke remaining words in every comment (5), but it would not have made any difference into an array (list) of word stems. since the score was calculated similarly for all of 3. For every comment calculated an intersec- the comments. So I decided to keep it simple. tion with every other comment. 4. Recorder a score for every possible intersec- Once I had all of intersection calculated, I tion. saved all comments in the nodes.csv file, that had 5. Removed all intersections with a score of 0.3 the following format: or less. 6. Saved all comments as nodes in Gephi graph Id;Label and every intersection score as an undirect- 1;Mark, I love you ed edge. I’ve saved all intersection in the edges.csv file, By now you might be wondering how the that had the following format: intersection score was calculated. You may also wonder what the heck is Gephi graph, but I’ll get Source;Target;Weight;Type to it a bit later. 1;2;0.4;"Undirected" Calculating intersection score Analyzing the network Let say we have two comments: This was all that was needed to import my data into Gephi as a Network Graph. You can read ["mark", "love"] # From "Mark, I love you" more about Gephi file formats here and this video # and provides a good introduction to Gephi and how it ["mark", "love", "more"] # From "Mark, I love you can be used. more" Once I imported my data into Gephi, I ran a network analysis algorithm called “Force Atlas 2”, We can find the score as follows: which resulted in the following network graph. I’ve manually added the text in red to sum- def findIntersection(first, second): marize some of the clusters. If you click on the # Find a sub set of words that is present image, you will be taken to a full screen represen- # in both lists tation of the graph. It is pretty big, so you might

56 hacker bits have to zoom out and scroll for a while before you ably too high and caused the error. On the flip see some data. side, it has helped to reduce the number of edges, focusing on the most important connections. Some notes on the results Please let me know in the comments if you find anything else noteworthy or if you have suggestions on how intersection scores can be improved.

Conclusion It is hard being a celebrity. I started this journey naively assuming that I can get Mark’s attention by simply posting a com- ment on his timeline. I did not realize the amount of social media attention an average celebrity gets. It would probably take a dedicated data scien- tist working fulltime just to get insight into all of the comments that Mark receives. While Mark can afford to hire such a person, my bet is that he is using his resources for more meaningful things. That being said, this has been a great learning experience for me. Gephi is a magical tool, and I I was really happy to see my approach finally highly recommend checking it out. working (after many days of trying). If you want some inspiration for automating I have been staring at those comments for a things, I highly recommend reading Automate the long time and I’ve seen some references to “mon- Boring Stuff with Python. ey”. Therefore, I was not surprised to see a couple If you are looking for a good entry-level text of clusters asking Mark “Where is my Money?” on data science, I found Data Smart to be an infor- I was very surprised, however, to see a cluster mative read, although hard to follow at times. of comments mentioning a specific number – 4.5 Also note that I’ve destroyed all of my data million to be exact. I had no idea where this num- sets to comply as best as I can with Facebook’s ber was coming from, but a quick Google search Terms of Service. Scraping content without per- pointed me to this hoax. Turns out a lot of people mission is also against Facebook’s Terms of Ser- were duped into believing that Mark would give vice, but I’ve avoided thinking about it until after away 4.5 million to 1000 lucky people. All you had I’ve done all of my analysis. to do was to post a “Thank you” message of sorts. I am hoping that Facebook will overlook my Other than that, I didn’t see anything very transgression, but I wanted to make sure I don’t interesting. There were some spammers and some send anybody else down the wrong path without a people asking Mark to ban some people from proper warning. Facebook. Some aggression towards Mark and a If all else fails, you can always follow me on lot more of the general types of comments that I Twitter. :)  did not filter out. I’ve also noticed some weaknesses in my approach. For example, there were two clusters around the word “precious”. It was probably caused by removing relationships that did not have intersection score of at least 0.3. Since I did not use the average length for two comments, the threshold of 0.3 really meant that the two com- ments were at least 60% similar, and it was prob-

Reprinted with permission of the original author. First appeared at alexkras.com.

hacker bits 57 food bit * Ketchup’s fishy origins…

elieve it or not, this ubiquitous sauce was once made from pickled fish parts. Ketchup began life as a Southern Chinese Bcondiment made from fish, salt and spices thousands of years ago. The sauce was a hit with 18th century British sailors, who brought it to Europe as a prized delicacy. British cooks soon came up with their own version of ketchup, which included savory ingredients like oysters, mushrooms and nuts. After countless it- eration, ketchup eventually evolved into the tomato-based version that we know and love.

* FOOD BIT is where we, enthusiasts of all edibles, sneak in a fun fact about food.

HACKER BITS is the monthly magazine that gives you the hottest technology and startup stories crowdsourced by the readers of Hacker News. We select from the top voted stories for you and publish them in an easy-to-read magazine format.

Get HACKER BITS delivered to your inbox every month! For more, visit hackerbits.com.