Issue 35 April 2013
The Absolute Beginner’s Guide to Arduino Engineers rebuilding the infrastructure that powers finance. careers.addepar.com package DDG::Goodie::Unidecode;# ABSTRACT: return an ASCII version of the search query use DDG::Goodie; use Text::Unidecode; zci is_cached => 1; zci answer_type => "convert to ascii"; triggers startend => "unidecode"; handle remainder => sub { my $u = unidecode $_; # unidecode output some- times contains trailing spaces $u =~ s/\s+$//; return $u; }; 1; package DDG::Goodie::Rot13; # ABSTRACT: Rotate chars by 13 letters use DDG::Goodie; triggers start => 'rot13'; handle remainder => sub { if ($_) { $_ =~ tr[a-zA-Z][n-za-mN-ZA-M]; return "ROT13: $_"; }; return }; zci is_cached => 1; 1; package DDG::Goodie::Base64; use DDG::Goodie; use MIME::Base64; use Encode; triggers startend => "base64"; zci answer_type => "base64_conversion"; zci is_cached => 1; handle remainder => sub { return unless $_ =~ /^(encode|decode|)\s*(.*)$/i; my $command = $1 || ''; my $str = $2 || ''; if ($str) { if ( $command && $command eq 'decode' ) { $str = decode_base64($str); $str = decode( "UTF-8", $str ); return "Base64 decoded: $str"; } else { $str = encode_base64( encode( "UTF-8", $str ) ); return "Base64 encoded: $str"; } } return; }; 1; package DDG::Goodie::Chars; # ABSTRACT: Give the number of charac- ters (length) of the query. use DDG::Goodie; triggers start => 'chars'; zci is_cached => 1; zci answer_type => "chars"; handle remainder => sub { return "Chars: " .length $_ if $_; return; }; 1; pack- age DDG::Goodie::ABC; use DDG::Goodie; triggers any => "or"; zci answer_type => "rand"; handle query_parts => sub { my @choices; my @collected_parts; while (my $part = shift) { if ( lc($part) eq 'or' ) { return unless @collected_parts; push @choices, join(' ', @collected_parts); my $length = @collected_parts; return if $length > 1; @collected_parts = (); } elsif ( $part ) { push @collected_parts, $part; } } push @choices, join(' ', @collected_parts) if @choices && @collected_parts; return if scalar(@choices) <= 1; my $choice = int(rand(@choices)); if (my @duck = grep { $_ eq 'duckduckgo' || $_ eq 'duck' || $_ eq 'ddg' } @choices) { return $duck[0]." (not random)", answer_type => 'egg'; } return $choices[$choice]." (random)"; return; }; 1; package DDG::Goodie::PublicDNS; use DDG::Goodie; use
Now you can hack on DuckDuckGo DuckDuckHack
Create instant answer plugins for DuckDuckGo
duckduckhack.com
3 Curator Hacker Monthly is the print magazine version Lim Cheng Soon of Hacker News — news.ycombinator.com, a social news website wildly popular among programmers and startup Contributors founders. The submission guidelines state that content Martin Legeer can be “anything that gratifies one’s intellectual curios- Andrew Chalkley ity.” Every month, we select from the top voted articles Grant Mathews on Hacker News and print them in magazine format. Bryan Kennedy For more, visit hackermonthly.com Patrick Wyatt Pete Keen Craig Kerstiens Amber Feng Advertising Published by Alex Baldwin [email protected] Netizens Media John Biesnecker 46, Taylor Road, Rachel Kroll 11600 Penang, Contact Malaysia. Proofreaders [email protected] Emily Griffin Sigmarie Soto
Ebook Conversion Ashish Kumar Jha
Printer MagCloud
Cover Photo: Beraldo Leal [flickr.com/photos/beraldoleal/6297076850]
Hacker Monthly is published by Netizens Media and not affiliated with Y Combinator in any way.
4 Contents
FEATURES 06 How I Created a Matrix Bullet Time-Style Rig By Martin Legeer
12 The Absolute Beginner’s Guide to Arduino By Andrew Chalkley
PROGRAMMING SPECIAL
16 From AS3 to Haxe 36 Goldeneye 64’s Inspirational Startup Story By Grant Mathews By Alex Baldwin
19 My First 5 Minutes On A Server 37 The Joys of Having a Forever Project By Bryan Kennedy By John Biesnecker
22 Whose Bug Is This Anyway? 38 Avoiding “The Stupid Hour” By Patrick Wyatt By Rachel Kroll
28 How I Run My Own DNS Servers By Pete Keen
30 How I Work With Postgres By Craig Kerstiens
34 Building Stripe’s API By Amber Feng
Arduino Photo: Pete Prodoehl [flickr.com/photos/raster/5933659066/]
For links to Hacker News dicussions, visit hackermonthly.com/issue-35
5 FEATURES
How I Created a Matrix Bullet Time-Style Rig
By Martin Legeer
6 FEATURES ack in March, a client for laugh for a little while. Then I asked This was a bit funny because the whom I’ve done some light if he was serious. I saw from the only thing I know that makes con- Bconsulting work asked me look on his face that he really did trast (light-wise, not color-wise) is a if it was possible to capture a 360- want me to build the rig there, so shadow. But sure, let’s make history. degree image that can be rotated my next reaction was, “I have to Another request he made was afterwards. I said of course, but I think about it.” to make it as easy as possible for didn’t think that much about the Don’t get me wrong, I’m up for further retouches, since he planned consequences — it’s a project that any badness, but this was beyond to use the rig extensively. would wake me up at night for the crazy and bananas. I took a few days to think about next few months. The next day, we sat down in what direction I would go with Everything was fine until the a restaurant and started talking the project — things like lighting moment he showed me the room I about this a little more — what his the subject, the background, the was supposed to make the rig in — vision was, what look he wanted to fluency between the photos — the it was his villa. achieve, and so on. What I ended distance between the cameras, This type of project is generally up with was that he would like to subject, camera count — just about done in warehouse-sized spaces, have a shadowless photo on a white everything. I’ve seen quite a few and there are good reasons for that background with maybe some results from “bullet time” systems (lighting being one of them). Well, contrast in there, but definitely all around the world, so I knew this was a garage about 130-square- shadowless. what things and results I would like feet in area and a ceiling about 8 to avoid, especially with my white feet tall. My first reaction was to background.
7 I came up with some ideas, but the outside with big octas around. light would bounce around like just to be sure, I called my friend However, we moved on from this crazy (eliminating shadows). Daniel over (he’s pretty much at idea because there were so many We started putting together the the same technical level as I am) to issues we had to address (possible precise lighting setup. We wanted to have somebody to bounce ideas off. flares, the quality of the image due give the client the option of using From that moment on, the concept to the lighting, how to spread the other styles of lighting for non- just grew. light evenly from the close distance, shadowless photos, so we arranged The next day, I called the client tension of the diffusing textile, the 6 lights on the ceiling in a circular and accepted the challenge. I asked possibility of someone damaging formation. for an advance before putting the textile, etc.). For those of you wondering everything together, making phone I called another colleague and about the brand of the lights, I calls, etc. just to be sure there we came up with the idea of normally shoot with Profoto, but would be no trouble afterwards. I making a cylinder, which we could I chose to go with Elinchrom this got the advance the very same day paint from the inside and repaint time (to save some money). Elin- (to my surprise) and then we got if necessary in the future. The only chrom satisfies the requirements for started. problem we had to solve with professional use, since it has such a Since we had so little space to this new design was how to light consistent amount of light, consis- make the rig on (mainly due to the it from within. Since the request tent color temperature, etc. low ceiling), my initial idea was to was to have a subject on a white put up some kind of a diffusing tent background, we decided to build around the subject and light it from it completely in white so that the
8 FEATURES So that was step one. Step two the cameras, and there is currently once, it has a slight delay between was putting together a list of no software on the market (that I the cameras — like 1/5s between the gear we needed to buy. You am aware of) that can do the job (at each camera. wouldn’t believe how much trouble least at the time we did this proj- This was a big issue for us it was to simply find a seller who ect). Maybe there is, but I couldn’t since we needed to sync the flash was able to send us 50 cameras find it. between the 50 cameras as well (we and prime lenses to the middle There were potential programs can’t light it with other types of of Europe. It was unbelievable. If we could use, but they were written lights, lest we fry the person in the it were Canon 5D MKIIIs, Nikon for Linux. I also could have had cylinder). D800s, or something of that class, programmers around me write the Just like Leonardo DiCaprio in that would be understandable, but software, but that would have taken Inception, we had to go deeper. Canon 600Ds? Come on! weeks or months, which we didn’t Finally, we reached the best and We chose the cheaper Canon have. simplest solution. Since each DSLRs over other brands simply We eventually found a single camera can be fired via a cable due to budget constraints. program for Canon DSLRs, which trigger, we created a net of cables I originally wanted to go for turned out to be a huge waste of that does just that — triggers all the Nikon (since it’s the brand I shoot money. The trial version had limited cameras at once. Two buttons are with), but we had trouble solving options, so we had to buy the full located at the end of the net: the certain issues with Nikon techni- software. After we did, we found focus/wakeup and trigger buttons. cians. We had to fire all 50 cameras out that although the software is Voila! Firing problem solved. at once and transfer all the data off able to trigger all the cameras at
9 The software allows us to download the RAW images onto computers (there has to be 4 laptops since the software can operate only up to 14 or 16 cameras and there are also bandwidth issues). That’s pretty much the entire build process. Of course, I simplified quite a few things to not bore you with my thoughts throughout the whole project, but all in all, I hope you enjoyed the ride. I know I did, and I would love to do it again — perhaps in a larger space. The last thing I have to say will be a disap- pointment to many of you: I don’t have any resulting images to share with you due to the client’s request to not share any sample photos outside. Perhaps in the future some of his photos will begin to appear on the web. n
Martin Legeer is a photographer who went from shooting events for companies like AXE, RedBull, Ferrero to commercial photography. He is trying to Reprinted with permission of the original author. make his mark in fashion industry and taking some First appeared in hn.my/bullet (petapixel.com) challenges along the way.
10 FEATURES Accept payments online. The Absolute Beginner’s Guide to Arduino
By Andrew Chalkley
ver the Christmas designers, hobbyists, and anyone sensor, a distance sensor, a switch, break from work I interested in creating interactive and so forth. wanted to learn some- objects or environments. O Outputs thing new. Source: arduino.cc Some examples of outputs would I’ve been eyeing up Arduino for Microcontroller be a light, a screen, a motor and so some time now, and for Christmas I Arduino is a microcontroller on a forth. got an Arduino UNO R3 board. circuit board which makes it easy to TL;DR receive inputs and drive outputs. What is Arduino? Arduino is a small computer that A microcontroller is an integrated you can program to read and Arduino is an open-source electron- computer on a chip. control electrical components con- ics prototyping platform based on Inputs nected to it. flexible, easy-to-use hardware and Some examples of inputs would software. It is intended for artists, be a temperature sensor, a motion
12 FEATURES Obtaining an Arduino Board the default program stored on the The Code There are several online distributors chip. We’re going to override this. The code you write for your Ardu- that stock Arduino boards. The USB cable powers the ino are known as sketches. They are Often boards are bundled up device. Arduinos can run stand- written in C++. with starter kits. Kits include a wide alone by using a power supply in Every sketch needs two void type variety of inputs, outputs, resistors, the bottom left of the board. Once functions, setup() and loop(). A wires and breadboards. Breadboards you’re done programming and void type function doesn’t return are solderless circuit prototyping don’t require it to be constantly any value. boards that you can plug wires and connected to your machine, you The setup() method is run once components into. can opt to power it separately. This just after the Arduino is powered Arduinos come in different fla- is entirely dependant on the use up and the loop() method runs vors. Most people starting off go for case and circumstances you want to continuously afterwards. The the UNO board. Its current revision use the device in. setup() is where you want to is the third, hence the R3 listed by do any initialization steps, and in stockists. Download Arduino Software loop() you run the code you want Most enthusiasts use sites like You’ll need to download the to run over and over again. Adafruit [adafruit.com] and Ele- Arduino Software package for your So, your basic sketch or program ment14 [element14.com]. operating system from the Arduino should look like this: You can even pick one up from download page [hn.my/adl]. void setup() your local RadioShack. When you’ve downloaded and { If you’re just getting a single opened the application you should
Arduino board or starter kit, be see something like this: } sure you have a USB A-to-B cable.
Most, if not all, starter kits come void loop() with the USB A-to-B cable. Most { printers have this type of interface so you may have this cable already } lying around. You need the cable to program the device, so it’s best to Now that we have the basic skel- double check when ordering. eton in place, we can do the Hello, World program of microcontrollers, Programming Arduino a blinking an LED. For the example I’m showing, you’ll Headers and Pins only need the Arduino UNO R3 If you notice on the top edge of the board itself and the required USB board there’s two black rectangles cable to transfer the program from with several squares in it. These your computer to the board. are called headers. Headers make On the board left of the Arduino it easy to connect components to logo there’s an LED, short for Light This is where you type the code the Arduino. The places where they Emitting Diode, a small light, with you want to compile and send to connect to the board are called the letter L next to it. the Arduino board. pins. Knowing what pin something We’re going to switch it on and The Initial Setup is connected to is essential for pro- off and then look into making it We need to setup the environment gramming an Arduino. blink on and off for 2 seconds at a to Tools menu and select Board. The pin numbers are listed next time. Then select the type of Arduino to the headers on the board in When you first plug your USB you want to program, in our case white. cable into your Arduino and your it’s the Arduino Uno. The onboard LED we want to computer, you may notice that this control is on pin 13. LED is blinking. Not to worry! It’s
13 In our code above the setup() We do this by calling The tty and cu are two ways method let’s create a variable called another special method called that computers can talk over a serial ledPin. In C++ we need to state digitalWrite(). This also takes port. Both seem to work with the what type our variable is before- two values, the pin number and the Arduino software so I selected the hand, in this case it’s an integer, so level, HIGH or the on state or LOW tty.* one. On Windows you should it’s of type int. the off state. see COM followed by a number. Select the new one that appears. int ledPin = 13; int ledPin = 13; Once you have selected your
serial or COM port, you can then void setup() void setup() press the button with the arrow { { pointing to the right. pinMode(ledPin, OUTPUT); } } void loop() void loop() { { digitalWrite(ledPin, LOW); } } Each line ends with a semicolon Next we want to compile to (;). machine code and deploy or upload In the setup() method, we want it to the Arduino. to set the ledPin to the output Compiling the Code mode. We do this by calling a If this is your first time you’ve ever special function called pinMode() compiled code to your Arduino, which takes two variables, the before plugging it in to the com- first the pin number, and second, puter, go to the Tools menu, then whether it’s an input or output pin. Once that happens you should Serial Port and take note of what Since we’re dealing with an output, see the TX and RX LEDs below appears there. we need to set it to a constant the L LED flash.T his is the com- Here’s what mine looks like called OUTPUT. If you were working munication going on between the before plugging in the Arduino with a sensor or input it would be computer and the Arduino. The L UNO: INPUT. may flicker, too. Once this dance is complete, your program should be int ledPin = 13; running, and your LED should be
off. void setup() Now let’s try to switch it on { using the HIGH constant. pinMode(ledPin, OUTPUT); Plug your Arduino UNO board } into the USB cable and into your int ledPin = 13; computer. Now go back to the void loop() Tools > Serial Port menu and you void setup() { should see at least 1 new option. { On my Mac 2 new serial ports pinMode(ledPin, OUTPUT); } appear. }
In our loop we are going to first void loop() switch off the LED to make sure { our program is being transferred to digitalWrite(ledPin, HIGH); the chip and overriding the default. }
14 FEATURES Press Upload again, and you should see your LED is now on! Let’s make this a little more interesting now. We’re going to use another method called delay(), which takes an integer of a time inter- val in milliseconds, meaning the integer of 1000 is 1 second. So after we switch the LED on, let’s add delay(2000), which is two seconds, then digitalWrite(ledPin, LOW) to switch it off and delay(2000) again. int ledPin = 13; void setup() { pinMode(ledPin, OUTPUT); } void loop() { digitalWrite(ledPin, HIGH); delay(2000); digitalWrite(ledPin, LOW); delay(2000); } Press Upload and you should see your LED blinking!
What next? The Arduino platform is an incredibly easy and versatile platform to get started with. It’s open-source hardware, meaning that people can collaborate to improve, remix and build on it. It’s the brains to some of the most popular devices that are driving the next Industrial Revolution, the 3D printer. [makerbot.com] And as Massimo Banzi says, “You don’t need anybody’s permission to create something great.” So what you waiting for? n
Andrew Chalkley is an Expert Teacher at Treehouse, Co-founder of iOS app development company Secret Monkey Science and technical writer on Screencasts.org. In his spare time he hacks around with hardware such as Arduino, Raspberry Pi and Kinect.
Reprinted with permission of the original author. First appeared in hn.my/arduino (forefront.io)
15 PROGRAMMING
From AS3 to Haxe
By Grant Mathews
recently converted a codebase of about 5000 Language differences lines from ActionScript 3 to Haxe [haxe.org]. Stronger type system. Here’s what I learned. I ➊ Generics This is a huge, huge win for Haxe. If you’ve used AS3, Initial impressions: you might be familiar with how they have a parameter- Haxe compiles really fast. ized Vector.
The AS3 target is solid. myObject[1] = "hi"; ➌ Through the entire translation phase, I didn’t myObject["somekey"] = 4.3; encounter a single Haxe bug. Debugging was a breeze Obviously if you do a loop through that object, you because the backtrace given was relative to the Haxe couldn’t specify the type of the key. Haxe gets around files (as you should expect). this by splitting Object into two types that encompass all of its expected functionality.
16 PROGRAMMING The first isTypedDictionary
17 Using Overriding setters/getters is tricky. ➏ The using keyword allows you to add additional ➋ Essentially, you can override a variable setter and methods onto existing types. The classic example of getter, but only if you know the name of the functions using is the Lambda class. The Lambda class has a bunch you’re overriding (which rules out extending some of static methods on it. We’ll use Lambda.exists as an built-ins), and you’re not permitted to use super to example. The definition looks like this: access the parent’s property. From what I understand, these limitations stem from problems with target static function exists
of Lisp, of course). Haxe is impressive. var myArray:Array
with using NME, which is admittedly a bit shakier than if (myArray.exists(is3)) { using the AS3 libraries, but it’s there, and it’s exciting. trace("I found a 3 in the array!"); I no longer feel nervous about the world moving to } HTML5. Nicolas Cannasse and the Haxe team move Nice, huh? incredibly fast. Just the other day I noticed they were writing a Haxe shader language and a set of generic Problems 3D bindings that will interoperate between Flash’s Stage3D, HTML5’s WebGL, and more. Wow. No cross-platform Dictionary type. I have to feel like one of the big reasons that Haxe ➊ The AS3 target has TypedDictionary, but sadly hasn’t seen more widespread attention is that it’s not it doesn’t exist on all platforms. The NME target has English. The documentation is full of imprecise word- ObjectHash, but the problem with ObjectHash is that it ing that feels amateur. (In fact, I spent some time clean- can’t have primitive types (Int,String, Float, Bool) ing it up the other day.) It’s easy to draw the conclu- as keys. sion that the language is like the docs — mismatched To solve this problem, I wrote SuperObjectHash.hx and awkward — but it’s not. [hn.my/soh] which combines ObjectHash and Hash Check it out. The possibilities are wild. n into a single interface that you can use without having to worry about having primitive typed values. Grant Mathews is a 22-year-old senior currently attending Stan- (It was pointed out on #haxe that ObjectHash is ford University. He wants to prove that games can be art, and planned to be introduced to Haxe, and will make it in invent the tools to make it happen. by Haxe 3. Then my SuperObjectHash won’t even be necessary.) Reprinted with permission of the original author. First appeared in hn.my/haxe (grantmathews.com)
18 PROGRAMMING My First 5 Minutes On A Server Essential Security for Linux Servers
By Bryan Kennedy
erver security doesn’t need Our servers are configured with Let’s Get Started to be complicated. My secu- two accounts: root and deploy. The Our box is freshly hatched, virgin Srity philosophy is simple: deploy user has sudo access via an pixels at the prompt. I favor adopt principles that will protect arbitrarily long password and is the Ubuntu; if you use another version you from the most frequent attack account that developers log into. of linux, your commands may vary. vectors, while keeping administra- Developers log in with their public Five minutes to go: tion efficient enough that you won’t keys, not passwords, so administra- passwd develop “security cruft.” If you use tion is as simple as keeping the your first 5 minutes on a server authorized_keys file up-to-date Change the root password to wisely, I believe you can do that. across servers. Root login over ssh something long and complex. You Any seasoned sysadmin can is disabled, and the deploy user can won’t need to remember it, just tell you that as you grow and add only log in from our office IP block. store it somewhere secure. This more servers and developers, user The downside to our approach is password will only be needed if you administration inevitably becomes that if an authorized_keys file gets lose the ability to log in over ssh or a burden. Maintaining conventional clobbered or mis-permissioned, I lose your sudo password. access grants in the environment of need to log into the remote termi- apt-get update a fast-growing startup is an uphill nal to fix it (Linode offers some- apt-get upgrade battle— you’re bound to end up thing called Lish, which runs in the with stale passwords, abandoned browser). If you take appropriate The above gets us started on the intern accounts, and a myriad of caution, you shouldn’t need to do right foot. “I have sudo access to Server A, this. but not Server B” issues. There are Note: I’m not advocating this as the account sync tools to help mitigate most secure approach— just that it this pain, but IMHO the incremen- balances security and management tal benefit isn’t worth the time nor simplicity for our small team. From the security downsides. Simplicity is my experience, most security breaches the heart of good security. are caused either by insufficient security procedures or sufficient proce- dures poorly maintained.
19 Install Fail2ban Lock Down SSH Configure ssh to prevent password and root logins and apt-get install fail2ban lock ssh to particular IPs: Fail2ban is a daemon that monitors login attempts to vim /etc/ssh/sshd_config a server and blocks suspicious activity as it occurs. It’s well configured out of the box. Add these lines to the file, inserting the IP address Now, let’s set up your login user. Feel free to name from where you will be connecting: the user something besides “deploy”, it’s just a conven- PermitRootLogin no tion we use: PasswordAuthentication no useradd deploy AllowUsers deploy@(your-ip) deploy@ mkdir /home/deploy (another-ip-if-any) mkdir /home/deploy/.ssh Now restart ssh: chmod 700 /home/deploy/.ssh service ssh restart Require Public Key Authentication The days of passwords are over. You’ll enhance security Setup A Firewall and ease of use in one fell swoop by ditching those No secure server is complete without a firewall. passwords and employing public key authentication for Ubuntu provides ufw, which makes firewall manage- your user accounts. ment easy. Run: vim /home/deploy/.ssh/authorized_keys ufw allow from {your-ip} to any port 22 ufw allow 80 Add the contents of the id_rsa.pub on your local ufw allow 443 machine and any other public keys that you want to ufw enable have access to this server to this file. This sets up a basic firewall and configures the server chmod 400 /home/deploy/.ssh/authorized_keys to accept traffic over port 80 and 443.Y ou may wish to chown deploy:deploy /home/deploy -R add more ports depending on what your server is going to do. Test the New User and Enable Sudo Now test your new account logging into your new server with the deploy user (keep the terminal window with the root login open). If you’re successful, switch back to the terminal with the root user active and set a sudo password for your login user: passwd deploy Set a complex password. You can either store it somewhere secure or make it something memorable to the team. This is the password you’ll use to sudo. visudo Comment all existing user/group grant lines and add: root ALL=(ALL) ALL deploy ALL=(ALL) ALL The above grants sudo access to the deploy user when they enter the proper password.
20 PROGRAMMING Enable Automatic Security Updates I think we’re at a solid place now. In just a few min- I’ve gotten into the apt-get update/upgrade habit over utes, we’ve locked down a server and set up a level of the years, but with a dozen servers, I found that servers security that should repel most attacks while being easy I logged into less frequently weren’t staying as fresh. to maintain. At the end of the day, it’s almost always Especially with load-balanced machines, it’s impor- user error that causes break-ins, so make sure you keep tant that they all stay up to date. Automated secu- those passwords long and safe! n rity updates scare me somewhat, but not as badly as unpatched security holes. Bryan Kennedy is the Co-Founder and CTO of Sincerely, helping to scale thoughtfulness across the world. Bryan is a YCombinator apt-get install unattended-upgrades alum and an angel investor. On warm summer nights he runs vim /etc/apt/apt.conf.d/10periodic MobMov.org, a worldwide collective of guerrilla drive-ins. Update the file to look like this: Reprinted with permission of the original author. APT::Periodic::Update-Package-Lists "1"; First appeared in hn.my/5mins (plusbryan.com) APT::Periodic::Download-Upgradeable-Packages "1"; APT::Periodic::AutocleanInterval "7"; APT::Periodic::Unattended-Upgrade "1"; One more config file to edit: vim /etc/apt/apt.conf.d/50unattended-upgrades Update the file to look like below.Y ou should prob- ably keep updates disabled and stick with security updates only:
Unattended-Upgrade::Allowed-Origins { "Ubuntu lucid-security"; // "Ubuntu lucid-updates"; };
Install Logwatch To Keep An Eye On Things Logwatch [hn.my/logwatch] is a daemon that moni- tors your logs and emails them to you. This is useful for tracking and detecting intrusion. If someone were to access your server, the logs that are emailed to you will be helpful in determining what happened and when, as the logs on your server might have been compromised. apt-get install logwatch vim /etc/cron.daily/00logwatch Add this line:
/usr/sbin/logwatch --output mail --mailto test@ gmail.com --detail high All Done!
21 Whose Bug Is This Anyway?
By Patrick Wyatt
t a certain point in clear that StarCraft needed more in the fog-map, and we meant every programmer’s “resources” (a.k.a. people), I joined to include line-of-sight visibility Acareer we each find a the effort. Because I came aboard calculations so that units on higher bug that seems impossible because late I didn’t have a defined role, so terrain would be invisible to those the code is right, dammit! So it instead I just “used the force” to on lower terrain, greatly increasing must be the operating system, the figure out what needed to happen the tactical complexity of the game: tools, or the computer that’s caus- to move the project forward. when you can’t see what the enemy ing the problem. Right?!? I got to write fun features like is doing, the game is far more Today’s story is about some of implementing parts of the com- complicated. Similarly, units around those bugs I’ve discovered in my puter AI, which was largely devel- a corner would be out of sight and career. oped by Bob Fitch. One was a couldn’t be detected. system to determine the best place The new fog of war was the most This bug is Microsoft’s fault… to create “strong-points” — places enjoyable part of the project for or not that AI players would gather units me, as I needed to do some quick Several months after the launch of for defense and staging areas for learning to make the system func- Diablo in late 1995, the StarCraft attacks. I was fortunate because tional and fast. Earlier efforts by team put on the hustle and started there were already well-designed another programmer were graphi- working extra long hours to get the APIs that I could query to learn cally displeasing and moreover, game done. Since the game was which map areas were joined ran so slowly as to be unworkable. “only two months from launch,” together by the path-finding algo- I learned about texture filtering it seemed to make sense to work rithm and where concentrations of algorithms and Gouraud shading, more hours every day (and some enemy units were located in order and wrote the best x386 assem- weekends, too). There was much to to select good strong-points, as it bly language of my career — a do, because even though the team would otherwise be embarrassing to skill now almost unnecessary for started with the Warcraft II game fortify positions that could be trivi- modern game development. Like engine, almost every system needed ally bypassed by opponents. many others I hope that StarCraft is rework. All of the scheduling I re-implemented some compo- eventually open-sourced, in my case estimates were willfully wrong (my nents like the “fog of war” system so I can look with fondness on my own included), so this extra effort I had written for previous incarna- coding efforts, though perhaps my kept on for over a year. tions of the ‘Craft series. StarCraft memories are better than seeing the I wasn’t originally part of the deserved to have a better fog-of- actual code! StarCraft dev team, but after war system than its predecessor, Diablo launched, when it became Warcraft II, with finer resolution
22 PROGRAMMING The trick for effective bug-fixing is to dis- “cover how to reliably reproduce a problem.”
But my greatest contribution to the Star- if (UnitIsHarvester(unit)) Craft code was fixing defects. With so many return X; folks working extreme hours writing brand new code, the entire development process if (UnitIsFlying(unit)) { was haunted by bugs: two steps forward, one if (UnitCannotAttack(unit)) step back. While most of the team coded new return Z; features, I spent my days hunting down the return Y; problems identified by our Quality Assurance } (QA) test team. The trick for effective bug-fixing is to dis- ... many more lines cover how to reliably reproduce a problem. Once you know how to replicate a bug, it’s if (! UnitIsHarvester(unit)) // "!" means "not" possible to discover why the bug occurs, and return Q; then it’s often straightforward to fix. Unfor- tunately reproducing a “will o’ the wisp” bug return R; <<< BUG: this code is never reached! that only occasionally deigns to show up can After staring at the problem for too many hours, I guessed it take days or weeks of work. Even worse is might be a compiler bug, so I looked at the assembly language that it is difficult or impossible to determine code. beforehand how long a bug will take to fix, so For the non-programmers out there, compilers are tools long hours investigating were the order of the that take the code that programmers write and convert it into day. My terse status updates to the team were “machine code”, which are the individual instructions executed along the lines of “yeah, still looking for it.” I’d by the CPU. sit down in the morning and basically spend all day cracking on, sometimes fixing hun- // Add two numbers in C, C#, C++ or Java dreds of issues, but many times fixing none. A = B + C One day I came across some code that wasn’t working: it was supposed to choose a ; Add two numbers in 80386 assembly behavior for a game unit based on the unit’s mov eax, [B] ; move B into a register class (“harvesting unit”, “flying unit”, “ground add eax, [C] ; add C to that register unit”, etc.) and state (“active”, “disabled”, mov [A], eax ; save results into A “under attack”, “busy”, “idle”, etc.). I don’t remember the specifics after so many years, but something along the lines of this:
23 Crunch time is a failed development “methodology; developers get tired and start making stupid mistakes.”
After looking at the assembly start making stupid mistakes. It’s that kind of problem fixed quickly. code I concluded that the compiler far more effective to work reason- Fortunately we were able to “roll was generating the wrong results, able hours, go home, have a life, back” the code changes and restore and sent a bug report off to Micro- and come back fresh the next day. the previous version of the code soft — the first compiler bug report When I started ArenaNet with in short order, but now we needed I’d ever submitted. And I received two of my friends, the “no crunch” to understand how we broke the a response in short order, which in philosophy was a cornerstone of build. Like many problems in pro- retrospect is surprising: consider- our development effort, and one of gramming, it turned out that several ing that Microsoft wrote the most the reasons we didn’t buy foosball issues taken together conspired to popular compiler in the world, it’s tables and arcade machines for the cause the bug. hard to imagine that my bug report office. Work, go home at a reason- There was a compiler bug got any attention at all, much less a able time, and come back fresh! in Microsoft Visual Studio 6 quick reply! (MSVC6), which we used to build You can probably guess — it This bug is actually Microsoft’s the game. Yes! Not our fault! Well, wasn’t a bug, there was a trivial fault except that our testing failed to error I had been staring at all along Several years later, while working uncover the problem. Whoops. but didn’t notice. In my exhaustion on Guild Wars, we discovered a Under certain circumstances, the — weeks of 12+ hour days — I had catastrophic bug that caused game compiler would generate incorrect failed to see that it was impossible servers to crash on startup. Unfortu- results when processing templates. for the code to work properly. It’s nately, this bug didn’t occur in the What are templates? They’re not possible for a unit to be neither “dev” (“development”) branch that useful, but they’ll blow your mind. “a harvester” nor “not a harvester”. the programming team used for [hn.my/fqa] The Microsoft tester who wrote everyday work, nor did it occur in C++ is a complex programming back politely explained my mistake. the “stage” (“staging”) branch used language, so it is no surprise that I felt crushed and humiliated at the by the game testers for final verifi- compilers that implement the time, only slightly mitigated by the cation, it only occurred in the “live” language have their own bugs. In knowledge that the bug was now branch which our players used to fact the C++ language is far more fixable. play the game. We had “pushed” complicated than other mainstream Incidentally, this is one of the a new build out to end-users, and languages. Ruby is a complex and reasons that crunch time is a failed now none of them could play the fully-featured language, but C++ development methodology, as game! WTF! is twice as complex, so we would I’ve mentioned in past posts on Having thousands of angry play- expect it to have twice as many this blog; developers get tired and ers amps up the pressure to get bugs, all other things being equal.
24 PROGRAMMING Everyone, programmers and build “servers alike, should be running the same version of the tools! ”
When we researched the com- We wanted to make sure that the The bug was easily fixed by piler bug it turned out to be one ArenaNet and NCsoft staff didn’t upgrading the build server, but in that we already knew about, and have access to cheat functions the end we decided to leave asser- that had already been fixed by the because we wanted to create a level tions enabled even for live builds. Microsoft dev team in MSVC6 playing field for all players. Many The anticipated cost-savings in Service Pack 5 (SP5). In fact all MMO companies have had to fire CPU utilization (or more correctly, of the programmers had already folks who abused their godlike “GM” the anticipated savings from being upgraded to SP5. Sadly, though we powers, so we thought to eliminate able to purchase fewer computers had each updated our work com- that problem by removing capability. in the future) were lost due to the puters, we neglected to upgrade the A further change was to elimi- programming effort required to build server, which is the computer nate some of the “sanity checking” identify the bug, so we felt it better that gathers the code, artwork, code that’s used to validate that to avoid similar issues in future. game maps, and other assets and the game is functioning properly. Lesson learned: everyone, pro- turns them into a playable game. So This type of code, known as asserts grammers and build servers alike, while the game would run perfectly or assertions by programmers, is should be running the same version on each programmers’ computer, it used to ensure that the game state of the tools! would fail horribly when built by is proper and correct before and the build server. But only in the live after a computation. These asser- Your computer is broken branch! tions come with a cost, however: After my experience reporting a Why only in live? Hmmm…. each additional check that has to non-bug to the folks at Microsoft, Well, ideally all branches (dev, be performed takes time; with I was notably shyer about suggest- stage, live) would be identical to enough assertions embedded in ing that bugs might be caused by eliminate the opportunity for bugs the code, the game can run quite anything other than the code I or just like this one, but in fact there slowly. We had decided to disable one of my teammates wrote. were a number of differences. For a assertions in the live code to reduce During the development of start, we disabled many debugging the CPU utilization of the game Guild Wars (GW), I had occa- capabilities for the live branch that servers, but this had the unintended sion to review many bug reports were used by the programming and consequence of causing the C++ sent in from players’ computers. test teams. These capabilities could compiler to generate the incorrect As GW players may remember, in be used to create gold and items, or results which led to the game crash. the (hopefully unlikely) event that spawn monsters, or even crash the A program that doesn’t run uses a the game crashed, it would offer game. lot less CPU, but that wasn’t actu- to send the bug report back to our ally the desired result. “lab” for analysis. When we received
25 Bugs come in all manner of shapes and sizes “and some don’t have a clear owner, so several of us would take turns at fixing these bugs.”
those bug reports we triaged to to a table of known answers. He and halt computation, but parity determine who should handle each encoded this stress-test into the RAM fell out of favor in the report, but of course bugs come in main game loop so that the com- early ’90s. Some computers use all manner of shapes and sizes and puter would perform this verifica- “Error Correcting Code” (ECC) some don’t have a clear owner, so tion step about 30-50 times per memory, but because of the addi- several of us would take turns at second. tional cost, it is more commonly fixing these bugs. On a properly functioning com- found on servers rather than Periodically we’d come across puter this stress test should never desktop computers. bugs that defied belief, and we’d fail, but surprisingly we discovered ■■ Overclocking: while less common be left scratching our heads. While that on about 1% of the comput- these days, many gamers used it wasn’t impossible for the bugs ers being used to play Guild Wars, to buy lower clock rate — and to occur, and we could construct it did fail! One percent might not hence less expensive — CPUs hypothetically plausible explana- sound like a big deal, but when one for their computers, and would tions that didn’t involve redefining million gamers play the game on then increase the clock frequency the space-time continuum, they any given day that means 10,000 to improve performance. Over- just “shouldn’t” have occurred. It would have at least one crash bug. clocking a CPU from 1.8 GHz was possible they could be memory Our programming team could to 1.9 GHz might work for corruption or thread race issues, but spend weeks researching the bugs one particular chip but not for given the information we had, it for just one day at that rate! another. I’ve overclocked com- just seemed unlikely. When the stress test failed, Guild puters myself without experienc- Mike O’Brien, one of the co- Wars would alert the user by clos- ing an increase in crash-rate, but founders and a crack programmer, ing the game and launching a web some users ratchet up the clock eventually came up with the idea browser to a Hardware Failure page frequency so high as to cause that they were related to com- which detailed the several common spectacular crashes as the signals puter hardware failures rather than causes that we discovered over bouncing around inside the CPU programming failures. More impor- time: don’t show up at the right time tantly, he had the bright idea for ■■ Memory failure: in the early days or place. how to test that hypothesis, which of the IBM PC, when hardware is the mark of an excellent scientist. ■■ Inadequate power supply: many failures were more common, He wrote a module (“OsStress”) gamers purchase new comput- computers used to have “RAM which would allocate a block of ers every few years, but pur- parity bits” so that in the event memory, perform calculations in chase new graphics cards more a portion of the memory failed, that memory block, and then com- frequently. Graphics cards are the computer hardware would pare the results of the calculation an inexpensive system upgrade be able to detect the problem
26 PROGRAMMING By focusing our efforts on the bugs that were “actually our fault, the team was able to spend time creating features that players.”
which generate remarkable hard-drive in the fridge year round. Ever more bugs improvements in game graph- No further problems! I don’t think that we’ll ever reach ics quality. During the era when Once the Guild Wars tech a stage where computer programs Guild Wars was released, many support team was alerted to the don’t have bugs — the increase in of these newer graphics cards overheating issue, they had success the expectations from users is rising had substantially higher power fixing many otherwise intractable faster than the technical abilities of needs than their predecessors, crash bugs. When they received programmers. The Warcraft 1 code and in some cases a computer certain types of crash reports, they base was approximately 200,000 power supply was unable to encouraged players to create more lines of code (including in-house provide enough power when the air flow by relocating furniture, tools), whereas Guild Wars 1 even- computer was “under load,” as adding external fans, or just blow- tually grew to 6.5 million lines of happens when playing games. ing out the accumulated dust that code (including tools). Even if it’s builds up over years, and that possible to write fewer bugs per ■■ Overheating: Computers don’t solved many problems. line of code, the vast increase in the much like to be hot and mal- While implementing the com- number of lines of code means it function more frequently in puter stress test solution seems is difficult to reduce the total bug those conditions, which is why beyond the call of duty, it had a count. But we’ll keep trying. computer datacenters are usu- huge payoff: we were able to iden- To close, I wanted to share one of ally cooled to 68-72F (20-22C). tify computers that were generating my favorite tongue-in-cheek quotes Computer games try to maximize bogus bug reports and ignore their from Bob Fitch, whom I worked video frame-rate to create better crashes. When millions of people with back in my Blizzard days. He visual fidelity; that increase in play a game in any given week, posited that, “All programs can be frame-rate can cause computer even a low defect rate can result in optimized, and all programs have temperatures to spike beyond more bug reports than the program- bugs; therefore all programs can be the tolerable range, causing game ming team can field. By focusing optimized to one line that doesn’t crashes. our efforts on the bugs that were work.” And that’s why we have In college I had an external actually our fault, the programming bugs. n hard-drive on my Mac that would team was able to spend time creat- frequently malfunction during ing features that players wanted Patrick Wyatt is a lifelong programmer, spring and summer when it got too instead of triaging unfixable bugs. game developer, and game-player, and as hot. I purchased a six-foot SCSI of 2004, a parent as well. He has worked on cable that was long enough to reach many popular games including Warcraft, from my desk to the mini-fridge Diablo, Starcraft, Guild Wars and TERA. (nicknamed Julio), and kept the Reprinted with permission of the original author. First appeared in hn.my/bug (codeofhonor.com)
27 How I Run My Own DNS Servers
By Pete Keen
Photo: flickr.com/photos/zagrobot/2731084578/
or the longest time I used Step 1: The Hardware I’m not even close to exploiting zoneedit.com as my DNS I decided that if I’m going to do these two machines. I’m planning Fprovider of choice. All of this, I’m going to go all out. To on moving more and more of my my important domains were hosted that end, I rented two VPSs, one apps and sites over to them, but there, and they never really did from RamNode.com in Atlanta and right now they’re mainly handling me wrong. A few months back I another from Prgmr.com in San Jose. this site and my email and DNS. decided that I wanted to learn how Overall I would say that my Ram- Why two machines? To host your DNS actually works in the real Node experience has been more own DNS servers, the registrars world though. Like, what does it positive than my Prgmr experi- require you to list two IP addresses actually take to run my own DNS ence. The network links have gone with the idea that you’ll be pro- servers? down twice in the past six months viding redundant service. The one at Prgmr, which isn’t the end of thing you don’t want is downtime Step 0: Why would you ever do the world when you’re running with DNS; it screws everything up. that?! a redundant service, but it’s still I’m mostly motivated by curios- pretty annoying. RamNode has had ity, but also by frustration. When 100% uptime so far. something isn’t going my way it Specs on these bad boys: just starts to make sense to do it ■■ prgmr (teroknor.bugsplat.info): 1 myself. My frustration with zone- core, 1024MiB ram, 24GiB Disk, edit wasn’t anything super specific. 160GiB transfer Their dynamic DNS system wasn’t too terribly dynamic and adding ■■ ramnode (empoknor.bugsplat. and editing zones through their info): 4 core, 2048MiB ram, web interface got to be pretty 30GiB SSD-backed Disk, tedious after awhile. I have a bunch 4000GiB transfer of zones (32 at last count), most of which are very simple setups. bugsplat.info is way more compli- cated, but we’ll get into that later.
28 PROGRAMMING Step 2: The Software Step 3: The Email Conclusion Once you decide to go down this One of the other problems I had So after all of that, what have I DNS rabbit hole, there are a bunch with zoneedit was their free email learned? Mostly that I’m a very of decisions to make on the soft- forwarding setup. It was slow. So particular person with regards to ware side. I considered PowerDNS slow. Slower than molasses spread this stuff. It’s fun right now, but I and BIND and finally settled on onto the back of the slowest dog. can see it getting kind of tedious tinydns.org managed via puppet and Even before this whole DNS down the line. We’ll find out! It’s supply drop. Tinydns is a proj- adventure started I knew I wanted been an interesting ride thus far and ect started by Daniel J. Bernstein to get rid of that. I’ve learned quite a bit, which is the many years ago and has proven to Each VPS runs its copy of my most important thing. n be extremely reliable when run as Postfix [postfix.org] setup (also intended (no axfr, configuration managed via puppet), which mostly Pete Keen is a software developer currently propogation via scp, etc). My setup just forwards incoming email into residing in Portland Oregon. He writes arti- is thus: my Gmail account. I don’t send cles about a variety of technology issues through it, since I haven’t quite at bugsplat.info ■■ Puppet [puppetlabs.com] manag- figured out all of the various DKIM ing the config for both boxes and DMARC and SenderID and Reprinted with permission of the original author. First appeared in hn.my/dns (bugsplat.info) ■■ Supply drop [hn.my/supplydrop] SPF things I need to do, and besides, deploys this configuration via Gmail won’t send out through my Capistrano [hn.my/cap] SMTP server anyway.
■■ Tinydns has a static config file Step 4: Logging checked into git, controlling most One of the more interesting aspects of my zones of this whole project has been ■■ Tinydns also has a dynamic file getting a comprehensive view of that does my dynamic DNS everything that goes on in my updates for the home router little empire. The other day I set up global logging using Papertrail bugsplat.info is my oldest and [papertrailapp.com], a hosted log- thus most complicated domain. ging service. It doesn’t do a whole It’s not even really that com- lot; mostly it just seeps up logs from plicated; it just handles a lot of all of my services, including these stuff. My Mac mini runs a cron two VPSs and a bunch of Heroku job every minute that ssh’s into apps, makes them searchable for both machines and rebuilds the a few days, and drops tarballs of tinydns config file if its IP has them onto S3 nightly. It has given changed. That IP is then assigned me really valuable insight into at to subspace.bugsplat.info, and least two things: my Gmail backup I have a wildcard CNAME for wasn’t working, and I get hit a lot *.bugsplat.info pointing at sub- by Chinese and India SSH breakin space. This lets me do things, like attempts. Still working on how to having various services running on deal with that one, but the Gmail that Mac mini with distinct host- backup is up and running. names, all hiding behind a common nginx. In addition, each VPS has a wildcard CNAME pointing to it from *.
29 How I Work With Postgres
By Craig Kerstiens
n at least a weekly basis and not uncom- Empowering developers, DBA’s, product people, monly multiple times in a single week I get marketers, and others to be comfortable with their Othis question: database will lead to more people taking advantage of what’s in their data. pg_stat_statements was a I’ve been hunting for a nice PG interface that works great start to this, laying a great foundation for valu- within other things. PGAdmin kinda works, except the able information being captured. Even with all of SQL editor is a piece of shit. the powerful stats being captured in the statistics of — @neilmiddleton PostgreSQL, so many are still terrified when they see Sometimes it leans more to, “what is the Sequel Pro something like: equivalent for Postgres?” My default answer is: I just use psql, though I do have to then go on to explain QUERY PLAN how I use it. For those who are interested, you can read ------more below or just get the highlights here: Hash Join (cost=4.25..8.62 rows=100 width=107) ■■ Set your default EDITOR then use \e (actual time=0.126..0.230 rows=100 loops=1) Hash Cond: (purchases.user_id = users.id) ■■ On postgres 9.2 and up \x auto is your friend -> Seq Scan on purchases (cost=0.00..3.00 ■■ Set history to unlimited rows=100 width=84) (actual time=0.012..0.035 rows=100 loops=1) ■■ \d all the things -> Hash (cost=3.00..3.00 rows=100 width=27) Before going into detail on why psql works perfectly (actual time=0.097..0.097 rows=100 loops=1) fine as an interface I want to rant for a minute about Buckets: 1024 Batches: 1 Memory what the problems with current editors are and where Usage: 6kB I expect them to go in the future. First this is not a -> Seq Scan on users (cost=0.00..3.00 knock on the work that’s been done on previous ones, rows=100 width=27) (actual time=0.007..0.042 for their time PgAdmin, phpPgAdmin, and others rows=100 loops=1) were valuable tools, but we’re coming to a point where Total runtime: 0.799 ms there’s a broader set of users of databases than ever (7 rows) before and empowering them is becoming ever more important.
30 PROGRAMMING Empowering more developers SELECT * by surfacing this information in FROM users a digestible form, such as build- LIMIT 1; ing on top of pg_stat_statements id | first_name | last_name | email | data tools such as datascope [datascope. 1 | Rosemary | Wassink | [email protected] | "sex"=>"F" heroku.com] by @leinweber and With toggling the output and re-running the same query, we can see getting this to be part of the default how it’s now formatted: admin we will truly begin empow- ering a new set of users. \x But enough of a detour, those Expanded display is on. tools aren’t available today. If you’re craig=# SELECT * from users limit 1; interested in helping build those -[ RECORD 1 ]------to make the community better, id | 1 please reach out. For now I live in a first_name | Rosemary world where I’m quite content with last_name | Wassink simple ole psql. Here’s how: email | [email protected] data | "sex"=>"F" Editor Using \x auto will automatically put this in what Postgres believes is the Ensuring you’ve exported your most intelligible format to read it in. preferred editor to the environ- ment variable EDITOR when you run psql history \e will allow you to view and edit Hopefully this needs no justification. Having an unlimited history of all your last run query in your editor of your queries is incredibly handy. Ensuring you set the following envi- choice. This works for vim, emacs, ronment variables will ensure you never lose that query you ran several or even sublime text. months ago again: export EDITOR=subl export HISTFILESIZE= psql export HISTSIZE= \e Gives me: \d The last item on the list of the first things I do when connecting to any database is check out what’s in it. I don’t do this by running a bunch of queries, but rather by checking out the schema and then poking at defini- tions of specific tables.\d and variations on it are incredibly handy for this. Here are a few highlights below: Listing all relations with simply \d: Note you need to make sure you con- \d nect with psql and have your editor List of relations set. Once you do that, saving and Schema | Name | Type | Owner exiting the file will then execute the ------+------+------+------query. public | products | table | craig \x auto public | products_id_seq | sequence | craig public | purchases | table | craig psql has long had a method of for- public | purchases_id_seq | sequence | craig matting output. You can toggle this public | redis_db0 | foreign table | craig on and off easily by just running public | users | table | craig the \x command. Running a basic public | users_id_seq | sequence | craig query you get the output: (7 rows)
31 List only all tables with dt:
\dt List of relations Schema | Name | Type | Owner ------+------+------+------public | products | table | craig public | purchases | table | craig public | users | table | craig (3 rows)
Describe a specific relation with\d RELATIONNAMEHERE:
\d users Table "public.users" Column | Type | Modifiers ------+------+------id | integer | not null default nextval('users_id_seq'::regclass) first_name | character varying(50) | last_name | character varying(50) | email | character varying(255) | data | hstore | created_at | timestamp without time zone | updated_at | timestamp without time zone | last_login | timestamp without time zone | n
Craig Kerstiens is part of the team at Heroku. He writes code in Python, curates Postgresguide.com and Postgres Weekly, and frequently speaks at conferences on those topics among others.
Reprinted with permission of the original author. First appeared in hn.my/postgres (craigkerstiens.com)
32 PROGRAMMING 33 Building Stripe’s API
By Amber Feng
thought it would be interest- Language-specific libraries and documentation and help our users ing to talk about Stripe’s API, documentation with any problems they might be Iparticularly lessons learned and Since Stripe’s API speaks HTTP and having with a particular integration. what kind of things we did to try JSON, you could easily integrate We decided that it was worth to make the API as easy to use as it into your application with any it, but this may not be the right possible. standard HTTP client library. How- answer for everyone. ever, this still requires constructing Make it easy to get started requests and parsing responses on Have a focused API, but allow It may sound like a no-brainer, but your own. flexibility the best way to get people to try We maintain and support open- We’ve found that it’s critically out (and hopefully eventually use) source libraries in some of today’s important to keep the API focused your API is to make it really easy to most popular web languages. It and simple. get started. turns out people are pretty attached It’s often tempting to add new To that end, we do things like to their favorite languages. features that are not obviously nec- including pastable code snippets We had a lot of internal discus- essary to the core API. For example, throughout our site and documen- sions about whether we actually our users frequently want us to add tation. One of the first things you’ll wanted to support our own client better analytics, tax calculations, or see on our front page is a curl snip- bindings or allow the community to send customers receipts. While pet you can paste into a terminal to to organically start and maintain these things are nice, every feature simulate charging a credit card. the projects themselves. Is it worth you add makes the API more com- Regardless of whether you have a owning the projects if it means that plex and cluttered. Stripe account or not (if logged in, you might have to maintain librar- You can instead give your users we fill in your test API key; other- ies for languages or frameworks in the tools to be able to write their wise, it’s a sample account’s API key), which you don’t have expertise? own extensions. We allow our users you can see the Stripe API in action. Maybe. (and third party applications) to All of our documentation code Official libraries have the benefit hook into Stripe in a couple of snippets are similarly possible to of being consistent: they all have ways: directly copy and paste — we try to the same level of quality, support embed as much information as pos- the same interface, and get updates sible (API keys, actual object IDs at the same time. Having our from the account, etc.) so our users own libraries also makes it easier don’t have to. for us to have language-specific
34 PROGRAMMING Webhooks In our test environment, we On the other hand, some errors Stripe uses webhooks to let our allow users to send test webhooks are harder to diagnose (especially users know when some interesting of any type and provide handy test from the API’s end, since you event has happened. This ranges card numbers that trigger certain have limited information about from events triggered by an API errors (like declines). what your user is actually trying to call, like charge.succeeded or This allows them to easily test accomplish). charge.refunded, to asynchronous the behavior of their own applica- Where possible, we absolutely events like customer.subscrip- tion in the face of different sce- think it’s worthwhile to try to tion.trial_will_end. narios instead of having to manually anticipate our users’ errors and help Our aim was to make it easy trigger things that are nondetermin- as much as we can. to layer additional logic on top of istic, like declines, or time-depen- Stripe events (like sending cus- dent, like expiring subscriptions. Dealing with Change tomer receipts or enabling push Lastly, dealing with change is never notifications). Giving our users the Help your users debug fun. As much as you hope you’ll ability to build this kind of custom- We’re developers too. We know never have to change the API, some- ized functionality allows them to from experience that debugging times you need to make changes control the entire experience for is a disproportionately large por- and sometimes those changes are their users as well. tion of the development cycle. We backwards-incompatible. also (unfortunately) know that There’s no easy answer for ver- Stripe Connect sometimes you spend a lot of time sioning APIs. We keep a per-user Stripe Connect, an API we released debugging something that eventu- version which reflects the state just last year, is another way of ally turns out to be really obvious of the API the first time the user building on top of the Stripe or silly. made an API request. Most of our platform. For common or easy errors, new features are additions that Connect is an OAuth2 API you (the API) likely know exactly aren’t backwards-incompatible, and [oauth.net/2] that allows a Stripe what’s wrong. So why not try to they just work automatically for user to authorize access to their help? everyone. Stripe account to a third-party Whenever we make a backwards- application. We’ve seen a variety of >> Stripe::Customer.create incompatible change, however, it applications built on top of Stripe Stripe::AuthenticationError: doesn’t affect the API behavior for so far: marketplaces and checkout No API key provided. (HINT: any of our current users. Users can pages let users “plug in” their Stripe set your API key using "Stripe. then choose to explicitly upgrade accounts to accept payments, and api_key = ". You can generate their version in the dashboard (after analytics dashboards fetch Stripe API keys from the Stripe web reviewing the detailed changelogs) data in order to show interesting interface. See https://stripe. or can send a version override graphs or patterns. com/api for details, or email header in any API request to test [email protected] if you have the behavior of a specific version. n Provide a testing environment any questions.) One of the most important Amber is an engineer at Stripe, and works things you need with an API is a >> Stripe.api_key = TEST_KEY primarily on the API. She loves all things great test/sandbox environment. => ... web and distributed, and enjoys hacking This is particularly important >> Stripe::Charge. on side projects and writing in her blog. for a payments API — our users retrieve(LIVE_CHARGE_ID) shouldn’t have to make live charges Stripe::InvalidRequestError: Reprinted with permission of the original author. when they’re trying to test their (Status 404) No such charge: First appeared in hn.my/stripeapi (amberonrails.com) integration. ch_17SOe5QQ2exd2S; a similar object exists in live mode, but a test mode key was used to make this request.
35 SPECIAL Goldeneye 64’s Inspirational Startup Story By Alex Baldwin
rowing up, GoldenEye developers had to estimate what on a game before, there was this had a special place in the finalized console’s capabilities joyful naïveté.” my heart; it was the first would be. G Scope was so slim that they game my parents wouldn’t let me Getting closer to the release didn’t even originally plan out the buy. I saved up allowances and dug date, the final platform specs were legendary multiplayer mode that up couch treasures for months to released and they had to make arguably made the game so success- taste the forbidden fruit. The effort significant graphic cuts to make it ful. It was done almost exclusively turned into one of the pillars of my work. by one guy as an afterthought. childhood experiences. I still vividly remember where to place the prox- The final Nintendo 64 hardware The game’s multiplayer mode was imity mines on Temple to get crazy could render polygons faster than added late in the development pro- spawn point kill streaks against my the SGI Onyx workstations they cess; Martin Hollis described it as little brother. Fifteen years later, it’s had been using, but the game’s “a complete afterthought.” According still inspiring me, but not for the textures had to be cut down by to David Doak, the majority of the proximity mines. half. Karl Hilton explained one work on the multiplayer mode was It’s hard to imagine that this method of improving the game’s done by Steve Ellis, who “sat in a game almost didn’t exist. Rare’s performance: “A lot of GoldenEye room with all the code written for a studio head, Mark Betteridge, was is in black and white. RGB color single-player game and turned Gold- quoted as saying, textures cost a lot more in terms enEye into a multiplayer game.” of processing power. You could do When Nintendo asked if we Despite everything, the game double the resolution if you used wanted to do it, we said, “well not went on to become the third greyscale, so a lot was done like really”…we were trying to build highest selling N64 game, inspire that. If I needed a bit of color, I’d our on IP, and film tie-ins meant console shooting games, and win a add it in the vertex.” a lot of ownership by the film crazy amount of awards. Next time company. While doing all this, their team you’re heading down the wrong had almost no idea what they were way of the entrepreneurial roller- The team faced insane amounts doing when they started out. Sound coaster, take a deep breath, make a of adversity and uncertainty. Start- familiar? cup of tea, and remember that you ing out, they didn’t even know can make it happen. Persevere and what the specs were for the new GoldenEye 007 was developed dominate. n platform. Wikipedia on the game’s by an inexperienced team, eight of development: whom had never previously worked Alex is a designer at thoughtbot in San on video games. David Doak com- Final N64 specifications and devel- Francisco, previously with 500 Startups, mented in 2004, “Looking back, opment workstations were not ini- Techstars, and Console.fm. He is a lover of there are things I’d be wary of tially available to Rare: a modified lattes and dope beats. attempting now, but as none of the Sega Saturn controller was used people working on the code, graph- for some early play testing, and the Reprinted with permission of the original author. ics, and game design had worked First appeared in hn.my/goldeneye (alexbaldwin.com)
36 SPECIAL The Joys of Having a Forever Project
By John Biesnecker
think most creative people Most people would say having a Stop beating yourself up about have something that I call a project that you can’t put down but not making progress on your “one I Forever Project — a project that you don’t make any substantial big goal.” Eating healthier, exercis- that, despite its audacity and seem- progress on is silly — the antithesis ing more, and being a better spouse, ing impossibility, simply will not of the various flavors of Getting friend, etc. are goals. Your Forever put itself to bed. A project that Things Done that spring up now Project is not. Your Forever Project comes creeping back into your and again — but I disagree. While is your Beacon on the Hill, pushing consciousness when you sit down I may not have finished (or even you to be better, to learn, to stretch, for a break from “real work.” A proj- really started) my game, poking to reach just a few finger widths ect that is hard to imagine actually around the edges of it have led me beyond your grasp, over and over embarking on, but whose mental to wonderful tangents during which again. cost of abandonment is far too high I’ve learned a lot about a lot of Embrace your Forever Project, to even consider. A project that things, things that I may have never and never stop dreaming. n you’d totally do if you had the time, touched if it weren’t for my Forever and the money, and the talent, and Project. Rather than be a source of John Biesnecker (@biesnecker) is an the… disappointment, my Forever Project American product designer and software I don’t know about you, but I is a source of constant inspiration. developer based in Shanghai, China. He adore my Forever Project (mine If I ever really completed it… enjoys thinking about hard problems, tell- happens to be a game that I’ve well…I’m not sure what I would ing good stories, and playing with his kids. been punting around in various do. Probably replace it with a better forms since the late 1990s, and I version of itself. But that’s silly Reprinted with permission of the original author. First appeared in hn.my/forever (dev.gd) wouldn’t be surprised if yours was because I’ll never actually complete also a game of some sort). I might it and because a Forever Project is not have made the progress on it like the speed of light — you can that I wish I would have, but just get infinitely close to it, but you can having it out there as something to never quite get there. It’s just the think about gives me a warm, fuzzy nature of the beast. feeling.
37 Avoiding “The Stupid Hour”
By Rachel Kroll
rom time to time there is a In my younger days, I used to There’s another good reason for romantic notion of teams feel this coming on and would just doing it this way. Have you ever Fpulling crazy hours and keep going. This was a spectacularly come back to a project and been working all-nighters frequently. The bad idea. The next morning, I’d get unsure of where to get started? idea is that you can cheat the night up and look at the code and would If you had left off just one item (or morning, for that matter) and have no idea how it ever worked. sooner the day or week before, continue coding, writing, or doing A function I had written during you’d already have a known starting whatever it is you that you do. the stupid hour might work for a point. Write it down on a post-it Sometimes this is driven by mania- specific test case, but I would have note and stick to your monitor, then cal managers, but other times it to sit and really dig at it to find out go do something else. comes from within. how. Then I would also discover The next day, not only will you Now, I’ve already written that it didn’t cover other test cases, have a nice place to resume, but [hn.my/wrong] about the occa- either. you’ll also have the benefit of sev- sional flashes of insight which lead Since it was ugly and unmain- eral hours (or days, if over a week- to a late evening here and there. tainable code, it needed to be end) of subconscious/background { That’s something else. That’s where fixed.T he fact it didn’t even work processing you didn’t even realize you have a fire burning inside of properly also meant it needed to was going on. It’ll make for a better you and you need to get that fire go. More often than not I’d have to result overall. join: 'Intensive Online Bootcamp', routed through your fingers and rip it out and redo that particular Don’t feed the stupid hour. It turned into code. You don’t do chunk of code. It was a net loss of never ends well. n learn: 'Web Development', this often. It’s just when things get time. I should have spent that time really good and all get flushed into the night before just sleeping rather Rachel Kroll lives and works in Silicon the computer at once. than trying to fight it while coding. Valley. Once a Googler, she now runs her goto: 'http://www.gotealeaf.com' This is more about the relentless In recent times, I’ve grown to own software consultancy business and push to keep working night after recognize this and appreciate it writes daily about software, technology, } night even when there’s nothing as a useful signal. I tend to stop sysadmin war stories, and productivity. special going on. Enough has been earlier than I would have before Her first book, The Bozo Loop, is a collec- tion of posts from 2011, with another on written about it, but it always seems and switch to other things after a the way. to get really complicated in how it’s certain point. Why write something described. I want to give it a simple that will have a good chance of Reprinted with permission of the original author. name that anyone can remember being broken and will require an First appeared in hn.my/stupidhour (rachelbythebay.com) and anyone else can understand. immediate fix? Leave it as a “to do” I call it “the stupid hour.” When item and come back to it the next talking about myself, I call it my day. stupid hour. It’s the point when I’ve been awake for too long and anything I create is sure to be sub- optimal. The late hour has drained enough out of me to where I turn stupid and my output shows this.
38 SPECIAL
Tealeaf Academy an online school for developers
Learn Ruby on Rails | Level up Skills | Launch Products | Get a Job { join: 'Intensive Online Bootcamp', learn: 'Web Development', goto: 'http://www.gotealeaf.com' }
{ join: 'Intensive Online Bootcamp', learn: 'Web Development', goto: 'http://www.gotealeaf.com' }
Tealeaf Academy an online school for developers
Learn Ruby on Rails | Level up Skills | Launch Products | Get a Job 39
Tealeaf Academy an online school for developers
Learn Ruby on Rails | Level up Skills | Launch Products | Get a Job HOSTING Rent your IT infrastructure from Memset and discover the incredible bene ts of cloud computing.
C
M
Y
CM MY $0.091/GByte/month or less CY 99.999999% object durability CMY From $0.020/hour 99.995% availability guarantee K to 4 x 2.9 GHz Xeon cores RESTful API, FTP/SFTP and CDN Service 31 GBytes RAM 2.5TB RAID(1) disk
SCAN THE CODE FOR MORE INFORMATION
HOSTING Find out more about us at www.memset.com
or chat to our sales team on 0800 634 9270.
hosting