the matrix linode’d

Today’s review is about a pretty decent hosting company called Linode. Here’s the three-day timeline from idea to implementation:

  1. Thursday: decided to create a new info website about 3D printing, bought the domain name on GoDaddy and waited for the changes to take effect at midnight
  2. Thursday: created a Github repository to store the source for the website since I’m open-source like that
  3. Thursday: created an account on Linode, purchasing a “linode” for that
  4. Thursday: designed/created the initial local/development website layout/framework, collected images and content
  5. Friday: created (provisioned) the basic linode (virtual machine) for the website on production, provisioned a virtual drive, deployed Ubuntu 16.04LTS onto that, booted and remoted into that, ran updates, installed the framework, added the website, setup security and the firewall
  6. Friday: adjusted the DNS at GoDaddy to point to the new server, added more content
  7. Saturday: launched the website on Linode with the initial version
  8. Saturday: tweaked the settings to make the Node.js—based website start on bootup
  9. Saturday: added more content to the website

site

Not bad for an open-sourced 10-page (44 files) responsive website, if I do say so myself.

Please note that when I say “hosting company”, I really mean a “virtual server provider” so this is more like Amazon EC2 as a service offering. I didn’t just rent website space (like on Wix.com or WordPress.com), I rented an entire virtual server, if you will.

Comparison of virtual server versus website space

There are some advantages/disadvantages of renting a virtual server over just some website slot on a server somewhere:

Pros:

  • In theory, you could run several websites from a virtual server
  • You can run services in the background (like Node.js) and manage them
  • You can run multiple threads on the same server, like helper routines which do something other than serving up pages
  • You’re not limited to the set of templates that are available from Wix.com, for example
  • Your website runs separately from other websites
  • You get an IP address which is only used for your website

Cons:

  • You have to setup security yourself since you’re responsible for the entire server
  • The learning curve is steeper
  • You have to know I.T. things like setting up servers and installing software

Framework/software

Here’s a list of what I used for this website:

  • Node.js: Probably the most famous event-driven JavaScript runtime engine out there
  • Express: A minimalistic Node.js framework for separating code from content on a website
  • Bootstrap.css: A responsive stylesheet and component library for styling a website
  • PM2: A handy process manager for Node-based applications on a server.  After pulling new code, I might run the command pm2 restart AppName to restart the service

Documentation

Kudos to Linode for providing a detailed Getting Started guide along with several tutorial videos on the subject.

And further, a note of thanks to PM2 which seems to satisfy the requirements of bringing up and managing a Node.js application as a service within a production environment and their ample documentation.

Suitability

Is Linode well-suited for most website designers/developers? Probably not. On an I.T. complexity scale from 1 to 10 potatoes, I’d say they’re probably seven potatoes, perhaps. In this case, you’d have to be comfortable doing the following:

  • Using a web-based console to allocate and bring up/down a virtual server
  • Using ssh to remote into your virtual server
  • Navigating within a command line interface on a Linux computer or similar
  • Using ssh-keygen to generate a keypair
  • Using apt-get to update things
  • Editing files using nano
  • Managing services, reading log files
  • Remotely rebooting your virtual server
  • Setting up a firewall, testing and managing same
  • Applying code using git
  • Testing a website to verify that there are no 404 (file not found) type of errors, for example
  • And obviously, creating/designing a website in the first place and using a repository like Github for storing those files

That said, it was a perfect fit for me since I can do those things. In fact, the Linode-related part of this took no more than two hours since this is the first time I’ve used their interface. My next one should go much faster.

Observations

I will say that I’m impressed. Unlike Amazon AWS, Microsoft and Google, the people at Linode haven’t created an interface that’s overly complicated. It seems to work simply and to do the things you need to do and those are: 1) buy a virtual server, 2) deploy something onto it, 3) turn it on and 4) remote into it. I don’t think the “big three” have figured this out yet; their interfaces and the assumed workflow requires too much research, in my humble opinion.

Additionally, the PM2 software does a great job of working with the git-based code distribution model, allowing you to restart the Node.js app when it’s required and to start up automatically each reboot. There’s an easy-to-remember command interface like pm2 show AppName which tells you what you usually want to know.

At a cost of $5/month, it compares favorably to most of the well-known hosting providers out there. The basic linode will likely satisfy the requirements of the average Node.js application up to a reasonable level of simultaneous users, I’d suggest.

price

Advertisements

blinking the raspi’s built-in LED

I’ve just added a repository of some JavaScript code to take over and exercise the built-in activity LED on a Raspberry Pi Zero W (and presumably other models). It’s called gpiozero-toggle-led and it’s a pretty simple interface with installation instructions and some sample code. It works with the underlying js-gpiozero JavaScript port of the popular original Python code. This would be an excellent way of simply demonstrating GPIO without any additional wiring, components, breadboards, extra power supplies or electrical knowledge (like finding a 330-ohm resistor using its color bands).

zero-wireless

Note that the “zero” in the title of the repository and in js-gpiozero does not refer to the Raspberry Pi Zero but to the original gpiozero Python library.

This should remove some of the guess work when attempting to use the relatively-new library since their documentation examples at the moment are taking a back seat to their code port from the more-extensive Python offering.

This approach can easily be modified to instead exercise external LEDs (as soldered or otherwise attached to the header pin locations seen below).  Note that you’ll use “BCM numbering” for APIs such as this one. For external LEDs, you would need to connect it inline with a resistor from a selected pin to one of the grounds for this to work with correct orientation of the LED’s anode/cathode, of course.

raspberry-pi-pinout

If you’re trying to use this with a Raspberry Pi of a different model, you’ll likely want to adjust the JavaScript slightly as seen below.

/routes/index.js:

// Existing code, for a Raspberry Pi Zero
var ledActivity = new LED(47, false);
// For Raspberry Pi 3, for example
var ledActivity = new LED(47);

And that’s it. Since the Raspberry Pi Zero assumes an opposite value for true/false than the bigger models, it’s necessary to configure this in the device constructor to make things work as expected. Since BCM pin 47 is the activity light on the board itself, this will allow you to control it.

how cool is electron?

I’ve been working the past couple of days with Electron, a Node.js cross-platform desktop app tool which uses JavaScript, HTML and CSS to create what look like native OS-style applications for Windows, OS X and Linux.

electron_atomelectron

Cool stuff, indeed. Out-of-the-box, it looks like you publish your Electron-based app like you would anything on github:

git clone https://github.com/Somebody/Repository.git
cd Repository
npm install
npm start

But there’s also a way of downloading OS-specific images and then adding your own app into this subdirectory structure. The result is a stand-alone EXE and folderset which reasonably looks like a drop-in replacement for something you normally would build locally using Microsoft Visual Studio perhaps. In this version though, you’d run Electron.exe but there are instructions on their website for renaming your application, updating the icon’s, etc.

I’ve just used it today to build a basic music player. I wouldn’t say that the layout is as responsive as a typical mobile app’s ability to move content but I did tweak things so that it can squash down to a mini-player and it stills looks great.

mplayer

I can thank KeithIG/museeks for the open-source code behind this. They have several OS-specific downloads available if you don’t want to build this yourself.

Pros

  • This allows you to build cross-platform desktop apps in much the same way that you’d use Adobe PhoneGap, say, to build for mobile apps.
  • You code in the familiar HTML/JavaScript/CSS trilogy of disciplines and it’s Node.js centric. It is also React.js-friendly, as I’m finding on this project.
  • So far, it seems to be well-behaved.
  • If you don’t want others to easily see your code, there’s a step where you can use asar to zip-up everything into a tidy package.
  • I didn’t have to digitally-sign anything like you might have to for a Windows 10 application or for OS X, say.
  • For people who have git and npm, the install is as easy as anything you’ve seen in the open-source space and a familiar workflow.

Cons

  • Currently, I don’t see any support for mobile platforms.
  • The complete foldedset comes in a 216MB which strikes me as a little big for what it’s doing.  The app itself for the music player weighs in at 84MB of this so the remainder is everything that Electron is doing to present all this.
  • You would need to setup three different build sites to maintain a specific download for your own app.  (It’s not like PhoneGap in which you just submit the common code and Adobe builds it in the cloud.)
  • Given that you’re not digitally-signing your code, you might have to talk your users through the hurdles of having the user “trust” the content within their particular OS.
  • This might be so popular soon that none of us can really afford to just use Electron.exe by default to serve up our app; we’ll need to rename it before publishing, in other words.

Overall

I can see myself wanting to really learn this one deeply. It has a lot of potential for delivering a more native-app experience for users.

despicable me—themed supercomputer

I gave a talk on Tuesday to an eager group of 155 attendees at the monthly SanDiego.js meetup on the topic of “Supercomputing in JavaScript”. I had an opportunity to show the new Raspberry Pi 3 supercomputer which I’d built and took it through its paces.

I think they mostly loved the audio events for assembling the minions and sending them to bed (shutting off the remote nodes). There was just enough time to also show the obligatory “Hello, Minions!” demo program to exercise the Message Passing Interface. I received a wide variety of questions and compliments from the group. And of course afterward, everyone who owned a Raspberry Pi came over to discuss their own projects, which was cool.

Here’s the PowerPoint presentation from that talk, in case you’re interested.

e-mc2 repository with step-by-step instructions

open-source minion supercomputer

It seems like every villian these days has their own plan for world domination. So I thought I’d better get started on my own by developing an open-sourced supercomputer design using the affordable Raspberry Pi 3 computer as the basic building block.

Here, I list some of the pertinent points regarding that design.

Minion Theme

The usual master-and-slaves paradigm for most supercomputer systems is replaced here with a gru-and-minions theme. I intend to add sound events at some point in the future, given the usual “headless computer” arrangement for servers. This should help to flag certain events as they’re happening and add some fun to the project as a whole.

Modular

Each squad of four computers would be contained in such a way that it could connect together with the next set of four.  The name for each squad is E=MC2 which stands for Expandable Minion Cube Cluster.

Self-Contained

Each E=MC2 has its own internal Ethernet switch and a USB charging hub to power things inside. The first in the system also includes a shared USB-based thumbdrive for common resources as well as the code that you wish to run in parallel.

Low-Power

Everything internally is powered with a USB charger so that’s about 15W per computer (or core). The power cost then is less than half the per-core cost of a typical supercomputer for electricity, noting that your average supercomputer must also be cooled.

Low-Cost

Weighing in at less-than-$300, the squad of computers has a per-core cost of less than $75. Throughout, I’m using open-source software. I hope to further minimize costs involved by renting time on a 3D printer locally rather than purchasing one.

Mobile

Since I hope to show this project to others, the entire thing should be relatively mobile by design.

In the “known wi-fi” mode, a single power cable could be the only connection to the device, allowing a laptop to connect to it wirelessly. It would be necessary to first educate the gru node and the laptop with the wi-fi credentials, however.

In the “unknown wi-fi” mode, you would add an Ethernet cable to a laptop computer to connect to the system. It would be necessary to configure the laptop’s Ethernet adapter to match that of the internal wired network.

When the 3D printing work has been completed I envision a clear, self-contained, 6″ cube weighing in a under two pounds (minus the power cord). We’ll see how that plays out soon enough.

Technical Savvy

I’m trying to make this as easy as possible for others to repeat. I’m carefully documenting the hardware list to make this easier to source and so that the same set of 3D printing designs I use might accommodate your own attempt at this.

I do, however, assume that you would have some basic electronics experience, have some tools and a small budget to work with and that you understand your way around Linux terminal commands like ls, cp, mv, mkdir, cd, chmod, chown, sudo, apt-get install as well as how to map a drive on a network in either OS X or Windows, say.

You’d also need to know how git and how github works in order to pull the code and design files.

It will be helpful if you understand DNS, DHCP, the concept of IP addresses and how to configure these on a computer.

Documentation in Real-Time

I’ve purchased the new hardware for this, having done some work already preparing the working image for gru, the master node of the system. I’ve created a repository for everything at e-mc2, noting that there are more than one README.md file within the collection of instructions so far.

As I install the full set of software and dependencies again I will be updating the documentation so that everyone may follow along and be assured that I’ve not glossed over any of the details.

Design Hurdles

I had some problems with the first (several) attempts to power everything with a chassis-based switching power supply but the connections themselves proved to be too problematic. And given the open-sourced nature of the project I would hope that others would build their own so I deemed this attempt “too fussy”. Additionally, the “green” Ethernet switch’s tendency to lower its power demands before any nodes came on then confused this power supply which looked something like this:

  1. The “Green” Ethernet switch would power on and the switching power supply would adjust the input current and output power
  2. The switch would determine that none of the nodes had “linked in” to their Ethernet connections (yet)
  3. The switch then would go into green mode and lower its power consumption and the switching power supply then would attempt to toggle down the output power
  4. The switch and other devices wouldn’t get enough power and then would power cycle, repeating the pattern

This next iteration then changes the power system completely which so far promises to be much easier, resulting in solid connections to the devices.

Common Protocols and Code

I’ve decided to use the Message Passing Interface (MPI) for inter-node communications and the Samba software so that the coder may map a drive to the shared data/code area.

Since I like Node.js, I’ll be developing something so that this is a choice as well. The initial “Hello, Gru!” demo may likely be written in Python, however.

Plastic

I’ve found a local place that rents 3D printers. Hopefully I can get some help designing the files necessary to house all this in a clear cube.

As far as external connections/openings to the cube, there should be one rectangular cutout for the IEC320C5 three-prong power plug and a Keystone receptacle for the Ethernet connector so that it can snap into place. Also, there should be some holes adjacent to an internally-mounted speaker. Optionally, there might be openings on both sides if a fan is added. Overall, it should be a pair of clear plastic parts which snap together to form a cube.

Internally, there should be screw holes and possibly slide rails or bumpers for the Ethernet switch, the USB charging hub and the four Raspberry Pi 3 computers. There should be room for five 1′ Ethernet cables as well as five or six 1′ micro-USB cables for power. As preferences go, it should be relatively easy to open up, remove a computer and its micro-SD card for the sake of maintenance, say.

Although it should operate well without a fan, I’m trying to source a usable 5V-powered fan for the design. Since some of you will want to overclock, this is probably a good idea, I’d guess. That being the case, a cube-to-cube side orientation of venting would create a continous flow of air through all. It may be difficult for everyone to source something like a 5V-to-12V boost module but that’s probably the route to go, combining this then with an inexpensive and readily-available 12V computer fan.

Using the Last USB Port Wisely

The existing design calls for four USB cables to power the computers and a fifth to power the Ethernet switch. That leaves one last USB port to power something. Here are some options.

  1. The existing design utilizes a tiny USB thumbdrive which goes directly into the gru node and is shared by all. Instead, this could be a USB-powered external drive with faster throughput, if so desired. One USB cable would connect the drive to gru and the second would provide power over USB to the charging hub.
  2. A small speaker attached to the grunode should allow sound events to be played in conjunction with code execution. Some speakers are USB-powered and this may be an option.
  3. A USB-powered fan would be the logical choice, however, for the remaining port.

Repository

In case you missed the link above, the repository is now at e-mc2 on github. As of today, it’s probably a bit premature to start your own supercomputer from these designs. I personally would wait until I’ve got more of the MPI-based coding in place and have added the step-by-step instructions for some of the dependencies.

Update 3/27/2017

Things are looking great for the repository and the supercomputer itself. The MPI-based code is now working as well as the website for managing the minions.


raspberry pi 3 with node.js

First day with the Raspberry Pi 3 and I’m already having way too much fun with this! Today, I decided to see how fast I could get a Node.js—based webserver running on it and was not disappointed.

Out-of-the-box it comes with the NOOBS installation manager at bootup which politely asks you to install an operating system from the selection. I chose the default Raspian operating system because it’s basically a Debian project and well-trusted.

My task was made simpler because I’d purchased one of the Raspberry Pi 3 kits which included a microSD card pre-loaded with NOOBS. I’d suggest doing the same since the markup over a standard 16GB microSD wasn’t much, to be honest. The empty Raspberry Pi 3 board costs about $35 these days but the kit at $60 includes the NOOBS-preloaded microSD and a 5V power adapter. My kit also included a basic/clear case but I opted for a $9 red case which seems to be more standard.

Setup

For the initial setup, you need spare Ethernet and HDMI-to-DVI cables, USB mouse, USB keyboard and a monitor. Everything gets plugged in before adding the power adapter to boot it.

One thing to note if you’ve just purchased one and are confused, you’ll need to remove the microSD adapter from the included SD adapter before inserting the former into the Pi. They include the larger SD adapter because some people might need that, in theory, to insert it into their workstation if the need arrives. Store the adapter for future use.

microsd

I found that the latest release of Raspian somewhat hides the wi-fi configuration compared to earlier versions. You’ll find what you need by left-mouse clicking the network symbol in the upper taskbar. I also found it necessary to manually edit the /etc/network/interfaces file in a terminal session to change the wi-fi from manual to dhcp. I would imagine that subsequent versions of Raspian will fix this however, you might not be able to easily update the operating system until you’ve achieved a working network connection in the first place.

Headless Boot

Confirming that the networking was happy, I shutdown the Pi, removed everything but the power adapter and booted it again, doing everything next remotely via ssh or RDP.  First I updated the software via a terminal session and then installed tightvncserver, xrdp and samba. I configured a shared drive for use later in /etc/samba/smb.conf file and created a Samba network user for the default pi operating system user. Next, I tested remote connectivity from both a Mac and a Windows-based computer and all looked good for the share.

Next, I installed Node.js globally from a Nodesource.com—based distribution as well as the express-generator. I created a ~/share/sites subfolder and then used Express to generate a new hello application below this. I then started up the newly-created website.  Fortunately, the Samba client recently installed publishes its own hostname to the DHCP server so that I could then surf the website from Windows without knowing its IP address as seen here. The default port for an Express website is 3000, btw.

raspberrypi_webserver

Since I’d created a network share earlier I simply edited the default Express-generated files using Wordpad on my Windows—based computer remotely.

What Next?

From here, I’ll likely be investigating earlier projects like the magnetic card swipe visitor station to convert it to using a Raspberry Pi 3 instead of a larger workstation. By adding a USB-based drive you could likely repeat the earlier Kloudspeaker project with a much smaller footprint.

I intend to create a squad of four Raspberry Pi 3’s in a housing which includes an Ethernet switch, a common power supply and a fan all in a self-contained housing which promotes good airflow and optional stacking. In this case, I’d need to change over to an Ubuntu operating system, I think, since some of the provisioning tools there might make something like this easier. I’ll ultimately have a supercomputer for a fraction of the cost of what you might expect to pay.

In a follow-up post, I’ll be discussing my attempts to install and evaluate the Ubuntu Core for the Raspberry Pi 3 operating system using a second microSD which I purchased for this purpose.

a rose by any other name…

Unfortunately for anyone who uses the express generator for Node.js, the very popular (default) jade template language’s owners are being forced by the threat of lawyers to relinquish the name.

It turns out that JADE happens to also be the trademarked name of a software development system that nobody’s ever heard of before this.  It was created by Jade Software Corp and presumably, they have lawyers whose children need braces and their own Mercedes Benz.

At the time of writing this, the jade Node.js module is hugely popular and had over three million downloads in the last month alone and over 40 million downloads since last year. But since the developer of jade is being forced to change the name that means that everyone who uses it is being forced to change their own projects with the new name. Note that express has been downloaded more than 72 million times and this is a project generator (capable of being run multiple times for multiple projects per coder). It’s staggering to try to quantify in terms of dollars how much labor is about to be wasted on this effort for literally a hundred million projects or more.

“But since the developer [of jade] is being forced to change the name that means that everyone who uses it is being forced to change their own projects with the new name. It’s staggering to try to quantify in terms of dollars how much labor is about to be wasted on this effort for literally a hundred million projects or more.”

The module jade has now been re-introduced as pug… or at least until someone who has trademarked that term doesn’t threaten them with a lawsuit, too.

I think it’s time the open-source community push back. This is ridiculous, to think that owning a trademark can prevent another person from using the term for something as unrelated as this. Conservatively, they’ve just cost the open-source community over US$1B in fixing this name change throughout the collection of projects out there.

hacking agar.io, part 5

I guess now anyone who’s been following will also want to a chance to play Agar.io without ads. Here are the step-by-step instructions.

Note: Throughout, I’ll use 1.2.3.4 as the IP address of the DNS server you’ll be creating. Assume that every time you see this, you’ll be substituting your own server’s private IP address. Any other IP address you see should be typed in exactly as I’ve shown.

I’ll be including instructions for two different DNS servers. Choose the one that makes more sense for you based upon your experience.

Node.js DNS server version

Since I like JavaScript, here’s a Node.js implementation which may be augmented to include a nice HTML administrative interface if you’d like. I haven’t gotten quite that far yet but you can see what it takes to host a DNS server and a webserver all in one application.

  1. I assume that you already have Node.js installed, as well as npm and the express-generator. If not, you’ll need to install each first.
  2. Open a terminal
  3. Change to your home directory and optionally, change into a subfolder like ~/Sites like I did. Create one if necessary with: mkdir ~/Sites
  4. Run the express command to generate a new project:  express one-trick-pony
  5. If that ran correctly, change into the newly-created folder:  cd one-trick-pony
  6. Run the npm command to install the dependencies:  npm install
  7. Determine the IP address of your server and save this information for later: ifconfig | grep en1
  8. Run the npm command to install dnsd into your project (those are two hyphens without a space between them):  npm install dense –-save
  9. Edit the www file:  vi ./bin/www
    1. After this line var http = require(‘http’); add the indicated text seen in the block quote below
    2. After this line server.on(‘listening’, onListening);, optionally add the line:  console.log(‘Webserver running at *:3000’);
  10. Determine the path of the node command you usually use and save this information for later:  which node
    1. Run the su command to elevate into superuser (root) mode:  su
    2. Change to the working folder from before: cd /Users/yourname/Sites/one-trick-pony
    3. Run the node command giving a full path to the executable, which you found in the earlier step: ../../local/node/bin/node ./bin/www
    4. At this point, you should see that the server is running, indicating that it’s listening to two different ports:  53 (DNS) and 3000 (HTTP).
  11. From a workstation you can verify that the DNS server is running with the indicated command, noting that the server should still be logging requests:  dig @1.2.3.4 www.agar.io
  12. Now from the iPad, for example, go to Settings -> Wi-Fi -> select the i logo next to your connected local wi-fi zone -> DHCP -> DNS -> (write down everything here and save it), overwrite it with 1.2.3.4 (your server’s private IP address)
  13. Press the Home button twice and if Agar.io is running, swipe up to remove it from memory
  14. Start up the Agar.io app and verify that it logs in (even with Facebook), it works AND it no longer displays advertisements.
  15. When you’re finished, in Settings -> Wi-Fi, either “Forget This Network” your existing local wi-fi profile (re-entering your password) or manually re-enter the earlier DNS information that you wrote down from an earlier step.  Your iPad is now ready to behave like before.
  16. When you’re completely finished, go back to the server’s terminal session and press Ctrl-C to end Node and then enter the exit command to leave the su session.

Code to add into the ./bin/www file:

var dnsd = require(‘dnsd’);

function dns_handler(req, res) {
console.log(‘%s:%s/%s %j’,
req.connection.remoteAddress,
req.connection.remotePort,
req.connection.type,
req);

var question =
res.question[0],
hostname = question.name,
length = hostname.length,
ttl = Math.floor(Math.random() * 3600);

if (question.type == ‘A’) {
// Agar.io website
if (hostname == ‘agar.io’ || hostname == ‘www.agar.io’ || hostname == ‘m.agar.io’) {
res.answer.push({name:hostname, type:’A’, data:”104.20.26.122″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”104.20.25.122″, ‘ttl’:ttl});
}
// Facebook.com authentication
if (hostname == ‘facebook.com’) {
res.answer.push({name:hostname, type:’A’, data:”31.13.69.228″, ‘ttl’:ttl});
}
if (hostname == ‘www.facebook.com’) {
res.answer.push({name:hostname, type:’A’, data:”31.13.77.36″, ‘ttl’:ttl});
}
if (hostname == ‘graph.facebook.com’) {
res.answer.push({name:hostname, type:’A’, data:”31.13.77.6″, ‘ttl’:ttl});
}
// AmazonAWS
if (hostname == ‘prod-miniclip-v3-881814867.us-west-2.elb.amazonaws.com’) {
res.answer.push({name:hostname, type:’A’, data:”52.42.253.135″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.43.226.3″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.39.93.232″, ‘ttl’:ttl});
}
// Miniclippt.com
if (hostname == ‘mobile-live-v5-0.agario.miniclippt.com’) {
res.answer.push({name:hostname, type:’A’, data:”52.8.170.192″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.9.37.138″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”54.183.177.123″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.52.55.140″, ‘ttl’:ttl});
}
}
res.end();
}

var dnsServer = dnsd.createServer(dns_handler);
dnsServer.zone(‘agar.io’,
‘ns1.agar.io’, ‘root@agar.io’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.zone(‘facebook.com’,
‘ns1.facebook.com’, ‘root@facebook.com’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.zone(‘amazonaws.com’,
‘ns1.amazonaws.com’, ‘root@amazonaws.com’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.zone(‘miniclippt.com’,
‘ns1.miniclippt.com’, ‘root@miniclippt.com’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.listen(53, ‘1.2.3.4’);
console.log(‘DNS server running at 1.2.3.4:53’);

Bind DNS server version

This version will assume that you have a Linux (Ubuntu, in this case) server or workstation that can run the bind9 service.

Here, I assume that you’re comfortable with commands in a terminal, know what sudo does and can use the vi editor to edit and save a file. You know what touch does. If any of these don’t sound familiar, then this probably isn’t the option for you.

On a Linux (Ubuntu) server, do the following:

  1. Make sure that your system is up-to-date:
    1. sudo apt-get update
    2. sudo apt-get upgrade
    3. sudo apt-get dist-upgrade
  2. Install the DNS service, noting that it will take a fair amount of configuration work
    1. sudo apt-get install bind9 bind9utils bind9-doc
  3. cd /etc/bind
  4. Create four empty files, one per “forward” zone. In the next steps you’ll be editing each, making sure to substitute your own server’s private IP address in each case.
    1. sudo touch for.agar.io
    2. sudo touch for.facebook.com
    3. sudo touch for.miniclippt.com
    4. sudo touch for.amazonaws.com
  5. sudo vi for.agar.io
    1. $TTL 86400

      @   IN  SOA     pri.agar.io. root.agar.io. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.agar.io.

      @       IN  A           104.20.25.122

      @       IN  A           104.20.26.122

      pri     IN  A           1.2.3.4

      www     IN  A           104.20.25.122

      www     IN  A           104.20.26.122

      m       IN  A           104.20.25.122

      m       IN  A           104.20.26.122

  6. sudo vi for.facebook.com
    1. $TTL 86400

      @   IN  SOA     pri.facebook.com. root.facebook.com. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.facebook.com.

      @       IN  A           31.13.69.228

      pri     IN  A           1.2.3.4

      www     IN  A           31.13.77.36

      graph   IN  A           31.13.77.6

  7. sudo vi for.miniclippt.com
    1. $TTL 86400

      @   IN  SOA     pri.miniclippt.com. root.miniclippt.com. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.miniclippt.com.

      pri     IN  A           1.2.3.4

      mobile-live-v5-0.agario     IN  A   52.52.55.140

      mobile-live-v5-0.agario     IN  A   54.183.177.123

      mobile-live-v5-0.agario     IN  A   52.8.170.192

      mobile-live-v5-0.agario     IN  A   52.9.37.138

  8. sudo vi for.amazonaws.com
    1. $TTL 86400

      @   IN  SOA     pri.amazonaws.com. root.amazonaws.com. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.amazonaws.com.

      pri     IN  A           1.2.3.4

      prod-miniclip-v3-881814867.us-west-2.elb  IN  A 52.42.253.135

      prod-miniclip-v3-881814867.us-west-2.elb  IN  A 52.39.93.232

      prod-miniclip-v3-881814867.us-west-2.elb  IN  A 52.43.226.3

  9. sudo vi named.conf.local
    1. # Append this to the file:

      zone “agar.io” {

      type master;

      file “/etc/bind/for.agar.io”;

      };

      zone “facebook.com” {

      type master;

      file “/etc/bind/for.facebook.com”;

      };

      zone “amazonaws.com” {

      type master;

      file “/etc/bind/for.amazonaws.com”;

      };

      zone “miniclippt.com” {

      type master;

      file “/etc/bind/for.miniclippt.com”;

      };

  10. sudo vi named.conf
    1. # Append this to file:

      logging {

      channel query.log {

      file “/var/log/query.log”;

      severity debug 3;

      };

      category queries { query.log; };

      };

  11. Make sure that the service can read/control its configuration files:
    1. sudo chmod -R 755 /etc/bind
    2. sudo chown -R bind:bind /etc/bind
  12. sudo vi /etc/apparmor.d/usr.sbin.named
    1. # Insert this line inside the “/usr/sbin/named {” section

      /var/log/query.log w,

  13. Create an empty log file, change ownership and make sure that the service can write to it
    1. sudo touch /var/log/query.log
    2. sudo chown bind /var/log/query.log
    3. cat /etc/apparmor.d/usr.sbin.named | sudo apparmor_parser -r
  14. Verify that the configuration files will parse correctly:
    1. sudo named-checkconf /etc/bind/named.conf
    2. sudo named-checkconf /etc/bind/named.conf.local
    3. sudo named-checkzone agar.io /etc/bind/for.agar.io (repeat for other zone files)
  15. Stop/start the DNS service:
    1. sudo systemctl restart bind9
  16. Follow the instructions from step 11 in the Node.js section to verify that the DNS server is running, substituting the IP address of the Ubuntu server.
  17. As before, configure the iPad to use your server’s IP address and test the Agar.io app
  18. You can watch what the app is querying from your server, giving you insight into how many ad servers are actually involved: tail -f /var/log/query.log
  19. When you are completely finished, you may stop the DNS server:  sudo systemctl stop bind9

That’s it. I’ve described how to setup two different DNS servers which should effectively cheat the ads you’d normally see during Agar.io game play.

And now, I think I’ll settle into some uninterrupted Agar.io and all without having to unnecessarily stop the game to shutdown some long-running/buggy ad attempt (losing my earned XP points).

hacking agar.io, part 2

This would be the second post in a series. You might want to read the first in the series if you haven’t already done so. Here, I continue with the work related to redirecting the game’s server traffic to my own website so that I can discover the interface.

DNS server

I first install Dnsmasq on my MacBook, add a single entry to its /etc/hosts file to redirect traffic for m.agar.io to my MacBook’s private IP address. Starting up Dnsmasq I then have a DNS server which will redirect game traffic to my own website. Make sure that the program is running by entering ps aux|grep dnsmasq|grep -v grep. You should see an entry for this program.

It’s probably a good idea to test your DNS server to verify that it returns the expected information.

nslookup
> server myip
> m.agar.io.
> exit

After entering the third line above you should see a DNS lookup which returns your server’s private IP address.

Our website

In my ~/sites folder, I run the following command to use Express to generate a generic website: express agar. As is usual for Express, I change into the newly-created agar directory and then run npm install to bring in the dependencies. Since the default installation binds to an upper TCP port and we want the standard port 80 instead, I then edit the bin/www file in this folder and replace the port number 3000 with 80 on a single line.

Note that Node.js, the underlying program that serves up an Express website, will not be able to bind to port 80 since it’s reserved unless I’m running as the root user. If your own user is setup to run the su command then you should be able to start this website with the command su npm start in the agar folder. Otherwise, you’ll have to run just su to become the root user, navigate back into your user folder area to find this folder and then just run npm start instead.

It’s probably a good idea to test the website by bringing up Safari and entering the address http://myip/ (substituting my private IP address) to see if it works.

Configuring the iPad

At this step, I’ll need to tell the iPad’s Wi-Fi configuration to use my own DNS server first and then the existing set of DNS servers next. You’ll find this under Settings -> Wi-Fi -> select the i button next to your own connected Wi-Fi network -> DHCP -> DNS -> prepend your own server’s private IP address and a comma at the beginning of the list.

This is the initial preparation for redirecting the game traffic to your own website. Note that the Node.js website while running will write to its log file and this will be our method of discovering the interface for Agar.io.

Discovery phase

By now attempting to play the Agar.io game on the iPad, it makes requests to what it thinks is the server. Only these requests are now being sent to my website instead. As each attempt is logged as a failure on my own website, I then make this call manually in another computer to the actual Agar.io website to see what it’s supposed to return.

For example, the game makes a request to the game server’s interface with just /info as the URL.

/info returns:

{"regions":{
  "CN-China":{"avgPlayersPerServer":368.5,"avgPlayersPerRealm":147.4,"numPlayers":737,"numServers":5,"numRealms":5},
  "US-Atlanta":{"avgPlayersPerServer":445.1034482758621,"avgPlayersPerRealm":203.2755905511811,"numPlayers":25816,"numServers":267,"numRealms":127},
  "EU-London":{"avgPlayersPerServer":430.6,"avgPlayersPerRealm":191.37777777777777,"numPlayers":8612,"numServers":179,"numRealms":45},
  "SG-Singapore":{"avgPlayersPerServer":546.0,"avgPlayersPerRealm":136.5,"numPlayers":1092,"numServers":9,"numRealms":8},
  "Unknown":{"numPlayers":0,"numServers":0,"numRealms":0},
  "BR-Brazil":{"avgPlayersPerServer":333.3220338983051,"avgPlayersPerRealm":200.6734693877551,"numPlayers":19666,"numServers":181,"numRealms":98},
  "RU-Russia":{"avgPlayersPerServer":473.25,"avgPlayersPerRealm":145.6153846153846,"numPlayers":1893,"numServers":45,"numRealms":13},
  "JP-Tokyo":{"avgPlayersPerServer":460.0,"avgPlayersPerRealm":172.5,"numPlayers":1380,"numServers":8,"numRealms":8},
  "TK-Turkey":{"avgPlayersPerServer":287.14285714285717,"avgPlayersPerRealm":154.6153846153846,"numPlayers":2010,"numServers":30,"numRealms":13}
  },
  "totals":{"numPlayers":61206,"numServers":724,"numEnabledServers":317,"numRealms":317}
}

As you can see, this is a fair bit of information. The format is known as json in case it’s not familiar to you. As of my writing this, there appear to be over 61,000 players in the game right now and well over 700 servers with almost half of those enabled. So this would be why it’s difficult to get a simultaneous FFA game with your friends—the odds are against you.

Without further ado, here are the other queries which I discovered.

/ returns:

37.187.171.110:1523
8QJP8

This appears to be your issued server and port on the first line and what is likely its instance alias from whichever cloud-based company they’re using.

/getLatestID returns:

131

I know, not very impressive. But it appears to be the highest user ID for your issued server.

/findServer returns:

{"ip":"151.80.98.52:1516","token":"86JYH"}

Another json response, this appears to also be issuing you a server and port. It’s possible that the first home query is asked at the beginning of the game and then /findServer is called each time your die in the game.

So far, this appears to be everything I’ve learned from this redirection technique.

Status

At this point, I now have the game interface which the Agar.io app uses to communicate with the server. It likely makes more requests but that’s good for now. I could have enough to go on in order to work up something so that multiple iOS people could join the same FFA game, for example, since we know this issuing mechanism.