raspberry pi 3 with node.js

First day with the Raspberry Pi 3 and I’m already having way too much fun with this! Today, I decided to see how fast I could get a Node.js—based webserver running on it and was not disappointed.

Out-of-the-box it comes with the NOOBS installation manager at bootup which politely asks you to install an operating system from the selection. I chose the default Raspian operating system because it’s basically a Debian project and well-trusted.

My task was made simpler because I’d purchased one of the Raspberry Pi 3 kits which included a microSD card pre-loaded with NOOBS. I’d suggest doing the same since the markup over a standard 16GB microSD wasn’t much, to be honest. The empty Raspberry Pi 3 board costs about $35 these days but the kit at $60 includes the NOOBS-preloaded microSD and a 5V power adapter. My kit also included a basic/clear case but I opted for a $9 red case which seems to be more standard.

Setup

For the initial setup, you need spare Ethernet and HDMI-to-DVI cables, USB mouse, USB keyboard and a monitor. Everything gets plugged in before adding the power adapter to boot it.

One thing to note if you’ve just purchased one and are confused, you’ll need to remove the microSD adapter from the included SD adapter before inserting the former into the Pi. They include the larger SD adapter because some people might need that, in theory, to insert it into their workstation if the need arrives. Store the adapter for future use.

microsd

I found that the latest release of Raspian somewhat hides the wi-fi configuration compared to earlier versions. You’ll find what you need by left-mouse clicking the network symbol in the upper taskbar. I also found it necessary to manually edit the /etc/network/interfaces file in a terminal session to change the wi-fi from manual to dhcp. I would imagine that subsequent versions of Raspian will fix this however, you might not be able to easily update the operating system until you’ve achieved a working network connection in the first place.

Headless Boot

Confirming that the networking was happy, I shutdown the Pi, removed everything but the power adapter and booted it again, doing everything next remotely via ssh or RDP.  First I updated the software via a terminal session and then installed tightvncserver, xrdp and samba. I configured a shared drive for use later in /etc/samba/smb.conf file and created a Samba network user for the default pi operating system user. Next, I tested remote connectivity from both a Mac and a Windows-based computer and all looked good for the share.

Next, I installed Node.js globally from a Nodesource.com—based distribution as well as the express-generator. I created a ~/share/sites subfolder and then used Express to generate a new hello application below this. I then started up the newly-created website.  Fortunately, the Samba client recently installed publishes its own hostname to the DHCP server so that I could then surf the website from Windows without knowing its IP address as seen here. The default port for an Express website is 3000, btw.

raspberrypi_webserver

Since I’d created a network share earlier I simply edited the default Express-generated files using Wordpad on my Windows—based computer remotely.

What Next?

From here, I’ll likely be investigating earlier projects like the magnetic card swipe visitor station to convert it to using a Raspberry Pi 3 instead of a larger workstation. By adding a USB-based drive you could likely repeat the earlier Kloudspeaker project with a much smaller footprint.

I intend to create a squad of four Raspberry Pi 3’s in a housing which includes an Ethernet switch, a common power supply and a fan all in a self-contained housing which promotes good airflow and optional stacking. In this case, I’d need to change over to an Ubuntu operating system, I think, since some of the provisioning tools there might make something like this easier. I’ll ultimately have a supercomputer for a fraction of the cost of what you might expect to pay.

In a follow-up post, I’ll be discussing my attempts to install and evaluate the Ubuntu Core for the Raspberry Pi 3 operating system using a second microSD which I purchased for this purpose.

Advertisements

a rose by any other name…

Unfortunately for anyone who uses the express generator for Node.js, the very popular (default) jade template language’s owners are being forced by the threat of lawyers to relinquish the name.

It turns out that JADE happens to also be the trademarked name of a software development system that nobody’s ever heard of before this.  It was created by Jade Software Corp and presumably, they have lawyers whose children need braces and their own Mercedes Benz.

At the time of writing this, the jade Node.js module is hugely popular and had over three million downloads in the last month alone and over 40 million downloads since last year. But since the developer of jade is being forced to change the name that means that everyone who uses it is being forced to change their own projects with the new name. Note that express has been downloaded more than 72 million times and this is a project generator (capable of being run multiple times for multiple projects per coder). It’s staggering to try to quantify in terms of dollars how much labor is about to be wasted on this effort for literally a hundred million projects or more.

“But since the developer [of jade] is being forced to change the name that means that everyone who uses it is being forced to change their own projects with the new name. It’s staggering to try to quantify in terms of dollars how much labor is about to be wasted on this effort for literally a hundred million projects or more.”

The module jade has now been re-introduced as pug… or at least until someone who has trademarked that term doesn’t threaten them with a lawsuit, too.

I think it’s time the open-source community push back. This is ridiculous, to think that owning a trademark can prevent another person from using the term for something as unrelated as this. Conservatively, they’ve just cost the open-source community over US$1B in fixing this name change throughout the collection of projects out there.

magnetic card readers

Continuing in the series of fun recycling projects for these Dell Vostro 200 computers, I purchased a MagTek Dynamag USB-based swipe card reader for a new project to track visitors in the office’s entrance.

dynamag

Overview

The solution includes an Ubuntu server which only runs a single application to receive the card swipe details, to find the Track 1 data and to then submit this information to a Microsoft IIS—based website. The pages here then offer up an API for the incoming card swipe details and an administrative page for registering new visitors, reviewing the log details and the visitors as seen, to include the ability to export those details to Excel. The data is stored in a Microsoft SQL Server database.

To save cost, I decided to have each visitor just use a magnetic swipe card already in their wallet. In this way, I wouldn’t have to buy a magnetic card encoder, purchase card blanks nor worry about designing or issuing them (or trying to get them back later).

Typical Cost of an Access Control System for Visitors

From this webpage 2016 Average Card Access Cost

“Expect to pay an average of $1,500 to $2,500 per door for a high-quality system for up to 150 people. … It will cost an average of $1,000 to $1,500 for the hardware for the door. It will cost an average of $3 to $5 per keycard. Monthly service fees can range from $10 to $100 per month.”

My total cost for this project was just eight hours’ labor plus the $48.36 for the card reader! This included the database and website coding plus the I.T. work to setup the Ubuntu server and to create the Python script to talk to the USB-based reader, to test everything and to write up the documentation. This isn’t bad, considering the cost of an average system.

kloudspeaker

In my quest for recycling old Dell Vostro 200 computers here at work I’ve recently had the pleasure of creating a document management system using the open-sourced Kloudspeaker software. It appears to be a mature project and installs easily.

It’s important to read through the documentation. Out-of-the-box it doesn’t do file uploads, for example, but the configuration may be modified to do so. And I found an issue with files larger than 2MB which again was fixed by reading the docs. It helps to have some reasonable system experience on your UNIX server because some of these issues are dealt with outside of the Kloudspeaker software’s configuration itself.

kloudspeaker_login

The interface is attractive throughout.  The upper menu is responsive to changes in window size, buttons (like Upload) will optionally appear if the feature is turned on and if the user has the correct rights.  Overall it just seems to work without problems.

kloudspeaker_documents.png

Typical Cost of a Document Management System

From the webpage How Much Does a Document Management System Cost?

“[For a cloud-based system] …you should expect to invest between $20 and $50 per person per month, with the major variable being the level of service you subscribe to.

“[For an on-premised system] To start, you’ll have an up-front charge for your server and desktop licensing. This licensing tends run between $350 and $500 per person on average. Depending upon which features you sign up for, you’ll need to purchase the appropriate number of servers at the average rate of $5,000 to $6,000 per device. Ongoing support and maintenance will take the form of an annual per person fee, which will typically cost you around $40 per seat.”

Installation

Below is the list of things I needed to do to get the server running. I managed to do all of this in just under two hours. It then took me another two hours of finding over 300 contracts so that I could upload them into the system.  All-in-all, that’s not bad (four hours’ labor + $0.00) for an excellent document management system that could have cost a lot of money!

  1. Boot from a (free) GParted disc on a Dell Vostro 200 computer to remove any existing partitions
  2. In BIOS, set the computer so that it will run without a keyboard without stopping during POST
  3. Boot from a (free) Ubuntu disc and install it as a server
    1. Choose server install
    2. Use entire disk with LVM
    3. Install GRUB
    4. Install OpenSSH server and LAMP
  4. Edit the /etc/network/interfaces file to give the server a static IP address
  5. Edit the /etc/apache2/apache2.conf file to give the webserver its name
  6. Create a CNAME record in my DNS server of contracts for this Intranet server
  7. Run sudo apt-get update, sudo apt-get upgrade and sudo do-release-upgrade to make sure that I’m using the latest software
  8. Run sudo apt-get install unzip to install this necessary software
  9. To pull the latest zip file from their website, in my home directory run the command  wget http://www.kloudspeaker.com/download/latest.php, run unzip to expand the file structure and then copy this recursively to the /var/www/html area under a kloudspeaker folder name, remembering to use the -a option for the sudo cp command to get hidden files as well
  10. Log into MySQL as the root user with mysql -u root -p and then from this console, run create database kloudspeaker; and then run exit; to come out of the MySQL client
  11. Edit the /var/www/html/loudspeaker/backend/configuration.php file, setting the appropriate database credentials for the newly-created kloudspeaker database, as well as adding "EventLogging" => array() to the list of plugins
  12. Edit the /etc/apache2/sites-available/000-default.conf configuration file to change the DocumentRoot to /var/www/html/kloudspeaker
  13. From another computer, visit the http://contracts/backend/install link to start their installation, verifying the database credentials and next, creating an administrative user for the software
  14. Back in a terminal, manually create a folder at the root with sudo mkdir /files and subfolders below this to hold each subgroup
  15. Run sudo chown -R www-data:www-data /files to make sure that the Apache service user can control the files uploaded into the system
  16. Edit /etc/php/7.0/apache2/php.ini to add these two lines: upload_max_filesize = 40M and post_max_size = 40M
  17. Run sudo apt-get update and sudo apt-get upgrade again to verify that your system is up-to-date after bringing in software
  18. Again from another computer, log into the website interface, creating users, groups, adjusting the access rights to the folders you’ve defined, making sure to test as a non-administrator user to verify the results
  19. Shut down the Ubuntu server, remove all peripherals and put the computer into the server room with only Ethernet cable and power cord connections, turning it back on

Using Kloudspeaker

Now that everything’s installed and setup, you add the user accounts for the people who also need access. You use their email address so it will invite them to create their own passwords.

I’m very pleased with the way this has turned out and how little effort was required to get it going. I’d highly recommend this software for businesses.

hacking agar.io, part 5

I guess now anyone who’s been following will also want to a chance to play Agar.io without ads. Here are the step-by-step instructions.

Note: Throughout, I’ll use 1.2.3.4 as the IP address of the DNS server you’ll be creating. Assume that every time you see this, you’ll be substituting your own server’s private IP address. Any other IP address you see should be typed in exactly as I’ve shown.

I’ll be including instructions for two different DNS servers. Choose the one that makes more sense for you based upon your experience.

Node.js DNS server version

Since I like JavaScript, here’s a Node.js implementation which may be augmented to include a nice HTML administrative interface if you’d like. I haven’t gotten quite that far yet but you can see what it takes to host a DNS server and a webserver all in one application.

  1. I assume that you already have Node.js installed, as well as npm and the express-generator. If not, you’ll need to install each first.
  2. Open a terminal
  3. Change to your home directory and optionally, change into a subfolder like ~/Sites like I did. Create one if necessary with: mkdir ~/Sites
  4. Run the express command to generate a new project:  express one-trick-pony
  5. If that ran correctly, change into the newly-created folder:  cd one-trick-pony
  6. Run the npm command to install the dependencies:  npm install
  7. Determine the IP address of your server and save this information for later: ifconfig | grep en1
  8. Run the npm command to install dnsd into your project (those are two hyphens without a space between them):  npm install dense –-save
  9. Edit the www file:  vi ./bin/www
    1. After this line var http = require(‘http’); add the indicated text seen in the block quote below
    2. After this line server.on(‘listening’, onListening);, optionally add the line:  console.log(‘Webserver running at *:3000’);
  10. Determine the path of the node command you usually use and save this information for later:  which node
    1. Run the su command to elevate into superuser (root) mode:  su
    2. Change to the working folder from before: cd /Users/yourname/Sites/one-trick-pony
    3. Run the node command giving a full path to the executable, which you found in the earlier step: ../../local/node/bin/node ./bin/www
    4. At this point, you should see that the server is running, indicating that it’s listening to two different ports:  53 (DNS) and 3000 (HTTP).
  11. From a workstation you can verify that the DNS server is running with the indicated command, noting that the server should still be logging requests:  dig @1.2.3.4 www.agar.io
  12. Now from the iPad, for example, go to Settings -> Wi-Fi -> select the i logo next to your connected local wi-fi zone -> DHCP -> DNS -> (write down everything here and save it), overwrite it with 1.2.3.4 (your server’s private IP address)
  13. Press the Home button twice and if Agar.io is running, swipe up to remove it from memory
  14. Start up the Agar.io app and verify that it logs in (even with Facebook), it works AND it no longer displays advertisements.
  15. When you’re finished, in Settings -> Wi-Fi, either “Forget This Network” your existing local wi-fi profile (re-entering your password) or manually re-enter the earlier DNS information that you wrote down from an earlier step.  Your iPad is now ready to behave like before.
  16. When you’re completely finished, go back to the server’s terminal session and press Ctrl-C to end Node and then enter the exit command to leave the su session.

Code to add into the ./bin/www file:

var dnsd = require(‘dnsd’);

function dns_handler(req, res) {
console.log(‘%s:%s/%s %j’,
req.connection.remoteAddress,
req.connection.remotePort,
req.connection.type,
req);

var question =
res.question[0],
hostname = question.name,
length = hostname.length,
ttl = Math.floor(Math.random() * 3600);

if (question.type == ‘A’) {
// Agar.io website
if (hostname == ‘agar.io’ || hostname == ‘www.agar.io’ || hostname == ‘m.agar.io’) {
res.answer.push({name:hostname, type:’A’, data:”104.20.26.122″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”104.20.25.122″, ‘ttl’:ttl});
}
// Facebook.com authentication
if (hostname == ‘facebook.com’) {
res.answer.push({name:hostname, type:’A’, data:”31.13.69.228″, ‘ttl’:ttl});
}
if (hostname == ‘www.facebook.com’) {
res.answer.push({name:hostname, type:’A’, data:”31.13.77.36″, ‘ttl’:ttl});
}
if (hostname == ‘graph.facebook.com’) {
res.answer.push({name:hostname, type:’A’, data:”31.13.77.6″, ‘ttl’:ttl});
}
// AmazonAWS
if (hostname == ‘prod-miniclip-v3-881814867.us-west-2.elb.amazonaws.com’) {
res.answer.push({name:hostname, type:’A’, data:”52.42.253.135″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.43.226.3″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.39.93.232″, ‘ttl’:ttl});
}
// Miniclippt.com
if (hostname == ‘mobile-live-v5-0.agario.miniclippt.com’) {
res.answer.push({name:hostname, type:’A’, data:”52.8.170.192″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.9.37.138″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”54.183.177.123″, ‘ttl’:ttl});
res.answer.push({name:hostname, type:’A’, data:”52.52.55.140″, ‘ttl’:ttl});
}
}
res.end();
}

var dnsServer = dnsd.createServer(dns_handler);
dnsServer.zone(‘agar.io’,
‘ns1.agar.io’, ‘root@agar.io’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.zone(‘facebook.com’,
‘ns1.facebook.com’, ‘root@facebook.com’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.zone(‘amazonaws.com’,
‘ns1.amazonaws.com’, ‘root@amazonaws.com’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.zone(‘miniclippt.com’,
‘ns1.miniclippt.com’, ‘root@miniclippt.com’, ‘now’, ‘2h’, ’30m’, ‘2w’, ’10m’);
dnsServer.listen(53, ‘1.2.3.4’);
console.log(‘DNS server running at 1.2.3.4:53’);

Bind DNS server version

This version will assume that you have a Linux (Ubuntu, in this case) server or workstation that can run the bind9 service.

Here, I assume that you’re comfortable with commands in a terminal, know what sudo does and can use the vi editor to edit and save a file. You know what touch does. If any of these don’t sound familiar, then this probably isn’t the option for you.

On a Linux (Ubuntu) server, do the following:

  1. Make sure that your system is up-to-date:
    1. sudo apt-get update
    2. sudo apt-get upgrade
    3. sudo apt-get dist-upgrade
  2. Install the DNS service, noting that it will take a fair amount of configuration work
    1. sudo apt-get install bind9 bind9utils bind9-doc
  3. cd /etc/bind
  4. Create four empty files, one per “forward” zone. In the next steps you’ll be editing each, making sure to substitute your own server’s private IP address in each case.
    1. sudo touch for.agar.io
    2. sudo touch for.facebook.com
    3. sudo touch for.miniclippt.com
    4. sudo touch for.amazonaws.com
  5. sudo vi for.agar.io
    1. $TTL 86400

      @   IN  SOA     pri.agar.io. root.agar.io. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.agar.io.

      @       IN  A           104.20.25.122

      @       IN  A           104.20.26.122

      pri     IN  A           1.2.3.4

      www     IN  A           104.20.25.122

      www     IN  A           104.20.26.122

      m       IN  A           104.20.25.122

      m       IN  A           104.20.26.122

  6. sudo vi for.facebook.com
    1. $TTL 86400

      @   IN  SOA     pri.facebook.com. root.facebook.com. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.facebook.com.

      @       IN  A           31.13.69.228

      pri     IN  A           1.2.3.4

      www     IN  A           31.13.77.36

      graph   IN  A           31.13.77.6

  7. sudo vi for.miniclippt.com
    1. $TTL 86400

      @   IN  SOA     pri.miniclippt.com. root.miniclippt.com. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.miniclippt.com.

      pri     IN  A           1.2.3.4

      mobile-live-v5-0.agario     IN  A   52.52.55.140

      mobile-live-v5-0.agario     IN  A   54.183.177.123

      mobile-live-v5-0.agario     IN  A   52.8.170.192

      mobile-live-v5-0.agario     IN  A   52.9.37.138

  8. sudo vi for.amazonaws.com
    1. $TTL 86400

      @   IN  SOA     pri.amazonaws.com. root.amazonaws.com. (

      2011071001  ;Serial

      3600        ;Refresh

      1800        ;Retry

      604800      ;Expire

      86400       ;Minimum TTL

      )

      @       IN  NS          pri.amazonaws.com.

      pri     IN  A           1.2.3.4

      prod-miniclip-v3-881814867.us-west-2.elb  IN  A 52.42.253.135

      prod-miniclip-v3-881814867.us-west-2.elb  IN  A 52.39.93.232

      prod-miniclip-v3-881814867.us-west-2.elb  IN  A 52.43.226.3

  9. sudo vi named.conf.local
    1. # Append this to the file:

      zone “agar.io” {

      type master;

      file “/etc/bind/for.agar.io”;

      };

      zone “facebook.com” {

      type master;

      file “/etc/bind/for.facebook.com”;

      };

      zone “amazonaws.com” {

      type master;

      file “/etc/bind/for.amazonaws.com”;

      };

      zone “miniclippt.com” {

      type master;

      file “/etc/bind/for.miniclippt.com”;

      };

  10. sudo vi named.conf
    1. # Append this to file:

      logging {

      channel query.log {

      file “/var/log/query.log”;

      severity debug 3;

      };

      category queries { query.log; };

      };

  11. Make sure that the service can read/control its configuration files:
    1. sudo chmod -R 755 /etc/bind
    2. sudo chown -R bind:bind /etc/bind
  12. sudo vi /etc/apparmor.d/usr.sbin.named
    1. # Insert this line inside the “/usr/sbin/named {” section

      /var/log/query.log w,

  13. Create an empty log file, change ownership and make sure that the service can write to it
    1. sudo touch /var/log/query.log
    2. sudo chown bind /var/log/query.log
    3. cat /etc/apparmor.d/usr.sbin.named | sudo apparmor_parser -r
  14. Verify that the configuration files will parse correctly:
    1. sudo named-checkconf /etc/bind/named.conf
    2. sudo named-checkconf /etc/bind/named.conf.local
    3. sudo named-checkzone agar.io /etc/bind/for.agar.io (repeat for other zone files)
  15. Stop/start the DNS service:
    1. sudo systemctl restart bind9
  16. Follow the instructions from step 11 in the Node.js section to verify that the DNS server is running, substituting the IP address of the Ubuntu server.
  17. As before, configure the iPad to use your server’s IP address and test the Agar.io app
  18. You can watch what the app is querying from your server, giving you insight into how many ad servers are actually involved: tail -f /var/log/query.log
  19. When you are completely finished, you may stop the DNS server:  sudo systemctl stop bind9

That’s it. I’ve described how to setup two different DNS servers which should effectively cheat the ads you’d normally see during Agar.io game play.

And now, I think I’ll settle into some uninterrupted Agar.io and all without having to unnecessarily stop the game to shutdown some long-running/buggy ad attempt (losing my earned XP points).

hacking agar.io, part 4!

Eureka! I’ve managed to totally cheat the ads and still play Agar.io on my iPad! Yesh.

For those of you following along, here are the entries that I had to create in my surrogate DNS server:

  • agar.io (with A records @, www and m)
  • facebook.com (with A records @, www and graph)
  • amazonaws.com (with A record prod-miniclip-v3-881814867.us-west-2.elb)
  • miniclippt.com (with A record mobile-live-v5-0.agario)

This allows the game to startup, authenticate via Facebook’s mechanism and to start the game. Gone are the ads in their entirety.

hacking agar.io, part 3

This would be the third post in a series. You might want to read the first in the series if you haven’t already done so. Here, I continue with the work related to rendering the game server’s ads so that they don’t display at all.

DNS server

It struck me that if I could build a relatively-ignorant DNS server of my own and point my iPad to it then I could control which servers my computer talks to. Remove everything but the minimum and this should work for killing the Agar.io advertisements.

So I would need to use nslookup to find the IP addresses of the servers the game talks to. And since I’m authenticating via Facebook’s mechanism I’d need to educate the DNS server to this as well.

I happen to have an Ubuntu server which is under-utilized at the moment since I’m using it to develop WordPress plugins. So I installed bind9 (the DNS service) to this server and then configured it with some entries and zones:

  • agar.io @, www and m
  • facebook.com @ and www

For each of the entries above, I just used the nslookup command to determine what the normal IP address(es) would be and used those values.

The next step would be to make sure that this bind service does not do recursive lookups, in other words, it won’t ask for help if it doesn’t know the zone in question. So in theory, it will only give answers to the entries I’ve made; anything else will fail a lookup.

Next, on the iPad -> Settings -> Wi-fi -> my zone area, I manually set the only DNS server to be the IP address of my private server. I then confirmed that in a browser I could not resolve Google.com but could see the webpage for Agar.io.

Did it work? Not yet. Upon startup it either is missing a critical server it needs to talk to and the app now can’t do the DNS lookup to find that server. It’s okay. I’ll keep hacking away at this to find out what server that is. I should be able to get this approach to work, I just need to go to school on the application some more to find out.