the rise and fall of the microsoft empire

1975-1980

Our historical timeline begins in 1975 when an unlikely duo—Paul Allen as Batman and Bill Gates as his awkward “Boy Wonder”—started Microsoft Corporation.  I’m guessing that ro-sham-bo was involved in this decision but incredibly somehow Bill was made the CEO when the company got its start.  Maybe dropping out of Harvard gives you that kind of confidence.

1981BillPaul

1981-2000

Nothing really significant happened until they managed to modify an existing operating system for the IBM PC in 1981 from another company and rename this to MS-DOS. Significant sales of the IBM series of computers and those of their competitors then launched a thirty-year stretch of dominance in the business world in the area of operating systems, software and development platforms.

For most of us, we reasonably dismissed Apple’s hardware and the MacIntosh operating systems as nothing we could seriously use in business outside of the marketing department.

Consumers bought new versions of software and that license was good for life.  It could often be transferred from one computer to the next as long as the last one was de-registered first.  If you built software for Windows, you likely used a Microsoft compiler to do so and you paid for that.  In fact, the Microsoft Technet collection of CDs was quite expensive.

2001

About six years into the “Internet Tidal Wave” as Bill would call it, Microsoft was starting to lose its way.  They tried to dominate in the browser wars but never quite managed to quash the competition.  Others saw their efforts in this area as annoying.  Their software for creating programs, Visual Studio, first hit the scene about four years prior to this.

Google was founded some five years prior and was just beginning to get attention from an investor before they had anything real yet.  In 1999 they moved from their garage to an actual building in Palo Alto.  Yahoo’s popularity as a search engine from a decade ago was waning.  Google’s ad-based revenue from keywords was paying off; they’d planted a money tree which eventually created an entire forest of money trees for them.  It wouldn’t be long until Microsoft’s executives behind closed doors would consider Google their biggest threat.

About this time Apple created a very clever method of provisioning content for one-and-only-one device within the music-delivery space.  The iTunes store would turn out to be the goose that laid the golden egg, as seen in the following revenues.  And yet, it would take years for either Microsoft or Google to realize the beauty in this fulfillment model and to come up with their own versions.

showmethemoney

The “Internet of Things” concept started gaining in popularity at this time.

2009

Microsoft’s attempts at copying Google’s success (MSN Search, Windows Live Search, Live Search) now culminated in the introduction of Bing as their default search engine destination for all things Microsoft.

Apple introduced the first iPhone and the first iPad about this time, noting that the same provisioning model from iTunes was incorporated into both via iOS.  The subscription model of sofware licensing was born with this, if you think about it.  If you wanted to write a program for either, you needed to use Apple’s software to do so.

Google has just introduced Chrome as a browser and would begin their campaign to slowly break Internet Explorer.  The same was true of the Android phone and its related operating system.  It would take a few years for Microsoft to catch up to either the iPhone or the Android before releasing their own app-savvy smartphone offering.

Amazon some three years prior had introduced the beginning of what would be a full complement of cloud-based services to support web development.  It would take Microsoft two full years to realize that they needed to be in this space and they didn’t have their offering ready for a few years more, too late to effectively compete.

Github.com had just celebrated their first year online, hosting over 46,000 repositories by then.  The world of open source was the very antonym to the way that software had been developed prior to this.

The free Ubuntu operating system was released about four years prior to this, backed by the well-funded company Canonical.

2015

Microsoft releases Windows 10, “the last version of Windows” (they claimed).  Rumors suggested that Windows would eventually go from a version-based license model to an annual-subscription model with respect to pricing.  I think it’s safe to say that the market hasn’t really embraced either Windows 8 or Windows 10.

The subscription-based model for Office 365 was introduced four years prior to this so the writing was definitely on the wall:  Microsoft wanted to depart from their former methods of making money and to chase the monthly subscription model.

2015-popular-coding-languages

The world of open source was offering new programmers a wealth of free code.  All they had to do was to take it and make it their own.  Formerly, Microsoft-friendly coding languages like C, C++, C#, VB and .NET dominated the playing field but this graphic shows how the game had changed.

2017

And here we are, present-day.  That curious number 42 now describes the number of years that Microsoft has been around.

Yesterday evening, I attended a very geeky meetup of perhaps fifty or sixty coders and only saw one Windows-based laptop.  Almost everyone had a MacBook of some kind.

I just spent about two hours today installing the free Visual Studio Community 2017 software so that I could—in theory, anyway—alter a free copy of the source code for TightVNC software.  Out of the box, so-to-speak, Visual Studio doesn’t want me to build this project since it uses an earlier target platform (Windows 7 or 8, one would assume).

Microsoft only wants me to make things for Windows 10.

So rather than making it easy for me to build a program that will happily work with Windows 7, they’re forcing me to jump through hoops in order to add the necessary pieces for this to happen.

Add two more hours to this and I find that my installation does not want to download the earlier pieces to allow this to happen.  I’m forced to then upgrade the code to Windows 10 compatibility mode… only to find that the build fails with 528 errors.

The main crux of all these errors appear to be:  “we can’t find common files”.  It’s a very amateur sort of error from a company that’s been providing compilers for several decades now.

I have to think that Microsoft doesn’t want me to do anything with Visual Studio unless it benefits Microsoft.  And this is the core of the reason why I suggest that they’re doomed.

Every time a coder like myself runs into obstacles like these, the usual seed that’s planted inside their head is “this would be easier with another free compiler or another language from someone else”.

2022

Fast-forward another five years and Microsoft will have lost ground on many fronts.  New software development here, there and everywhere will be via some language which wasn’t popularized by Microsoft on computers which aren’t Windows and with browsers which aren’t Internet Explorer or Edge.  Our toasters and refrigerators and our cars will be powered by the Ubuntu operating system or perhaps Debian, a similar free Linux flavor.  These appliances will be connected to our wi-fi and even to the Internet but there won’t be a scrap of anything Microsoft about them.  They’ll be coded up with something that isn’t C#, doesn’t use .NET and doesn’t need Visual Studio in order to compile it.

The only thing with a Microsoft pedigree with some staying power could be some of the websites and services currently served up at Microsoft’s datacenters via Azure.  But Amazon or Google could kill that by simply lowering their own prices for cloud-based services.

Advertisements

the 21st century digital résumé

It used to be that a programmer’s résumé was a single sheet of onionskin paper (expensive, semi-transparent) with a carefully-selected collection of one’s job history and such. Parts of what you were trying to “sell” to the would-be employer were your wordprocessing and layout design skills.

Now, everyone can type, has access to printers/computers and Microsoft Word. In fact, they can even select an attractive template from the many offered so it’s not like much skill is now involved in those areas. At one time, spellchecking was an activity that involved a Webster’s dictionary. The world has changed.

Github

To be an open-source programmer, you must now have a public set of repositories on github.com or so it seems.  My collection of repositories on github.

jsfiddle

As of today, I now have my very own jsfiddle.net collection. My public dashboard on jsfiddle.

WordPress blog

More and more, programmers are encouraged to be social and outgoing enough to want to communicate to others. Obviously, you’re here already so you have my blog’s address.

Slack

And part of that “being social” requirement now seemingly includes spending a fair amount of time during your life chatting with others within the coding space. Since slack.com projects appear to be project-centric rather than coder-centric, there doesn’t appear to be a way of publicizing your identity outside of a particular team URL.

Website portfolio

And then of course, potential employers want you to highlight several existing website concepts in which you either participated or you directly own them.

sexism in the codespace

Lately, there’s been a push to convince girls to learn how to be software developers. Inside these initiatives, the individuals who are part of the movement are attempting to empower people to achieve more with their lives. I would suggest that this is the best part of this new idea.

But what if you pulled back the curtain to reveal that there’s a secret business reason behind all of this outside these initiatives? What if big business is the real driving force here? What could they possibly want out of potentially doubling the workforce within the software development space? Obviously they want what they already have in the clothing industry: sweatshops.

“What could they [big business] possibly want out of potentially doubling the workforce within the software development space? Obviously they want what they already have in the clothing industry: sweatshops.”

sweatshopnoun

A factory, especially in the clothing industry, where manual workers are employed at very low wages for long hours and under poor conditions.

Think about it for a moment. Is Google [US$90B/year] such an Andy Griffith—friendly company that it wants to empower young girls to realize their potential… or does it want more profits?

Simple economics at work

The economics of supply and demand suggest that if you double the supply of software developers then the price for software development projects will reasonably be half. If Google can flood the marketplace with coders then the price for development should plummet as a result.

The following photo is a typical Nike factory. You might ask yourself why all of the factory workers are young girls. Well, they work for a fraction of the cost of their male or older counterparts in the workforce. And given what we know about dominance and submissiveness, young girls will more likely just do what you tell them to do without rebelling (against horrible working conditions, for example).

nike

Sexism

Imagine how you’d feel if suddenly boys and men were being “sold” this idea that they should be coding right now at the same rate that young girls are being advertised to. It would seem weird and creepy, actually. A quick Google search will show you just how many of these girls + coding websites, organizations and meetups have recently been spawned.

A great litmus test for sexism, racism or any “-ism” is to just change out the race and gender, for example. If the result sounds wrong then by definition the original was also wrong for the same reasons.

Examples from actual websites

Here, I make those changeouts to highlight what I’m seeing:

http://www.girlswhocode.com => http://www.boyswhocode.com

http://www.girldevelopit.com => http://www.boydevelopit.com

http://www.blackgirlscode.com => http://www.whiteboyscode.com

If the changed-out version sounds politically-incorrect then the original is by definition just as wrong.

Who to trust

So who do you trust at the end of the day? On one side you have a number of friendly-sounding organizations who seem to be working to make things better for girls and women. On the other side you have a collection of corporations who have a habit of maximizing their profits by using unsustainable working conditions.

Feel free to join the software development field regardless of your gender or race. Just don’t be fooled by big business into working in a coding sweatshop, if you will, because you’re desperate for work and because you lack the confidence that some of your co-workers possess.

magnetic card readers

Continuing in the series of fun recycling projects for these Dell Vostro 200 computers, I purchased a MagTek Dynamag USB-based swipe card reader for a new project to track visitors in the office’s entrance.

dynamag

Overview

The solution includes an Ubuntu server which only runs a single application to receive the card swipe details, to find the Track 1 data and to then submit this information to a Microsoft IIS—based website. The pages here then offer up an API for the incoming card swipe details and an administrative page for registering new visitors, reviewing the log details and the visitors as seen, to include the ability to export those details to Excel. The data is stored in a Microsoft SQL Server database.

To save cost, I decided to have each visitor just use a magnetic swipe card already in their wallet. In this way, I wouldn’t have to buy a magnetic card encoder, purchase card blanks nor worry about designing or issuing them (or trying to get them back later).

Typical Cost of an Access Control System for Visitors

From this webpage 2016 Average Card Access Cost

“Expect to pay an average of $1,500 to $2,500 per door for a high-quality system for up to 150 people. … It will cost an average of $1,000 to $1,500 for the hardware for the door. It will cost an average of $3 to $5 per keycard. Monthly service fees can range from $10 to $100 per month.”

My total cost for this project was just eight hours’ labor plus the $48.36 for the card reader! This included the database and website coding plus the I.T. work to setup the Ubuntu server and to create the Python script to talk to the USB-based reader, to test everything and to write up the documentation. This isn’t bad, considering the cost of an average system.

digital inflation

I spent most of the morning retiring an old Compaq Presario server; it’s perhaps fifteen years old. It was in with some things in storage and I thought I’d get rid of it since the hardware wasn’t even compatible with an Ubuntu server install attempt.

Less Was More

I realize in going through the motions of archiving all my many coding projects from years ago just how much bloat we’ve taken into our computers and our computer languages these days. I think the laptop I’m on right now has 8GB of RAM and this NT 4.0 Server only had 384MB of RAM, running instances of IIS, SQL Server, NT Server, WINS, DNS Server as well as a VSS server. It also hosted QuickBooks Pro 99, Adobe Photoshop 5.0 LE and Visual Studio. Come to think of it, it also easily ran my own NT services, custom-made IIS ISAPI filters and custom SQL Server extended stored procedures that I’d written. It had Microsoft Office, Adobe Acrobat and Illustrator, Flash-development tools by Macromedia.

It ran all this on 384MB of memory.  And those fifteen prolific years’ worth of accumulated everything only resulted in about 4GB of storage, perhaps the equivalent of a mere three movies on my laptop now. I almost have to laugh as I store it on a terabyte external drive.

Digital Inflation

graph

Seriously, though, what have we gained by making everything so heavy? I recall being able to accomplish anything I needed to do in that older version of Adobe Photoshop and using a fraction of that 384MB of memory. Just now, it took Adobe Photoshop CC about five minutes to load up so that I could paste this graphic in and start to work on it. I select the Spot Healing tool and begin clicking.  One, two, three… and then I wait as the tool freezes up and I have to wait for the spinning cursor to resolve itself. I then Step Backward to remove the garbage that the tool added and try to repeat, only this time slower. This sucks. I know that older version from over ten years ago didn’t do this.

So now, every day we get faster and faster microprocessors with multiple cores. But that Windows 10 upgrade from this year demanded that we have no less than 4GB of RAM just to install it. Why?

The answer is that we’re under a form of inflation that’s taken over the digital world. The same resources we had last year just aren’t good enough.

Back in this post I suggested that open source projects are suffering from this bloat, too. Big companies like Google believe that programs have to be big to be good. I disagree. Sometimes quality and project size are in direct opposition with each other. The more code you have, the more code that could be potentially bad.

Who Do We Blame?

Is it the microprocessor manufacturers who are behind this? I don’t think so. How about the operating system makers like Microsoft and Apple? Probably. I do know that .Net is a huge, bloated layer of code that’s supposed to be Microsoft’s version of Java. But the reason for both Java and .Net is to write machine-independent code. And since nobody really writes .Net code to run on an Apple computer or on a Unix box then what’s the point?

And now that .Net has seen its heyday Microsoft is ready to do the “new, new thing” which is to chase Apple’s app-based iTunes-delivered store. So we as consumers picked up this thick layer of code in the form of .Net which honestly does little for us. And yet most of the software written for a Windows-based computer has to use this foundation. No wonder it takes so much to do so little.

I’d like to blame Adobe’s bloat on all the code which is designed to permission their new subscription-based model. Try to buy Adobe Acrobat now and you’re left with the choice of paying $500 or something like $20-per-month. Neither option is worth it, in my humble opinion. Much of that startup lag I mentioned before could be the client app talking to mothership Adobe to see if my licensing this month is paid.

Predictions

Honestly, the pendulum has to swing back the other way. Consumers will reject subscription-based pricing models, will turn in greater numbers to open source operating systems and desktop tools and eventually the big players will come back with their apologies and revised ideas about how to win back their former customers.

In earlier times you couldn’t expect an average computer person to use a command line interface. But younger computer users are trained in public school and they’re not so timid. Strangely enough, Microsoft is turning to a similar mechanism to do advanced things in their software using PowerShell commands. And there is even an option to install their server software without a GUI environment at all… like a UNIX server, if you think about it.

The original IBM PC didn’t have a Windows interface since that didn’t come until many years later. The very popular (free) Ubuntu server software now does the same. You’d be surprised how much work the computer can do when it’s not unnecessarily displaying graphics.

I believe that we’ll eventually build simpler interfaces. The Windows 10 “Metro” menu and those of smartphones now are visually simpler if you think about it. They’re essentially flat squares that you can push with your finger rather than the fussy-little, 3D-styled buttons from twenty years ago in Windows 95.

Hardware like Google Glass may remove the need for such specialized interfaces. Since this hardware doesn’t include a keyboard your ability to interact with the interface is limited to pointing your head toward a spot on the screen and holding it until it’s selected. When they add voice commands to this interface we’ll see yet another revolution in how we expect software to behave. Hopefully we’ll get to the point where there are no more buttons to push—all commands would be accomplished verbally in your own language of choice.

Back to simpler interfaces, however, must we have all the visual candy? Could we not focus on the work to be done, the spoken commands to trigger that work and remove everything else but the text-based status? On today’s hardware we could do that now if we really wanted to. But since companies like Apple/Microsoft/Google want their store-like delivery model we’ll likely not get what we want unless we build it ourselves.

trial version annoyances

I had a quick-and-dirty task to do today at work:  I wanted to write a very simple program which would split an Adobe PDF document into its individual pages. It didn’t sound like a difficult thing to accomplish, to be honest. By the end of the day, however, I find myself in hacker mode, putting much more effort into doing an end-run around someone’s idea of security.

split-pdf

Options

Of course, this is relatively easy on OS X in the Automator utility. You can create a service, associate it with a folder, say, and then drag/drop a PDF into that folder. Done.

But this needed to be for Windows-based computers and I had a preference to do this in C# within Visual Studio if there wasn’t an easier way of doing it otherwise. Researching a bit I confirmed that there weren’t any native tools within Windows which would take care of this. Next, I then looked for free libraries or similar. This search turned up:

  1. iText (ruled since it’s just a .Net wrapper over Java)
  2. PdfBox.net (ruled out since it’s just a .Net wrapper over Java)
  3. Spire.pdf
  4. Aspose.pdf

And yet, each of these seems to expect money from me in order to build a solution. Granted, somebody probably put a lot of effort into these libraries. I remember myself creating a very nice one-pass XML-to-PDF compiler perhaps ten years ago and was very fond of it. Perhaps it was that experience that led me to the solution I chose: I decided to use Aspose.pdf and then programmatically render their trial-version watermark void.

You might be thinking, “why don’t you just pay for the library?” That’s a good question. The people who wrote Aspose.net expect me to minimally pay $799 per year just to be a developer. And then, presumably, each client would also need to pay this amount for a licensed DLL. They have seven even higher pricing tiers into the many-thousand area. Given the need to simply split a PDF file, I don’t see the value.

The Difficulty of Starting From Scratch

Granted, I could begin from scratch and write a PDF “tree-walker”, find the pages, iterating through them to re-create the content page by page. Since I understand the underlying storage method in a PDF file this could be done in under a month. I could then build this into my own library and charge money for it, presumably cutting the knees out from under these players in the market space.

That said, splitting a PDF file isn’t an $800 problem nor is it a one-man-month problem. A program which splits a PDF file should cost about… $10 tops.

The Problem With the Trial Version of Aspose.pdf-generated PDFs

Unfortunately, the trial version of the Aspose.pdf library places an obtrusive watermark at the top of each page.

AsposeWatermark
Example output of the  trial version of the Aspose library

 

 

Programmatically-Removing Watermarks From PDFs

So then, I researched to see if there were any available/free methods of removing watermarks from PDF files. There doesn’t appear to be. I would need to write it myself.

One challenge is the problem is patching a binary file in-place with C#. To be honest, I expected the .Net framework to have something like this but that doesn’t appear to be the case. In addition to hacking the PDF object code I would need to write a rudimentary binary search-and-replace routine for C#.

Hacking the PDF File

It’s good to be familiar with the object storage model for PDF files in order to understand what approach I then took.

A typical PDF file includes many objects and a table at the end which is essentially a table of contents for those objects. If you’re familiar with a Rich Text Format (RTF) file, then it’s much like this except for the catalog at the end.

It’s that catalog at the end that provides the first challenge, when editing a binary PDF file you can’t change the size of an object or move it. Doing so would break the catalog.

The second biggest challenge when editing a binary PDF file is the frequent use of inline compression/encoding. You can’t easily find the actual object that you’d like to overwrite. And yet, with a simple PDF file you can accomplish this by using a hexadecimal editor and iteratively change one character per object until you “break” the object in question, that pesky watermark.

AsposePDF.png
Typical PDF file contents

 

 

The Achilles Heal of Watermark-based Prevention

So now, what would it take to nuke that watermark? One method would be to find the object, physically remove the entire object from the file and remove its reference from the catalog. And yet, then I’d need to update the file offsets for half of the other objects within the file itself.

Inside the body of the PDF file each of these compressed-content objects includes the key to its own demise:  FlateDecode. This is the protocol for compressing the included text within an object and I believe it’s the ZLib (Limpel-Ziv) compression at work. And that usually includes an Adler-32 checksum at the end of it. Replace even a single byte of that compressed stream—presumably without updating the checksum—and that object content is broken.

But what does Adobe Reader do with a broken object? It silently swallows it without displaying it, which is exactly what we want to do here! Replace even a single encoded byte in that unwanted watermark and it’s effectively gone.

“Replace even a single encoded byte in that unwanted watermark and it’s effectively gone.”

So the hack then was a few lines of code. As I mentioned before, I used a trial-and-error method of temporarily editing one compressed section of PDF after another until I’d broken the watermark. At this point, I then determined that the text for my target search was “xœ}OM” or more simply “}OM”. Confirming that the watermark included the only occurrence in the file of this combination of characters allowed me to do a binary comparison and replacement.

// Above this was the Aspose sample code to write each page
// to a file. I inserted this code on a per-page basis to
// then modify that newly-created PDF file.

// This is our own code to find/replace their watermark
string fileToModify = pdfDocument.FileName.Substring(
	0, pdfDocument.FileName.IndexOf('.')
	) + "_p" + pageCount + ".pdf";
string fileModified = pdfDocument.FileName.Substring(
	0, pdfDocument.FileName.IndexOf('.')
	) + "_p" + pageCount + "_no-watermark.pdf";
using (var reader = new BinaryReader(
	new FileStream(fileToModify, FileMode.Open)))
	{
	using (var writer = new BinaryWriter(
		new FileStream(fileModified, FileMode.Create)))
		{
		byte[] buffer = new byte[1024];
		int count;
		while ((count = reader.Read(buffer, 0, buffer.Length)) != 0) {
			// Now look for our sequence
			for (int j = 0; j < (count - 3); j++) {
				if (	buffer[j] == '}' &&
					buffer[j + 1] == 'O' &&
					buffer[j + 2] == 'M')
					{
					buffer[j] =     0x31; // 1
					buffer[j+1] =   0x32; // 2
					buffer[j+2] =   0x33; // 3
					}
				}
			// Optionally having patched in place,
			// write to the destination file
			writer.Write(buffer, 0, count);
			// Empty out our buffer for another run
			for (int i = 0; i < buffer.Length; i++) {
				buffer[i] = 0x00;
			}
		}
	}

I’m sure there are prettier ways of searching a buffer but this was easy enough. Note that I only actually need to change, say, the first character at “buffer[j]” which is sufficient to break that checksum mechanism.

And the rest, as we say, is history.

AsposeWatermarkGone
Same example, after breaking the watermark

You might ask why I’d post about such things. I do it for the sake of my own curiosity and I assume that others like you are curious as well. Just as little kids build sand castles and then smash them to bits we bigger kids like to build security and then smash that as well. One of the reasons why this is good practice is that it teaches us what is “good enough” security and what is “better” security. Just because you think something is secure because you can’t think of a way around it, that doesn’t mean that some other clever person can’t work their magic.

hacking agar.io, part 2

This would be the second post in a series. You might want to read the first in the series if you haven’t already done so. Here, I continue with the work related to redirecting the game’s server traffic to my own website so that I can discover the interface.

DNS server

I first install Dnsmasq on my MacBook, add a single entry to its /etc/hosts file to redirect traffic for m.agar.io to my MacBook’s private IP address. Starting up Dnsmasq I then have a DNS server which will redirect game traffic to my own website. Make sure that the program is running by entering ps aux|grep dnsmasq|grep -v grep. You should see an entry for this program.

It’s probably a good idea to test your DNS server to verify that it returns the expected information.

nslookup
> server myip
> m.agar.io.
> exit

After entering the third line above you should see a DNS lookup which returns your server’s private IP address.

Our website

In my ~/sites folder, I run the following command to use Express to generate a generic website: express agar. As is usual for Express, I change into the newly-created agar directory and then run npm install to bring in the dependencies. Since the default installation binds to an upper TCP port and we want the standard port 80 instead, I then edit the bin/www file in this folder and replace the port number 3000 with 80 on a single line.

Note that Node.js, the underlying program that serves up an Express website, will not be able to bind to port 80 since it’s reserved unless I’m running as the root user. If your own user is setup to run the su command then you should be able to start this website with the command su npm start in the agar folder. Otherwise, you’ll have to run just su to become the root user, navigate back into your user folder area to find this folder and then just run npm start instead.

It’s probably a good idea to test the website by bringing up Safari and entering the address http://myip/ (substituting my private IP address) to see if it works.

Configuring the iPad

At this step, I’ll need to tell the iPad’s Wi-Fi configuration to use my own DNS server first and then the existing set of DNS servers next. You’ll find this under Settings -> Wi-Fi -> select the i button next to your own connected Wi-Fi network -> DHCP -> DNS -> prepend your own server’s private IP address and a comma at the beginning of the list.

This is the initial preparation for redirecting the game traffic to your own website. Note that the Node.js website while running will write to its log file and this will be our method of discovering the interface for Agar.io.

Discovery phase

By now attempting to play the Agar.io game on the iPad, it makes requests to what it thinks is the server. Only these requests are now being sent to my website instead. As each attempt is logged as a failure on my own website, I then make this call manually in another computer to the actual Agar.io website to see what it’s supposed to return.

For example, the game makes a request to the game server’s interface with just /info as the URL.

/info returns:

{"regions":{
  "CN-China":{"avgPlayersPerServer":368.5,"avgPlayersPerRealm":147.4,"numPlayers":737,"numServers":5,"numRealms":5},
  "US-Atlanta":{"avgPlayersPerServer":445.1034482758621,"avgPlayersPerRealm":203.2755905511811,"numPlayers":25816,"numServers":267,"numRealms":127},
  "EU-London":{"avgPlayersPerServer":430.6,"avgPlayersPerRealm":191.37777777777777,"numPlayers":8612,"numServers":179,"numRealms":45},
  "SG-Singapore":{"avgPlayersPerServer":546.0,"avgPlayersPerRealm":136.5,"numPlayers":1092,"numServers":9,"numRealms":8},
  "Unknown":{"numPlayers":0,"numServers":0,"numRealms":0},
  "BR-Brazil":{"avgPlayersPerServer":333.3220338983051,"avgPlayersPerRealm":200.6734693877551,"numPlayers":19666,"numServers":181,"numRealms":98},
  "RU-Russia":{"avgPlayersPerServer":473.25,"avgPlayersPerRealm":145.6153846153846,"numPlayers":1893,"numServers":45,"numRealms":13},
  "JP-Tokyo":{"avgPlayersPerServer":460.0,"avgPlayersPerRealm":172.5,"numPlayers":1380,"numServers":8,"numRealms":8},
  "TK-Turkey":{"avgPlayersPerServer":287.14285714285717,"avgPlayersPerRealm":154.6153846153846,"numPlayers":2010,"numServers":30,"numRealms":13}
  },
  "totals":{"numPlayers":61206,"numServers":724,"numEnabledServers":317,"numRealms":317}
}

As you can see, this is a fair bit of information. The format is known as json in case it’s not familiar to you. As of my writing this, there appear to be over 61,000 players in the game right now and well over 700 servers with almost half of those enabled. So this would be why it’s difficult to get a simultaneous FFA game with your friends—the odds are against you.

Without further ado, here are the other queries which I discovered.

/ returns:

37.187.171.110:1523
8QJP8

This appears to be your issued server and port on the first line and what is likely its instance alias from whichever cloud-based company they’re using.

/getLatestID returns:

131

I know, not very impressive. But it appears to be the highest user ID for your issued server.

/findServer returns:

{"ip":"151.80.98.52:1516","token":"86JYH"}

Another json response, this appears to also be issuing you a server and port. It’s possible that the first home query is asked at the beginning of the game and then /findServer is called each time your die in the game.

So far, this appears to be everything I’ve learned from this redirection technique.

Status

At this point, I now have the game interface which the Agar.io app uses to communicate with the server. It likely makes more requests but that’s good for now. I could have enough to go on in order to work up something so that multiple iOS people could join the same FFA game, for example, since we know this issuing mechanism.

hacking agar.io

In an earlier post I described an addictive game called Agar.io, an interactive eat-or-be-eaten game involving graphical dots. In this series of posts, I’ll be attempting to hack the game to see what I can get away with.

Agar-top

Define:  hacking

I suppose there are several ways of interpreting the term hack here. In the movies, some character will “hack the mainframe” or some other nonsense. And we’re also familiar with someone who attempts to use techniques to hack a website, perhaps injecting SQL code into an innocent-looking HTML form. Here, I refer to one of the original uses of the word, to hack away at a problem until it is solved. I’m interested in the game itself, how it talks to the server and I’d like to go to school on their efforts. As a coder of smartphones myself I’d call that part of the learning curve.

Goals

Ultimately, I would like to learn how the game works behind-the-scenes. I do have some secondary goals though. It would be interesting to see if it is in fact possible to edit an existing iOS app and have it still work and all without the original coder’s digital certificate. If successful, I think the first order of business would be to remove the ads you might see during game play. Another personal goal would be to allow multiple friends on iOS devices to play the FFA (free-for-all) mode of the game with each other; this could be made possible with a proxy server, I’d propose.

The platform

Currently, I play the Agar.io game on an iPad II since I prefer the interface over a browser-based version that’s available. So I will be attempting to hack the Apple store app ultimately.

This may turn out to be impossible since an app that runs on iOS is supposed to be digitally signed to prevent tampering. And yet this is what I intend to do, nonetheless. I’ll be testing that assertion to see if a hacked app will still work.

Concepts

Here, I’ll discuss some of the concepts of the approaches I’ll take.

  • Patching:  Patching is an old-school technique in which binary code, for example, is edited in place with a script. Individual characters or code is replaced in the original to create a new file. The patch program itself works together with another program called diff, used to calculate the differences between two files.
  • DNS:  This service is responsible for looking up a name like m.agar.io and replacing it with an IP address.
  • Redirection:  Using your own DNS server so that you can redirect requests to your own website instead of the intended one.
  • iOS app:  An iOS app might seem a little daunting if you’re not a coder. It’s actually a collection or manifest of files all rolled up into one .ipa file. I think it’s safe to say that the app was written in Apple’s Xcode using a computer language like Objective-C or Swift.
  • Ad-based add-ons:  It’s clear that Agar.io has many opportunities to display ads within the game itself. The programming interface to these (for the Agar.io developer) is almost always JavaScript-based.
  • Tethering:  Connecting a smartphone—or the iPad in this case—to a computer to allow for interaction (like development testing) to occur.

Throughout this series of posts keep in mind that if I’m indicating a command, it’s often being done on a MacBook with OS X 10.11.5 El Capitan at a shell prompt. Otherwise, I could be referring to something I’m doing on an iPad II with iOS 9.3.2 installed.

DNS server

I’ll be using the Dnsmasq easy-to-implement DNS server for redirecting Agar.io’s server requests to my own website. I’ll then configure my iPad to use this server first when doing DNS lookups.

Discovery website

And since I’m familiar with Node.js and Express I’ll be using this to mockup a website for those redirected app requests. When the iPad makes a request to what it thinks is the Agar.io website, I will see that request in my website’s logs.

This could be technically called a man-in-the-middle technique since I could then have my own website forward the request to Agar.io’s actual server and then answer the iPad with that response, adjusting it if I wanted to. I guess technically you could also call this a proxy approach.

Binary editor

I’ll likely also use Hex Fiend at least minimally to find the location within the main program app where I’ll be patching the code.

Installing a modified app

Normally, you would download an app directly to our iPad straight from the Apple iTunes store. Technically, I suppose, I could have taken advantage of the redirection concept from before to steer the iPad to my own website to deliver the edited content but it’s not that difficult. There appears to be a mechanism so that you can download iOS applications on an OS X computer and then, while tethered, install them remotely using iTunes. This actually allows us to use a MacBook in this case to snag the code package itself and to start all the fun. We’ll be taking advantage of this in order to then try to push a modified app package to the iPad.

If you’re on a standard OS X computer and you get the Agar.io app, it won’t seemingly do anything after the download; you’re not presented with the usual Open button after it has downloaded. It does, however, get silently copied to your hard drive under your user folder in /Users/username/Music/iTunes/iTunes Media/Mobile Applications. Having downloaded it, you should find a file called Agar.io 1.3.0.ipa which is the app (collection) itself.

Expanding the app

From here, you might not know that an .ipa file is little more than a .zip file. I’d suggest copying the Agar.io app file somewhere else (like creating a folder called AgarIO) and then open a shell so that you can decompress it.

MacBook:AgarIO$ unzip "Agar.io 1.3.0.ipa"

This command then will decompress the collection of files for you.

What’s inside the .ipa file

There are a lot of files inside this package, just like you’d find with most store apps. The first I’ll discuss is iTunesMetadata.plist which is perhaps the most aggravating of all. A .plist file is like a database for a coder, it usually stores configuration options. Opening it with TextEdit then shows me that this is the file responsible for knowing who downloaded it (myself) and how I’m then authorized to use it. I’m sure there’s a similar mechanism inside any music file you download from iTunes to prevent you from playing it on an unauthorized device. So in other words, I couldn’t just patch the Agar.io application and then make it available for download for others. Each person interested in this would need to go through the motions themselves.

Next, there is a META-INF folder which contains two files. I haven’t fully investigated them yet but the first is com.apple.FixedZipMetadata.bin which appears to again be a compressed collection of files. And the second is com.apple.ZipMetadata.plist. It appears to have some indication of how the actual program was zipped up into an .ipa file.

The final folder is Payload which includes what appears to be a single file, Agar.io. Or, is it a single file? Knowing what I do about making iOS apps, it’s actually another compressed file. In Finder, you’ll want to rename this Agar.io file to Agar.zip, for example. Back in your shell, then unzip it as you did before to expand its contents.

What’s inside the Payload file

So now we’re getting down to the actual programming itself. Everything we have seen up until now is just a wrapper so that iTunes and Apple can provision an app to you and just your device(s).

Surprisingly, there are a total of 1,111 .png graphic files inside. Most seem to represent the many skins that you’ll see in the game. There are 153 .plist files which are used to store anything from advertisement configuration information, to promotions, to language localization information and collections of available skins by category. With respect to my goals, I’m not really interested in these. And there is a single .db file for the Vundle advertising platform.

There is a folder called _CodeSignature which appears to include hashes of the collection of graphics, presumably to prevent them from being edited perhaps.

There are 65 .ccbi files which appear to be another form of .plist files. There are 15 .json files which appear to have different localized versions.

Finally, there is agar.io which is the actual program file itself. I’ll save the actual editing for a follow-up post to this one.

Status

That’s a good start so far. We’ve downloaded the Agar.io app and performed two decompression steps to get at the actual executable itself. Next, I think I’ll switch gears and build the discovery website and DNS server so that I can get at the app’s server interface.

Update

Skip to the final post in this six-part series if you’re looking for the code. Enjoy!

a new app pricing model

Personally, I don’t like the idea of “in app purchases” which seems to be the norm these days in the current app pricing model. You get the app for free, try to use it and then find out that you can’t save your results unless you pay the author for this feature. If you’re like me then you usually feel like they’ve wasted your time.

I would suggest that a better model is how we now buy music.

Changes in the recording industry

You used to have to purchase the entire album just to get a single song, that was just how things were. If an artist or band had one killer song that was enjoying a lot of airplay on the radio and assuming that you really wanted that song then you had to pay the $12 or so for the entire album. And you just crossed your fingers that one or two of the other songs made it worthwhile.

With the advent of iTunes and similar websites, we now have the ability to sample and purchase exactly which songs we want to pay for. If half of the album isn’t worth it, you don’t have to buy all of it. If the artist only has one good song then there’s no reward for them to pad the album with a lot of junk.

A new model for app pricing

So why shouldn’t we just show our app’s prices up front instead of hiding them inside? Currently, a new customer can’t see how much of the app is crippled and how much is functional until it’s been downloaded and used. How many times have you downloaded and demo’d two or even three different free apps of the same kind, trying to find one that was reasonably useful?

What we need is a venue for selling our apps like musicians sell their songs. Theoretically, it might look something like this:

PayAsYouGo_734x466.png

More feature transparency

The biggest benefit to a model like this is that it shows the potential customer what’s included in the full program and the cost of each feature. If, like me, they’re not interested in the social connector feature then they simply don’t purchase it. You pay for what you need and nothing more.

As developers, we would distribute modules of functionality and charge the user on a per-module basis. I would suggest that features be priced differently based upon the perceived value. In fact, there’s nothing to prevent the price of a popular feature from increasing over time.

Like in the iTunes model, clicking a play button next to the feature ought to bring up a demo or screenshot of the feature in action.