Strategies for Scaling Real-Time Web Apps

Using Polling
Don't setup any cacheing and run a normal server. Poll for whatever data you need at any time, by as many users as possible. This puts load on your app and your database server. You can probably do a couple of thousand updates per minute before you get into trouble with a standard node.js / mongodb setup. On our Rackspace 1gb (of ram) cloud servers, we can handle a max of around 2000/second per box. Even a hefty Mongo setup won't be able to sustain that kind of load very long and the DB connections will start piling up and your servers will melt after a few minutes. With our setup of regular short polling every second (which is crazy) you'll want to scale things up around 200 simultaneous users.


Using Polling and Varnish 
The strategy here is simply to setup polling between set periods of time. So you can say get me everything that happened between this 10 second chunk and that 10 second chunk. Setup varnish to cache those calls , so everyone gets the same stuff. This puts the main responsibility for handling the load on the cacheing layer, which are usually built to handle a lot of load. Your database will only have to do one update every 10 seconds. It can handle that really easily. If there is some initial data you need to load to start your app that may get out of date, you can cache that bigger query for like 1 minute and again your database will be happy as pie. You can use Varnish here as a load balancer as well, but you probably will have to scale the load balancer before you'll need to scale up your app servers. A hefty Varnish box (say 8gb of ram) should be able to handle thousands of hits per second of cached data.

Using Socket.IO for polling

If your app server can safely handle ~2000 users per second, but your database cannot, you may want to try a setup like this. Let your app server "cache" the data from your database and push updates using web-sockets/long polling with socket.io. When the user initially connects to your page you can send down the cached data (or even a buffer of say the last 1000 messages). As the database gets updated using  normal POSTs and things the frequent polling of the database for updates can happen at the app-server (say one 5-second poll per app server), which will then send updates to all the users they know about. Obviously, this depends on how much data your pushing through (and how much ram you have), but with small amounts of stuff you can do the caching at the app level instead of on a special cache server, sparing both a new server and your database. Theoretically this setup would scale horizontally as well with multiple app servers, either through a load-balancer or some other round-robining approach. 

Notes on a failed launch with node.js and mongodb

Intro

What follows is a play-by-play of i.TV's launch of viWEer with Entertainment Weekly. We were using all of the latest technologies: node.js, MongoDB, nginx, Varnish. And yet our initial launch was a complete disaster. If you want you can skip straight to the lessons learned.

Day 1

After a hurried, but exciting 4 weeks we were finally ready to launch our app. We had spent plenty of time tweaking the CSS3 box-shadows, working through all the CORS (cross-origin AJAX) bugs in Safari and IE, and were ready to launch our awesome production ready app. We passed our partner's load test requirement like pros, the security audit was likewise a success. Primetime here we come.

Launch party ensures. It was a pretty mild get together. Brad (CEO) hands everyone's wive's gifts for our hard work (See's Candy, delicious). We all get to feel good about ourselves and there we go live in 3..2..1..........crap. NGINX errors or Varnish errors. jQuery AJAX errors. Yeah, we're screwed.

So while the family and friends chatted away, we all got into a room and tried to figure out why our servers were dying. I mean Mixpanel is telling us we have maybe 400 people hitting the servers. I'm pretty sure my 256mb LAMP box could bench press that many people without breaking a sweat. But we've got VARNISH and MONGODB and NODEJS, NOSQL / JAVASCRIPT to the ends of the earth.  Why do we suck so much? I mean we even have an API for our iOS app built on the same infrastructure that gets tons of hits without any of these problems.

Okay, turns out, we needed to learn a little bit more about Varnish. Varnish hates cookies. We originally built the API powering our app with sessions, because it was going to serve itself up. Instead, we decided to go the CORS-based single-page-web-app approach. Our API and our front-end would be totally separate. Just throw our HTML/JS up anywhere and we're good to go. So scalable and awesome and un-coupled. We're amazing! So we have a backend that's supposed to be de-coupled, but ends up relying on cookies and sessions. Not so amazing.

Day 2

So the problem is we can't handle the load. Our not very good load testing, wasn't very good. Varnish can't cache things well that use cookies. So we quickly write up something that tells express to not use the cookies middleware on certain routes. We also figured out how to look at the AGE header from varnish to make sure that things are really caching. Our tests show things are caching. We're pretty confident things are going to be awesome today. So we chill out.

Nighttime comes (app gets used from like 6-9PM) and yeah it's basically like yesterday, which is complete crap. Not as complete crap, but basically it sucks. A lot of errors in different browsers, generally a lot of things go down from time to time, but overall they work throughout the night. We're live troubleshooting with the partner who we're launching this with. They have some few problems, but after refreshing their browser or retrying things, things basically worked.

We still aren't very expert at Varnish. We're not sure things are really cacheing properly. It looks like things are caching a little bit, but not very much. What the heck. varnishstat, be my friend, please. Our database is being murdered. Mongostat is showing hundreds of concurrents and it's really slow. We also use that database to do a lot of hard queries for our iPhone app. db.currentOp() on mongo shows tons of things being blocked. We go to our servers, we find out that nginx, which is in front of all of our node servers (to handle SSL, forwarding, and some other stuff) is running out of connections. 

We quickly hypothesize that our database is too small to handle both a barrage of connections AND our really intense map reduces that are bogging it down every 20 minutes. Maybe the (fairly small) database is blocking, so NGINX is queuing up the connection, waiting for them to return and because they never return it eventually just has thousands of opened connections. So sometimes things work, but mostly it sucks.

Day 3

We make the database HUGE, first thing. Rackspace was like, screw you, you can't even make a database that big in your "huddle", but we said, no way, we want it. 16gb of ram. In your face wait times and blocked servers and NGINX too many connections / timeout errors, because we were waiting on the database. After a few hours of working with Rackspace we got our new production-ready database in order. Copy the image of the old db. Start up the new one. Manually move over any critical updates that happened during the process and we're good to go. Hallalujah. Finally off to NOSQL heaven!

Come showtime no one can post or do anything, except us. Our partners who we're launching this experience with can't do anything. We can't reproduce it. This sucks. We get our whole team on Skype, finally some people start having the problem. The sessions are just basically constantly restarting. It appeared to work, because you could login and do one thing that required sessions. After that your session would restart and you couldn't do it anymore. We had two things going on that we're possibly giving us grief:

1) We're still doing our auth over CORS, which is generally weird and send about 4 times more requests than is necessary.
2) We migrated our database to a new server and didn't clean out our old sessions collection.

We struggled, we stayed up really late. Our partner on this launch calls our CEO. They're about to drop our partnership. Another conference call with the whole team. We have basically have a day to show them that we're not complete idiots or the entire thing falls through.

Our CTO mounts our totally de-coupled api server on-top of our web server using some express middleware craziness app.use('/viewer', require('api').createServer()). No more CORS.

Next up we clear the sessions collection on our new database.

Wow, everything seems to work now. Too bad we've had 3 days of crap. So we decide that to prove that we're awesome we need to get a bunch of users to test this NOW. We split the team in 2. Half the team stays up all night and tries to get people from the Internets to test this / put in place some legit load testing. The other team shows up at work at 5:30AM and carries on making sure that everything is solid.

I'm pretty sure at this point that I'm going to lose my job.

Day 4

First off: don't try mechanical turk at 4 in the morning to try and get a bunch of users to test your site. It doesn't work.

Secondly, the middle of the night team setup a PhantomJS server that's able to properly emulate hunderds of concurrent users, actually using the app. It's pretty awesome and it really was the first thing we were able to use to make sure our scaling was working properly.

I'm part of the early in the morning team. We get their too early. We can't open the door, so we hang out in Einstein's Bagel's nearby talking about what we're going to do. We weren't able to get online (other than our phones) for like 30 minutes, so we talked and talked and talked.

Here are my ideas:
- Get more app servers
- Get NGINX out of the way of node?
- Figure out Varnish, varnishstat, etc
- Lots and lots and lots and lots of load testing
- Get a less hacky way to get varnish to ignore sessions

So we did all of those. I learned how to read varnishstat. I did some basic apache-bench testing. I figured out that things still weren't being cached. We found the limits of our server setup. I added 5 app servers. I changed the varnish config to round-robin across all of our servers. We figured out how to get varnish and not express to handle routing /viewer to the api server. We made a plan to completely git rid of sessions, because they were the only thing left in the stack that we didn't build ourselves. Things are looking pretty good.

We're still nervous about how things are going to go with our partner. We're expecting a call with them to hear about how they're canceling the partnership.  The call never came. The night comes. The app seems to work okay.

By this point though, we've lost most of our users :-/

Day 4 and beyond


I did a lot of testing and found that NGINX was no more performant than node.js directly on our small Rackspace servers. In fact, things were a lot better when NGINX was out of the situation.

After this, our partner's amazing Load Testing team helps us really kill our servers. We tweak a few things with our client code, add more caching, and are able to consistently handle thousands of concurrent connections. Finally we're getting somewhere.

For the next few days we dig into everything we can, constantly working on improving our load capacity and learning about. We find bottlenecks all over the place, but under the highest loads, Varnish, on a a cloud server with 4gb of ram, eventually starts to choke, but we're still pretty happy with say 2000 concurrents connections (which roughly translates to maybe 10,000 users at a time).

Finally, the ad campaigns run, we get back to features instead of bug fixes and things start to slowly settle down. For the next few weeks we're paranoid and really careful that we're going to break things. After a while, they just keep working. We're happy and we can sleep at night.

Lessons Learned


1. It's really important to understand your stack.

At the begging we didn't know if Varnish was caching anything. We couldn't tell you why we should or shouldn't have NGINX in front of Node. We weren't sure why the sessions collection was growing endlessly. It would have been really helpful to understand that stuff before we launched. It was really hard to learn how to debug and load test these components on the fly. You need to understand the tech before you run into problems with it.

2. Load testing is really freaking important.

Without knowing the limits of our stack it was really hard for us to know how to scale when the time came. Day after day we made assumptions about how to improve things (grow the database, put static files on a CDN). They were useful, but they may not have been necessary had we figured out before we tried to launch where things were likely to break. We know now that 5 app servers is complete overkill for us. That's a couple grand we probably didn't need to spend on hosting. Also a lot of problems show under heavy load that don't show up otherwise. PhantomJS, httpperf, and apachebench were all helpful tools to help us figure out our server stack.

3. Real-Time is hard. 

Part of our problem was that we were polling for updates every single second. With 500 users, that meant our database was getting hit 30000 times per minute from this one app. Later we changed the polling interval to 5 seconds and got Varnish to cache those requests properly. Next time, we might just look at using Socket.io.

Analytics For Web Apps

i.TV recently launched a hot new web app called Viewer along with Entertainment Weekly. This app required some pretty specific analytics. The kind that a typical page-views style tracking system won't count.

The big diff with webapps vs. typical web pages is it's usually only one page, so instead of capturing pages you have to capture events. Logins, posts, filtering, replies, replay in our case. We also need to know how long people are using the app. These are all events (or at least can be).

Thankfully we found MixPanel, which provides some awesome graphs along with an easy to use event-capturing system.

In addition to some pretty awesome support. MixPanel lets you bookmark reports that you create and they e-mail them to you regularly. You can't yet bookmark funneled reports (the fancy kind that show you the drop-off between various events: log in, post, etc), but you can send multiple reports.

The monthly prices start at free, which is what we started with! That can log up to 25,000 events.

All in all MixPanel is awesome for doing analytics for web apps!

Mongoose and MongoHQ

A while ago I wrote a post about how to hook up node.js with the MongoHQ database hosting service. I still love MongoHQ, but my mongo wrapper around the node mongodb native driver sucked! I recently switched wereviewutah.com over to use Mongoose and just wanted to share with everyone how easy it is to use Mongoose with MongoHQ.

Sample Model File (models.js)


Sample Config File (./config/db.js)


Sample Route File (server.js)

Remove passphrase from SSL key

We have NGINX with SSL and automatically deploy with a script. We recently changed SSL certs and they were setup with a password, which makes it pretty hard to automatically deploy. I found this awesome tip on this blog http://blog.sosedoff.com/category/nginx/ for removing the password:

$ cp server.key server.key.org
$ openssl rsa -in server.key.org -out server.key
 
The newly created server.key file has no more passphrase in it.

NGINX Proxy solves CORS dilema

Hi guys,

This is a really special announcement:

We worked around our CORS IE problem, by adding some simple NGINX proxy's.

The problem we had was that CORS in IE doesn't support the with credentials directive, which is required for using sessions. (See fancy explanation here: http://hacks.mozilla.org/2009/07/cross-site-xmlhttprequest-with-cors/)

First we use $.support.cors from jQuery, which confirms that they have credentials support./

If they don't support it we modify their URLs to have the base server name in the path. (In our case links to something.i.tv would result in i.tv/something/)

Then in our NGINX conf, we used proxy_pass to send people that go to those URLs to the real server:
http://wiki.nginx.org/HttpProxyModule#proxy_pass

Here is the code:
location /something/ {
    proxy_pass http://something.i.tv/;
}

We had previously looked at node-httpd-proxy, building our own proxy server, trying a million hacks to get CORS to work with IE, wrapping everything in JSONP (doesn't work well with long POSTs), but this ended up working the best (and it's by far the easiest)!

Go Node.js, Express and NGINX! We love these things at i.tv.

Things I Want To Learn Someday (Soon)

1) CouchDB
2) Canvas Gaming
3) Unity 3D

ServerSide JavaScript Date Nonsense

Hi.

I want to format some dates in my Express views.

First of all, my employer, i.TV, has a DateTime library that we wrote that is alright:
https://gist.github.com/1125717

But I was hoping this would be something that's more in the view layer than than in the model/controller level.

Here are some libraries and functions that do some version of that:
http://www.datejs.com/
https://github.com/Flamefork/underscore.date
https://github.com/cdcarter/commonjs-date-formatting
http://timeago.yarp.com/

However, none of them did what I wanted. Here is one that did:
http://ejohn.org/blog/javascript-pretty-date/

This is what I did to make it work in my express views:

Coda Wish List

Hi Panic,

I do a lot of node.js and front-end web dev with Coda.

This is my unsolicited wish list for the next version:

1) I want support for syntax highlighting in files without extensions like "Jakefile".

2) I want to not have to use type="text/javascript" and type="text/css" and to get syntax highlighting of js/css within html files

3) I want better JavaScript completion including support for all of the natives like Date and Math.

4) Escape should consistently cancel the global search

5) Ctrl-Shift-F should always call the global search

6) Git integration

Sincerely,

Jamund Ferguson

http://www.jamund.com

MongoDB Performance: Group vs. Find vs. MapReduce

Lately, I've been playing a lot with MongoDB's group command. It's a powerful command that allows you to write some really interesting queries quickly. Here's an example of one that calculates "Tags" on a blog site:


In my tests it also proved to be about 4x faster than a similar MapReduce, however, it comes with a severe cost: It blocks all reading from the collection. This is a huge problem and basically makes it worthless for doing serious queries on a database with say hundreds of thousands of users like I have in my day job. From what I can tell finds, distincts, and mapReduces don't block and some combination of those provide non-blocking alternatives.

Here's a simple map reduce example:


One more note, I recently read MongoDB: The Definitive Guide, which had a lot of examples and clarification that are not readily available in the online documentation. I highly suggest checking it out!

Mobile vs. Native

I just wanted to comment on Ray C. Morgan's JSConf 2011 talk about web vs. native when it comes to mobile apps.

He started off his talk with a 15 minutes Objective-C lesson, which probably threw pretty much everyone off, but it was perfect for me. I recently had a client ask me to convert a mobile web app I had developed for them to a native applications, so I was knee deep in Objective-C. The talk really hit on a lot of things I had been thinking about and I'd like to re-iterate some of his points and then take issue with a few others.

Building Native Apps is Not that Hard

This maybe more than anything stuck out to me as I was learning Objective-C at the moment, because as a web programmer trying to learn how to do native iPhone apps, I could pretty much see where he was coming from. It's not impossible to learn Objective-C, and your apps are going to have some instant speed benefits as well as feel like native apps. People are used to these and if you're building apps you shouldn't assume that you can't do native even if you don't know how to do it yet. My experience with Android tells me pretty much the same things. It's definitely possible, even for us web-devs.

Build web pages in semantic HTML

While he kept saying "remember the good ol' days of semantic html?", and I think most people were thinking, no actually, we're still getting there. I think his point was pretty valid. If you're building a website, mobile or not, start with HTML, not a super fancy fake mobile framework. People expect web pages to act like webpages, so think about your mobile website the same way. It's not a mobile application, it's a web page. This gives you the best likelihood of your site being able to work on multiple mobile platforms and for the most part lets you take advantage of what you already know. Even media queries can be used really effectively in a lot of situations. Again, think about what people are doing with your site and what you're trying to accomplish.

Missing from his talk: Use fancy frameworks for prototyping

If you're a web developer and you want to do a proof of concept of a mobile application before you invest heavily in learning native language, use something like Sencha Touch to build it. It will look pretty native and give you a good idea if it's worth building. This is basically what happened to me, when I initially decided to build a Sencha Touch app for a client, who had an iPhone 3GS. The app is great and usable, but ultimately we decided to make it native. I could never have gotten so far along without starting the JavaScript framework, so I highly recommend using them as a prototyping tool.

Overall, I loved his talk and appreciated some of the points that were brought up. Yay for JSConf. Hopefully, I'll get around to posting more re-caps like this soon!

Mongo Capped Collections

We have a huge capped collection over here at i.tv that we're using for logging purposes. It's pretty great, except that it's so big that it's too slow to query, so
here's what you can do to get useful data out of the system:

db.requests.find().skip(10000).limit(300).toArray()

Capped collections don't usually have an index, so they're not good for searching when they get large, (you set how big you want them), but they are in chronological order by default, so our little query will find the records between 10,000 to 10,300 which will correspond with an actual block of time.

It's not a bad way to get information out of the system, but beware, because the logs are always growing, you'll need to move your skip back relatively frequently.

no.de, github, one repo

Are you like me and use github for your repo, but also use Joyent's awesome no.de service to host your website. That's exactly what I'm doing with this CollabPaintJS project I'm working on.

It was silly to have two separate local directories when the only difference between the two repos was the README file github uses, so I decided to put them together. Here's my workflow now.

All I needed to do was add was update my .git/config file to look include both of the following:

[remote "node"]
        fetch = +refs/heads/*:refs/remotes/origin/*
        url = node@64.30.136.179:repo
[remote "github"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    url = git@github.com:xjamundx/CollabPaintJS.git

Note: At first things were a little bit weird. Both of the repos had changes initally that were not in the other. Fortunately in my example the differences didn't really matter and I just went with one of them. I did this by branching the no.de version into a node branch and then checking it out ontop of master after merging in the ghithub branch. Once I did this everything was fine. Pulling, if you need to do it, is a little different as well, because you might be pulling from two different places:

git pull github master

Overall, for a small project that's being hosted both by github and Joyent, this solution seems to work pretty darn well. Let me know how if it works for you or alternatively, why it's a bad idea!

SQLite and iOS Quirks

This caused me a lot of confusion over the last couple of days.

When selecting you start on 0, using statements that look like this:

self.commentId = sqlite3_column_int(selectstmt, 0);
self.email = [NSString stringWithUTF8String:(char *)sqlite3_column_text(selectStmt, 1)];

But when inserting you start on 1 using similar statements:

sqlite3_bind_text(insertStmt, 1, [self.contactName UTF8String], -1, SQLITE_TRANSIENT);

Another quirk:
It's SQLite, not SQLLite!

It's a bit quirky, but once you get to know how to use SQLite with iOS it becomes really helpful!

P.S.: Here is a tutorial that I found helpful:
http://www.iphonesdkarticles.com/2008/10/sqlite-tutorial-selecting-data.html

Express Pagination Middleware Simplicity

Inspired by watching TJ Hollawaychuk's supremely awesome screencast about using route-specific middleware at http://expressjs.com/screencasts.html, I threw together a simple use case for myself: pagination. While pagination might not be an obvious choice for middleware, it works here because it doesn't need to know anything about the route that's using it, though in this case it may take advantage of some passed in variables if they exist!



Here is a partial using EJS syntax that will display some previous / next arrows wherever you choose to place them. Notice I'm using locals.prev and locals.next here, which works, becuse I'm not sure those variables will even exist. In these cases you access them as properties off of the locals object in the view, which means you don't have to do this if (typeof prev !== "undefined" && prev), which is a little more verbose if you ask me:



I liked how easy the middleware approach was over having specific pagination logic in each route that used pagination. Express is fantastic for letting me send in multiple functions that I can chain together by calling next(). In this example, without using the middleware I would have had to wrap each paginated function in the count() database call, which would give me an un-neccessarily long callback chain. I prefer the simple middleware approach allowed by express.

JS Codings Standards

Here are some JavaScript coding standards/conventions I've been thinking about lately. I put this together pretty quickly, so it may contain some errors. What you think about these? Are there other conventions you support? Please post your feedback and/or corrections and suggestions!

MongoHQ and Node

THIS IS OUT OF DATE CHECK OUT THIS NEW ARTICLE INSTEAD

MongoHQ

Free MongoDB hosting for playing around with Mongo. Free for 16mb of storage. $5/month for 256mb. It's good for small stuff and getting started with Mongo.

Difficulties

One of the problems I found while connecting to MongoHQ from node was the authentication. Most of the MongoDB libraries for node are very simple to use, but cut some of the features from the mongodb native library like authentication (or at least make it difficult to access), so I ended up rolling my own wrapper to make the authentication step as easy as possible.

Custom Wrapper

I wrote a simple wrapper to the Node MongoDB native driver that made it easier to hook into MongoHQ, which requires authentication. It may be useful to other people as well:

Connecting the pieces

To get this custom driver up an running it's very simple.

• Initialize it.
• Register your collections.
• User it

Docs and Help

• Node Mongo Tutorial (with a printout of the node-mongo-native api)
• MongoHQ
• MongoDB Manual
• Node Mongo Native Driver

Locals in ExpressJS

Using Local Variables in Express

ExpressJS allows you to define "locals" which are variables available to the view in multiple places and in multiple ways. Here are a few examples.

Incrementing a Count Variable With Mongo

The main line that you need to look for is the following one:
db.logs.update({}, {$inc:{count:1}});

Adding colors to git diff by default

~/.gitconfig


[user]
name = Jamund Ferguson
email = jamund@gmail.com
[color]
branch = auto
diff = auto
status = auto
[color "branch"]
current = yellow reverse
local = yellow
remote = green
[color "diff"]
meta = yellow bold
frag = magenta bold
old = red
new = cyan
[color "status"]
added = yellow
changed = green
untracked = cyan

Objective-C Basics

Hi Friends,


I'm trying to learn some Objective-c. Here's what I've got so far.


Creating dictionaries


JavaScript Version:
var jamund = {
    pic: "http://a0.twimg.com/profile_images/1164437235/IMG_0803.jpg",
    name: "Jamund Ferguson",
    position: "Sr. JavaScript Developer",
    yearsEmployed: "1",
    twtiter: "xjamundx",
    email: "jamund@gmail.com"
};


Objective-C Version:

NSDictionary *jamund = [NSDictionary dictionaryWithObjectsAndKeys:

                        @"http://a0.twimg.com/profile_images/1164437235/IMG_0803.jpg", @"pic",

                        @"Jamund Ferguson", @"name",

                        @"Sr. JavaScript Developer", @"position",

                        @"1", @"yearsEmployed",

                        @"xjamund", @"twitter",

                        @"jamund@gmail.com", @"email", nil];

Splitting a string into an array of parts


JavaScript Version:
var myString = "This is a test";
var myWords = myString.split(" "); // ["this", "is", "a", "test"]
console.log("First word: " + myWords[0]); // First word: This

Objective-C Version:

NSString *myString = @"This is a test";

NSArray *myWords = [myString componentsSeparatedByString:@" "];

NSLog(@"First word: %@", [myWords objectAtIndex:0]); // First word: This

http://iphonedevelopertips.com/cocoa/nsrange-and-nsstring-objects.html

Getting the first name in a full name string

JavaScript Version:

var name = "Jamund Ferguson";

console.log("Location: " + name.indexOf(" ")); // Location: 6

var firstName = name.slice(0, name.indexOf(" ")); // Jamund

Objective-C Version:

NSString *name = @"Jamund Ferguson";

NSRange r = [name rangeOfString:@" "]; 

NSLog(@"Location: %i", r.location); // Location: 6

NSLog(@"Range: %@", NSStringFromRange(r));

NSLog(@"First name: %@", [name substringToIndex:r.location]);// First name: Jamund

http://stackoverflow.com/questions/256460/nsstring-indexof-in-objective-c


Creating a Simple Array of Strings


Objective-C Version:

NSArray *friends = [NSArray arrayWithObjects: @"Cat", @"Dog", @"Mouse", @"House", @"Pals", @"Rob", nil];

JavaScript Version:
var friends = ["Cat", "Dog", "Mouse", "House", "Pals", "Rob"];

WTFs!?

"Missing sentinel in function call" -  Just means that you must end your array with a nil (see above).

Adding the same subview to a view multiple times in row leads to it only being added once:

int j = [[self.view subviews] count];

NSLog(@"You have %i subviews loaded", j); // You have 2 subviews loaded

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

[self.view addSubview:loadingView];

j = [[self.view subviews] count];

NSLog(@"You have %i subviews loaded", j); // You have 3 subviews loaded

This means that we don't have to worry about checking if a view has been added before adding it again, which is actually pretty convenient!

Resources


A great tutorial:
http://cocoadevcentral.com/d/learn_objectivec/

Another tutorial:

http://developer.apple.com/library/ios/#referencelibrary/GettingStarted/Learning_Objective-C_A_Primer/

Sencha Touch iPad Signature JavaScript

We'll be extending an Ext.Panel object from Sencha Touch to create an iPad signature pad. When you click submit it will tie in to the server-side PHP script in this post. It works pretty well, though there are many ways that it could be enhanced.

We are using the Sencha Touch MVC style approach outlined here:
http://www.slideshare.net/senchainc/structuring-your-sencha-touch-application
http://www.sencha.com/learn/Tutorial:A_Sencha_Touch_MVC_application_with_PhoneGap

Go ahead and try it out here (iPad/Safari/Chrome/Webkit only): http://www.jamund.com/signature

Save a Base64 Encoded Canvas image to a png file using PHP

This and the next post will demonstrate how to draw on a canvas and then save the file on a server using PHP. Here is the very simple server side code:



The next post will be the slightly more complicated front-end code. This was particularly designed for a Sencha Touch iPad app that will allow people to sign with their finger. It has some flaws, but the basics are pretty reusable and good!

Your result may look something like this:

Timing the EcmaScript 5 Date.now() function

While looking at this awesome list of browser support for EcmaScript 5 features. I decided to time the difference between using Date.now(), a static method to access the current timestamp, and new Date().getTime(), the previous way to get a current timestamp in JavasScript.

The results are this: As expected, the static method wins!

The testing code can be found here:
http://jsbin.com/eheyi3/2 (I have no idea how long this will stay up):

Or here:
http://jamund.com/tests/date.html

Here are the results running in Chrome 9:

Testing Date.now() 10000000 times
0.908s
Testing new Date().getTime() 10000000 times
1.241s

Here are the results running in FireFox 3.6:

Testing Date.now() 10000000 times
7.144s
Testing new Date().getTime() 10000000 times
12.016s


Here are the reuslts running in FireFox 4:

Testing Date.now() 10000000 times
1.73s
Testing new Date().getTime() 10000000 times
3.677s

Here are the results running in Safari 5:

Testing 'Date.now()' 10,000,000 times
0.74s
Testing 'new Date().getTime()' 10,000,000 times
2.817s

Drawing in the lines

So I'm working on this really sweet HTML5 monster game, where you get to draw your own 8-bit style monsters and fight them against one another. So far I've only got a tiny bit of the monster creation part up and running. Don't worry I'll throw the whole thing up on on github when I'm done.

Anyway, one of the problems I ran into with the canvas was how to keep people's drawings within the predefined lines. The canvas is 500px wide and it's a 20x20 square of 25px blocks.

To create the lines it's very simple.

 ctx = document.getElementById('monster').getContext('2d');


 ctx.fillStyle = "black";
 for (i = 25; i < 500; i = i + 25) {
    ctx.fillRect (i, 0, 1, 500);
    ctx.fillRect (0, i, 500, 1);
 }

       That will draw some simple lines every 25px. Now how about keeping between the lines. It's simple. We just need to round our x, y to the nearest 25px, like so, that way it will always start inside one of our blocks. Then we'll actually be drawing 24px blocks to fit inside the lines completely.

I thought it was pretty neat!

        var draw = false;

 window.onmousedown(function() {
  draw = true;
 });

 window.onmouseup(function() {
  draw = false;
 });

 document.getElementById('monster').onmousemove(function(e) {
  var x = e.clientX - offsetX,
   y = e.clientY - offsetY;
   x = Math.round(x / 25) * 25 + 1;
   y = Math.round(y / 25) * 25 + 1;
  if (draw) ctx.fillRect(x, y, 24, 24); // keep things in the lines
 });

p.s. follow the progress over at http://www.jamund.com/monsters

iScroll 3.0 Select Box Hacks

iScroll is pretty neat. It was created by cubiq as an attempt to create a workaround for position: fixed on iOS. Sencha Touch has a much more complete solution, but as a simple solution iScroll works fairly well. 

This could be a great time to rant about how mobile safari is the new IE6, and how having to use JavaScript to fix issues that have already been fixed on the web for a long time is ridiculous. (Seriously, just let position: fixed work as long as the viewport and zoom-levels are set properly.) 

Check out the code here:
http://code.google.com/p/iscroll-js/

Anyway, look no further than iScroll.....unless you want to use a select box in the scroller.

I followed this thread about the problem here:
http://code.google.com/p/iscroll-js/issues/detail?id=14

And finally found a solution here:
http://groups.google.com/group/iscroll/browse_thread/thread/5b2fbad6aa667907

Here is my select box:

    Option 1    Option 2    Option 3

And it all seems pretty good with one additional problem. It seems that the screen gets nudged down just a little bit when you select things, so I had to add this one bit of JavaScript to bring it back to the top after the we choose our item from the select box.

document.getElementById("showing").addEventListener("change", scrollToTop);

function scrollToTop() {

  window.scrollTo(0,1);

}


The result looks something like this (everything under the sort by stuff scrolls):

Regex beats For loop for JavaScript search and replace

Here's an interesting test Rob and I did today at i.tv trying to figure out the best way to replace html entities (&) with their equivalent (&). We were hoping to basically replace the html_entity_decode function in php, but we're only worried about a limited subset of characters.

The basic approach is something like this:
str = str.replace(/&/g, '&');

There was something I found on stack overflow about this that is kind of start:
http://stackoverflow.com/questions/784586/convert-special-characters-to-html-in-javascript

We needed to do many search and replaces and compared two methods. Loop through a list of characters and replace them individually or use a big regular expression and do only one match. I wasn't sure what would be best, but from the results clearly the regex wins.

Code:

http://jsbin.com/inime4/2/edit

Results (Safari 5 on top, Firefox 3.6 to the right, and Chrome 8 on the bottom):

Hopefully the timing comparison demonstrates how much faster the regex is over a normal loop! Oh and Firefox 3.6 hates this test, but Chrome really rocks it.

Final solution:
var htmlEntityDecode = (function() {
  var specialChars = {
      "&":"&",
      "&eacute":"é",
      "©":"©",
      "ñ":"ñ",
      "æ":"æ",
      "Æ":"Æ",
      "¿":"¿",
      "£":"£",
      "¢":"¢",
      "®":"®"
    }, 
    inside = Object.keys(specialChars).join("|"),
    regex =  new RegExp(inside, 'g');
    
  return function(str) {
    return str.replace(regex, function(html) {
      return specialChars[html];
    });
  }
})();
  

Update: it looks like someone else also came up with a similar answer:

http://stackoverflow.com/questions/1229518/javascript-regex-replace-html-chars