kristopolous

The Transcendance of Empathy

2012-01-26T12:28:00.000-08:00

Does this offend you?

Awesome. Just don't shoot the messenger. Here's what that message is:

Tacit dissapproval of Apple's advertising. Apple has co-opted the greatest thinkers of the 20th century, in public domain images, to peddle electronics. To illustrate the true horror of the campaign, Mother Teresa is directly, as opposed to Apple's preferred implicitly, saying "I approve of my image being used to sell plastic stuff from China."
When you mouth the words, "I Approve", and look at the image, you are reminded of The Culture of Consumption, especially of excess, which is Apple's modus operandi. It means that ones excess income, which is, after all, excess by definition, goes to luxury items with full knowledge of global starvation. In so doing, you are an indirect deciding factor in people going to sleep hungry and Part of the Problem.
The iPad being coveted by Mother Teresa is supposed to show the line between humans and the possessions we have and how we decide to take on the responsibilities and time commitment of things and not people.
The real child in her arms being the background to the iPad emits a feeling that this could easily be your child, but isn't and how the notion of those that one cares for is a fleeting subjective game, crass, and indifferent.

This is the transcendence of empathy.

80s mtv

2012-01-19T11:28:00.000-08:00

my new project, 80s mtv

Don't Pay For Content

2012-01-18T18:27:00.000-08:00

They Don't Deserve Your Money
Content providers do not trust you. They bully you into complying with a narrow view of acceptable use, shackling you to the glacial pace of policy-making.

They do not offer any of the confidence, security, or assurance that you have come to expect in return for your money.

Their are $60 devices that store 6,000 songs. At $1 per song, this $60 device can store $6,000 of content. It is easily stolen or lost and the songs can be deactivated at any time, for any reason.

This is like paying $6,000 for a highly-delicate, non-refundable, non-replaceable $6,000 "money card" that you are supposed to carry around everywhere and hope you don't drop it, damage it, or forget it. And oh yeah, the balance can be decreased arbitrarily without notice.

Sounds stupid, right?

It is, and that's the best deal they offer. You'll probably pony up for the same thing multiple times. Sounds like a sweetheart deal for them, doesn't it? That's right, say it with me:

They Are Scamming You
They want you to pay for a restricted license to play a downloaded file that can be remotely deleted at any time. Then, if you lose it and want another copy, they will act like they have no record that you already paid them for it; as if an accounting system is some technically insurmountable pipe dream. That's stupid, they already have every purchase on file and have probably already sold your data ten times over.

Really, they are lying to you and are just looking for an excuse to charge you money.

You should not be doing business with someone like that, who fundamentally mistrusts you and is not willing to give you control of what you pay for. So because the content provider no longer trusts you, you must no longer trust them.

The Old Way
This wasn't always this case. 15 years ago, if you purchased an audio CD, you were permitted the following:

The right to duplicate and convert.
You could copy the CD onto a cassette or CD-R for personal use and archiving.
The right to device agnosticism.
That CD would work in your car, handheld player, boombox, and home stereo system. They were not keyed to a class of system and were fundamentally generic.
The ad-free right.
As a quid-pro-quo for your money, you could enjoy the CD without arbitrary advertisements.
The right of merchantability.
If you mailed in a damaged CD in case, many record labels would send you a new copy for a nominal fee, ensuring you that you wouldn't have to double-pay to re-access what you already purchased.
The right of satisfaction.
Many music stores permitted you to listen to the entire CD in the store prior to purchase or even return it if you simply didn't enjoy it.

You Get Nothing For Your Money
In the digital age, all of these rights will ideally (in the eyes of the provider), be removed; because there is a fundamental culture of distrust.

Fortunately, there is still one mechanism that guarantees you such rights.

Piracy Is The Answer
Piracy gives you the consumer confidence you have become accustomed to in the 90s that the content provider refuses to offer in the internet age:

You can backup the content freely in the way of your choosing.
You can enjoy the content on future technology or any existing one.
The content is pure and functional. It comes without restrictions or advertising.
The content is readily replaceable.
If you are not satisfied, you can revoke ownership of it without any fear of an unrecoverable monetary loss.

Until the content providers give you the same guarantees in the digital age that they gave in the physical one, they will not be offering a product worthy of your time or money so there is no reason to give them any of it.

So, for a product you can trust in, become a Pirate today.

Flash Sucks

2011-12-30T15:55:00.000-08:00

But where is the alternative to the write-once run-anywhere platform that has a rich UI toolkit, is installed on almost all machines ... what? Java you say? Really, come now; Loading anything in Java in the browser feels like I'm booting up a Virtual Machine. What about Silverlight? No, not that either.

Standards you say? Ok, let's roll. You can do most of what Flash does with:

CSS3
HTML Video
WebGL
Adaptive Bitrate Streaming (progress)
HTML Canvas
Imaginary Partial Loading Support That Doesn't Exist (maybe the partially supported <script defer="defer">)
SVG
The Imaginary Sandboxing for Separation of Applications standard
HTML Audio
HTTPS
WebSockets
Imaginary Crossdomain.xml Equivalent That Doesn't Exist
LocalStorage
ECMAScript 5
IndexedDB

And even if you got the imaginary things working and the drafts finalized you still need a solution that

Works on all major platforms
Works on all major browsers
Has no nuanced implementation details
Has non-conflicting, non-ambiguous standards
Is well understood
Has a usable IDE for graphic designers
Has a wide, cheap, coder base.

Flash does this. In fact, I had streaming, synchronized, animation and audio in 1996, on Netscape 2.0, on a Pentium 1 @ 120Mhz with 16MB of RAM, on a 28.8Kbps connection, on Windows 95. All I had to do was download a 160KB add-on and restart my browser; back before DOM 0, and when the W3C was moving from SGML to the fancy new "XML" standard.

A decade and a half later, going to the CSS3 equivalent I have to carefully choose the browser, then see my cpu hosed and still have frame drop, have audio sync problems, and have to load ALL of it before seeing ANY of it.

The Flash hate is totally misplaced. What they have done is absolutely phenomenal. I'm sorry that you see teeth whitening ads and porno site popups with it, but don't blame the technologists; that's like blaming Honda Of Japan because some asshole cut you off on the Freeway.

And besides, what will the ad-haters use in this future world of 2022? Some amalgamated FlashBlock equivalent that can easily just turn off all the annoying stuff? This line will be blurred and it won't be possible.

Adobe has done, and continues to do what Microsoft Failed at, what Apple Failed at, what Sun Failed at, and what Google is almost, but not quite, succeeding in, decades later.

What a Nice Logo you have, Where Did You Find it?

2011-11-27T19:47:00.000-08:00

The small record label Balance Underground (or Alliance) has apparently lifted the Debian swirl as their logo for the past few years (actually legal, but only if the jackets have a disclaimer). It's a sub-label of another swirly label called Balance Recordings that appears to have different artwork.

_{Apparently they were also Spelling-Bee Champions.}

Here's some releases, most of them link back to discogs; the ones at the bottom are from their Facebook page.

About the author
Chris Mckenzie writes Free software.

Edit 1: The Debian swirl appears to be trivial to construct and has been used by other organizations.
Edit 2: A representative from the organization in question has some replies below concerning the logo! :)

Augmented Wiki Model

2011-11-23T13:49:00.000-08:00

Augmented Wiki Model

Statement of problem

An artifact of the traditional wiki model is that malice, bias, and fiction (un-valued content) can masquerade itself as factual information. The mechanisms deployed to prevent this; banning users, protecting, and locking pages each require manual intervention by a member of a privileged group of users.

Objective

To minimize the amount of un-valued content on a wiki by augmentation of the wiki model.

Assumptions

The value of a user's recent contributions can be used to predict the value of a user's contributions in the near future.
The value of a page of content currently to a wiki community can be used to predict the value of the content in the future under a certain guideline (outlined below).

Methodology

Given the above axiomatic assumptions, I propose to augment the wiki model as follows:

Assign a variable to every user and to every page.

The user variable will be called the "integrity" of the user and be a signed number starting at 0.

The page variable will be called the "value" of the page and be a signed number starting at 0.

Every user will be allowed to state a fully retractable, single "+1" "0" or "-1" on another user's or page's value. In theory this will create the following:

The value of the user's contributions will be directly reflected by the sum of the value of their integrity, assigned by other users.

The value of the page's content will be directly reflected by the sum of the value of the page, assigned by users.

Under the above constraints, the following limitations, that by assertion preserve the spirit of the wiki, can be imposed:

There will be two types of edits: moderated, and un-moderated.

Moderated edits

Another user has to approve the edit

Any edit where a user's integrity is less than a page's value becomes a moderated edit

Only users whose integrity is greater than or equal to the value of the page can approve an edit queued for moderation

Un-moderated edits

An edit that does not need approval.

Any edit where a user's integrity is greater than or equal to the value of the page becomes an un-moderated edit

Furthermore, the following safeguards will be used

Since certain accounts will now be of more value than others, hijacking will become a problem.

A user's integrity will only be viewable by a user with more integrity.

Page values will always be viewable.

Only a user with an integrity greater than or equal to the a page's value can view the queue for moderation with the exception of

A user who submitted an edit for moderation can purge their submission from the queue

The history of a page will not reveal to a user with insufficient integrity which edits were moderated and which edits were not

Since people can gain integrity through deceit, and then only disclose malicious intent after accumulation of power, any vote placed upon a user or a page is fully retractable.

Hypothesis 1

Under the augmented wiki model as proposed above, the quantity of un-valued content on a wiki will decrease.

Hypothesis 2

The augmented wiki model as proposed above eliminates the need for the traditional manual protection mechanisms employed to prevent un-valued content being committed.

The winners are: Opera, IE, Firefox, Chrome, Safari; in that order.

2011-11-11T20:22:00.000-08:00

This is a response to a contentious blog entry I wrote two days ago.

In under 36 hours, a lot has changed:

ECMAscript.org has updated their tests
Opera 12 Alpha is out.
Firefox 10 Alpha is out.

Also, many said I didn't give Opera a fair shake. So now the beta and alpha builds have been included.

Here's the updated (2011-11-11) results. Each browser's data is compressed with bzip2.

Browser	Fails	The photo	The data	Release Status
Opera 11.60 beta 1147	1	Screenshot	XML Results	Beta
IE 10 10.0.8102.0	6	Screenshot	XML Results	Alpha
Opera 12.00 alpha 1116	11	Screenshot	XML Results	Alpha
Firefox 10.0a2 2011-11-11	160	Screenshot	XML Results	Alpha
Firefox 9.0	164	Screenshot	XML Results	Beta
Firefox 8.0	164	Screenshot	XML Results	Release
IE 9 9.0.8112.16421	322	Screenshot	XML Results	Release
Chrome 17 17.0.932.0 dev-m	415	Screenshot	XML Results	Alpha
Chrome 16 16.0.912.36 beta-m	415	Screenshot	XML Results	Beta
Chrome 15 15.0.874.120 m	416	Screenshot	XML Results	Release
Safari 5.1.1 6534.51.22	772	Screenshot	XML Results	Release
Opera 11.52 1100	3751	Screenshot	XML Results	Release
IE 8 8.0.6001.18702	N/A^[1]	Screenshot	N/A	Release

¹The test fails to run in IE8, much like the last time. But after the error is hit, the status bar quickly goes back to saying "done". The developers tools, however, shows the error in the same place.

Well, Opera wins. and the old becomes new again. IE10 still takes the number 2 slot with everything else staying nearly the same.

about:
Tests were done on Windows 8, XP, and Vista; to try to main consistency. If you want to contact the author, reply here below. The author is Chris McKenzie; a programmer dedicated to truth, no matter how crazy it gets. Check out his projects on github.

The Acid3 of JS has a few surprises.

2011-11-09T19:29:00.001-08:00

Edit: Updated Results Are Available Here

Wikipedia has an article on a Javascript conformance test called sputnik from Google.

It has the results from various Desktop browsers on how standards-compliant they are. Now I know what you are thinking, probably #1 is Chrome because the test is from Google, followed by Opera, Safari, and Firefox; with IE trailing behind in a pitiful display of brokenness.

But that's not what Wikipedia shows. In fact, it shows IE as the most standards-compliant browser.

Microsoft has quite brilliant engineers and recently they've began to take their browser seriously again, but to say that Internet Explorer has the most standards compliant JS engine? This would be quite a departure from history.

I needed to verify these results, for my own sanity.

According to Wikipedia, the test has changed hands and is now hosted at test262.ecmascript.org. You can go there now, it's on a domain registered by Mozilla Corporation. So this doesn't look like some wonky unfair comparison that almost looks to be designed to make Microsoft look good. Let's get questions of legitimacy out of the way.

Here's the updated (2011-11-09) results. Each browser's data is compressed with bzip2.

Browser	Fails	The photo	The data
IE 10 10.0.8102.0	6	Screenshot	XML Results
Firefox 8.0	164	Screenshot	XML Results
Firefox 9.0 (2011-11-08)	164	Screenshot	XML Results
IE 9 9.0.8112.16421	322	Screenshot	XML Results
Chrome 16.0.912.32 beta-m	415	Screenshot	XML Results
Chrome 15.0.874.106 m	416	Screenshot	XML Results
Opera 11.52	3750	Screenshot	XML Results
IE 8 8.0.6001.18702	N/A^[1]	Screenshot	N/A

¹The test fails to run in IE8 and hits a Javascript error fairly early on.

IE 10 did the best by far. Surprised? Me too. The next up was Firefox. Also surprised? Yeah, here too. Then IE 9 ... IE 9? The world is crazy sometimes. Chrome comes in right above Opera, which basically eats shit ... that's a new one. What has this world come to?

One of the important things to note is that failures seem to happen in batches. That is to say it's successful for a few hundred (or a few thousand) tests, then a bunch of tests fail consecutively. Almost makes me think of that math exam you once got where questions 2-8 depended on you getting question 1 right.

These tests also don't speak to how performant the browsers are on, well, anything. Their are more factors than compliance-to-standards when it comes to providing the best web experience. Regardless, these results certainly support the on-going notion that IE is finally back in the running.

Further Work

I just looked at the number of failed tests; not focusing on the content of what tests failed. I'm not familiar enough (and just plain not smart enough) with the internals of ECMAScript to know whether the results warrant further investigation or concern. I do know from my own experience that I've fallen victim to many more IE Javascript bugs then other browsers. And perhaps, that's the important thing; not whether a feature is implemented, or passes a smoke screen test, but whether the feature has problems. It would be great to hear what some of the browser developers have to say about these results.

about:
Tests were done on Windows 8, XP, and Vista; to try to main consistency. If you want to contact the author, reply here below. The author is Chris McKenzie; a programmer dedicated to truth, no matter how crazy it gets. Check out his projects on github.

PHP? LOLZ!

2011-11-04T13:57:00.000-07:00

First, complaints from an imagined developer:

OMG PHP,
You are not nerdporn. When I do my contracting as a developer, I can no longer take my fixie to the local programmers coffee shop to do work and look cool. Since I don't go out otherwise, how the hell am I going to get a girlfriend in Sunnyvale with such crap?
You are so easy to use that your community is chock full of people who can't code. Like a business guy fooled into the idea he can skimp out on hiring one of us expensive nerds by coding that brilliant facegrouptube idea himself (with an iphone app of course, it'll make millions!). How hard can it possibly be to write a product and platform? Pshh!, piece-o-cake!
You are simply not expressive enough. My coding boner goes way flaccid when I see how not purty PHP is. When my emacs window doesn't follow the rule of thirds when looking at code I get pissed off and it makes baby jesus cry.
Sincerely,
Programmers of the Internet.

With all the hate, why is it still hands down, the most popular language for websites that people actually use?

Things get done when bickering pedantic coders stop caring about process and focus on product. Picky nerd debates about refactoring, design, coding approach ... get tossed, because they are more annoyed by the environment then they are by all those lost Dr Who episodes. This speeds things up enormously (because nobody is arguing any more) and like magic, shit actually gets done.

This makes PHP an extremely productive, fluff-free language. It's not going to change any time soon and nobody is excited about it; people don't assemble in church basements on a Tuesday night like narcotics addicts to talk about how awesome it is at solving all your problems.

It becomes just a tool; which is the ideal position of any language.

Americas wealthiest company put out a press release today.

2011-10-05T22:32:00.000-07:00

iPhone 4S reviews weren't what we expected

So we pulled the plug. There will be no Jobs. No Jobs today, no Jobs tomorrow; no Jobs forever! Now go away because without Jobs, we'll never again have anything you'll want.

Droid X Gingerbread Review

2011-10-03T15:06:00.000-07:00

I "upgraded" my Motorola Droid X recently to Gingerbread:

Recording videos doesn't work. Instead my phone skillfully convinces me it's recording but then has no stop button, no way to leave the application, and stays in the video mode until I pull the battery.
The phone now runs hot and the battery lasts about 1/3 the time as before. Might as well carry around an assortment of extra batteries just so I can use it throughout a regular business day.
Swype now prefers really obscure words like "tutu" when I mean to type "you" and half the time fails to put a space in between words. The browser gets real confused when that happens and clears the input, not permitting me to manually add a space after I realize the mistake.
Although the experience is now shinier with gradients and catchy animations, it is effectively immaterial when applications are hanging left and right and the actual interface for major components (like marketplace) have discarded basic features and transformed themselves from applications to advertisements.

Now here's some big questions:

Why is task switching still unsolved? I need to download third party apps just to go in between two running processes. Really?
Why is task killing still so difficult? To kill a task, Simply follow these 10 basic steps: (1) Go to the home screen, (2) Click the left button, (3) Press "Settings", (4) Scroll the menu down, (5) Press "Applications", (6) Press "Manage applications", (7) Press the "All" tab, (8) Scroll through the application list, (9) Press the menu item of the application you want to stop, (10) Press "Force stop".
What the hell is that? 10 steps to exit a program? I'd hope that making this easier would be somewhat a priority.

Useful things that shouldn't have been removed

Was moving an icon from the application list to the home screen so unbelievably hard and counterintuitive that you had to axe it and replace it with a 3 step process (which doesn't work if you have a task switcher btw)
What is the white gradient near the bottom of the screen? My guess is that 1 out of every 10,000 users had some phone background that was just a montage of android icons and then they went in to their carrier, complaining that they couldn't find the buttons on the interface. It really harkens back to the old days when there was a d-pad with an "ok" button in the center wherein the default functions for the d-pad were operations that incurred a charge, like calling your voicemail or depositing money directly into your carriers bank account.
The interface is now branded everywhere with my mobile carrier. Are they so insecure that I need to be reminded every screen in the largest letters on the screen, who I pay every month? My carrier has already opted to plaster their logo on both sides of the phone (in fact, it is etched in the back).

Features that nobody wants
When I press the home key slightly too quickly, a very slow and clunky application pops up that overrides my silent volume setting and chimes "Say a command". I usually get here when I am trying to exit a hung application in order to go through the ten stop process to kill it. However, I get redirected to conversation mode. I've tried "Go fuck yourself" or "Eat shit" but it just responds with "Did you say ... Call Mom?"

And finally, Blockbuster. Costumers are literally going way out of their way, with the possibility of bricking their phone in order to remove their application from the default install list. That's how bad blockbuster actually is. People hack their phone to uninstall it.

Conclusion
I know how hard it is to make an Operating System; but having things like the video recorder crashing the phone should have been caught in early QA testing; a user trying swype for five minutes would notice the expansive vocabulary that the phone guesses when you are trying to say "I am outside near the gas station", and they should have noticed way early that they literally cut the phones battery life by 70% ... and then should have been like "hrmm ... let's not release this."

This version is a major disappointment. It's such an important time too; Google has an opportunity to really snatch the marketshare from Apple; but it's not going to happen with crap like this.

Sometimes Old Ideas just need Good Execution

2011-09-01T15:15:00.001-07:00

When ideas that have never been successful keep coming up over and over again it's usually because they simply haven't been implemented right and are just waiting for someone to think up the right sauce.

Ambitious successful technology is usually a culmination of decades of failure. For example:

1. Touch screen computing (iPad)
2. On-demand video (YouTube, Hulu)
3. Video phones (Skype)
4. The all-in-one fits-in-the pocket pda/phone (Android/iPhone)

If you talked to say, a YouTube exec in 2004 about their ideas you'd probably walk away thinking "Oh yeah, that idea; the one that has failed so hard, so many times, that I was looking for the hidden cameras".

And you'd be dead wrong.

This isn't new, if you go back 40 years you'll find that Credit Cards, Bar Codes, ISBN numbers, the home theater, all had predecessors that sucked. Even the personal computer and the home video game system had years of failed starts.

The Lottery Analogy

2011-08-29T13:07:00.000-07:00

I've told you the lottery ticket analogy, right? It's really important that everyone understands this:

If you want to win the lottery, you need to do something that makes that possible; in this case it's buy a ticket. But you'll probably lose and that ticket will probably be worthless. However, you cannot win the lottery without it. And who knows? You might, actually, indeed, win.

Here's the rules:

You can't win without playing.
Every time you play, you get a new chance to win, independent of the previous chances.
Doing the right thing on the wrong day is worthless. Even if you do everything perfect and you get really lucky, there are still externalities that will almost always snatch the victory from you. You will then see someone doing the exact same thing as you on another day and succeeding.
When you realize that you picked the right numbers on the wrong day, get over it and Play Again. Don't allow your past to paralyzed you.
You will officially lose far before you personally know it. Make sure you know what failure and loss looks like and then accept it. Never fool yourself into thinking you have a winner, acknowledge the loss and move on as quickly as possible.

This is true in all aspects of life: you want to have a famous pop song? You need to do something that makes that possible; in this case work quite hard and make a song. But you'll probably lose. That song, will probably be listened to by nobody. However, you can't have a famous song without it.

Want to be a successful author or get a good job or make money on the stock market or be a famous scientist? The same rules apply.

on getting a job

2011-07-20T04:46:00.000-07:00

know what you can tolerate

if you don't think you could put up with being a sysadmin every day, don't ever apply as one; regardless of skill; you won't get the job unless you interview at a dysfunctional company that you won't want to work at. know what role you would be willing to put up with and apply strictly for those positions.

getting an interview
resumes are important, github is more. you are pitching yourself, over email. silence isn't a problem. keep trying until someone explicitly says no. remember you are talking to humans and not grammar professors. be cordial but formal; assertive, eager, and well-mannered.

interviewing
smile, agree with everything, answer slowly and clearly, make eye contact, laugh at their jokes, gloss over their mistakes, don't bring in any baggage, and don't worry too much.

after the interview
it's critical to send an email after about 4 but less then 24 hours later. repeat the same pleasantries.

about pay
understand what you need to leverage before it comes on the table. you should make the first offer, always. do it at the point of accepting the job after the employer has vested in you as a future employee. make it implicitly but not explicitly clear that you will quickly back out if conditions aren't met.

remember why are you working
most programmer I know would rather be working on their pet project (which is The Next Big Thing) full-time instead of for a company. If this is your goal, remember this at the negotiation table; 5 day work weeks, coming in around noon, leaving at 5 strictly every day; all of these accommodations are best set at this point.

Abandon your cake

2011-05-27T13:08:00.000-07:00

Pretend you spend all day making your spouse a cake. You carefully research the ingredients and then painstakingly try to the best of your effort to do everything well. However, it comes out slightly too brittle; the hand-made frosting is a little tart. But you spent a lot of time on it.

You present the cake. After the overjoyed response, you proceed to cut the first slice and your spouse takes the first bite, rolls the taste around, ponders a bit and very seriously states, "A bit bland, the frosting needs some work. Let's go to the grocery store and get a better one."

You are shocked. "Let's go to the grocery store?!", you spent all day on that damned thing; the store wasn't even open when you started; there were not even cakes available for sale at the time you began.

But sure, you aren't a pastry chef and you probably made a few mistakes along the way, but it's your cake. You made it specifically for your spouse.

But now, of all things, your spouse looks impartially at it, throws out the idea that you had anything to do with it and does some objective comparison. The conclusion is inescapable; the store is in fact open now, and does sell better cakes than the one that you made and instead of making it yourself you probably should have just gone out and bought one when the store opened, spending all of 20 minutes.

However painful it may be, you must acknowledge that if you want to enjoy a great cake, you should toss yours in the garbage, forget about it, and grab one from off the shelf.

Software is the same way. Oftentimes you will find that a solution baked in-house by a colleague you know and respect is not as mature as one that is open source.

Sometimes although the in-house solution may predate the open source one by many years, the open source solution appears to have raced ahead in quality, stability, and features.

You know intellectually that you will get a better product if you go forward with the open source solution that is well supported and well written. You know that you will free up the time of all parties involved. It looks like a pure business and logical decision. It's easy to forget the fact that you are suggesting to toss out someone's handmade cake by effectively saying:

Hi. I just wanted to say that you wasted your time and your solution is inferior to something I found in 30 seconds on google. We should be using this instead.

Breaking this news is one of the most difficult things to do at a personnel level on any programming team. You look like the new kid tossing out personal insults, disparaging the quality of your colleagues works; suggesting that they produced inferior code that simply is not good enough to be put in any project that you want to work on.

The truth is though, adopting open source (ie, off-the-shelf) components over in-house solutions can often be a leading factor in whether a project is successful and done on schedule. Off-loading as much responsibility as possible permits your team to focus on the product and not the dependencies.

When you are the cake maker, this reality is a very difficult thing to accept. "My cake is worthless?", you incredulously pout. You skeptically go over the open source project with a fine-toothed comb. "But wait", you insist, "It can't do xyz, and I can. Ha!" or "Let's run some performance tests and see how this POS does".

However, you acknowledge that it is well done. They are catching errors that you were too lazy to check for; they have active mailing lists and people around the world fixing bugs while you are at home sleeping. Intellectually, you know what to do:

Abandon Your Cake.
Implement the stuff you have that the open-source project does not and submit a patch. Probably introduce yourself as someone who has written a similar project but has done the grief of switching over.

Once you get over yourself and toss your cake away, you can hop on the winning team with enough courage and strength.

Don't look at it as if you need to match the feature-set of the project but as a collaborative project with open-membership that you have specific expertise on.

You came up with a solution to a generic problem that many people face and you learned a lot by building it out yourself. You have a lot of knowledge to contribute. You were good enough to do the solution single-handed; you are certainly good enough to contribute to a group effort.

So someone else won the internet lottery fame game this time, it happens. Go join them and you will be a valued member of their team. After a little while you will be able to faithfully consider it your project as much as any other contributor.

It will be easier, you'll get more exposure, and you can claim part ownership of something that people actually have heard of instead of filling your resume with github projects that are only watched by yourself.

Sharing the fame beats pounding the pavement any day.

hold on ... even better

2011-04-17T17:18:00.000-07:00


  if($level=="Level 1" && $month=="Month 1")
  {
    $newLevel="Level 1";
    $newMonth="Month 2";
  }
  else if($level=="Level 1" && $month=="Month 2")
  {
    $newLevel="Level 1";
    $newMonth="Month 3";
  }
  else if($level=="Level 1" && $month=="Month 3")
  {
    $newLevel="Level 1";
    $newMonth="Month 4";
  }
  else if($level=="Level 1" && $month=="Month 4")
  {
    $newLevel="Level 1";
    $newMonth="Month 5";
  }
  else if($level=="Level 1" && $month=="Month 5")
  {
    $newLevel="Level 1";
    $newMonth="Month 6";
  }
  else if($level=="Level 1" && $month=="Month 6")
  {
    $newLevel="Level 2";
    $newMonth="Month 1";
  }
  else if($level=="Level 2" && $month=="Month 1")
  {
    $newLevel="Level 2";
    $newMonth="Month 2";
  }
  else if($level=="Level 2" && $month=="Month 2")
  {
    $newLevel="Level 2";
    $newMonth="Month 3";
  }
  else if($level=="Level 2" && $month=="Month 3")
  {
    $newLevel="Level 2";
    $newMonth="Month 4";
  }
  else if($level=="Level 2" && $month=="Month 4")
  {
    $newLevel="Level 2";
    $newMonth="Month 5";
  }
  else if($level=="Level 2" && $month=="Month 5")
  {
    $newLevel="Level 2";
    $newMonth="Month 6";
  }
  else if($level=="Level 2" && $month=="Month 6")
  {
    $newLevel="Level 3";
    $newMonth="Month 1";
  }
  else if($level=="Level 3" && $month=="Month 1")
  {
    $newLevel="Level 3";
    $newMonth="Month 2";
  }
  else if($level=="Level 3" && $month=="Month 2")
  {
    $newLevel="Level 3";
    $newMonth="Month 3";
  }
  else if($level=="Level 3" && $month=="Month 3")
  {
    $newLevel="Level 3";
    $newMonth="Month 4";
  }
  else if($level=="Level 3" && $month=="Month 4")
  {
    $newLevel="Level 3";
    $newMonth="Month 5";
  }
  else if($level=="Level 3" && $month=="Month 5")
  {
    $newLevel="Level 3";
    $newMonth="Month 6";
  }
  else if($level=="Level 3" && $month=="Month 6")
  {
    $newLevel="Level 4";
    $newMonth="Month 1";
  }
  else if($level=="Level 4" && $month=="Month 1")
  {
    $newLevel="Level 4";
    $newMonth="Month 2";
  }
  else if($level=="Level 4" && $month=="Month 2")
  {
    $newLevel="Level 4";
    $newMonth="Month 3";
  }
  else if($level=="Level 4" && $month=="Month 3")
  {
    $newLevel="Level 4";
    $newMonth="Month 4";
  }
  else if($level=="Level 4" && $month=="Month 4")
  {
    $newLevel="Level 4";
    $newMonth="Month 5";
  }
  else if($level=="Level 4" && $month=="Month 5")
  {
    $newLevel="Level 4";
    $newMonth="Month 6";
  }

That must have taken like days or something ... I don't know if the programmer even knew about the advanced "copy" and "paste" concepts.

wait wait

2011-04-16T22:23:00.001-07:00

I can top that (below)


  switch($month)
  {
    case'1':
    $date = calculateenddate(1,$weakly,7,$unixtime);
    break;
    case '2':
    $date = calculateenddate(2,$weakly,7,$unixtime);
    break;
    case '3':
    $date = calculateenddate(3,$weakly,7,$unixtime);
    break;
    case '4':
    $date = calculateenddate(4,$weakly,7,$unixtime);
    break;
    case '5':
    $date = calculateenddate(5,$weakly,7,$unixtime);
    break;
    case '6':
    $date = calculateenddate(6,$weakly,7,$unixtime);
    break;
    case '7':
    $date = calculateenddate(7,$weakly,7,$unixtime);
    break;
    case '8':
    $date = calculateenddate(8,$weakly,7,$unixtime);
    break;
    case '9':
    $date = calculateenddate(9,$weakly,7,$unixtime);
    break;
    case '10':
    $date = calculateenddate(10,$weakly,7,$unixtime);
    break;
    case '11':
    $date = calculateenddate(11,$weakly,7,$unixtime);
    break;
    case '12':
    $date = calculateenddate(12,$weakly,7,$unixtime);
  }
  return $date;

What is a loop?

2011-04-16T22:15:00.000-07:00


function duedates($programStartDate, $month, $request) {
    if ($request == "accelarate")
    {
     $weakcycle = '2';
    }
    else if ($request == "deaccelerate")
    {
     $weakcycle = '4';
    }
    else
    {
     $weakcycle = '4';
    }
    $db = new databasemanager();
    $unixtime = strtotime($programStartDate);

    switch($month)
    {
     case'2':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     break;
     case'3':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     break;
     case'4':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     break;
     case '5':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     break;
     case '6':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     break;
     case '7':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(6,$weakcycle,7,$unixtime);
     break;
     case '8':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(6,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(7,$weakcycle,7,$unixtime);
     break;
     case '9':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(6,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(7,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(8,$weakcycle,7,$unixtime);
     break;
     case '10':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(6,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(7,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(8,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(9,$weakcycle,7,$unixtime);
     break;
     case '11':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(6,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(7,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(8,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(9,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(10,$weakcycle,7,$unixtime);
     break;
     case '12':
     $date[] = calculateenddate(1,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(2,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(3,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(4,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(5,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(6,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(7,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(8,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(9,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(10,$weakcycle,7,$unixtime);
     $date[] = calculateenddate(11,$weakcycle,7,$unixtime);
     break;
    }
    return $date;
  }

... Solid Gold I tell you ... solid gold ....

horrible css today

2011-04-10T13:40:00.000-07:00


 <div class="heading18 width250 brdrbtm padbtm_10">Search for 
Trainings</div>
 <div class="gr10New width250 pad_top">
<strong class="bluetext11">
Search for Upcoming Teacher Training Programs
</strong></div>
 <div class="pad_b3 width250 pad_top"><strong>Select Level:</strong></div>

Now look what you did; you broke the internet.

Horrible SQL

2011-04-10T10:12:00.000-07:00


$db->query("delete from email_campain_settings where UserID='".$_POST['objID']."' and
TagValue='Active' and TagID='free' ");
$db->query("delete from email_campain_settings where UserID='".$_POST['objID']."' and
TagValue='Active' and TagID='paid' ");
$db->query("delete from email_campain_settings where UserID='".$_POST['objID']."' and
TagValue='Active' and TagID='GOLD' ");
$db->query("delete from email_campain_settings where UserID='".$_POST['objID']."' and
TagValue='Active' and TagID='SILVER' ");
$db->query("delete from email_campain_settings where UserID='".$_POST['objID']."' and
TagValue='Active' and TagID='PLATINUM' ");

Fantastic. Just go home, ok?

Javascript Database

2011-04-08T17:31:00.001-07:00

Throwing my hat into this one: A Generic Javascript Database for all.

abusing php header redirect

2011-03-19T16:31:00.000-07:00

header('location:confirm.php?error=true&CN='.$_REQUEST['CNumber'].'&EY='.$_REQUEST['ex
pYear'].'&EM='.$_REQUEST['expMonth'].'&CHF='.$_REQUEST['CHolderFName'].'&CHL='.$_REQUEST['CHolderLName'].'&Cvv='.$_REQUEST['CV
V2'].'&CT='.$_REQUEST['CardType'].'&Message='.$myRsMessage);

Wow. Just, wow.

horrible css again

2011-01-17T21:01:00.000-08:00

<div class="clear"></div>
<div class="text_center pad10">- or -</div>
<div class="clear"></div>
...
.pad10 {
padding:10px;
}

.text_center {
text-align:center;
}

.text_right {
text-align:right;
}

.bg_white {
background-color:white;
}

.show {
display:block;
}

.hide {
display:none;
}
...

sigh

CSS is not a macro programming language.

Look, I know CSS is hard. That's why you get paid well; cause you are doing difficult things. Now for the love of god, do it right. Because all you are doing here is creating a mess.

Crazy PHP round 2

2011-01-16T18:22:00.001-08:00

function createARB($refId, $name, $length, $unit, $startDate, $totalOccurrences, $trialOccurrences, $amount, $trialAmount, $cardNumber, $expirationDate, $firstName, $lastName,$loginname, $transactionkey, $host,$path,$address,$city,$state,$zip,$country,$phone,$email,$address2,$city2,$state2,$zip2,$country2) {

^^^ THAT ^^^ has a bigger brother:

function fetchRequestString($login, $password, $trans_id, $x_first_name, $x_last_name, $x_address, $x_city, $x_state, $x_zip, $x_country, $x_phone, $x_email, $x_Amount, $x_card_num, $x_card_code, $x_exp_date, $desc, $merchantEmail, $transKey,$x_invoice_num,$x_ship_to_first_name,$x_ship_to_last_name,$x_ship_to_address,$x_ship_to_city,$x_ship_to_state,$x_ship_to_zip,$x_ship_to_country, $x_tax, $x_freight,$x_line_items = false){

You know, modern languages can pass in structured data, right? We don't have to push things on to the stack like we did in 1965; you know this, right?

USPS API is horrible

2011-01-16T16:12:00.000-08:00

Garbage Names

On January 2nd, USPS changed shipment option names from things like

First Class Mail International Package

to

First-Class Mail<sup>&reg;</sup> International Package**

This is not a bug. This is what USPS intentionally wants to use going forward.

The solution suggested to fix the apis that break on this is to do something like "strip <sup>&reg;</sup> and the **".

That is to say, that programmers should now put in a data sanitizer because USPS has decided to supply a bunch of garbage tags in their latest API. The recommended solution is to drop the new garbage.

Let me take some time and try to figure out how this could have ever possibly went down:

In December there was a board meeting:
Manager: Our brand name needs to be protected. Are we putting a ^® everywhere that our brand "First-Class Mail" appears? Programmer: Well, not exactly. Our programmer facing API... Manager (interrupts): I've heard enough. When should we do this? Now, or after we fire you? Programmer: B..but, this will break everything! Manager: Well then, we will find someone that will respect our brand. Programmer (under her breath): Holy fuck... Programmer: Alright ... no problem sir. We'll have to send an email out. Manager: The email has already been sent. We can't send out a second. Programmer: Again, this will break everything, internatio... Manager (interrupts): One email only. It's already been sent. People will adapt. Programmer: Let's at least update our documentation by that week. Manager: The documentation server are due for a scheduled downtime that week. We can't change this - the policy has been set. (yes, they went down that week)

Really odd requirements

Ordering of things that probably ought to not matter, really do for USPS. For instance, This is correct:
<Length>12</Length> <Height>8</Height>
This is wrong:
<Height>8</Height> <Length>12</Length>
You have to do Width, then Length, then Height ... and only in that order. Similarly, you have to do Pounds, then Ounces, then Machinable, then MailType, then GXG, then ValueOfContents (mind the case), then Country, then Container, then Size, ... etc... in this order, and only this order.

And when it says it's optional on the documentation, don't count on it. It's required.

Here's how this may have happened, maybe.

Programmer: Let's use XML for our API. Manager: What if someone specified Ounces first, then Pounds, instead of Pounds then Ounces? Programmer: The keys are unique, that ordering will not be a problem. Manager: What if it was a mistake? Programmer: XML shouldn't care. The siblings orders should be interchangeable in the case of Key/Value assignment like this. Manager: There should be only one correct way? Programmer: We would have to go way out of our way to make this. Manager: We pay you well, right? Programmer: Ok Ok, let's at least state this in our documentation. Manager: What? That there is only one correct way? Isn't this assumed? You are a programmer right? Aren't computers supposed to be this pedantic? Programmer: Yeah well ... Manager: We pay you well, don't worry about it.

Horrid Documentation

And why on earth does http://www.usps.com/webtools/htm/Rate-Calculators-v1-2a.htm take sooo damned long to load?

A peak at the source reveals:
<meta name=ProgId content=Word.Document>
<meta name=Generator content="Microsoft Word 11">

Yes. They write there technical APIs in Microsoft Word and then just save them as HTML and post them directly to the site. Genius. At least we can scrape a few things:
<o:Subject>Web Tools API Reference</o:Subject> <o:Author>john.f.johnson@hp.</o:Author> ... <o:LastAuthor>Amanda Loser</o:LastAuthor> ... <o:_AuthorEmail dt:dt="string">Sherry.McNeill@hp.com</o:_AuthorEmail> <o:_AuthorEmailDisplayName dt:dt="string">McNeill, Sherry</o:_AuthorEmailDisplayName>
My first tone of complaint will be the fact that they don't talk about the changes, although they seem to suggest there has been some. You see stuff like this in there example code:
First-Class Mail Interna)
So their example code, isn't actually an example; it's a neutered copy and paste that doesn't actually illustrate a full exchange. It fails to serve its most fundamental purpose; which is to illustratively show what one means.

Horrible CSS

2010-12-14T02:16:00.000-08:00

This is wrong.


<div class="wid200 text_right border_top pad3">
  <div class="clear">
   ...

Basically, authoring CSS like this is the same way people authored HTML before stylesheets. You are explicitly itemizing style hierarchically and ignoring encapsulation and the semantic bridge that CSS is meant to provide. If you prefer to code like this, You should probably actually write like this


<big>
 <big>
  <br>  <br>  <br>  <br>
  <font color=red>
   <strong>
    Markup like this, in circa 1996 style --- because when you are explicit 
    with every style declaration, you are doing the Exact Same  
    Thing.
   </strong>
  </font> 
  and I wrote code like this, it was the mid 90s ... I used Navigator 3.04 gold,
  and connected to the internet via 14400 bps. We've moved one.
 </big> 
 If you ever profess that you do "MVC" and then code like this, you are
 just using buzzwords and totally mistaken.  End of story.
</big>

Here is what a good piece of CSS would look like

<span class="profile cta container">
 <button onclick=dosomething()>Something</button>

The Most Important Thing here is to notice that the class does not dictate a layout or a style - it dictates a higher level, encapsulated, semantic meaning for the content. If you want consistency in layout, design, and theme; this is what it should look like. Then when you do your JS coding, using your fancy jquery, you do


$(".profile .cta button")

And you don't have to add an additional id tag to reference the section semantically. The principle here is simple:

De-couple, but do not de-correlate

Enterprise Database is

2010-12-05T16:13:00.000-08:00

Using a number in a state column that corresponds to a 5 field state table, that is cross referenced to a country table that has 8 fields so that you have to 3 table joins in order to get from "20183" to "CA". Here? you want the billing address I know how to do that!


     select users.FirstName || ' ' || users.LastName, 
         Ship.AddressLine1, Ship.AddressLine2,
         Country.State, State.PostalRegion, Country.name from
       users as users
         join ShippingAddress as Ship on users.ID = Ship.UserID
         join State as state on Ship.StateCode = state.stateID
         join Country as country on state.countryId = Country.code
       where users.ID = '101';

That was so easy and totally necessary. You know things are done Right^™ when a 12-core machine takes 15 seconds to produce a page full of addresses; cause that's how long it took in 1975; when a kilobyte of memory set you back $10,000.

What's revision control?

2010-10-19T17:23:00.001-07:00

This is the css directory for one of my clients, just for one file:


 ++main.css
 __main.css
 main-17-october.css
 main-1st-nov.css
 main-23rdfeb.css
 main-3-8-9.css
 main-bk1.css
 main-extra.css
 main.css
 main.css.bk
 main14April2009.css
 main15April2009.css
 main_bk.css
 main_old.css
 main.css.bak

Not only are their 14 versions of that file, but their are 5 separate conventions for assigning dates. Truly remarkable.

Horrible Javascript Code

2010-10-19T16:58:00.000-07:00

      var class_schedules_text = '';
       class_schedules.each(function() {
               class_schedules_text =  class_schedules_text + $(this).html();
       });
       parts = class_schedules_text.split('</strong>');
       class_schedules_text = parts[1];
       class_schedules_text = '<div style="width: 280px; margin-bottom: 10px; float: left; margin-left: 
       5px;font-size: 12px;">'+class_schedules_text+'</div>';

What it was replaced with

var class_schedules_text = class_schedules.html().replace(/<[^b].*?>/g,'');

Remember, there are two ways of doing layout

Know what you are doing and do it
Use excessive markup to eventually coerce the elements into place

For instance,


  box_content += "<p><div style='overflow: auto;'>
  <label style='float: left; margin-right:3px;font
  -size: 12px;'>Schedule:</label><ul style='margin
  :0; padding:0; float: left; font-size:12px;'>"+c
  lass_schedules_text+"</ul></div></p>";

Becomes:


   box_content += "<p>"+class_schedules_text+"</p>";

Stupidity example: Whm

2010-10-18T18:07:00.000-07:00

So you have this things "WHM" installed. It looks like the default apache page. All you need to do is put things into /usr/local/apache/htdocs/ and you are done right? Override the default script and that's that.

NO, Bitch, You Are Wrong

If you do this, then you will get a bunch of errors; apache is looking for sys_cpanel or something somewhere. It will not actually feed the index.php file. It will feed a subdirectory and it's content listings. So if you wanted to look at /usr/local/apache/htdocs/mydocs, you will see the contents of the directory, but as soon as you click
on a single file, KABOOM!

How the fuck do you make it work then?

Good questions.

Let's back up

The old way, clearly too difficult

# touch /usr/local/apache/htdocs/index.html
Point your web browser to http://ip/

The new way, clearly easier

Point web browser to http://ip_address/whm
Log in
Click on the 14th icon, "Account Functions" between "Account Information" and "Multi-Account Functions"
Click on the 2nd icon, "Create a New Account"
For the domain, give it the eventual domain you'd like to use
Enter a Username
Enter a Password
Retype the Password
Enter in an email and scroll down to the bottom
Click Create
SSH to the ip address using the new username/password pair
Run touch public_html/index.html
Go to the machine you need to access it on and open up your etc/hosts file. On Windows its c:\%WINDOWS%\system32\etc\hosts, on MacOS and Linux is /etc/hosts.
Add the domain name you used in the above step and the ip address. This is a temporary mapping so that you can see the empty file
Save the file
Restart your web browser to clear out your dns cache
Enter in the domain you used above

It's so much easier the new way, right?

How did you figure this out?

It took days. Literally. Who would have thought that you needed to create an account through WHM in order to view a php file on the web server? What kind of stupid is this?

Fuck Cpanel

2010-10-18T17:27:00.000-07:00

Is it really excessively hard to learn how your website stack works? Really? Do you realize that all the complexities of the site are just presented to you in different packaging through Cpanel? It doesn't actually make things easier, all it does is rephrase the same shit in a new way.

While, at the same time, overriding all of the traditional ways of doing things.

If you have the slightest idea what you are doing, this bludgeoning piece of crap software will just randomly regenerate its wrong and broken configuration files over the ones that you manually configured.

Furthermore, they insist on protecting me for some reason and requiring me to do things there way ... instead of doing things the way that has worked since about 1992 - no that wasn't good enough; finding a file through locate, and editing it directly just was way too difficult. Now we need 5 levels of colorful icons amongst a collection of 140 and I have to remember "Services Panel" > "Web Services Panel" > "Web Server Configuration" > "Permissions for Web Server" or whatever the fuck they have decided to come up with.

It's called "httpd.conf" it's in the /usr/local/etc directory or the /etc directory. It's under the apache config, I use vim to edit it, just like I did in 2005, 2000, and 1995. Why must you insist on splicing the file into a spewed out mess of 85 different menu options.

Why?

Because the old way didn't use a fucking mouse. That's why. It wasn't user friendly enough, I'm guessing because it didn't look like the Windows Explorer shell - the epitome of ease of use, of course.

Now I need to find this shit that accesses that single fucking line of the file, and still type in the same damn configuration directives as if I was editing it manually in vim; but now it's through a web app; That's Progress!

I still actually have to have the knowledge of what to change to what; but now I need to know a bunch of additional information so that Cpanel won't overwrite it next time it decides to regenerate all the files from some stupid innocuous command, like an apache restart.

What files does Cpanel like to rewrite?

Who the fuck knows! Really. I don't know at all. I'm thinking that the entire /etc directory is becoming a black box where I would need to start up KDE4 on tightvnc in order to edit like, /etc/hosts in a way that won't get reverted every 5 minutes. If I knew any better, I'd know that I'd have to go into 6 configuration files and rewrite a bunch of Python and XML to disable it, along with running a bunch of distribution specific command line tools. Because you know, putting 4 layers of abstraction on a key/value pair text file is totally awesome; and people are totally capable of modifying the value of two input boxes, but are totally incapable of modifying the value of a single line of a file.

This shit is totally moronic. What is wrong with people?

I see where you are going. But what, pray tell, does Cpanel actually do?

That's easy!

All it does is that it takes a previously rather straight-forward Layout and piles shitpile after shitpile on it to create this crazy obfuscated interface where the obvious and important things are littered 4 menus deep and strewn about like some fucking 12 year old with ADHD and Photoshop decided to start making webpages last tuesday and then jizzed out some sorry excuse for a piece of software and called it Cpanel.

It takes very straightforward technical models and tosses them out the window, instead opting for some exponentially more complex "natural model" which is actually still a model, thus an abstraction, thus providing a barrier to entry, thus requiring a learning curve, and thus being counterproductive and extraordinarily detrimental to the intended purpose of further trying to simplify an already simple task.

Oh I see, what should I learn from this triage?

When webdevs get the rapt as being the stupidest, densest, dumbest programmers in the industry, I can clearly and emphatically agree when they turn to, and speak anything but with the most vile of contempt for this awful veneer that just smears and blurs otherwise obvious, straightforward processes.

For the love of God, if you want to be a programmer, learn to use a fucking computer.

For a more detailed example of this type of mentality, check out this posting on WHM
---

Javascript CSV Viewer

2010-05-04T01:57:00.000-07:00

I had a csv file, you know, the typical... but I didn't have a viewer installed. I wanted something straight forward. I ended up using google docs but really, I was looking for copy and paste, something that would easily facilitate CSV files

Image Fading in linux

2010-04-22T12:46:00.000-07:00

Please look here: fade.html. I'm going to move away from this horrible platform soon.

CPAN

2010-04-21T18:37:00.000-07:00


I like CPAN.  Do you like it?
 Would you like to answer now? [yes]
 Is your answer in English? [yes]
 Is your answer in ASCII? [yes]
 Does UTF-8 look right to you? [yes]
 Would you like to answer via keyboard input? [yes]
 Would you like to continue? [yes]
Checking if you have a computer ...... ok
Checking if it is running ............ ok
Checking if it is done booting ....... ok
Checking if operating system loaded .. ok
Checking if perl is installed ........ ok
...
Error: terminal type "Linux" unknown. Aborting.
cpan>

Apophnia, an image server

2010-03-27T16:24:00.001-07:00

So I was presented with a problem like this:

Hiya, I have a bazillion images and I want to um you know like:
Be able to use new resolutions on the fly
Have these dynamically created images cached
Incur almost zero overhead in the process
Have a dedicated image web server or a web server module to do it ... images are important

Basically, given file /myimage.jpg I want to do say, /myimage_500x500.jpg and then say,
Look for myimage_500x500.jpg.
If not found, back up, try myimage.jpg
See that (_500x500) is a resize directive
Dynamically resize myimage.jpg to 500x500, serve that image
Save a new file to disk myimage_500x500.jpg so that when it is requested again ... it's easy

Oh, and I want some other things
When the image is resized the first time, I want it to be the fastest algorithm possible so that the user sees the resized image with minimal delay.
But I also want a different, slow, high quality algorithm to run to save it to disk

One more thing, if it's not too hard
I want to specify things like this:
"medium": {
    "resize": [1024,1024],
    "quality": 50,
    "cropwhitespace": true
}
so I can do something stupid like request /myimage_medium.jpg

If only this existed! [ Download it here ]

Current implementation details

Written in C.
Uses Mongoose as a webserver
Uses Imagemagick for sizing
Doesn't actually do *everything* above yet, but that is the goal.

Youtube music searcher

2010-02-24T23:52:00.000-08:00

http://qaa.ath.cx/music.html

Try to copy and paste a list like...

+Run DMC - Peter Piper 
Salt N' Pepa - Push It
Slick Rick - A Children’s Story
Sugarhill Gang - Rapper’s Delight
Sugarhill Gang - Apache
Tone Loc - Funky Cold Medina 
Treacherous Three - Feel The Heart Beat 
Trouble Funk - Pump Me Up 
Ultramagnetic MC's - Ego Trippin' 
West Street Mob - Breakdance Electric Boogie 
Whodini - Freaks Come Out At Night 
Whodini - Friends

Online Porter Stemmer

2010-01-08T16:42:00.000-08:00

I was looking for an online version of the Porter Stemming algorithm. I couldn't find one, and then realized that I am the person most recently credited with the official JS version here: http://tartarus.org/~martin/PorterStemmer/

So I guess then, it's my responsibility to make a demo: http://qaa.ath.cx/porter_js_demo.html

There it is.

yawn, merry christmas

2009-12-25T05:39:00.001-08:00

Starting a repository, as I always intended

JS Array to object

2009-12-17T05:00:00.000-08:00

So I wanted to be able to use an object in JSON but transfer it as an array.

The issue is that the syntax of el.name is nice, but the {name:"joe"} is longer then just ["joe"].

The way around it is very simple actually.

We'll start with a normal object

{
 firstname: "Joe",
 lastname: "Shmoe",
 address: "123 Twiddledeedoo Lane",
}

Now we do this in javacript:


var keys = ["firstname", "lastname", "address"], obj = {};

And we transfer ["Joe","Shmoe","123 Twiddledeedoo Lane"] as the payload.

Now the code is fairly straight forward. Assuming:


var values = ["Joe","Shmoe","123 Twiddledeedoo Lane"];

To efficiently get back to the object we do the following:

do{
 obj[keys.pop()] = values.pop();
} while (keys.length);

And that's it. It's so straight forward ... I don't know why it was a mystery for so long...

Easy HTML

2009-12-15T22:26:00.002-08:00

Try it out: http://qaa.ath.cx/easy.html

Stupid ssh script

2009-12-11T19:48:00.001-08:00

I'm usually too lazy to go through the dance of sending over an auth key to the host. So I wrote a dumb script to help me, here it is: http://qaa.ath.cx/authadd.txt

There's a little bit of scary logic in there to avoid duplicates... You may want to take it out.

Journeys through PHP

2009-12-11T01:17:00.001-08:00

Over the years I've used a bit of server side languages: C, Perl, C Shell; I've even wrote my own web server for kicks once. Sounds very 1997, right?

Back to 2009...

So I had to make this website for this company, a big idea company, from scratch. I was a systems programmer who wanted to become familiar with web stuff.

The feasible options in my subjective mind were:

Java

Java was a pain to use in college, and I never really understood Tomcat. Also, I just personally feel like Java requires more thoughtful design and more work. I hate doing both those things.

Scala

Too young still. No really, I don't believe it has proven itself yet. I'm sure it's full of crazy bugs. Would you consider twitter stable? It uses Scala.

Perl

Too old. No I mean ... this would have been ok, I guess. Slashdot is probably still in Perl ... as is, I dunno. It's a good language - probably wouldn't have been a bad choice ... oh well. So much for that.

Ruby

I don't know what is up with this language. People love it one day then hate it the next. I don't think I've heard of anyone at like, the big 50 sites talking about it; so until that happens, until I see an article entitled something like

How we reduced latency and increased load on ebay general search with a distributed Ruby based approach

Until that day, it's really not worth looking into.

Python

Oh a language where print had to be redesigned to be more 'pythonic'. These people actually get bent out of shape on some imaginary philosophy that constitute "Good Programming Language Design". They are like the Libertarians of the programming world. Maybe there is some manifesto nobody has showed me or some magazine I'm not subscribed to.

It's a fine language, but when basic things require the runaround (like a switch statement), I'm told, "But this; this is 'proper'"; the language purists argue, referencing some invisible playbook from which all their assertions come.

I have to give it some credit however; for when you want to do non-trivial advanced things Python handles them almost as well as Perl. Only I guess using Python instead of Perl will make me more friends amongst the wine sipping, dining philosophers of the programming world.

PHP

Ah, the 'bottom rung', 'used dishrag', 'tragically maldesigned' accident of computer science. Why would anyone ever engage themselves in such a huge waste of time?

Well,

It's easy
It's usually fast
It's relatively bug free
It's philosophy-free
It can do almost anything that takes less then a page of math to explain
It integrates nicely with webpages

That sounds like a decent language. But for some reason, it's totally uncool.

Maybe because it's such an easy choice, the large amount of bad code from amateur programmers in PHP has given the language itself a bad reputation --- you know, like what happened with Java. Sorry Sun. :-(

Some of the distaste for PHP however, is well-founded.

The Bad Parts of PHP

Ok, so PHP is a fine language. There. I said it. Go unfriend me on facebook if you want. But really, it grabs things from a database and can easily emit html. That usually covers it.

The truth is, most of the time, if you are doing webpages, you can get by with any language that supports the following:

Conditionals (if/then/else)
Variables that vary (haha - take that Haskell)
Data structures (array)
Data resource connections (like pg_query)
Easy to use I/O routines (sorry Erlang)
Iterators (for, foreach, while, do)

And that's it. You can totally make a huge ass website if you just have those things in your language. This isn't asking much, and when you don't ask for much, PHP delivers!

Alas, their are some problems:

Concurrency Support

Concurrency isn't something that is an after-thought. You can't just slap it on to a language like you can with SHA1 support. Well, you can, but then you get a mess. Ideally, you need to have a language that was carefully thought out (read Python) or something designed for this mess (read Erlang).

Overhead

Object memory overhead in PHP is obscene. Just ridiculous. Most web objects are small, like on the order of kilobytes --- which PHP happily translates into megabytes. But if you want to do some Data Warehousing applications and you have a huge framework already in PHP, then PHP for the DW sounds like a reasonable choice. Oh, my friend, be prepared for a surprise.

PHP's hash tables don't scale well. Aggregated O(1) my ass --- yeah if 1 means 1 second.
PHP's memory limit is a 32 bit signed int --- that wraps around and does some ABS or something on itself --- so if you make it like 2049M then you have a 1MB limit. You can try it yourself if you like
Object oriented-ness was tacked on ... once again, if it's going to be there, then it has to be a core language design feature, not an after-thought. When you try to retrofit a traditional language into an object oriented syntax, you usually end up with something much more complex then need be, i.e. C++.

As a result, doing anything more then C-style software engineering in PHP is just going to lead to bad news. In PHP, you need to embrace globals, carefully name your variables, and sort your code like it's a C program from back in the day. Then when you hit the scaling point of multiple databases spanning across multiple servers you realize a problem:

No serious backend software is in PHP

Why does this matter? Well let's look at the DW again. If you are using a Java based system like say, Solr (Lucene), HBase (Hadoop), or Cassandra, then you have two options:

Use a restful API, or in the case of Cassandra, Thrift.
Program in Java

The problem with number 1, is that unless you have scaled your application over sooo many systems, so poorly, that your data is far away from your processing, then you are duplicating data in memory. Sure it's fast, but it's also stupid. What you are saying is this:

As a matter of convenience and stubbornness and my decision to program in language X when using a library in language Y, I will now translate all the data from language Y to X through some crazy interface before I do my mapping and reducing steps.

And that, my dear, is horribly irresponsible.

Despite all this, I'm guilty of all the above and use PHP extensively. :-p

adding exclusionary search to craigslist

2009-12-09T05:06:00.000-08:00

So I'm looking for a car, but craigslist doesn't support the - sign in search

Try doing -salvage in the car search ... doesn't work.

So I made a filter bar to exclude things from a craigslist search:

Get it here

Use it to type in obvious things...
Screen shot

Simplifying libpcap filter creation

2009-12-08T20:44:00.000-08:00

Making a capture filter for tcpdump from wireshark has always been a pain in my mind. Maybe there is a nice tool out there to do it for me --- but I really don't know. So the useful syntax for tcpdump is basically:

(network protocol)[offset] == (decimal value)

For instance:

ether[100] == 123 and ether[102] == 124

I mean sure, some wankers probably want to do more with their packets but I'm not one of them. I simply have a typical packet like this:

 0000  02 00 00 00 45 00 02 8a  db 46 40 00 40 06 b0 6c   ....E... .F@.@..l
 0010  43 7f e9 5a 4a 7d 35 64  fa 4c 00 50 a0 b6 33 f4   C..ZJ}5d .L.P..3.
 0020  6b 36 b9 97 80 18 20 8a  ca 1e 00 00 01 01 08 0a   k6.... . ........
 0030  5b 11 94 89 ca d6 48 ed  47 45 54 20 2f 20 48 54   [.....H. GET / HT
 0040  54 50 2f 31 2e 31 0d 0a  48 6f 73 74 3a 20 67 6f   TP/1.1.. Host: go
 0050  6f 67 6c 65 2e 63 6f 6d  0d 0a 55 73 65 72 2d 41   ogle.com ..User-A
 0060  67 65 6e 74 3a 20 4c 69  6e 6b 73 20 28 32 2e 31   gent: Li nks (2.1
 0070  70 72 65 33 37 3b 20 46  72 65 65 42 53 44 20 37   pre37; F reeBSD 7
 0080  2e 30 2d 52 45 4c 45 41  53 45 20 69 33 38 36 3b   .0-RELEA SE i386;
 0090  20 38 30 78 32 34 29 0d  0a 41 63 63 65 70 74 3a    80x24). .Accept:
 00a0  20 2a 2f 2a 0d 0a 41 63  63 65 70 74 2d 45 6e 63    */*..Ac cept-Enc
 00b0  6f 64 69 6e 67 3a 20 67  7a 69 70 2c 20 64 65 66   oding: g zip, def
 00c0  6c 61 74 65 2c 20 62 7a  69 70 32 0d 0a 41 63 63   late, bz ip2..Acc
 00d0  65 70 74 2d 43 68 61 72  73 65 74 3a 20 75 73 2d   ept-Char set: us-
 00e0  61 73 63 69 69 2c 20 49  53 4f 2d 38 38 35 39 2d   ascii, I SO-8859-
 00f0  31 2c 20 49 53 4f 2d 38  38 35 39 2d 32 2c 20 49   1, ISO-8 859-2, I
 0100  53 4f 2d 38 38 35 39 2d  33 2c 20 49 53 4f 2d 38   SO-8859- 3, ISO-8
 0110  38 35 39 2d 34 2c 20 49  53 4f 2d 38 38 35 39 2d   859-4, I SO-8859-
 0120  35 2c 20 49 53 4f 2d 38  38 35 39 2d 36 2c 20 49   5, ISO-8 859-6, I
 0130  53 4f 2d 38 38 35 39 2d  37 2c 20 49 53 4f 2d 38   SO-8859- 7, ISO-8
 0140  38 35 39 2d 38 2c 20 49  53 4f 2d 38 38 35 39 2d   859-8, I SO-8859-
 0150  39 2c 20 49 53 4f 2d 38  38 35 39 2d 31 30 2c 20   9, ISO-8 859-10, 
 0160  49 53 4f 2d 38 38 35 39  2d 31 33 2c 20 49 53 4f   ISO-8859 -13, ISO
 0170  2d 38 38 35 39 2d 31 34  2c 20 49 53 4f 2d 38 38   -8859-14 , ISO-88
 0180  35 39 2d 31 35 2c 20 49  53 4f 2d 38 38 35 39 2d   59-15, I SO-8859-
 0190  31 36 2c 20 77 69 6e 64  6f 77 73 2d 31 32 35 30   16, wind ows-1250
 01a0  2c 20 77 69 6e 64 6f 77  73 2d 31 32 35 31 2c 20   , window s-1251, 
 01b0  77 69 6e 64 6f 77 73 2d  31 32 35 32 2c 20 77 69   windows- 1252, wi
 01c0  6e 64 6f 77 73 2d 31 32  35 36 2c 20 77 69 6e 64   ndows-12 56, wind
 01d0  6f 77 73 2d 31 32 35 37  2c 20 63 70 34 33 37 2c   ows-1257 , cp437,
 01e0  20 63 70 37 33 37 2c 20  63 70 38 35 30 2c 20 63    cp737,  cp850, c
 01f0  70 38 35 32 2c 20 63 70  38 36 36 2c 20 78 2d 63   p852, cp 866, x-c
 0200  70 38 36 36 2d 75 2c 20  78 2d 6d 61 63 2c 20 78   p866-u,  x-mac, x
 0210  2d 6d 61 63 2d 63 65 2c  20 78 2d 6b 61 6d 2d 63   -mac-ce,  x-kam-c
 0220  73 2c 20 6b 6f 69 38 2d  72 2c 20 6b 6f 69 38 2d   s, koi8- r, koi8-
 0230  75 2c 20 6b 6f 69 38 2d  72 75 2c 20 54 43 56 4e   u, koi8- ru, TCVN
 0240  2d 35 37 31 32 2c 20 56  49 53 43 49 49 2c 20 75   -5712, V ISCII, u
 0250  74 66 2d 38 0d 0a 41 63  63 65 70 74 2d 4c 61 6e   tf-8..Ac cept-Lan
 0260  67 75 61 67 65 3a 20 65  6e 2c 20 2a 3b 71 3d 30   guage: e n, *;q=0
 0270  2e 31 0d 0a 43 6f 6e 6e  65 63 74 69 6f 6e 3a 20   .1..Conn ection: 
 0280  4b 65 65 70 2d 41 6c 69  76 65 0d 0a 0d 0a         Keep-Ali ve....

And now I want to like, create a libpcap filter on the "GET" to detect whether it's a GET request.

Now I know what the wankers are saying:

"Well, what if this was some crazy ass packet over here with the boundary of one layer of the network stack just happening to translate into the ASCII character 'G' and then say, the signature or magic number of the next layer down translating into 'ET'. THEN WHAT? Then what?"

Um, then, I get those packets too; all zero of them. Really. I mean, get a life. That shit doesn't happen.

So let's go back to the real problem. I want to use wireshark and figure out how to write this ...

So I fire up wireshark and then go to the three pane view. I expand the "Hypertext Transfer Protocol" node, then the "GET" node and right click on "Request Method: GET". I go up to apply as filter then click "selected":

This seems quite reasonable. After clicking on it I do indeed get something in the filter syntax:

Absolutely stunning. I get this:

http.request.method == "GET"

That's not libpcap. Totally not it. You totally lose. I cannot pass that string to tcpdump. Luckily, the wireshark people have a command line capture form of wireshark, called tshark. Fantastic, let's see how to use it! :-)


$ tshark -h
TShark 1.0.3
Dump and analyze network traffic.
See http://www.wireshark.org for more information.

Copyright 1998-2008 Gerald Combs (email withheld by me) and contributors.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Usage: tshark [options] ...

Capture interface:
  -i <interface>           name or idx of interface (def: first non-loopback)
  -f <capture filter>      packet filter in libpcap filter syntax
  -s <snaplen>             packet snapshot length (def: 65535)
  -p                       don't capture in promiscuous mode
  -y <link type>           link layer type (def: first appropriate)
  -D                       print list of interfaces and exit
  -L                       print list of link-layer types of iface and exit

Great, just great. You can't even accept your own syntax as a capture filter, just as a display one. I'm trying to reduce file size here...

What's the point?

So anyway, to make life easy I did a nice javascript app, as I love to do sooooo much and you can use it here.

So we go back to my initial packet and copy and paste it in the gargantuan textarea and click the only button there 'process'.

After you click on the obvious blue links, something magical starts to happen:

The filter syntax shows up for you. You don't have to play CSI Miami and count the bytes with VB by hand or something any more. Welcome to the 1980s ... you can use a mouse and construct a filter with ease. Enjoy.

It's worth mentioning that there is actually no protocol level analysis being done. If you have a radiotap header, the silly javascript app won't know. If you are listening on a device without an ethernet layer, such as tun0 or tun1, it will also not know. You basically need to be listening on wired ethernet. But that isn't hard now, is it!

JS Challenge #1

2009-12-06T07:38:00.000-08:00

Self Evaluating JS
Here is an example:

<div id=toexec>
"Hello World<!--".replace("Hello World",'"Goodbye World"');//-->"
</div>
<button onclick=exec()>Exec</button>
<script>

function exec(){
      var n = document.getElementById('toexec');
      n.innerHTML = eval(n.innerHTML);
}
</script>

The Challenge
Actually there isn't one. I just really like that code

http://qaa.ath.cx/pexec.html

CSS Challenge #3

2009-12-06T07:14:00.001-08:00

No diagram for this.

The challenge is to define some CSS so that you can do basic math operations. I have a nice working example hidden away.

The specs are

class="number operator number"

where number is one of [one, two, three, four, five, six, seven, eight, nine]

and operator is either [plus, minus]

So that if you do
<div class='three plus eight' />

you'll see say, 11 squares or some other kind of unit.

Extra points if you can make negatives work too.

You ought to be able to do it in 12 definitions or less...

good luck!

CSS Challenge #2

2009-12-06T06:54:00.001-08:00

Same HTML as #1, but the CSS must be within this:


div { /* You probably need at least this */ }
body{ /* and you can use this too */ }

CSS Challenge #1

2009-12-06T06:45:00.000-08:00

So I give you the HTML, you give me the CSS:


 <div style=border-color:lime />
 <div style=border-color:yellow />
 <div style=border-color:red />

Programming Challenge #2

2009-12-06T06:27:00.000-08:00

If you can solve 1, you can solve 2


#include<stdio.h>
main(){
        char ByModifyingThis[]="What happened?";
        char printThis[]="You won!";

        // your space
        
        printf("%s",ByModifyingThis);
}

Programming Challenge #1

2009-12-06T05:32:00.000-08:00

Ok, first to post a correct 1 line solution wins...


#include<stdio.h>
void gettohere()
{
        printf("You win!");
}

int main(void) 
{
        // From Here
        // Without calling gettohere();

        return 0;
}

Trimming JSON

2009-12-04T22:58:00.000-08:00

JSON is a fairly succinct markup language. For instance, say I have two people Bob and Alice, and I have very private records on them:

{

"Bob":
{
"SSN": "502-80-2012",
"CCNumber": "3842-4234-1023",
"Mother's Maiden Name": "Swanson"
},

"Alice":
{
"SSN": "312-45-1231",
"CCNumber": "8481-3231-2234",
"Mother's Maiden Name": "Swathmore"
}

}

Now say I wanted to put this imaginary pseudo structure in "JSON format". Conveniently enough, it already is. No really. I'm not kidding. That's it. We are actually done. Or, in a more compact form:

{"Bob":{"SSN": "502-80-2012","CCNumber": "3842-4234-1023","Mother's Maiden Name": "Swanson"},"Alice":{"SSN": "312-45-1231","CCNumber": "8481-3231-2234","Mother's Maiden Name": "Swathmore"}}

Unnecessary Syntax
Now that's all gravy and really useful, but wait; did you know that there are only a few reserved words in Javascript and that everything else can safely be used as labels; as long as it doesn't have a space or some other reserved character. The stuff above then could be:

Before: (188b)
{"Bob":{"SSN": "502-80-2012","CCNumber": "3842-4234-1023","Mother's Maiden Name": "Swanson"},"Alice":{"SSN": "312-45-1231",CCNumber": "8481-3231-2234","Mother's Maiden Name": "Swathmore"}}

After: (173b)
{Bob:{SSN:"502-80-2012",CCNumber:"3842-4234-1023","Mother's Maiden Name": "Swanson"},Alice:{SSN:"312-45-1231",CCNumber:"8481-3231-2234","Mother's Maiden Name": "Swathmore"}}

Those extra bytes don't need to be transferred, really. So don't do it.

Zero-information data
Some information above offers Zero information. I could, for example, specify that I will always get the fields in this order:

SSN
CCNumber
Mother's Maiden Name

As I did above. Given this then, those labels contain Zero information and ought to be eliminated if you care about space. The datastructures then, could be converted into arrays as follows:

Before: (173b)
{Bob:{SSN:"502-80-2012",CCNumber:"3842-4234-1023","Mother's Maiden Name": "Swanson"},Alice:{SSN:"312-45-1231",CCNumber:"8481-3231-2234","Mother's Maiden Name": "Swathmore"}}

After: (101b)
{Bob:["502-80-2012","3842-4234-1023","Swanson"],Alice:["312-45-1231","8481-3231-2234","Swathmore"]}

There! Removing Zero-information data is quite a space saver - as long as you are competent enough to know how to keep your code and your data separate.

Conceptual Run-time Analysis
The above optimizations will cost you a few regular expressions on the server side before emitting it to the client but this will be saved on the client side a few ways

Less data on the wire
Smaller string to parse
Array indexes are faster then hashmap lookups --- they just are.

So it's the cumulative time of (server) and (client):

extra regex - data transfer - parsing - hash overhead

And seeing as the server side environment is much more diverse and gives you much more options then clientside (where it's javascript) - there's probably more reason to believe that this is a feasible net gain then not.

Questionable Optimizations
There are further steps that you can do for byte-count reduction --- but it is fairly questionable whether there will be a cumulative performance gain. I've broken the further steps down to a few methods:

Data Structure Flattening
Some background
In C, when you declare a multi-dimensional array, say 50x50, you can either index the array as you defined it, say using something like [4][20]; or you could just ignore it and treat the datastructure as it exists in memory, just a sequence of 250 units. You can cast the datastructure to the generic pointer type and then index it at [4 * 50 + 20] - sometimes it's easier, sometimes it isn't.

Flattening JSON
Let's revisit our previous datastructure and the principles of Zero Information. If we know that the structure will be something like this:

{ name: [3 fields], name: [3 fields], ... name: [3 fields] }

Then we could utilize a trick very similar to the C one discussed above. That is to say, just make the datastructure one gigantic array:

Before: (101b)
{Bob:["502-80-2012","3842-4234-1023","Swanson"],Alice:["312-45-1231","8481-3231-2234","Swathmore"]}

After: (99b)
["Bob","502-80-2012","3842-4234-1023","Swanson","Alice",
"312-45-1231","8481-3231-2234","Swathmore"]

There's a few important things we have to give up: First we no longer have 'labels' so the quotes have to go back in, adding a few bytes. But as a result, we can simplify the syntax a bit. Iterating through this can be done in many ways and the optimal one depends largely on the context of the data usage. It is however questionable, whether having to do additional math overhead is worth the extra 2 bytes. But there is, however, one more optimization we can do in this field

Stringifying
If we are willing to take the math overhead in stride, why don't we just go all the way then and simply make the data CSV - wherein you only need to quote things that have the reserved characters:

Before: (99b)
["Bob","502-80-2012","3842-4234-1023","Swanson","Alice",
"312-45-1231","8481-3231-2234","Swathmore"]

After: (83b)
"Bob,502-80-2012,3842-4234-1023,Swanson,Alice,312-45-1231,8481-3231-2234,Swathmore"

Transferring the output into a place usable for Javascript is debatable. You can't just split on the ',' since you need to accommodate for the escaped \'.

So you make yourself a nice regex to do the split and come up with an array at the end. But this is foolish. What you really ought to do is use the exec method and work your way through the string. This is of course, quite a bit more work then the code above.

This more work means more javascript code, more lines to execute, more lines to compile, etc. Is it worth it? Eh, don't know ... really, I don't.

Tables
Another thing we could do is to have substitution tables. For instance, let's say that when you analyze your data-structure after you've done the Zero-information removal techniques above, you still see a lot of redundancy; simply because there is a user that is very active, or something else of that nature. You can then make a static lookup table and implement it in JS and your server side scripting language of choice. During the JSON generation you simply swap things out. For instance, pretend we have this data set:

{Alice:["Marina Del Rey","California"],Bob:["Marina Del Rey","California"],Eve:["Playa Del Rey", "California"],Doug:["Playa Del Rey","California"]}

This would be likely say, if you are a local company and have local clients in these areas. You already have the fancy gzip compression and decompression on the server and client side, so you think you are good there, but not really:

gzip still needs to pass over the strings on both sides; parse them, create the tables, allocate the memory - etc. It's real CPU time that you don't need to spend
Javascript needs to do the same stuff.

So you just make a simple table:

var C = {
MDR:"Marina Del Rey",
CA:"California",
PDR:"Playa Del Rey"
};

Expose this on both ends and put it in your JSON encoder on the server side:

Before: (149b)
{Alice:["Marina Del Rey","California"],Bob:["Marina Del Rey","California"],Eve:["Playa Del Rey", "California"],Doug:["Playa Del Rey","California"]}

After: (74b)
{Alice:[C.MDR,C.CA],Bob:[C.MDR,C.CA],Eve:[C.PDR,C.CA],Doug:[C.PDR,C.CA]}

Not only have you saved significant bytes, but now those entries are just pointers to pre-existing strings in a table, so JS doesn't have to allocate new memory for it.

Dynamic Tables
If you really enjoy hitting your server CPU hard, you could create dynamic tables:

Before: (149b)
{Alice:["Marina Del Rey","California"],Bob:["Marina Del Rey","California"],Eve:["Playa Del Rey", "California"],Doug:["Playa Del Rey","California"]}

After: (136b)
{C:{MDR:"Marina Del Rey",CA:"California",PDR:"Playa Del Rey"}
,Alice:[C.MDR,C.CA],Bob:[C.MDR,C.CA],Eve:[C.PDR,C.CA],Doug:[C.PDR,C.CA]}

But this is not recommended. Because you aren't doing the zero-information principle which is to remove, not just rearrange. What you have done above is just complicate things for a what may actually work out to be no byte gain at all, and will certainly be more CPU intensive to generate and parse on both the client and the server side.

Geoiplookup script

2009-12-04T21:43:00.001-08:00

I found a syntax error in one of our javascript files for the insertion of Google Analytics. This of course meant that the results were flatlined:

On December 4th, however we officially 'launched'. Their are now two problems:

How much traffic did we get on Dec 2 - 4?
Where did this traffic come from?

The first question is relatively easy to answer. Since we use nginx as a proxy for apache, I went over to /var/log/nginx and gunzip'd a few log files... nothing fancy.

How much traffic did we get on Dec 2 - 4
To find out how many unique visitors there were over the span of four log files we do the following pseudo-code:

print logs | show only the ip address for each line | find the unique ones | give me the count

This becomes

$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | awk ' { print $1 } ' | sort | uniq | wc -l
315

Not bad. To do it date based, we just add a grep line in the stack:

December 2nd:
$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | grep 02\/Dec | awk ' { print $1 } ' | sort | uniq | wc -l
52

December 3rd:
$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | grep 03\/Dec | awk ' { print $1 } ' | sort | uniq | wc -l
59

December 4th:
$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | grep 04\/Dec | awk ' { print $1 } ' | sort | uniq | wc -l
204

Nice healthy jump there. But wait, there's more.

Where did this traffic come from?
In order to do this I found a tool called geoiplookup - which is available in the apt repositories.

$ apt-cache search geo | grep IP
libgeoip-dev - Development files for the GeoIP library
libgeoip1 - A non-DNS IP-to-country resolver library
python-geoip - python bindings for the GeoIP IP-to-country resolver library
python-geoip-dbg - python bindings for the GeoIP IP-to-country resolver library (debug extension)
geoip-bin - IP lookup command line tools that use the GeoIP library
kipi-plugins - image manipulation/handling plugins for KIPI aware programs
libapache2-mod-geoip - GeoIP support for apache2
libgeo-ip-perl - Perl bindings for GeoIP library
php5-geoip - GeoIP module for php5
tclgeoip - Tcl extension implementing GeoIP lookup functions
tor-geoipdb - geoIP database for Tor
$ sudo apt-get install geoip-bin

The geoiplookup tool only comes with a database that narrows the IP address to a specific country --- which is not very interesting. However, using the magic oracle, I discovered a much more specific city-based database at http://geolite.maxmind.com/download/geoip/database/

so then I did a nice:
$ cd
$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz

Thought a few seconds... then gave up:

$ man geoiplookup

OPTIONS
-f Specify a custom path to a single GeoIP datafile.

-d Specify a custom directory containing GeoIP datafile(s). By default geoiplookup looks in /usr/share/GeoIP

Wow, that's damn confusing. Is this right?:

$ mkdir geo
$ mv GeoLiteCity.dat.gz geo
$ cd geo
$ gunzip GeoLiteCity.dat.gz
$ geoiplookup -f ~/geo/ 4.2.2.4
Error Traversing Database for ipnum = 67240452 - Perhaps database is corrupt?
Segmentation fault
$

Lovely. So the next step

$ strace !! |& less
...
brk(0) = 0x603000
brk(0x624000) = 0x624000
open("/home/chris/geo/", O_RDONLY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f866a21f000
fstat(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
...

It's rather pathetic when strace gives you better documentation then the man page. It clearly wants the full path.

$ geoiplookup -f ~/geo/GeoLiteCity.dat 4.2.2.4
GeoIP City Edition, Rev 1: US, (null), (null), (null), 38.000000, -97.000000, 0, 0

Final Packaging
Ok, that's the ticket. Now we just do a little bit of xargs:
$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | grep 04\/Dec | awk ' { print $1 } ' | sort | uniq | xargs -n 1 geoiplookup -f ~/geo/GeoLiteCity.dat

That does the trick, but now we have this huge ass "GeoIP City Edition, Rev 1:" in front of everything. That's ok, I know sed ... let's make it ordered by country too:

$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | grep 04\/Dec | awk ' { print $1 } ' | sort | uniq | xargs -n 1 geoiplookup -f ~/geo/GeoLiteCity.dat | sort | sed s/^.\*:\ //g

Ah, almost complete. Now we just need to mail it off so I can forward it to the boss

$ cat www.izuu.com.access.log www.izuu.com.access.log.[1-3] | grep 04\/Dec | awk ' { print $1 } ' | sort | uniq | xargs -n 1 geoiplookup -f ~/geo/GeoLiteCity.dat | sort | sed s/^.\*:\ //g | mail cmckenzie

And there. Then I have a nice list of stuff to give to the boss that albeit, not graphical, is still digestible and better then a line at 0... sw33t.