The disruptive power of URL shorteners

The popularity of URL shorteners has taken off ever since Twitter launched. We are increasingly seeing the short URLs everywhere. In fact, nobody would ever want to send a regular link again - even via email. Why? Simply because you get the tracking ability when you shorten the URL. For example, bit.ly gives you the click count on a URL and it gives you an idea about the reach of your network.

However, this blog post is not about the benefit of shortening the URL. Instead, I see in the URL shorteners the power to disrupt the status quo in the search industry. Yes, that's right! Google can be toppled! By these tiny URL shortener companies. I hope Google is reading this.

How you may ask? Here's how. Have you noticed this little button becoming more ubiquitous:




This button is a service provided by Tweetmeme and helps readers do a 1-click tweet of the web page, and I have used this on my website at www.mashedge.com.

Guess what happens when you add this button to your web page? The very first step in generating the button is to shorten the URL by a service like bit.ly. So, the URL shortener is the first one to see your new web page or blog entry as soon as it is born. And guess what, it can be hours or days before this web page shows up in the search results. This is the gap that the URL shorteners can exploit to provide a unique service, and a lucrative one too. Thus, the URL shorteners are becoming the gatekeepers to new web pages. They have new information that nobody else has. And with it, comes the power to disrupt.

Now, a real-time search service (probably there is one already that I am not aware of?) can be built on top of this stream of brand new URLs popping up all over the web. And nobody can beat this service in terms of freshness, not Twitter and not Google. When the results of this real-time service are aggregated with the conventional search engine results, you will have a new order in the search engines. Unless of course, Google buys up all the URL shorteners.

And the search service can be built at a fraction of the cost of Google. You don't need the armies of robots to discover these new URLs. They are being offered to the URL shorteners on a golden platter.

So, if the URL shorteners don't get into the search business, they will be very quickly acquired by one of the search companies. Expect these acquisitions to be have a huge price tag!

Follow kiran_achyutuni on Twitter.

Twitter vs Google: Twitter Winning

I am simply blown away by the power of real-time search in Twitter. Provided you choose to follow the right people, you can get a wealth of high-quality relevant real-time information from Twitter.

Google prides itself on providing relevant and fresh results but the little blue bird is beating the gorilla hands down. So, why is this? It's because of millions of users constantly searching their favorite corners of the web and tweeting about interesting articles. The collective search power of humans is trumping the computer algorithms of Google.

The question arises -can this be automated? Yes, parts of it can be automated - as in RSS feeds etc. The search engines can be informed in real-time about the existence of new content. So, why are Google and other search engines lacking? The answer is that a human being is the ultimate judge in quality and relevance, and that cannot be automated. Whenever a Tweet happens about a particular story, the user has voted on it about its quality. The Tweeter has put his/her reputation on the line by recommending a link. That's why it is a high-quality recommendation. The relevant users pick it up and it spreads like wild fire. It's the user community that is making the difference in Twitter and putting it over the top.

By the way, my use of Google search has decreased after discovering the power of Twitter! Has yours too?

Some useful links

1. Official Twitter 101 for business
   
2. Top 10 Twitter tips for Beginners (from PC Magazine)
3. Technorati's @twitter_tips (articles from the blogsphere that are tagged as twitter_tips)
4. Alltop twitter Stories (Keep track of the latest stories on Twitter)
5. 105 Twitter apps for PR (although it is directed to PR people, this is a useful list of all audiences).

IE Browser caching issues

Internet Explorer (IE) caches web pages differently when compared to Firefox and Safari. And so, the website developer must take extra care to ensure that their website works as expected in IE.

Here are some tips:

1. Make sure you set the expiration date for every page. If yours is a dynamic web site, make sure to set the expiration date to 'now' so that the browser is forced to refetch the page.
2. If you are using AJAX, make sure your XMLHttp requests that use the 'get' method use an URL that changes with time.

I have added links to good webpages that talk about this issue in this web book Mashedge Chapter: Browser Compatibility.

Why Google wins

Ever wonder how Google maintains its #1 position among search engines? Here's one reason: they send their robots to find places where stuff changes.

I first put up my website www.mashedge.com on a limited alpha test over a month ago. The first day I put it up, Googlebot was there and has been a regular visitor since. Only in the last 1 week, I have seen the MSN bot come by. There has been no trace of the Yahoo bot even now. If Microsoft and Yahoo want to catch up with Google, they better get their bots working right.

Setting up sendmail relay with Airtel India as the ISP

This post is meant for those users who meet these requirements:

1. You have obtained a static IP address from Airtel India.
2. You have set up your host correctly - see my previous post.
   
3. You plan to host a mailserver on your host with the ip address from step 1.
4. You plan to use sendmail as the MTA (mail transport agent).
   

In order for sendmail to successfully relay messages to your ISP, and then onto the recipients, there are a number of steps to configure sendmail correctly. Some of these are required to eliminate spam (a very honorable goal).

First, you need to ask Airtel to enable your sendmail daemon to connect to their sendmail daemon on port 25. This is accomplished by sending a email to their customer service care.karnataka@airtel.in with cc: to d.blr@airtel.in. (Note these e-mail addresses will be different if you are in a different region in India). This step is mainly to get written confirmation from you that you will not send spam and have anti-virus measures properly done on your host.

Second, once port 25 is enabled, go to relay.airtelbroadband.in and register the e-mail address for which you need the relay. Let's say the webserver on your host will send mails from email address foo@blah.com, then you do the following:

1. use the DSL userid and DSL password that Airtel gave you.
2. In the relay addresses form, add "foo@blah.com". In the password field, enter the auth password of your choice for this email address, let's say it is "mypwd". Only emails from this address will be relayed. Everything else will be rejected by Airtel. Note that you should choose this email address to be the login account of your mailserver on your mailserver machine. In case you want to change this address later, you can login to relay.airtelbroadband.in with "foo@blah.com" and "mypwd" and then update the email address and/or passwd.

You should see that the email has been successfully registered. This completes the setup at Airtel ISP.

Now the important part on your side.

1. cd to /etc/mail
2. edit authinfo file and add this line: AuthInfo:relay.airtelbroadband.in "U:foo@blah.com" "I:foo@blah.com" "P:mypwd" "M:DIGEST-MD5 LOGIN PLAIN"
3. edit sendmail.mc and add these lines. For more info about this, see this sendmail chapter at Mashedge:

define(`SMART_HOST', `esmtp:relay.airtelbroadband.in')dnl
FEATURE(authinfo)dnl
define(`confAUTH_OPTIONS', `A')dnl
TRUST_AUTH_MECH(`EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_REALM',`blah.com')dnl
define(`confCACERT_PATH', `/')dnl
define(`confCACERT', `ca-bundle.crt')dnl
define(`confSERVER_CERT', `sendmail.pem')dnl
define(`confSERVER_KEY', `sendmail.pem')dnl
define(`confCLIENT_CERT', `sendmail.pem')dnl

After you do these, do the following:

• make -C /etc/mail (This will regenerate all the files required for sendmail)
• /usr/sbin/service sendmail restart

This completes the setup.

You can test if sendmail is actually sending email by sending one test email as foo@blah.com from your host. Check the /var/log/maillog for any errors.

Sendmail from hosts with static IP addresses

Configuring sendmail to relay mails is straightforward. Look at this chapter on sendmail at Mashedge for suggestions by folks on the internet. The implicit assumption is that your host is allowed to send e-mail.

So, why would your host not be allowed to send mail? Due to the proliferation of spam, e-mail providers (such as Google, Yahoo etc) will refuse to relay mail from sendmail from a host unless both the DNS lookup and the reverse DNS lookup match for that host sending e-mail.

Let's say you have a domain name "blahblah.com". You get an static IP address from your ISP a.b.c.d.

You fix the DNS by adding the "A" entries using the domain management tools provided by the vendor from who you purchased the domain name. When you do "nslookup blahblah.com", you should see that "blahblah.com" resolves to "a.b.c.d". This is the easy part.

The next part is the reverse DNS. In this case, a lookup via the unix command "dig -x a.b.c.d" should resolve to "blahblah.com". Instead it will resolve to your ISP provider. Unless this part is fixed, you will NOT be able to send mails via sendmail from your host "blahblah.com". Fixing this is not in your control and you must contact your ISP to do the needful.

If you attempt to send mails without fixing the reverse DNS, your SMTP provider will reject the e-mail with an error 550 5.7.1 (IP name may be forged). Yikes! See here for more details about relaying denied.

Open Source for your PC - Slowly advancing forward.

After using Apple Mac for years, I reluctantly bought a PC (S0ny VAIO NS VGN-NS25G) with windows vista on it! I was forced to because an accounting software that I planned to use (Tally) runs only on windows.

Not wanting to fork out more money for other Microsoft programs, I explored open source out there.

Here are the recommendations:

1. OpenOffice is quite an excellent replacement for Microsoft Office.
2. Mozilla's Thunderbird with Lightning and Google Contacts add-ons is a no-brainer replacement for Microsoft Outlook.

OpenOffice is getting major traction. Even some big banks in India have begun to use OpenOffice and it is real easy to download and set it up.

It is easy to configure Thunderbird to fetch e-mail from your e-mail provider. Using Google mail with IMAP was quite trivial. On the whole it took me about 3-4 hours to download and set up my laptop PC.

However, can my grandfather or father do this? Not likely. Can a non-computer professional set up their own PC with open source software? Not likely.

While all the great pieces of open source software exist and together can easily give Microsoft a run for their money, what is lacking is a comprehensive one click solution to install and set up all the various pieces of open source software into a complete workable solution on a PC running Windows.