Quick-and-Dirty Command line submission of form data

by Pawan Jaitly

Neal Stephenson, the voice of the l33tg33ksp33k generation wrote a fine piece about OSs and how they compared with each other. The title of it being the grandiloquent "In the beginning there was the command line". Every budding nerd should read it.

While the essay is a delightful read about OSs and their history, it also talks about how the command line is often the most powerful way of doing things. Point-and-click has its place, but when you know what you want to do, a GUI just gets in the way. Perhaps the worst manifestation of point-and-click is web forms - the google search interface, for example. For all its vaunted cleanness, google requires you to fire up a heavyweight browser, type in the google url if you have not got it bookmarked or as a toolbar, perhaps some clicking to position your text, and then you type in the text. If we could skip some of these steps, we could be faster.

Surfraw is one way to do it. From their man page:

"Surfraw provides a fast unix command line interface to a variety of popular WWW search engines and other artifacts of power. It reclaims google, altavista, dejanews, freshmeat, research index, slashdot and many others from the false-prophet, pox-infested heathen lands of html-forms, placing these wonders where they belong, deep in unix heartland, as god loving extensions to the shell."

An example of how surfraw does a google search: in your xterm, if you type:

google neal stephenson command line

your browser starts up and presents the results of a search on the text "neal stephenson command line". The default surfraw browser is the w3m text mode browser, but you can choose GUI ones too.

Surfraw has a number of "elvis". These are scripts that parse your command line input and make it work with google, slashdot etc. You can create your own too, with a modicum of bash knowledge and a look through the ones already available (the ixquick elvi was done by me that way).

But, sometimes we want to submit data repetitively to a form on the net and get the result out in a very specific, quick and dirty way, without building a fullblown elvi. Well, we can leave surfraw and its elvis aside in that case, and put together something that uses the power of lynx. This technique is perhaps best illustrated by going through a particular example:

The indian railways system, one of the wonders of mass transportation, has what is called a PNR (Passenger Name Record). This is assigned to you when you book your ticket, and is used to let you know what your ticket status is - eg if it is confirmed, wait-listed etc.

There is an easy web interface to see your status. Most people access it from from http://www.indianrail.gov.in. But you have to navigate your way through things and put up with adverts. So it would be nice to be able to script it, so that you can run if from cron etc.

1. The first step is to find the html form that takes your pnr. This one is buried in a frame inside http://www.indianrail.gov.in/, at http://www.indianrail.gov.in/pnr_stat.html.

2. Take the source of this form, save it on your localhost, find the "action" part, and change

<FORM NAME="pnr_stat" METHOD="post" ACTION="http://www.indianrail.gov.in/cgi_bin/inet_pnrstat_cgi.cgi">

to

<FORM NAME="pnr_stat" METHOD="post" ACTION="mailto: me@localhost">

This will now mail the form with any data you put in to me@localhost, instead of feeding it to http://www.indianrail.gov.in/cgi_bin/inet_pnrstat_cgi.cgi when you submit the form.

3. Then take a look at the mailed form in your mail box. Strip it to its bare essentials (the key=value pairs) eg:

lccp_pnrno1=433&lccp_pnrno2=1835205&submitpnr=Get+PNR+status

4. Make a file called, say, pnrdata with this bare essential part.

5. Then feed it into lynx -post_data like this, which will talk to the cgi script on the website with this data, and send the result back to us on the command line. We may as well grep the interesting bit too:

lynx -post_data http://www.indianrail.gov.in/cgi_bin/inet_pnrstat_cgi.cgi < ./pnrdata | grep -A32 PNR

6. In fact, stick the above line in a file called, say, pnrstatus, and make the file executable with a chmod.

#!/bin/bash
 lynx -post_data http://www.indianrail.gov.in/cgi_bin/inet_pnrstat_cgi.cgi < ./pnrdata | grep -A32 PNR

Now, ./pnrstatus will give you your pnrstatus without annoying ads, popups etc. The raw power of the command line triumphs yet again!