ADAPTING FORM-BASED WEB ENVIRONMENTAL DATA/INFORMATION FOR LOCAL NEEDS

Peter Brueggeman
Scripps Institution of Oceanography Library

Reprinted from: ERMD News, Newsletter of the Environment & Resource Management Division, Special Libraries Association 4(3):8-11, 1998.

There are many environmental Web sites that deliver environmental data ("data" meaning numerical or textual information). Sites I regularly visit in an oceanographic library deliver climatic and oceanographic data, tide predictions, etc; you have your own preferences: water resources, earthquakes, geology, etc. Many of these Web sites commonly present their environmental data via forms (pull-down menus, checkboxes, radio buttons) and/or image maps. You connect to the Web site, step through a selection process to specify various environmental parameters, and then submit a data request to the server. A subset of the server's data is then presented to you by the server on a Web page.

Subsets of environmental data may be something you regularly access and you would appreciate decreased effort in accessing it frequently. You may wish to present this subset of environmental data to your users collocated with similar resources on one Web page of your design rather than forcing users to link to several Web sites and step through each site's selection process. Tips are presented to dissemble environmental data Web sites to more directly access local environmental information. Sometimes the multistep path followed to the desired data corresponds to a URL that can be used for subsequent direct access to that data. Sometimes you can modify some of the site's HTML coding so that the HTML code can live on your Web page on your Web server and successfully query that other server for the data subset needed. This doesn't require a detailed expertise with Web form coding or CGI scripting. The forms at some Web sites can be dissembled relatively easily. Others will prove difficult due to time constraints or your expertise. Tinker with it and see. However there is a maintenance cost. You will have to check your access to those servers occasionally to ensure that your local HTML code or links continue to work. A change at the server site can cripple your bypass to the data you wish to access.

A simple example of bypassing image map steps in order to directly access local environmental data is illustrated with the US National Weather Service Tallahassee's "Interactive Marine Observations" site at http://www.nws.fsu.edu/buoy/ . The user steps through two levels of image maps (world and region) in order to view local weather data. When the local data is finally displayed, the URL associated with that data is reusable for direct access to that local data. Here's an example of that site's San Diego County URLs pulled out for incorporation into a local Web page:




<B>Interactive Marine Observations</B>: Current Weather at
    <A HREF="http://www.nws.fsu.edu/B/uscg?station=l13">Point Loma</A>,
    <A HREF="http://www.nws.fsu.edu/B/uscg?station=l58">Mission Beach</A>,
    <A HREF="http://www.nws.fsu.edu/B/uscg?station=2l7">Solana Beach</A>, &
    <A HREF="http://www.nws.fsu.edu/B/uscg?station=l34">Oceanside Harbor</A>.
    [<A HREF="http://www.nws.fsu.edu/">NWS Tallahassee</A>]

The US Naval Observatory Astronomical Applications Department's "Sun and Moon Data for One Day" at http://aa.usno.navy.mil/AA/data/docs/RS_OneDay.html presents sunrise/sunset and moonrise/moonset/moonphase data for single days for US cities and worldwide. The user selects date, state and city to see local data for one day. This Web page's HTML form coding can be adapted to preselect the state and city so that a local Web page presents the user with simply the date selection for a prespecified city. The target will be creating a Web page for "Sun and Moon Data for One Day in San Diego" so that the user doesn't have to specify state and city.

Step down to the form to be appropriated for your local info page and save it to your hard disk (with your browser) for editing and adapting. As you edit the form, you can reload it again and again into your browser to test it and ensure it continues to work in retrieving the desired environmental data. Test continually at each major editing step in order to ensure that you are not heading down a blind alley with non-functional HTML coding. Make every URL <A HREF=""> complete in the form being edited. Include the data server's Internet address (the base URL seen as http://machine.domain) in all incomplete URLs (known as relative URLs). Relative URLs do not have an Internet address and consist of a directory name on the server's hard disk and also possibly a file name within that directory. Relative URLs will not work on your locally-served Web page unless the data server's base http address is grafted onto the front of each relative URL. The base server's address may be in the <HEAD> section of the HTML file where it is referenced as <BASE HREF="http://riemann.usno.navy.mil/AA/"> in this example. You may also be able to determine the base server's http address from your browser's URL address line.

Edit out all unwanted onscreen text including this site's worldwide data entry FORM. The original "Sun and Moon Data for One Day" page has relative URLs for sun and moon term definitions i.e. /faq/docs/faq12.html#top which would be appropriate to incorporate as links into a local Web page. The base server's address (http://riemann.usno.navy.mil/AA/) has to be grafted onto the front of the relative URLs that will be used in the local Web page to link to definitions i.e. http://riemann.usno.navy.mil/AA//faq/docs/faq12.html#top . Be sure to include a credit link to the original Web page of the data server. Users need to know the source of the data being delivered from your Web page and you need a quick path for a return visit in order to revise your form when the data server changes.

A FORM ACTION URL is the URL of the data server on which the environmental data FORM will act in making its request. Frequently, the URL invoked by the ACTION of the data FORM is a relative URL; this will work only on the data server. If the HTML coding used on your local Web page has an incomplete or relative FORM ACTION URL, then your local FORM won't know the data server's address in order to make a successful data request (the FORM ACTION). FORM ACTION URLs have to be complete on your local Web page in order to successfully access the data on the data server. For example, an incomplete or relative FORM ACTION URL is <FORM ACTION="/cgi-bin/tcweb.exe" METHOD="POST">. A complete FORM ACTION URL is <FORM ACTION="http://www.tides/com/cgi-bin/tcweb.exe" METHOD="POST">.

Since only one state is needed (California) for this "Sun and Moon Data for One Day in San Diego", there is no need for the user to pull down a menu for all the states and select California. Change the original user-specified state selection process to an automatic hidden input process that will specify California behind-the-scene to the data server. Do this by editing the state selection portion of the FORM which displays the pull-down state menu. Change SELECT to INPUT; this will automatically input a prespecified state rather than present a state selection opportunity to the user. Add a "hidden" INPUT TYPE= value so that state selection will be accomplished behind the scenes. Put the OPTIONal VALUE for California "CA" into the INPUT VALUE= specification. Be sure to remove onscreen text associated with the state INPUT since the user doesn't need to see it any longer.

Before: <SELECT NAME="st"><OPTION SELECTED VALUE="CA">California</SELECT>

After: <INPUT TYPE="hidden" NAME="st" VALUE="CA">

See Figure One and its HTML source code at ermd1.html for the editing progress so far.

Now the Web page functions to find sun and moon data for the data server's California locations. San Diego is the target location for this "Sun and Moon Data for One Day in San Diego" example and still has to be input manually into a text-input box at this stage. Again, San Diego needs to be changed to a prespecified input hidden from the user. Change the INPUT TYPE from "text" to "hidden" and edit out the size and length specifications of the text-input box. Add the "San Diego" VALUE for the hidden INPUT.

BEFORE: <INPUT TYPE="text" NAME="place" SIZE="32" MAXLENGTH="32">

AFTER: <INPUT TYPE="hidden" NAME="place" VALUE="San Diego">

See Figure Two and its HTML source code at ermd2.html for the editing progress so far.

The state and city input has now been removed from the user; the user need only input the date. You now have a kernel of HTML coding for use in a local Web page collocated with other links to environmental data.

The next editing example will use Nautical Software's tide and tidal current prediction site at http://www.tides.com/ . The specific Web page for querying their prediction server is http://www.tides.com/tcpred.htm (which has to be determined by close attention since the home page starts up FRAMES). This site presents the user with a pulldown menu for state/region selection, radio buttons for tide/current prediction, today/tomorrow, and unit selection, and a textual input box for specifying a specific location within a state/region. If the state/region is selected in the pulldown menu and then the site list is queried, the user is presented with a pulldown submenu for the sites in that state/region and with radio buttons for today/tomorrow in order to query the server . Incorporating selections from the state/region site list into a local Web page while allowing for the user to specify today/tommorrow and tide/current prediction is the target example. This cannot be done on the server which first presents a tide/current prediction selection and then allows the user to select today/tomorrow for a local site list.

Edit the original http://www.tides.com/tcpred.htm Web page to replace relative URLs with absolute URLs. Edit out unwanted text. Leave in a reference to the company/data server as the data credit.

See Figure Three and its HTML source code at ermd3.html for the editing progress so far.

For local use, you would want to prespecify the state/region and present the user with the local site pulldown menu for the state/region of interest. However you want to incorporate that within the Web page you previously edited in order to present your local user with the ability to select tide or current predictions as well as units. Make your state/region selection and click on the "get list" button to see the local site pulldown menu page. View its HTML source and save it as a file on your hard disk for editing. Copy and paste its FORM section at the bottom of your edited local Web page so that you can graft the two forms into one form. Edit down the local site pulldown menu to just the sites you want to present in a pulldown menu. Both forms have the same "Select day: <INPUT TYPE="radio" HTML form coding so delete the duplicate that was just pasted. Now you have the starting point to combine the two forms into one.

See Figure Four and its HTML source code at ermd4.html for the editing progress so far.

Both forms have an identical FORM ACTION= statement for the data server so edit out the reference to it in the second newly-pasted form. The first form had the tide and current radio buttons present whereas the newly-pasted second form is prespecified for tide or current depending on which you had selected. The second form has <INPUT TYPE="hidden" for this prespecification and you want to edit it out thereby retaining the first form's tide or current specification opportunity. The first form has a <SELECT NAME="Region"><OPTION> section that presents the pulldown menu for states/regions. The second form has an <INPUT TYPE="hidden" action to preselect your state/region; you want to substitute this preselected <INPUT TYPE="hidden" action for the <SELECT NAME="Region"><OPTION> section. Finally you want to substitute the <SELECT NAME="Location"><OPTION> pulldown menu of local sites in the second form for the corresponding INPUT box and GET LIST button in the first form. Though you will be tempted, do not revise the names of the local sites since those exact names are usually being passed to the server for action. If you change the local site names, then the server won't recognize them. Another catch can be if there are different local site lists depending on the selection of other parameters. In this example, tide or current prediction was selected first and then a local site list was presented. There are differences between the local site lists which will be addressed later. For the purposes of example, ignore it for now.

See Figure Five and its HTML source code at ermd5.html for the editing progress so far.

This form is now ready for incorporation into your local Web page. You may wish to bypass this form selection process for frequently requested data. A relevant example for the author would be direct access via two URLs to the tide predictions for today and tomorrow in feet for Scripps Institution of Oceanography Pier. Everything is prespecified so there is no need to present a pulldown menu of San Diego sites plus radio buttons for today/tomorrow, tide/current, and units. However if one uses the pulldown menus to specify "La Jolla (Scripps Institution Wharf)", a CGI program is run on the server to deliver the Web page associated with that data; the result is that you do not see a direct URL to that data. To get the actual URL for that local data, change the FORM METHOD from POST to GET. There are two types of FORM submission that differ in the way that they are processed. The POST method passes data using environment variables, is commonly used, but it hides the URL that takes you to the data requested. The GET method passes data on the command line and displays the the URL used to get to the data requested. In this example, <FORM ACTION="http://www.tides.com/cgi-bin/tcweb.exe" METHOD="POST"> is changed to <FORM ACTION="http://www.tides.com/cgi-bin/tcweb.exe" METHOD="GET">. Reload your edited Web page, check off the data you want prespecified, and copy-and-paste the direct URLs retrieved for incorporation into your local Web page. The direct URLs will look familiar since they incorporate the parameters queried in the forms you have been editing. For example, the direct URL for today's tide in feet for La Jolla (where Scripps Institution of Oceanography's pier is located) is

http://www.tides.com/cgi-bin/tcweb.exe?Day=Today&TideCurrent=t&Unit=f&Regio n=California+%28NOAA%29&Location=La+Jolla+%28Scripps+Institution+Wharf%29&D oit=1&a_submit=+SEARCH+

Prespecified parameters for day, tide or current, unit, region, and location are present in the URL. That URL can be used within your local Web page to take the user directly to the pier tide prediction and bypass the selection process.

See Figure Six and its HTML source code at ermd6.html for the final result.

At the bottom, it incorporates two direct URLs for Scripps Pier tide predictions (obtained via METHOD=GET). It also brings together into a Web page all the other environmental data queries discussed above and collectively represents the beginning of a local Web page linking to environmental data. The dilemma with different site lists for tide and current predictions are dealt with by separate forms. The METHOD="GET" is used for tide predictions and METHOD="POST" is used for tide currents in order to contrast the difference in URLs retrieved by the two methods. The tips presented will get you pretty far in adapting many environmental data servers to your local needs. There is no risk in tinkering since you are prototyping your development from a Web page on your hard disk. If you don't like it, don't put it on your Web server. If you do put it on your Web server, test it out occasionally to ensure it continues to work.