Project #4: The Common Gateway Interface

• College front door

• Department front door

Course links

• Front door

• General information

• Schedule of topics

• Readings

• Labs

• Programming assignments

• Revised5 report on the algorithmic language Scheme

• PLT DrScheme manual

As some of you may know, the documents for this course that are accessible on the World Wide Web are simply text files that include some additional text that is not usually displayed by the browser. This additional text is called ``markup,'' and it gives the browser some information about the structure of the document -- where to put paragraph breaks and horizontal rules, which parts of the text should be treated as samples of Scheme source code, and so on.

Writing up the entire document in advance and storing it in a file is fine for many applications, but it means that the document is inert. Whenever the document is accessed, exactly the same text (and the accompanying markup) will be transferred from the server to the reader's computer and displayed. Sometimes, however, one would like to modify the text of the document on the basis of information that one finds out about when it is accessed. When a browser accesses a World Wide Web document, it is required to supply quite a bit of information to the server (the computer on which the document is stored). To take advantage of this information, one writes a program that the server can run when it is asked for the document. The program should construct and write out the text of the document, including any markup that it needs; the server collects this output and forwards it to the browser, which displays it. The advantage is that the program can examine information supplied by the browser and write out different text depending on what it learns.

The Common Gateway Interface (CGI) is a set of conventions, supported by software, that facilitate the writing of programs that generate World Wide Web documents. CGI programs can be written in almost any programming language; naturally, though, we'll use Scheme.

Here is what an inert document intended for the World Wide Web looks like when it is stored in a file. Each little piece of markup is enclosed between less-than and greater-than signs; this is a convention of the Hypertext Markup Language (HTML) that is used. (Don't be distracted by the actual text in this example. If you must know, it's a quotation from Kant's Critique of pure reason. Feel free to substitute any preferred text of your own.) 

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html>
<head>
<title>Sample HTML page</title>
</head>

<body>
<p>
If by merely intelligible objects we mean those
things which are thought through pure categories,
without any schema of sensibility, such objects
are impossible.  For the condition of the
objective employment of all our concepts of
understanding is merely the mode of our sensible
intuition, by which objects are given us; if we
abstract from these objects, the concepts have no
relation to any object.  Even if we were willing
to assume a kind of intuition other than this our
sensible kind, the functions of our thought would
still be without meaning in respect to it.
<p>
</body>
</html>
Follow this link to see how this page looks in your browser.

Here's the explanation for the various pieces of markup: The first two lines are declarations that identify the markup standards and conventions and the character encoding that will be used in the rest of the document. The <html> in the third line and the </html> at the end form a pair of tags; they mark the beginning and end of the part of the text that is directed to the browser and describes the document's layout and content. Another pair of tags, <body> and </body>, enclose the text that is actually to be displayed in the interior of the browser window; <head> and </head> enclose ``header'' information that the browser may or may not choose to display. The paired tags <title> and </title> within the header enclose the title of the document; our Firefox browser, for instance, chooses to display this title at the top of the frame of the browser window. Finally, the paired tags <p> and </p> mark the beginning and the end of a paragraph.

One other pair of tags, not used in the preceding document but frequently helpful, is <pre> and </pre>. Any text placed between these tags is considered ``preformatted''; the browser is not supposed to change the spacing of the lines or the positions of the breaks between lines. This is handy for printing the source code of a computer program or other text that is carefully laid out before the browser gets it.

Suppose, now, that we wanted a Scheme program to generate the ``Sample HTML page'' document, so that the server would run the program instead of recovering the document from a file. All we would need to do is ask Scheme to print out the appropriate strings and halt. The heart of the program looks like this: 

(display "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>")
(newline)
(display "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\"")
(display " \"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">")
(newline)
(display "<html>")
(newline)
(display "<head>")
(newline)
(display "<title>Sample HTML page</title>")
(newline)
(display "</head>")
(newline)
(newline)
(display "<body>")
(newline)
(display "<p>")
(newline)
(display "If by merely intelligible objects we mean those")
(newline)
(display "things which are thought through pure categories,")
(newline)
(display "without any schema of sensibility, such objects")
(newline)
(display "are impossible.  For the condition of the")
(newline)
(display "objective employment of all our concepts of")
(newline)
(display "understanding is merely the mode of our sensible")
(newline)
(display "intuition, by which objects are given us; if we")
(newline)
(display "abstract from these objects, the concepts have no")
(newline)
(display "relation to any object.  Even if we were willing")
(newline)
(display "to assume a kind of intuition other than this our")
(newline)
(display "sensible kind, the functions of our thought would")
(newline)
(display "still be without meaning in respect to it.")
(newline)
(display "</p>")
(newline)
(display "</body>")
(newline)
(display "</html>")
(newline)

In other words, we basically just tell Scheme what we want the program to print out, making each line, as a string literal, an argument to display, and dropping in the appropriate calls to newline. The only difficult part is remembering that double-quotation marks that occur inside string literals have to be ``escaped,'' that is, protected by backslashes that appear in the string literals but not in the actual string values.

To make this Scheme program executable by the document server, so that the document that it outputs will be available on the World Wide Web, we need to take the following steps:

  1. Before anything in your MathLAN account can be accessed on the Web, you must make your home directory accessible. To do this, open a terminal window (left-click on the icon of a computer monitor on your front panel) and give the command 

    chmod 755 ~

    at the prompt. (The symbol ~ stands for your home directory.)

  2. Any materials related to the World Wide Web belong in a subdirectory of your home directory named public_html. If you have no such subdirectory, create one by giving the command 

    mkdir ~/public_html

    in the terminal window. This directory, too, must be accessible; give the command 

    chmod 755 ~/public_html

    to make it so.

  3. CGI programs, in particular, belong in a subdirectory of ~/public_html named cgi-bin. If you have no such subdirectory, create it with the command 

    mkdir ~/public_html/cgi-bin

    and make it accessible with the command 

    chmod 755 ~/public_html/cgi-bin

  4. Start up DrScheme and copy the CGI program into the Definitions subwindow,

  5. Add the following lines at the top of the file: 

    #!/bin/sh
":"; exec /usr/bin/mzscheme -r $0 "$@"

    Don't leave any blank space before the # character.

    The effect of these lines is to tell the server what software must be started in order to process the program. In this case, we'll be using MzScheme, which is DrScheme without the graphical user interface and programming tools. (We'll continue to use DrScheme to develop the program, but the server will use MzScheme to run it.)

  6. Add the following lines immediately after the ones added above: 

    (display "Content-type: text/html")
(newline)
(newline)

    The effect of these lines is to advise the browser (somewhat redundantly) that text containing HTML markup is coming.

  7. Save the program in your cgi-bin directory. A good name for the file would be ~/public_html/cgi-bin/sample.cgi. Notice that, when we expect to use the Common Gateway Interface, the file name ends in .cgi rather than the usual .ss.

  8. In the terminal window again, make your program file accessible: 

    chmod 755 ~/public_html/cgi-bin/sample.cgi

  9. Congratulations -- you've written your first CGI program in Scheme! To see it in action, ask the browser to load the URL 

    http://www.cs.grinnell.edu/~spelvin/cgi-bin/sample.cgi

    (except of course with your username in place of spelvin).

Now for something more interesting. The Common Gateway Interface ensures that the information that the browser gives to the server when it requests a document can be recovered by a CGI program. On our system, using either MzScheme or DrScheme, there are actually two ways for the browser to exchange information with your program. One is through environment variables that are maintained by the server; the non-standard procedure getenv can be used to access these. It takes one argument, which must be a string that names an environment variable, and returns a string giving the value of that environment variable: 

> (getenv "USERNAME")
"stone"

Two environment variables that are particularly useful in writing CGI programs are "REMOTE_ADDR", which is the Internet Protocol number of the computer from which the browser request originated, and "HTTP_USER_AGENT", which gives a description of the browser that the reader is using.

The other mechanism by which a browser exchanges information with a program is the query string. This is a sequence of one or more equations, separated by ampersands, that is attached (with a question mark) at the end of the URL that brings up a page. Here is a typical query string: 

user=spelvin&commentary=phooey

In this example, the query string consists of two equations, user=spelvin and commentary=phooey. In each equation, the part preceding the equal sign is an attribute, and the part after the equal sign is a value.

The non-standard DrScheme procedure get-bindings takes no arguments and returns an association list in which the cars are the attributes supplied in the URL and the cdrs are the corresponding values. There is, in addition, a non-standard procedure extract-binding/single that looks up an attribute in the association list and returns the associated value.

In order to use the get-bindings and extract-binding/single procedures, your program must import them from a PLT Scheme library. To do this, add the line 

(require (lib "cgi.ss" "net"))

near the beginning of your program. The library also contains a few other procedures that you might find helpful. You can read about them by looking up ``CGI'' in the DrScheme Help Desk.

Here's a CGI program that uses both mechanisms. 

#!/bin/sh
":"; exec /usr/bin/mzscheme -r $0 "$@"

;;; A demonstration of the use of the Common Gateway Interface with Scheme.

;;; John David Stone
;;; Department of Mathematics and Computer Science
;;; Grinnell College
;;; stone@cs.grinnell.edu

;;; created April 17, 1997
;;; last revised April 13, 2005

;;; Import CGI-related procedures from the PLT Scheme library.

(require (lib "cgi.ss" "net"))

;;; The DISPLAY-LINE procedure takes any number of arguments and prints
;;; them all out, right next to one another, in order, using the
;;; human-readable format for string and character values.  It then
;;; terminates the output line.

(define display-line
  (lambda args
    (for-each display args)
    (newline)))

;;; Print the Hypertext Transfer Protocol header.

(display-line "Content-type: text/html")
(display-line)

;;; Print the XML and HTML document type declarations.

(display-line "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>")
(display-line "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\""
              " \"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">")

;;; Print the opening HTML tag and the HEAD element, giving the title for
;;; the document.

(display-line "<html>")
(display-line "<head>")
(display-line "<title>I know where you are!</title>")
(display-line "</head>")
(display-line)

;;; Print the BODY element.

(display-line "<body>")
(display-line "<p>")
(display-line "You're at " (getenv "REMOTE_ADDR") ", aren't you?")
(display-line "</p>")
(display-line)

(display-line "<p>")
(display-line "And you're running " (getenv "HTTP_USER_AGENT")
         " as your browser!")
(display-line "</p>")
(display-line)

(let* ((bindings (get-bindings))
       (user (extract-binding/single "user" bindings))
       (commentary (extract-binding/single "commentary" bindings)))
  (if (and commentary
           (not (string=? commentary "")))
    (begin
      (display-line "<p>")
      (display-line "And I say ``" commentary "'' to you, too, " user "!")
      (display-line "</p>"))))

(display-line "</body>")

;;; Close the HTML element.

(display-line "</html>")
Follow this link to see the document that this program generates when given the query string shown above.

Now we're ready to look at the project. I've written a Scheme program that culminates in the definition of a procedure called display-calendar. This procedure takes two arguments, a month number and a year, and displays a calendar for that month: 

> (display-calendar 4 2005)
April 2005

 Su  M Tu  W Th  F Sa
                 1  2
  3  4  5  6  7  8  9
 10 11 12 13 14 15 16
 17 18 19 20 21 22 23
 24 25 26 27 28 29 30

The project is to use this program as the basis for a CGI program that generates an HTML document containing the calendar for a specified month. The idea is that when a browser submits a URL such as 

http://www.cs.grinnell.edu/~yournamehere/cgi-bin/display-calendar.cgi?year=1951&month=7

the server should invoke your CGI program, which should generate the HTML for a page that looks like this one.

Your part of the job is to design, write, and test the additional Scheme code that recovers the year and month from the query string in the URL, prints out the necessary markup at the beginning and end of the document, and invokes display-calendar in exactly the right place and with the appropriate arguments. When you have saved the resulting program in a file with a name that ends in .cgi, added the magic server-instruction lines at the top, and shared the file with the appropriate chmod command, the resulting program will be ready to respond to requests from anyone on the Internet who needs to know what day of the week February 23, 2007 falls on.

Added exercise

For those of you who would like to experiment a little more with CGI, here's a second, similar exercise. The following Scheme program defines a procedure, called anagrams, which takes a string as its argument and returns a list of all the possible ways of rearranging the characters in that string. 

;;; Given a value NEW and a list BASE, the
;;; INSERT-EVERYWHERE procedure constructs and returns
;;; a list of lists, each of them formed from BASE by
;;; inserting NEW at a different position.

(define insert-everywhere
  (lambda (new base)
    (let kernel ((rest base))
      (if (null? rest)
          (list (list new))
          (cons (cons new rest)
                (map (lambda (result)
                       (cons (car rest) result))
                     (kernel (cdr rest))))))))

;;; The PERMUTATIONS procedure takes a list LS and
;;; returns a list of lists, each of which results from
;;; arranging the elements of LS in a different way.
;;; (If LS contains duplicate elements, the list that
;;; PERMUTATIONS returns will include permutations that
;;; differ only in the arrangement of the duplicated
;;; values and so appear identical.

(define permutations
  (lambda (ls)
    (if (null? ls)
        '(())
        (apply append (map (lambda (perm)
                             (insert-everywhere (car ls) perm))
                           (permutations (cdr ls)))))))

;;; The ANAGRAMS procedure takes a string STR and
;;; returns a list of strings, each of which results
;;; from arranging the characters in STR in a different
;;; way.

(define anagrams
  (lambda (str)
    (map list->string (permutations (string->list str)))))

Since the number of rearrangements rises very quickly as the length of the string increases, I recommend making it a precondition of this procedure that the length of str be seven characters or less.

The exercise is, again, to convert this program into a CGI program that displays on a Web page all of the anagrams of a sufficiently short string that the reader supplies through a query string in the URL.

This document is available on the World Wide Web as 

http://www.cs.grinnell.edu/~stone/courses/scheme/labs/cgi.xhtml

created April 16, 1997 ... last revised April 18, 2005

John David Stone (stone@cs.grinnell.edu)