Printer-friendly Version

On the web server, these are the subdirectories of the NCCI Web site. Note that not all the directories contain material for display on a web browser.

• /bin - Binary executables and perl scripts for use with this site. No files in this directory are viewed on the Web.

• /cfg - Configuration files. These files control how the site operates. In particular, note how the .mailto files work: Each .mailto contains a list of email addresses; The files feedback.mailto, membership.mailto, and survey.mailto are hand edited, the remainder are generated by scripts in the /src directory. No files in this directory are viewed directly on the Web, but the .mailto files are available on the web to NCCI staff.

• /cgi - CGI (Common Gateway Interface) scripts. These are Perl scripts which are executed by client browser requests to the web server. They typically serve up dynamic web pages, but can produce other format pages such as PDF and XLS files.

• /dat - Data files - typically CSV (Comma-Separated Value) files exported from spreadsheets containing source data. For example, the Membership database is maintained as an XLS file in the /xls directory. Before the business directory can be updateed, that spreadsheet must be saved as a "CSV" file in this directory, under the name Membership.csv. No files in this directory are viewed on the Web.

• /doc - Documents which have been exported for viewing on the web site.

• /events - The location of all generated web pages for NCCI events. Note that all files in this directory are generated from scripts and files in the /src directory.

• /htm - HTML documents for viewing on the web site. Note that the "home page" for the entire site is in the top level directory (above the htm sub-directory). It is called index.htm. Note that all files in this directory, as well as the ../index.htm file are generated from scripts and files in the /src directory.

• /htms - Secure HTML documents. This is simply a directory of html files which requires a username and password via Apache Basic Authentication. The files in this directory are intended for NCCI staff use. Note that the files in this directory are generated from scripts and files in the /src directory. One key item in this file is the .htaccess file. This file directs the web server to perform BASIC authentication, using the .passwd file in the same directory as it's password database. Note that the .htaccess file must contain the fully qualified path name to the .passwd file.

• /img - GIF and JPG image files used by the site.

• /lib - Library of Perl modules used by Perl scripts and CGIs throughout this site. These are .pl (Perl Library) and .pm (Perl Module) files. No files in this directory are viewed on the Web.

• /log - Log files generated as the web site is used. All CGIs that produce mail messages (eg. for feedback, on-line application for membership, etc.) append the same information that is sent in the mail message to a log in this directory. NOTE: This directory MUST BE WRITABLE by the web server. If you are having problems with the CGI scripts that send mail, or if the logs are not being updated or created, set the permissions of this directory to (in Unix parlance) drwxrwxrwx. The log files are actually viewable from the web interface, a feature intended to be used by staff and administrators.

• /man - Manual pages. This directory contains documentation for developers of the web site. In particular, there are Unix-style man pages for the Perl modules specific to this site.

• /members - The location of all generated web pages for members. Note that all files in this directory, as well as the ../index.htm file are generated from scripts and files in the /src directory.

• /pag - A directory for storing intermdiate .pag files which are produced during the generation of this web site. No files in this directory are viewed on the Web.

• /src - This directory is where all the scripts and templates for producing the web site reside. See the section below for a description of how the scripts and templates operate. No files in this directory are viewed on the Web.

• /ss_deployment - This directory contains JPG images of a slide show depicting the potential deployment models for the NCCI web site. These images are viewed by a small CGI which performs a very simple "slide show" function.

• /sys - This directory contains symbolic links to all the components which exist outside the web site. For example, there is a symbolic link named "perl" in this directory which points to the executable for perl. This extra level of indirection to access system components means that the hard-coded location of execution on the #! line of scripts does not need to be changed if the location of the executable changes - you only need to change the link in this directory. So, the shebang line of perl scripts in this site should read
  #!../sys/perl
  No files in this directory are viewed on the Web.

• /tmp - A directory for storing temporary files. These are typically produced during the generation of this web site. No files in this directory are viewed on the Web.

• /xls - Location of Excel spreadsheet which contain data for the web site. No files in this directory are viewed on the Web.

History and Deployment

A prototype web site for the NCCI was developed in March 2001 and deployed at goss.rho.net/ncci. It was intended to be a working model of a fully featured Chamber of Commerce web site, and a starting point for future development.

Any questions about this site, and in particular items that this document does not address, can be addressed to Clint Goss at clint@goss.com. You can also visit the goss.com web site.

Web Server Requirements

In order to run this web site, these components need to be loaded/available:

• Web server. The site was tested with the Apache web server version 1.3.12. The source code as well as binary version for many configurations are available at www.apache.org

• Perl. Perl version 5.6.0 is recommended as this was the test platform. Earlier version might work, but you're on your own. The source code as well as binary version for many configurations are available at www.perl.org

• mySQL. This component is not currently required. It was planned to be used, but the features of the web site as they have been implemented so far do not call for a heavy-duty database. However, this is my recommendation - it's free and it works.

• Mail. To implement feedback, on-line membership applications, and other mail-related features, this site sends emails back to NCCI staff. The assumption is that SMTP mail and /usr/sbin/sendmail is set up. If it is not, or some other mail facility is to be used, the CGI scripts in /cgi need to be modified.

How the Scripts Work

This site makes heavy use of "generation" techniques to produce the HTML web pages.

The scripts in the /src subdirectory do all the work. They all have the name gen_XXX.pl. Typically on a Windows box with ActiveState perl installed, you will have to run each of them with the command:

perl gen_XXX.pl

They can be run in any order. They typically combine information from the subdirectories /dat an /cfg, together with the .pag page files and the .htt template files, to produce pages in the /htm, /htms, /events, and /members subdirectories.

On an MS-Windows box, you can run the script gen.bat which will run all the gen_XXX.pl scripts.

How the Rebuild the Entire Web Site from Scratch

Here's an outline of how to generate the entire web site from scratch. This assumes that the key components - Apache, Perl, and SMTP mail - have been set up as described above.

1. Establish a symbolic link from /sys/perl to the executable for Perl on your system.

2. Ensure that the executable /usr/sbin/sendmail is present. If it is installed somewhere else, you will need to change the hard-coded reference in the file /src/genlib.pl.

3. Edit the file /cfg/config.tlh and set all the relevant parameters - the date and correct version of the site.

4. Edit the files /cfg/feedback.mailto, /cfg/membership.mailto, and /cfg/survey.mailto to specify the appropriate mail recipient(s) for each function.

5. Set up the correct access files for your web server so that the web server will execute all .cgi scripts in the /cgi directory.

6. Set up the correct access files for your web server so that the web server will authenticate access to any .htm files in the /htms directory. On an Apache web server with basic authentication, you will need to set a .passwd file in this directory.

7. Update up the source .xls files, Membership.xls, BusinessOpportunities.xls, and StandingCommittes.xls with current information.

8. Bring up each of the source .xls files in turn, and export them as .csv files with a .csv extension into the /dat directory. Use extreme caution not to overwrite the source .xls file with the CSV version.

9. Create empty files in the /log directory feedback.log, membership.log, and vatSurvey.log, and make sure they are writable by all.

10. Make sure the directories /log and /tmp are writable by all.

11. Run the script /src/gen.bat. If you are not on a Windows system, then execute each of the commands in that script by hand (there are only a few).

If you have any issues that you can't unravel yourself, contact Clint Goss at clint@goss.com.

Perl Module Man Pages

On-line man pages are available for the Perl Modules used for this site:

• BiHashSync
• CryptPasswordFile
• CSV
• DateMan
• Encode
• G
• GBasic
• HttpCookie
• NavigationBar
• OraConnection
• Random
• Synch

How the Clean up the Site

This section describes how to "clean up" the site. You might need to do this to minimize space before making a copy over a slow link, or before a backup.

All files in the /pag and /tmp subdirectories can be removed at any time with no effect on the site.

The log files in /log can be removed at any time, but you will lose the history of logged feedback items, submitted membership applications, etc.

In addition, all files in the following subdirectories can be removed. Removing these files will render the web site unusable, but these files can be regenerated (on a system with Perl installed).

• /dat
• /events
• /htm
• /htms
• /members

Format for Source Spreadsheets

Most of the "source" information for this web site is containined in a set of Excel (.xls) spreadsheets. These are exported to .csv files, which are read by the perl scripts to produce the .htm web pages.

In order for the perl scripts to work correctly, specific things in the spreadsheets must be maintained.

TBD

Secure User Management

Access to the secure documents in the /htms directory is via Apache basic authentication. The user-password database is contained in the file /htms/.passwd. This file uses the standard Unix username:encrypted-password format.

Management of this user password file is currently extremely rudimentary. The script /bin/addPassword opens a password file and appends a new entry on the end. It uses the CryptPasswordFile.pm Perl module.

If you want to modify a password entry, you would need to

• Execute the command in the /bin directory:
  ./addPassword ../htms/.passwd username newpassword

• Edit the file /htms/.passwd. It now has two entries for the username. Delete the earlier one containing the old password and leave the new entry at the end.

How the Mail-based Features Work

Many of the interactive features of the web site operate via Email. For example, feedback, on-line application for membership, and registration to receive email distributions all end up by sending an email message back to NCCI with the approriate information as entered on an HTML form by the person browsing the web site.

These emails are sent to one or more email addresses as specified in the file /cfg/XXX.mailto where XXX is particular to the feature. (eg. /cfg/feedback.mailto holds the recipient email addresses for the site feedback feature).

These .mailto files contain one or more email addresses, one email address per line. Make sure that there are no extra spaces on the line with the email address. Comments are NOT allowed in these files.

If You Have Problems

This section lists some things to check if you experience problems with this site.

• The links in the /sys directory might be missing. Read the description of the /sys directory above. In particular, check that there is a symbolic link from perl to the location of the perl interpreter on your system.

• Make sure that the permissions on all CGI files in the /cgi directory are executable for all. On a Unix system, the permissions should probably be
  rwxr-xr-x
  To fix this problem, the Unix command would be
  chmod a+x *.cgi
  

• The /tmp and /log directories must exist and be writable by all. On a Unix system, to fix this problem, the Unix command would be
  chmod a+w tmp log
  

• All files in the /log directory must be writable by all. On a Unix system, to fix this problem, the Unix command would be
  chmod a+w log/*
  

• If access to the secure documents in the /htms directory is not working, check the .htaccess file in that directory. This file must refer to the fully qualified path name of the .passwd file on the current host.

To Do

This section is a roster of items which remain to be done.

• Establish a make structure. The Unix make facility should really be use to generate, clean up, and perform other routine tasks on this site. However, development was undertaken under (gulp!) Windows on a laptop, so a make facility was not avaiable.

• Improve security. The staff functions are accessed via a web page in the quasi-secure /htms directory. However, the CGIs that are executed are in the un-secured /cgi directory.

• Fix the date reporting. All times listed in the log files and mail messages sent as a result of feedback, membership application, etc are the time on the web server host. If the host is not in the same timezone as the NCCI (as is happening with the prototype server on goss.rho.net), confusion can result. Times will be listed as PST or PDT (Pacific Standard or Daylight Time). They should be converted to AST (Africa Standard Time) for ease of use by the NCCI.

• Remove /tmp files automatically. They just build up over time.

Issues

This section lists some known problems and issues with the site. In particular, these are items which do not necessary belong in the "To Do" list above.

• Some of the code, especially the library code in the /lib directory, was not written originally for this site. So, you are likely to find "vestigal" code which has no bearing on this site and which is never executed. For that reason, it is recommended that you do not start with a complete reading of the library code in the /lib directory.