Instructions for maintaining the Detonation Database. Mike Kaneshige Last modified: 2/11/97 Contents: 1. How to update or add to the Detonation Database A. Highest level of automation B. Manual generation of LaTeX output C. Manual generation of HTML output 2. How to create a plot 3. How to add a data file to the database 4. How to add a plot to the database 5. How to add a reference to the database 6. Formatting rules for datalist 7. Formatting rules for data files. 8. Formatting rules for plotlist 9. Formatting rules for references 10. How to update the hyperlatex source 11. Rules 12. Notes <--- See here for setup requirements ***** 1. How to update or add to the Detonation Database: Updating the online and hardcopy versions of the database require an enormous number of individual tasks. These tasks have been very extensively automated, however, with the result that only a handful of actions need to be taken. In outline form, these are: i. Update the datafiles. The scanning and digitizing procedure has yet to be proceduralized. ii. Update the PC Excel database files (plotlist, datalist, and referenc). The format of these files is crucial to the success of later steps, and the quality of these files and the digitized data are prerequisites to a classy final product. iii. Upload the PC Excel files to the Unix network as tab delimited text files. iv. Remake the database by running the makefile. A. Highest level of automation. Once the datafiles in /home/vipera/detn_db/data/plotdata and the database files (plotlist, datalist, and references) in /home/vipera/detn_db/db are updated, the only thing left to do is run the makefile in /home/vipera/detn_db. To perform a full update, simply type 'make' in this directory. Due to current limitations of the TeX installation on kelvin, it is necessary to do the rebuild on blast. Some knowledge of make functioning is helpful to understand all of the possible uses of the makefile. Typing 'make' will cause it to update all parts of the database requiring it. Typing 'make latex' will update only the hardcopy version, and 'make html' will update only the HTML version. The latter does not need to be performed on blast. The makefile describes a set of dependencies between files (some are artificial, created for the sole purpose of keeping track of certain dependencies) and specifies what actions to take when a lower level file has been modified since the last build. 'make' has some command line options that can be useful. 'make -n' will list all the actions it finds necessary, without actually executing them. Since running make usually generates a lot of output and a useful error message may be buried in text that scrolls off the screen too quickly to be useful, try 'make | tee make.log' to write the output to the make.log file while displaying it on the screen. B. Manual generation of LaTeX output. Here is a full list of actions to take to bring the hardcopy output up to date, without using the makefile, assuming the datafiles in /home/vipera/detn_db/data/plotdata and the database files (plotlist, datalist, and references) in /home/vipera/detn_db/db are up to date. cd /home/vipera/detn_db/data/plotdata; ../../scripts/data2tex -d=../texdata *.txt (Aug 27, '99 changed by JA and JES ! Must use the -d arg or you clobber ALL data files!!) (Updates the LaTeX formatted data tables from the raw data files) cd /home/vipera/detn_db/scripts; mkplots; procplot ../plots/gpsrc/*.gs (Updates the gnuplot scripts used to generate the plots, and runs those requiring it. These scripts do some intelligent checking to determine which plot scripts and plots require updating, and this could be a source of trouble. The scripts would require a little editing to remove the checking. The checking saves large amounts of time during most rebuilds.) (Changed by JA and JES on 27 Aug - see Makefile for correct subdirectory arguments) cd /home/vipera/detn_db/db; xl2ddb (Updates the *.tex files and the references.bib file.) cd /home/vipera/detn_db/html; buildtex (Runs latex2e and bibtex. The script (which is a shell script) also manipulates the contents of the longtables.tex file by emptying it and writing "\T\setlongtables" in it to switch the \setlongtables off and on between latex2e runs. To perform its operations manually, see its contents. You need to be on blast for this to work, at present.) cd /home/vipera/detn_db/html/dvips -Pcmrps db -o (Creates the db.ps output from the db.dvi output.) C. Manual generation of HTML output. Here are the instructions for generating the HTML version of the database, using the same assumptions as in B, above. See above for some explanation. cd /home/vipera/detn_db/data/plotdata; ../../scripts/data2tex -d=../texdata *.txt cd /home/vipera/detn_db/scripts; mkplots; procplot ../plots/gpsrc/*.gs cd /home/vipera/detn_db/db; xl2ddb cd /home/vipera/detn_db/html; hyperlatex db (Note that if any changes have been made to the references list, the LaTeX output will need to be generated first, to reconstruct the db.bbl bibliography file.) 2. How to create a plot: The current plots are generated with gnuplot, because it's fairly easy to automate. Choose your data files in data/plotdata/, make sure they're correct (see 'Formatting rules for data files' below), and note which columns you want to plot in each. Then, just add a line to the plotlist file (see 'How to add a plot to the database' below) and make sure all fields are correct. To integrate the new plot into the compiled database, follow the procedure above to update the database. To see what the new plot looks like, only, upload the plotlist (and datalist and references if they have been modified) and run mkplots and procplot as described above (under Manual Generation of LaTeX Output). This will create postscript and gif versions in /home/vipera/detn_db/plots/gplots. 3. How to add a data file to the database: If there is an entry in the datalist file (datalist.xls) that matches your data file, use it; if not, create a new line. Fill in all columns carefully and accurately. Follow the formatting requirements in 'Formatting rules for datalist' below, strictly. Move on to 'How to update the hyperlatex source' below. 4. How to add a plot to the database: Create a new line or copy an existing line (in plotlist.xls). Fill in the columns in the plotlist file as specified in 'Formatting rules for plotlist' below. See 'How to create a plot' above. Make sure each data file is accurate and is accurately referenced in the datalist file. See 'How to add a data file to the database' above. Follow the formatting requirements in 'Formatting rules for plotlist' below, strictly. Move on to 'How to update the hyperlatex source' below. 5. How to add a reference to the database: Follow the example of the other references (in referenc.xls), following the formatting requirements in 'Formatting rules for references' below, strictly, when adding a reference to the references file. 6. Formatting rules for datalist: 1. Make sure the data "File" identifier (name) is unique. It should include the file creator's initials. Avoid funny characters. If you can follow a pattern, do so. The only real requirement is that each identifier be unique. Once you have chosen an identifier, you can use Excel to search for existing occurrences. 2. Put a 'Y' in the "Dig." column if the file containing the digitized data has been created and placed in the /home/vipera/detn_db/data/plotdata/ directory. See 'Formatting rules for data files' below. 3. Put a 'Y' in the "Acc." column if the data file is verified to be accurate. Watch out for multiple-column files, which can trick you when you plot them. 4. Put the unique identifier of the source in the "Reference Identifier" column. 5. The "Reference" and "Date" columns are only for human convenience, so use them as seems appropriate. They may be removed in the future. 6. Put the figure or table number from the source in the "Fig." column. This is for convenience during checking, and may be used in the document later. 7. Choose a category and sub-category(ies) from the document and fill in those columns. Use the same spelling and lower-case as the other entries. 8. Put the fuel, oxidizer, and diluent in those columns. Use chemical formulas when possible, in upper-case. 9. Use kPa and K for the "P0" and "T0" columns. Ranges are ok. 10. ER is self-explanatory. 11. Use percentages, not mole fractions, in the "% Diluent" column. 12. Keep comments concise and relevant. For now, the comments are for internal use to explain oddities in the data sets, but they may someday be printed in the document to add useful information that won't fit in the other columns. 7. Formatting rules for data files: The data must be formatted with comma-space delimiters. It can have a header; each line must be commented with a #. The header should be set up the same as the data columns - same number of columns, separated by comma-space delimiters. The header should specify what is in each column, with units if appropriate. Follow the example of the other data files. The data files appear raw in the WWW document, are processed to make Latex tables, and are used by gnuplot to create summary graphs. 8. Formatting rules for plotlist: 1. The "File Name" column specifies the name of the plot file that will be generated for each row. Don't change these names too much, since the program will leave the old plots in the plots directory and create new ones, cluttering the directory until it gets manually cleaned. 2. The "Title" column should contain a useful title for the plot. The actual title will include the mixture, also. 3. Put the category (as specified in the document) of the plot in the "Category" column. All lower case. Needs to be exact. 4. The "Mixture" column takes a description of the mixture. For now, it's not a strict thing. 5. 'X Label' and 'Y Label' specify labels to use for the x and y axes. 6. The contents of the "Log" column specify which axes, if any, are to be plotted log-scale. If x or y appear in the column, that axis will be log-scale. 7. 'X Range' and 'Y Range' optionally specify ranges for the x-axis and y-axis. If they are left blank, the range will be set automatically to fit the data. 8. The 'Details' column can contain a set of codes (upper case but otherwise unformatted) to determine what information is printed under the plot on the description line for each data set. The possible codes are currently P-pressure, T-temperature, E-equivalence ratio, D-diluent and percentage, F-fuel, O-oxidizer, S-subcategory. 7. In the rest of the columns (F_#), enter the names of the data files used in the plot (without the .txt extension), exactly. 8. Try to keep the number of data sets per plot below 7, since gnuplot can only generate 6 different symbol types. 9. Formatting rules for references: The processing script basically just takes the information in each column and assigns it to a BibTeX field. Most of the columns always go to the same field, e.g. title="title column", year="year column". One or two of the columns may be assigned to different fields depending on what kind of BibTeX entry it is. 1. The "ID" column should contain the unique identifier for the reference on that line. Make sure the identifier is really unique. Strictly, it doesn't matter what the identifier is (no spaces; avoid funny characters), but the standard format is first author first initial - underscore - first author last name (first four letters) - year. E.g. the unique identifier for Abid S., Dupre G., Paillard C. (1991) is s_abid91. 2. The year column should contain the year of publication. 3. The author column should contain all the authors if possible. This column gets processed by the master perl script into a bibtex file for processing into a LaTeX input file, so its format is important. For each author, list the last name followed by the first and middle initials. Use a space between the last name and the initials, with periods after each initial but no space between the initials. Separate each pair of authors with a comma and a space. This format gets converted into the format needed by bibtex. 4. The title column should contain the title of the article or section, as it should appear in the document bibliography. Capitals other than at the beginning of the title (and certain other places) need to be protected by placement within curly brackets ({}). Subscripts (as in chemical formulas) can be made in Latex format (e.g. H$_2$), but this will only convert to H_2 in the HTML version. This needs work. 5. 'Published in' is the journal or collection or proceedings. Use abbreviations as specified within the document, if possible. 6-8. Vol, No, and pp. refer to the 'volume', 'number', and 'pages' entries of LaTeX bibliographies. They should be self-explanatory, except, page ranges should be separated by a single '-', which gets converted to '--' for the bibtex file. For "incollection" types, which require a Publisher but have no Number, the Number column is used for the Publisher. 9. The 'Type' refers to the Latex citation type, which so far can be 'article', 'inproceedings', 'phdthesis', 'incollection', or 'techreport'. More types can be added but require additions to the perl scripts (xl2ddb and refparse). 10. How to update the hyperlatex source: Save the database files (plotlist.xls, datalist.xls, and referenc.xls) as "Text DOS or OS/2", within Excel 5.0 on the PC, or at least the files you have changed. Upload (ftp) the files plotlist.txt, datalist.txt, and referenc.txt to /home/vipera/detn_db/db/ on viper as plotlist, datalist, and references. If you don't use these filenames, you will need to change the names or specify the different names to the xl2ddb script (not an option when running automatically with make, see the xl2ddb script). The processing program 'xl2ddb' expects these names. If you are running xl2ddb directly (not through make), run the 'xl2ddb' perl script simply by typing the name. It will give some feedback as it processes the references file into a bibtex input file (references.bib), creates the latex files for the data tables, and creates the latex files for the plots. 11. Rules: A. Only use originally published data, except where the original source is utterly unavailable, e.g. where the reference is a 'private communication'. 12. Notes: A. tetryl conversion: 1 g tetryl = 4520 J B. Prerequisities for updating the database: 1. If the items below don't cut it, check out the .cshrc and maybe even .login of mikek for necessary PATH variables I assume everyone has. 2. Environment variables: a. PATH "${PATH}:/usr/local/www_utils" (suggested) C. Some more things I want to add: 1. Troubleshooting 2. What all the files are and what they do