

                               TRICKS WITH MERGE

      or WHAT YOU ALWAYS WANTED TO KNOW ABOUT MERGE BUT WERE AFRAID TO ASK
                    (and after reading this you'll know why)


  NOTE:  Although this file can be printed, it is not formatted for printing
         because some of the example command lines extend past 80 characters.


  This file started out at less than 12K, but more and more was added as new
  subjects were brought up, so I decided I'd better list the topics discussed.
  All the commands here refer to MERGESYS.  For MERGEPRO, always add /s and
  /f1 if no other field options are indicated.

  REDUCE THE SIZE OF DUPLICATE TRASH FILES (AND USE THE "/1K" SWITCH)
  PREPARE UNSORTED "MARKED" TRASH FILES USING THE "/1S" SWITCH
  LOG TRICKS (AND HOW TO MAKE ONE LINE DESCRIPTIONS)
  REPLACE DESCRIPTIONS ALREADY IN A DIR USING THE "/1M" SWITCH
  MERGE ONLY DESCRIPTIONS THAT DON'T MATCH (OPPOSITE OF "/1M")
  SWAP DESCRIPTIONS DURING COMPARES
  KEEP DUPLICATES BECAUSE THEY ARE REALLY DIFFERENT PROGRAMS
  GET A LIST OF FILE NAMES
  REMOVE OLDER VERSIONS OF PROGRAMS
  TAKE A CD-ROM OFF-LINE (AND HOW TO SORT CD-ROM DIRS WITH PCBFILER)
  ADD A NEW CD-ROM BY MERGING IT
  ANOTHER WAY TO "SORT" WITH MERGE


  Please note:  my apologies to anyone who tried to use this "sort" technique
  and found it didn't work right when testing the "errorlevels".  It was
  written using the NDOS (same as 4DOS) command processor, and I discovered
  later that COMMAND.COM doesn't respond the same when testing the errorlevel.
  I changed the errorlevel tests, and it should now work with COMMAND.COM.


  REDUCE THE SIZE OF DUPLICATE TRASH FILES (AND USE THE "/1K" SWITCH)

  The "tricks" described here refer to files created by CRSDUPE1.BAT, but
  these same principles can also be applied to files created with the other
  batch files, or for that matter, any time MERGE is used.  As mentioned at
  the end of CRSDUPE1.BAT, you can shrink CROSSDUP.DES (a sorted trash file of
  stripped duplicates) by selectively merging on different keys.  For example:

MERGE crossdup.des newdup.des /ns /eo /f1 /f2 /f3

  will remove all duplicates with the same name, size, and date (use /ns since
  you don't want to do the sequence test and /eo to not halt with an error if
  a key field is missing below; CROSSDUP.DES is not sorted by size and date in
  ascending order, but the duplicates are still removed because they are next
  to each other).  NEWDUP.DES will have only a single occurrence (the one that
  came first) of any names that had identical sizes and dates.  Or you could
  use /f1 /f2 to remove all with the same name and size, then /f1 /f3 for the
  same name and date.  You could even use /f1 /f4 /f5 to remove matches of the
  name and the first two words in the description (fields 4 and 5; I would not
  recommend /f1 /f4; an even more cautious approach would be /f1 /f4 /f5 /f6).
  You can repeat this process as many times as you like on NEWDUP.DES (merge
  it on itself after creating it the first time you merge CROSSDUP.DES) to
  successively eliminate descriptions that meet whatever criteria you choose.

  However, there is no assurance the description that remains is the best
  description; it will only be the latest date, or the one from the last DIR
  merged if names and dates are the same.  A further step you could take with
  the descriptions that matched whatever criteria you selected above is to
  get all of them together in one file (and at the same time have all the
  descriptions that weren't eliminated together; these are the ones that may
  be different programs with the same name).  Since you are only interested in
  keeping the best description, a quick scan with an editor is possible (e.g.,
  to see which is the longest and delete the others).  First merge NEWDUP.DES
  on itself to strip all remaining duplicates to a trash file (they will be
  the second or more occurrences that were left behind after the "shrinking"
  because they may not really be duplicates).  Then "compare" the trash file
  to the original CROSSDUP.DES with MERGE's new "/1K" switch, which will only
  keep descriptions that match and strip the rest to another trash file (you
  need to add /k so all matching descriptions are kept, otherwise the second
  or more occurrences would also be stripped to the trash file):

MERGE /o newdup.des /tnodup.des+n
MERGE /k /1k nodup.des crossdup.des diff.des /tsame.des+n

  SAME.DES now has all occurrences of duplicates that are probably the same
  program (so you can pick out the best description), DIFF.DES has those that
  may be different programs, and CROSSDUP.DES isn't changed because a separate
  output file was named.  You can also use these techniques on the trash files
  which were created when the DIRs were merged on themselves and compared to
  DIRs 60 and 61 to weed out descriptions that are probable duplicates (the
  retained description will have the "happy face").  This may all seem very
  tedious, but you can automate it with a batch file (if you use a replaceable
  parameter, the same batch file can process any of these trash files; see the
  end of LOG TRICKS further below for an example batch file).


  PREPARE UNSORTED "MARKED" TRASH FILES USING THE "/1S" SWITCH

  The trash files created by CRSDUPE2 and 3.BAT are not sorted, so they would
  need to be first if you wanted to do the "compare" as shown above.  However,
  the relative order of the "happy face" and stripped descriptions may change,
  and the "happy face" will be truncated if it is in column 79, but you can
  shrink them selectively before sorting (the sequence won't matter when using
  /eo /ns, and since the duplicates are paired, the shrinking process still
  works).  But there is another trick to get around PCBFiler's shortcomings
  by first separating the descriptions into two files, sorting each, and then
  merging the files back together.  Some "happy faces" may be missing, but the
  relative order will still be the same (well, actually the order between the
  descriptions that the DIR60 compare stripped from DIR61 are reversed because
  of the "switcheroo" where the kept description, now in DIR60 but originally
  from DIR61, followed DIR60's replaced description in the trash file, but the
  important thing for this usage is since the "happy face" may be missing, you
  can still identify the kept description because it will always come first).

  You can do this because when /1s is used with the trash option, "marked"
  descriptions are stripped to the trash file.  CRSDUPE3.BAT's trash files
  will only have one occurrence of each per duplicate because the DIRs already
  had all internal dupes removed and every successive "compare" DIR had any of
  its cross dupes from a lower numbered DIR stripped, but CRSDUPE2.BAT's trash
  files may have several occurrences, so add /k to keep all "unmarked" dupes
  (that's also why HAPPY.DUP is shown merged on itself before merging it back
  with CROSSDUP.D??:  to get only one occurrence of the kept description).
  Although an error in file 1 during compares won't cause a halt, since errors
  are still "swapped" and forced out, use /ns to not perform the sequence test
  because you don't want to change the order if there's a sequence error (/eo
  could also have been used instead, but out of sequence records should be the
  only type of error if there are any at all, so why do the test if it's not
  necessary and you are going to ignore it anyway).  /1s is different from the
  other "compares" in that you can merge a file on itself, or even strip the
  "marked" occurrences from file 1 and merge it with file 2 at the same time.

MERGE /o /k /ns /1s /thappy.dup+n crossdup.d??
sort both on name ascending
MERGE /o happy.dup
MERGE /o /k happy.dup crossdup.d??

  Now CROSSDUP.D?? can be treated just like the "marked" trash files from
  CRSDUPE1.BAT (the only difference is you can't safely use /1s again to strip
  the marked first occurrences since some "happy faces" may be missing).


  LOG TRICKS (AND HOW TO MAKE ONE LINE DESCRIPTIONS)

  Besides having descriptions that may be different programs, DIFF.DES above
  will still have many that are really the same.  In most cases, you can tell
  if a description is really a duplicate or a completely different program
  that just happens to have the same name by looking at the one line written
  to the log when /vl1 is used.  The log was rather unwieldy when it came to
  extricating a description from a trash file so it could be reinserted into a
  DIR, but not anymore.  Create a log using the command line below:

MERGE /o /k /vl1 /loneline.des+n diff.des

  Since /k keeps duplicates, you haven't changed DIFF.DES, but you have a log
  with the entire first line of the duplicated descriptions (/vl1 writes both
  the first occurrence and any following duplicates when a file is merged on
  itself or when cross file duplicates are encountered during the compare or
  merge of two files, while +n writes the log without a header or statistics,
  and when used with /vl1, the "type" of error or duplicate is also omitted).
  The line and record numbers, along with DIFF.DES's file name, still follow
  the description line, but the word "Duplicate" doesn't start the line, and
  since there is also no header or stats at the end, it can be "merged".  You
  can even "chop off" the extra stuff by running CHK4DES on ONELINE.DES, which
  will truncate the line at 80 characters, leaving only the description part.
  Now you can edit the much smaller log and delete all descriptions that are
  probable duplicates while keeping the descriptions that could be different
  programs with the same name (you may be surprised at how many there are; you
  only need the name once, so you can also delete any other occurrences, or
  merge ONELINE.DES on itself when you're done editing).  To extract the full
  descriptions from DIFF.DES, just merge them using the /1k switch.

MERGE /o /k /1k oneline.des diff.des /tmoresame.des
MERGE /o /k moresame.des same.des

  Now DIFF.DES has only the descriptions that may be different programs while
  SAME.DES has all the probable duplicates.  Instead of creating DIFF.DES and
  SAME.DES first by using the "shrinking" techniques above, it may be quicker
  to make a one line log file with CROSSDUP.DES, do the shrinking on it, and
  then create DIFF and SAME.DES (reverse the order in which you prepare them;
  since the one line files are smaller, "merges" won't take as long).  The
  sample batch file at the end of this section takes that approach.

  If you want to make sure you've really got only different programs with the
  same name you can use your editor again to check DIFF.DES (it shouldn't be
  that big).  Since DIFF.DES will be a bunch of duplicate names, you probably
  won't want them in the same directory, so you could also use the editor and
  save them to different files for merging into specific DIRs.  If you are
  using this method on the cross duplicates stripped from DIRs 60 and 61, it
  won't be as difficult getting the descriptions back into those DIRs because
  the trash file you are working on corresponds to the DIR that will receive
  the reinserted descriptions (they were stripped from their respective DIRs
  because a lower numbered DIR had the same name).  However, in this case, the
  equivalent to DIFF.DES will also contain the "happy face" first occurrences
  that were kept, so after you've verified that these are really different
  programs (you could also just delete the "happy face" occurrences with the
  editor), merge DIFF.DES's equivalent on itself with the /1s switch.

  The cross dupe trash files created with CRSDUPE3.BAT are just like those
  from CRSDUPE1.BAT, except they aren't sorted (unless you modified all those
  "compares" to be like the ones in CRSDUPE1.BAT for DIRs 60 and 61).  If you
  followed the "separate/sort/remerge" procedure described above, don't use
  /1s here because some "happy faces" are probably missing.  Your best bet may
  be to use the editor and delete the one that occurs first (it was the one
  marked by a "happy face" if there is none).  However, since there is only
  one occurrence of the kept description and it comes first, you could also
  merge the file on itself using the trash option, and the descriptions you
  want to put back in the DIR will be in the trash file.  CRSDUPE2's trash
  files aren't sorted either, but a duplicate description that is really a
  different program won't go back into the DIR that corresponds to the trash
  file you're working on since it was stripped from a higher numbered DIR (you
  would have to decide where it goes as with CROSSDUP.DES above).

  You may have noticed that you can use the log like this to generate a one
  line description for all your files (of course, some of the one-liners won't
  have anything useful because many authors have a penchant for using highbit
  characters or some other nonstandard design for the first line of their
  descriptions).  If you want one line descriptions for anything, copy a DIR
  to another file name and merge them with log's +n and /vl1, then merge the
  log on itself to get only one occurrence, and voila, one line descriptions
  (you'd still need to run CHK4DES to truncate the extra stuff; sorting with
  PCBFiler also truncates it, but then you'd have stray "description" parts to
  contend with; however, you could strip them using MERGE again with /ea).
  Well, that's sort of an obtuse method, but it is dependable.  A simpler way
  would be just to merge each DIR on itself to a second output file name using
  /s1 /ea (for MERGESYS; use /f1 /ea for MERGEPRO).  What this should do is
  strip all subsequent lines of a description as "key" errors (/s1 sets single
  line records and all the lines that start with a space are considered to be
  errors).  I haven't tried it on every DIR, but it worked on the ones I did.

  The commands below could be used in a batch file that shrinks and prepares
  descriptions for examination.  It makes a one line description first, then
  creates the desired description files from it.  It may also run a little
  faster than working on the full descriptions because the files are smaller.
  The trash files from CRSDUPE2 and 3.BAT will need sorting before they can be
  used here, otherwise just invoke this file with the trash file name.  Since
  MERGE defaults to case sensitive, line 5 is a case insensitive check of the
  first three words in the description.  CHK4DES just truncates ONELINE.DES to
  80 characters, and NO_EXIST.FIL is used because CHK4DES requires two file
  names to run (it won't even exist unless a description has CHK4DES's default
  "description not found" phrase; if you wanted to remove any names that don't
  have a description, as well as the ones where the first line is blank, add
  ++ after the file names for CHK4DES's "no description" phrase).

MERGE /o /k /vl1 /loneline.des+n %1
CHK4DES oneline.des no_exist.fil
MERGE /eo /ns /f1 /f2 oneline.des work.lin
MERGE /o /eo /ns /f1 /f3 work.lin
MERGE /o /eo /ns /f1 /f4 /f5 work.lin
MERGE /o /eo /ns /f1 /f4 /f5 /f6 /c work.lin
MERGE /o /tnodup.lin+n work.lin
MERGE /k /1k nodup.lin oneline.des diff.lin

  At this point, you could stop and use an editor to delete all except the
  descriptions that look like different programs from DIFF.LIN (there is no
  trash file to catch the stripped nonmatching one line descriptions because
  these would be probable duplicates and the main reason you'd want to examine
  their descriptions is to keep the best one, which you usually can't tell by
  looking at just one line).  DIFF.LIN is also not merged on itself since it
  may not have been checked with an editor yet, and having duplicates doesn't
  effect getting the full descriptions out of the cross duplicate file.  In
  any case, finish with (don't forget to delete the work files and rename DIFF
  and SAME.DES with something that corresponds to the processed trash file; if
  you haven't looked at DIFF.LIN yet, rename it too):

MERGE /k /1k diff.lin %1 diff.des /tsame.des+n


  REPLACE DESCRIPTIONS ALREADY IN A DIR USING THE "/1M" SWITCH

  There is a problem with trying to replace a description in a lower numbered
  DIR with a better one that was removed from DIR60 or 61, since their trash
  files contain all of their stripped descriptions and there is no way to tell
  which DIR caused it to be stripped out.  Well, MERGE has another new compare
  switch, "/1M", which will merge into file 2 all descriptions from file 1
  that match those in file 2, and only the ones that match, REPLACING the
  descriptions in file 2 with ones from file 1 (if you also used /k to keep
  duplicates, you'd get both descriptions into file 2).  SO, you don't need to
  know which DIR a description came from in order to replace it with a better
  one.  Just go through the trash files from DIRs 60 and 61 (do this before
  sorting with PCBFiler) and delete all the descriptions that you don't want
  (the "happy face" will mark whatever description is currently in a DIR).
  You could call the file with the descriptions you didn't delete REPLACE.DES,
  which you would then sort with PCBFiler on name ascending.  Next write a
  batch file with 59 lines using the command line below (the "?" = 1 to 59).
  If the DIR headings are in place, add /b2??h (they will also be kept):

MERGE /o /1m /td:\temp\replaced.dup+n /ld:\temp\replaced.log /vl1 d:\temp\replace.des d:\pcb\dirc\dir?

  The trash and log files are optional (REPLACED.DUP would get descriptions
  stripped out of the DIRs).  When this batch file has run, all the matching
  descriptions that were in the lower numbered DIRs will have been replaced
  with the ones from REPLACE.DES (since some of the descriptions you wanted to
  replace may have been removed from all the lower numbered DIRs because of
  the way CRSDUPE1.BAT makes CROSSDUP.DES, if you didn't put them back in a
  DIR but just added them to DIR61, you'll also need to include DIR61).


  MERGE ONLY DESCRIPTIONS THAT DON'T MATCH (OPPOSITE OF "/1M")

  I was wrong when I said the one compare MERGE couldn't do was merge in only
  records that don't match.  I forgot about the "/X" switch, and I wrote it!
  It's not a compare switch, but it accomplishes the same thing by making file
  1 the output file when there are two file names.  Suppose you had an older
  list of descriptions that you wanted to add to a DIR, but you didn't want
  any currently listed names to be replaced because they would be later dates.
  Since file 1 always has precedence and keeps a duplicate over file 2, you
  can make the DIR file 1 and have it also be the output file (you could also
  do it by naming a third file for the output file, but then you'd need to
  delete the old DIR and rename the output file with the DIR's name, whereas
  /x will accomplish it in one step).  Just use the command line below:

MERGE dir? old_dir? /x /o


  SWAP DESCRIPTIONS DURING COMPARES

  All the CRSDUPE?.BATs pull a fast one when comparing DIR60 to DIR61 and put
  DIR61's description, since it is probably the best and latest, into DIR60.
  You can also do that with all the other DIR compares, and in most cases the
  best will be kept.  The only problem here is that the description replaced
  in the categorized lower numbered DIR may NOT be the same program, just one
  with the same name (it didn't make any difference with DIR60 since it's just
  a "misc" category).  You can check the trash files after the first time any
  of these batch files run and decide whether the number of wrong replacements
  would be prohibitive.  Plus, since most of the files in DIR61 are "uploaded"
  by you, and many are from CD-ROM's you haven't combined with the rest of the
  DIRs yet, when they are added, their descriptions will end up in the correct
  DIRs.  Several methods are shown below for swapping descriptions.  The first
  technique is similar to that used in CRSDUPE1 and 2.BAT, except the "marked"
  kept description from the higher numbered DIR doesn't follow the stripped
  one from the lower numbered DIR as is done in the batch files, which also
  eliminates a step.  If the DIR heading is in place, add /b1?? /b2??h to line
  1 and /b2??h to line 3 in all three examples, and in the last add /b1??h to
  line 4 and /b1?? /b21 to line 5.

MERGE /o /1 /tkeep_d61.dup+n /lcrossdup.log dir?_low dir61
if not exist keep_d61.dup goto NEXT
MERGE /o /tstripdup.low+n /lcrossdup.log /vl1 keep_d61.dup dir?_low
MERGE /o /1t /tcrossdup.low+n /lcrossdup.log keep_d61.dup stripdup.low
del keep_d61.dup
del stripdup.low

:NEXT
...

  The first compare removes any cross file dupes from DIR61, but then they are
  merged back into the lower numbered DIR, which strips whatever it had to the
  trash file STRIPDUP.LOW.  Then KEEP_D61.DUP is compared using /1t to the
  stripped duplicates so that the kept descriptions are marked and added to
  the continuing trash file, CROSSDUP.LOW, along with the stripped ones (which
  came from DIR?_LOW; in this arrangement, the rest of the descriptions are
  ones that DIR?_LOW stripped from a higher numbered DIR).  The next method is
  like that in CRSDUPE3.BAT and uses the /1m switch, first replacing the lower
  numbered DIR's descriptions with any from DIR61 that match (the "compare"
  direction is reversed here, and the lower numbered DIR's stripped duplicates
  then remove from DIR61 the descriptions that replaced them).

MERGE /o /1m /tstripdup.low+n /lcrossdup.log /vl1 dir61 dir?_low
if not exist stripdup.low goto NEXT
MERGE /o /1 /tkept_d61.dup+n /lcrossdup.log stripdup.low dir61
MERGE /o /1t /tcrossdup.d61+n /lcrossdup.log kept_d61.dup stripdup.low
del kept_d61.dup
del stripdup.low

  CROSSDUP.D61 in this example normally has the descriptions that were removed
  from DIR61 by the lower numbered DIRs, but if all the DIR?_LOW compares were
  done as shown above, it would have the descriptions from lower numbered DIRs
  that DIR61 replaced.  The last method makes sure the kept description in the
  lower numbered DIR has the latest date (ties will keep the ones from DIR61).

MERGE /o /1 /tstripd61.dup+n /lcrossdup.log /vl1 dir?_low dir61
if not exist stripd61.dup goto NEXT
MERGE /o /k /f1 /f3ddt /lcrossdup.log stripd61.dup dir?_low
MERGE /o /tstripped.dup+n /lcrossdup.log /vl1 dir?_low
MERGE /o /1t /tcrossdup.???+n /lcrossdup.log dir?_low stripped.dup
del stripd61.dup
del stripped.dup

  There is an equivalent using the /1m switch, but it requires an extra step
  (a "sort" pass with /ef to get the latest date first; you can't just compare
  with /1m /f1 /f3ddt because only exact matches are merged, and if there were
  any, they would be name and date duplicates).


  KEEP DUPLICATES BECAUSE THEY ARE REALLY DIFFERENT PROGRAMS

  If you keep different programs with the same name, besides needing a method
  where a caller can specify which one to download, another problem is every
  time you add a new CD-ROM, the duplicated descriptions that you reinserting
  into the DIRs when the last CD-ROM was added will be stripped again and you
  would have to do them all over.  I don't know of an easy way to get around
  this, but I do have two suggestions.  You could make a new DIR, e.g., 62,
  where all you have is descriptions that are duplicated somewhere else and
  the rest of the DIRs would only have a single occurrence of the same name
  between them.  This may also make it easier for callers to download them as
  you could have a special "door" or something to access these files.

  Another idea is to have a "master" list of all dupes, plus a separate list
  for each DIR that has a name duplicated somewhere else.  After you add a new
  CD-ROM, you could "compare" the master duplicate list to every trash file of
  stripped dupes to remove their names and anything else that may have been
  added with the same name (use /1k /k as was done in the examples above):

MERGE /1k /k master.dup dirdup?? temp.dup /tnew??.dup+n
MERGE /o /k temp.dup old.dup
MERGE temp.dup /ttmpstrip.dup
MERGE /k tmpstrip.dup oldstrip.dup

  DIRDUP?? would be a sorted trash file of stripped dupes (the crossdupe trash
  files created by CRSDUPE2 and 3.BAT should be prepared with the method shown
  in PREPARE UNSORTED TRASH FILES above).  NEW??.DUP has only new duplicates
  that were caused by this CD-ROM and can be manipulated with the techniques
  previously discussed.  OLD.DUP is a continuing file which, after every trash
  file is processed, will contain all the dupes already on-line plus any new
  ones from this CD-ROM that also have the same name.  When TEMP.DUP is merged
  on itself, the dupes that were stripped from the DIR will be in TMPSTRIP.DUP
  (the kept first occurrence is still in TEMP.DUP), which is then merged with
  /k to keep duplicates into OLDSTRIP.DUP, another continuing file.  After all
  the DIRs are compared, merge OLDSTRIP.DUP on itself using the trash option.
  Any names from the CD-ROM that match a duplicate file currently on-line will
  be in the trash file (as well as any descriptions you might have that are
  already duplicated more than twice; you'd just have to keep a separate list
  of them if there are any, but it certainly wouldn't be a long one):

MERGE oldstrip.dup /tadded.dup

  If ADDED.DUP doesn't exist (or the only descriptions in it are the same as
  the list of names already occurring more than twice), all that's left to do
  is merge the separate lists of duplicates back into their respective DIRs.
  However, if it has other descriptions, compare it with /1k /k to OLD.DUP so
  you can see what was added (there may be cases where you have two different
  programs with the same name, the CD-ROM replaced one of them in a DIR, then
  stripped the other during the crossdupe compares, so that now both of the
  previous descriptions are stripped, or a CD-ROM replaced a DIR description,
  then the CD-ROM was stripped by a compare, etc.; you just won't know until
  you check).  When you examine the "compared" OLD.DUP, the sequence in which
  it was originally created is important.  If you first added the individual
  DIRs above, then the cross duplicate trash files, the very first occurrence
  will be the one that is kept (the descriptions in OLD.DUP will be paired and
  the first of each pair is the one that was retained, but a kept description
  from one DIR can then be stripped during the cross duplicate compares).  If
  you made it in the reverse order, the first of the last pair is the one that
  is currently in a DIR somewhere.  You may even have to check the original
  DIRDUP?? trash files to decide what belongs where.  This sounds like a lot
  of trouble, and about the only good thing is it shouldn't happen very often!


  GET A LIST OF FILE NAMES

  To get a list of every duplicate file name, first merge all the DIR trash
  files together (there may be 61 of them, DIR1 to 61.DUP):

MERGE dir1.dup dir2.dup dir.dup
MERGE /o dir3.dup dir.dup
...

  DIR.DUP will only have the "marked" first occurrence descriptions that were
  originally kept, but that isn't important here since you're just after the
  file names.  Now merge it with DIR61_60.DUP that was created previously:

MERGE dir.dup dir61_60.dup moredupe

  Next, copy MOREDUPE to another file (e.g., DUPE2) and merge them together
  with the trash file parameters +k+n:

MERGE moredupe dupe2 /tmoredupe.nam+k+n.

  Since all names will be duplicated, the trash file will be another list of
  just file names, this time with the duplicates obtained from merging the
  DIRs on themselves and "comparing" them to DIR60 and DIR61.  Finally, merge
  CROSSDUP.NAM and MOREDUPE.NAM (or whatever you called the trash file above)
  and you will have the name of every file that had a duplicate description,
  which may be useful for checking that there is only one occurrence of the
  file on your hard drive or between your hard drive and CD-ROMs (just make
  sure the file you keep matches the DIR description, otherwise someone could
  download SERIAL.ZIP, expecting a programming aid but instead end up with a
  program that checks serial ports).  You can use this same technique to make
  a list of just file names for all the files you have on-line by copying the
  master file list to another name and merging them with trash's +k+n.


  REMOVE OLDER VERSIONS OF PROGRAMS

  Suppose an author asks you to remove older versions of his or her programs
  from your DIR descriptions and provides you with a list of them.  How do you
  go about it now?  Let MERGE do it for you.  Put each file name on a line by
  itself and save them to a text file (all should be in uppercase and in name
  ascending order, just like the names would appear in the DIR descriptions).
  Then write a 61 line batch file (which could be used anytime you need to
  remove certain descriptions) to compare the list to each DIR.  If the list
  of names is in REMOVE.LST, the batch file might look like the example shown
  below.  The /b2??h assumes the DIR headings are still in place, and the DIRs
  need to be sorted in name ascending order, which the batch file can also do
  by running PCBFiler from the command line (see BATNOTES).  The trash file is
  optional but you may want to check that these are really the descriptions
  meant to be removed and not some other programs with the same names (you can
  also use a separate trash file for each DIR so if any of the descriptions
  need to be replaced, you know where they go, and a trash file won't exist if
  the DIR being compared had none of the names in the list).  The /ef is used
  in case you accidentally (like I did when testing these commands) get a name
  out of sequence in REMOVE.LST.  Although file 1 errors won't cause a "halt
  on error", out of sequence records can still be "swapped" so their order is
  changed, but /ef can't fix an out of sequence record that needs to be moved
  forward more than one position, so you may want to first merge REMOVE.LST on
  itself using /ef to correct any sequence errors (see BATNOTES for more).

MERGE /o /b2??h /1 /ef /td:\temp\removed.des+n d:\temp\remove.lst d:\pcb\dirc\dir1
MERGE /o /b2??h /1 /ef /td:\temp\removed.des+n d:\temp\remove.lst d:\pcb\dirc\dir2
MERGE /o /b2??h /1 /ef /td:\temp\removed.des+n d:\temp\remove.lst d:\pcb\dirc\dir3
...


  TAKE A CD-ROM OFF-LINE (AND HOW TO SORT CD-ROM DIRS WITH PCBFILER)

  You can easily remove a CD-ROM's listings from your DIRs when you take it
  off-line, providing it's sorted the same as the bbs DIRs (which may be a big
  catch).  The major purpose of BATNOTES is to keep from separately sorting a
  CD-ROM's DIR listings, but it can be done without making a special DIR.LST,
  although that is a possibility (you wouldn't need one for every CD-ROM, just
  have a special CD-ROM.LST and copy it over your regular DIR.LST, next copy
  the DIRs from whatever CD-ROM you are taking off-line to the directory named
  in CD-ROM.LST, renaming them appropriately as they are copied, and then run
  PCBFiler from the command line).  The method shown below temporarily renames
  your bbs DIRs and copies to their directory the CD-ROM DIR.  If they are not
  already called DIR1, etc., you can rename them while they are copied, e.g.,
  copy cd-rom\dirpath\list* d:\pcb\dirc\dir*.  Of course, this doesn't work if
  they have some unusual naming convention and you would need a separate copy
  line for each one (hint:  you could use "merge" batch files, if you write
  them so they call another batch file that has the actual merge commands, to
  do the copying since all the DIRs would be appropriately paired, then just
  replace the merge commands batch file with another one that does a "copy").

ren d:\pcb\dirc\dir* d:\pcb\dirc\tmp*
copy cd-rom\dirpath\* d:\pcb\dirc
PCBFILER /sort

  Since you already know what CD-ROM DIRs were combined with the bbs DIRs from
  when they were added, "compare" each CD-ROM DIR to the appropriate bbs DIR
  (now called TMP).  By using several fields for the key (along with /eo /ns),
  you can be fairly certain you'll remove only the listings from this CD-ROM
  (in case a CD-ROM added later replaced a name from this one, BUT if this and
  another CD-ROM have identical descriptions for the same file, you'll remove
  one that you meant to keep).  Then delete the CD-ROM DIRs and rename yours.
  Don't forget to add /b2??h if the DIR headings are in place, and you'll also
  need /b1?? if the CD-ROM's DIRs don't start on line 1 (if both sets of DIRs
  start on a line other than 1 and it's the same, you can just use /b1??, but
  if you want to keep the DIR heading, you would still need to add /b2??h).

MERGE /o /eo /ns /1 /f1 /f3ddt /f2 /f4 /f5 /f6 d:\pcb\dirc\dir? d:\pcb\dirc\tmp?
MERGE /o /eo /ns /1 /f1 /f3ddt /f2 /f4 /f5 /f6 ...
...
del d:\pcb\dirc\dir*
ren d:\pcb\dirc\tmp* d:\pcb\dirc\dir*

  If your bbs DIRs are in date descending order, put /f3ddt before the other
  fields (how they are sorted doesn't matter in this case, only that both are
  sorted the same, so you can also sort the CD-ROM in date descending order).
  To help promote the success of this method, always place a CD-ROM's DIR
  before an existing bbs DIR when adding a new CD-ROM (see below).  Then if
  the new CD-ROM has a program name that duplicates one already on-line, the
  new CD-ROM's description will replace the one currently in use.  However,
  what if 10 CD-ROMs all have identical descriptions for the same file, as
  could easily happen if the CD-ROMs used the FILE_ID.DIZ, and you took one of
  those CD-ROMs off-line:  you'd loose that listing, and I can't think of any
  way to keep it.  Still, I don't see how else you can take a CD-ROM off-line
  without comparing out its description.  You can't just add back to the bbs
  DIRs all the CD-ROMs minus the one going off-line since its descriptions
  would still be in the bbs DIRs.  What you'd need to have is a separate list
  of bbs files that doesn't contain any of the CD-ROM files for each DIR, then
  add to them the CD-ROMs staying on-line each time you remove one.

  Still, adding back 40 or so CD-ROMs every time you remove one is going to be
  time consuming.  Rather than doing that and having to keep separate bbs file
  lists, it may be simpler to compare out the descriptions using the technique
  below.  It uses the same method as above, except it saves each DIR's removed
  descriptions in a trash file that corresponds to the DIR.  Then, the on-line
  CD-ROMs are compared to the trash files and any matches are removed to other
  trash files, which will be merged back into the original DIRs.  If you are
  set up to "merge" CD-ROM DIRs (see further below and COMBINE.BAT), the merge
  batch files can do the work (when you write them, let them call a separate
  batch file that has the actual merge commands, which can be easily replaced
  with whatever commands are needed at the time, or just add extra replaceable
  parameters to the command line so you can feed the appropriate commands when
  necessary).  Ok, say you've got a WHIZBANG.BAT, the merge batch file for a
  CD-ROM you're going to take off-line, that looks like the lines below.  The
  file names may look backwards, but that's only because it's probably easier
  to edit names when they are on the end (MERGEROM.BAT places the parameters
  correctly; the second name will become the first file name and vice versa):

call MERGEROM.BAT dir1 e:\whizpath\whiz5
call MERGEROM.BAT dir2 e:\whizpath\whiz11
call MERGEROM.BAT dir3 e:\whizpath\whiz1
...

  It doesn't matter what technique the existing MERGEROM.BAT uses for adding
  CD-ROMs since it is going to be temporarily replaced with MERGEOFF.BAT (the
  line below), the nice part about calling batch files.  This also assumes all
  DIRs are sorted by name ascending.  If not, you can use one of the methods
  described to sort them when needed, although that will add some time.  The
  line below is actually the REALLY short example shown in COMBINE.BAT for
  merging the CD-ROMs, it just has the needed commands added to the end, which
  could also be done by passing replaceable parameters.  Since anything that's
  removed will be added back if another CD-ROM has it, only three key fields
  are used, which may speed things up slightly, but the files created should
  still be small, which is important too (/eo isn't needed for three fields).
  As discussed in COMBINE.BAT, if any of the CD-ROM DIRs have headings, it's
  better to pass /b1?? as a parameter from the appropriate merge batch file:

MERGE /o /b1?? /b2??h %2 d:\pcb\dirc\%1 /1 /td:\temp\%1.off+n /f1 /f3ddt /f2 /ns

  If the CD-ROM DIRs haven't had their errors fixed, you could add /e, but I
  can't say how successful this will be if unknown types of errors are present
  (file 1 errors during a compare won't halt MERGE, but they could interfere;
  with /e, hopefully things like stray description parts will be disregarded).
  Another possibility is to put the line below first, which strips any errors
  before the compare (similar to when the bbs DIRs are merged on themselves)
  and can also remove a CD-ROM DIR heading so /b1?? isn't needed afterwards:

MERGE /o /b1?? /ea /f1 /f3ddt %2

  Now, instead of calling WHIZBANG.BAT with COMBINE.BAT as when you merged it
  into the bbs DIRs, call it with REMOVE.BAT, which first sets everything up
  and could also do any sorting (don't forget the bbs DIRs), for example:

ren d:\pcb\dirc\dir* d:\pcb\dirc\tmp*
copy cd-rom\dirpath\* d:\pcb\dirc
PCBFILER /sort
copy d:\pcb\dirc\dir* cd-rom\dirpath
del d:\pcb\dirc\dir*
ren d:\pcb\dirc\tmp* d:\pcb\dirc\dir*

  If CD-ROM\DIRPATH is the actual CD-ROM, you obviously can't copy the sorted
  DIRs back, but you can cheat and make a directory with the same path on some
  hard disk, then use the ASSIGN or SUBST command (first copy the DIRs to your
  hard drive) to change the CD-ROM drive's letter to the hard drive's so that
  all further requests for the CD-ROM DIRs get faked out and are sent to your
  hard drive, and you can do all this in the batch file (also do during normal
  maintenance down-time; if you try it while callers can connect, no telling
  what might happen).  The example below assumes there are already different
  versions of MERGEROM.BAT and copies MERGEOFF.BAT over the existing one.

copy d:\pcb\dirc\mergeoff.bat d:\pcb\dirc\mergerom.bat
call WHIZBANG.BAT

  Now you need another MERGEROM.BAT with the lines shown below that compares
  these trash files to the rest of the CD-ROMs when their "merge" batch files
  are called (MERGEON.BAT also keeps the created DIR??.ON trash files sorted
  so that descriptions can be merged back in, otherwise as each new CD-ROM is
  compared, any stripped descriptions would be appended, and by overwriting
  the DIR??.OFF file each time, as matching descriptions are removed, it gets
  smaller and subsequent compares will take less time).  This also uses the
  new Compare "E" parameter, which deletes an input/output file if its size
  would be zero, so if it turns out that all the descriptions for a particular
  DIR will be returned, the OFF file won't exist and any further "merges" will
  just halt with a "file not found" error (see the discussion on FIXDIR.BAT in
  COMBINE.BAT for more information on the "E" parameter):

MERGE /o %2 /b1?? d:\pcb\dirc\%1.off /b21 /1e /td:\temp\temp.on+n /f1 /f3ddt /f2 /ns
if not exist d:\temp\temp.on goto END
MERGE /o /k d:\temp\temp.on d:\temp\%1.on
del d:\temp\temp.on
:END

  As with MERGEOFF.BAT earlier, if the CD-ROM DIRs could have errors, either
  add /e to the first line above or put the extra line to strip errors first.
  Any descriptions in DIR??.ON will result from another CD-ROM having it too
  and stripping it from the files WHIZBANG created (do any sorting as needed).

copy d:\pcb\dirc\mergeon.bat d:\pcb\dirc\mergerom.bat
call STILLON1.BAT
call STILLON2.BAT
call STILLON3.BAT
...
del d:\pcb\dirc\dir*.off
copy d:\pcb\dirc\mergeadd.bat d:\pcb\dirc\mergerom.bat

  MERGEADD.BAT has the commands needed to merge CD-ROM and bbs DIRs, although
  for this purpose, it could just have the command line below, which probably
  is faster (if you're keeping duplicates because they are really different
  programs and have any in the same DIR, add /f1 /f3ddt /f2 to the line):

MERGE /o %2 d:\pcb\dirc\%1 /b2??h

  Anything that needs to be returned to the bbs DIRs are in properly sorted
  files that are ready to be merged back.  REMOVE.BAT can do it in the next 61
  lines, or you can have another batch file it calls do the work, just as with
  the merge batch files.  In any case, the lines would look like:

call MERGEROM.BAT dir1 d:\pcb\dirc\dir1.on
call MERGEROM.BAT dir2 d:\pcb\dirc\dir2.on
call MERGEROM.BAT dir3 d:\pcb\dirc\dir3.on
...

  Finally, REMOVE.BAT deletes DIR*.ON and returns everything to normal.  Ok,
  so maybe it turns out that you can't easily take a CD-ROM off-line, but like
  all the other things MERGE can do, once the batch file is written you are
  home free.  You can use the same REMOVE.BAT any time you remove a CD-ROM by
  editing the names (most of the time there would only be two changes).  Just
  in case all these comments have obliterated what is going on, a repeat of
  the commands in REMOVE.BAT follows (add any extra ones where needed, such as
  for sorting, ASSIGNing, SUBSTituting, etc.):

copy d:\pcb\dirc\mergeoff.bat d:\pcb\dirc\mergerom.bat
call WHIZBANG.BAT
copy d:\pcb\dirc\mergeon.bat d:\pcb\dirc\mergerom.bat
call STILLON1.BAT
call STILLON2.BAT
call STILLON3.BAT
...
del d:\pcb\dirc\dir*.off
copy d:\pcb\dirc\mergeadd.bat d:\pcb\dirc\mergerom.bat
call MERGEROM.BAT dir1 d:\pcb\dirc\dir1.on
call MERGEROM.BAT dir2 d:\pcb\dirc\dir2.on
call MERGEROM.BAT dir3 d:\pcb\dirc\dir3.on
...
del d:\pcb\dirc\dir*.on


  ADD A NEW CD-ROM BY MERGING IT

  You can use a sort method similar to the above for adding CD-ROMs by first
  sorting your existing DIRs on name ascending, renaming them, copying the
  CD-ROM's and sorting them, then merging the two (you could use /ef and /ea
  together or separately with a "sort" loop, although that shouldn't be needed
  with only one occurrence of a name within a DIR).  If you are adding more
  than one CD-ROM, the first line below would be replaced with a "merge" line
  for each CD-ROM and you would probably want a sort loop (see DIRDUPE.BAT,
  COMBINE.BAT, and BATNOTES for more information).  Since these commands are
  the same for each DIR except the file names, you can save some typing by
  placing them in a separate batch file and using replaceable parameters, then
  call the batch file with the appropriate file names.  Add /b1?? to the first
  line if the CD-ROM DIR has a heading, and if your DIR headings are in place,
  add /b2??h to the first line, /b1??h to the fourth, and /b1?? /b21 to the
  sixth (if you remove your DIR heading in the first line by not using the "h"
  parameter, then don't add any /b's to the rest of the lines).

MERGE /o /k /ef /ea /f1 /f3ddt /tdir1.err+n /ldirerror.log /vl dir1 tmp1
del dir1
ren tmp1 dir1
MERGE /o /ttemp.dup+n /ldir_dupe.log /vl1 dir1
if not exist temp.dup goto DIR2
MERGE /o /1te /tdir1.dup+n /ltemp_dup.log dir1 temp.dup

:DIR2
MERGE /o /k /ef /ea /f1 /f3ddt /tdir2.err+n /ldirerror.log /vl dir2 tmp2
...


  ANOTHER WAY TO "SORT" WITH MERGE

  BATNOTES shows how to use the /ef switch and a "sort" loop to make MERGESYS
  perform a sort operation.  However, the method is only practical where a few
  records may have their order reversed (it took about an hour and 40 minutes
  to completely sort a previously unsorted 487K file with 6700 lines and 1300
  records).  There really is no efficient way to sort using MERGESYS (after
  all, it's a "merge" program), but with the method shown below, the same 487K
  file took a little over 4 minutes, a completely unsorted 1.7 meg file with
  23,000 lines and 4,000 records took 23 minutes, and a 3 meg file with 43,000
  lines and 19,000 records took 2 hours (as you can see, the time for sorting
  does not increase linearly, and it will also depend on just how much sorting
  the file needs when MERGESYS starts).  The improvement occurs because rather
  than simply merging the file on itself, where a record that is too far back
  can only be moved forward one position on each pass, the sequence errors are
  stripped to a trash file, which is then remerged (this is basically the same
  as the technique for fixing errors that is described in DIRDUPE.BAT, except
  here it is done automatically by MERGESYS).

MERGE /o /k /b1?? /f1 /f3ddt /ef /ea /tunsorted+n /lsort.log dir?
if errorlevel 3 goto END
if not errorlevel 2 goto NOSORT

:LOOP
ren unsorted remerge
MERGE /o /k /f1 /f3ddt /ef /ea /tunsorted+n /lsort.log remerge dir?
del remerge
if errorlevel 3 goto END
if errorlevel 2 goto LOOP

:NOSORT
if not exist unsorted goto CHKDUPE
ren unsorted dir?.err
MERGE /o /k /f1 /f3ddt /eo /ldirerror.log /vl dir?.err

:CHKDUPE
MERGE /o /tdir?.dup+n /ldir_dupe.log /vl1 dir?

:END

  Just as with the sort loop in BATNOTES, this also sorts alphabetic dates, so
  you would need to be certain there are none before running it.  If there are
  other types of errors, they will be in UNSORTED after all sequence errors
  have been corrected, which is then renamed to correspond to the DIR being
  processed and merged on itself with /vl so the log will show what errors
  occurred.  /eo is also used to not halt (these are errors!) or change their
  order.  It wouldn't be safe to do this without /f3ddt, because any stray
  description parts could end up being "sorted" into the DIR rather than being
  stripped.  The /b1?? in the first line assumes a DIR heading is in place
  when it starts, which is then removed since the "h" parameter isn't added.
  By making "dir?" a replaceable parameter, this can also be a separate batch
  file that is called with whatever DIR's name needs to be processed.  The
  example shown below is a general purpose sort loop.  The file to sort should
  be entered first after the batch file name, followed by the key fields (if
  there is a heading to skip, add /b1?? as needed to the lines below).

MERGE /o /k /ef /ea /tunsorted+n %1 %2 %3 %4 %5 %6 %7 %8 %9
if errorlevel 3 goto END
if not errorlevel 2 goto NOSORT

:LOOP
ren unsorted remerge
MERGE /o /k /ef /ea /tunsorted+n remerge %1 %2 %3 %4 %5 %6 %7 %8 %9
del remerge
if errorlevel 3 goto END
if errorlevel 2 goto LOOP

:NOSORT
if not exist unsorted goto CHKDUPE
ren unsorted error.rec
MERGE /o /k /eo /lerror.log /vl error.rec %2 %3 %4 %5 %6 %7 %8 %9

:CHKDUPE
MERGE /o /tdupe.rec+n /ldupe.log /vl1 %1 %2 %3 %4 %5 %6 %7 %8 %9

:END
