HowTo: Index a ColdFusion Site for Search with VSpider

So in my studies of learning ColdFusion and working on my first few projects, I came into an issue. I was redeveloping the public website for the company I am currently working for. The previous site was poorly built with little or no code reuse ability and quite a bit of static navigation and content.

I redesigned and developed the entire site. When I finished I entered the coldfusion administrator and re indexed the search collection. I performed a search to test it and was immediately alarmed. My search results were displaying code... I was confused and immediately grabbed my book on ColdFusion to learn that the built in search indexing collection could be prepared two ways, by reading the raw source code and indexing it (which is fine for completely static pages) or by using the search spider that is built in from Verity.

The clear solution is to use the Verity Spider. So this post is dedicated to how I was able to index my dynamically generated site with the Verity Spider.

1. Make your site available on localhost, even if your site will not stay there. For example, http://localhost/mysite/


2. Create a text file called mysiteCmds.txt and save it to the verity folder within the ColdFusion directory. For Example, C:\ColdFusion8\verity\mysiteCmds.txt


3. In this document will be all of your commands for the vspider such as starting point, files to exclude from the index and so on. For Example, mine looks like the following:
   

   
   -style c:\ColdFusion8\verity\Data\stylesets\ColdFusionVspider
   -collection c:\ColdFusion8\verity\collections\mysite
   -exclude http://localhost/mysite/admin/*
   -indmimeexclude text/css
   -start http://localhost/mysite/index.cfm
   -cgiok



4. Then I open up the command prompt (start->run->cmd) and run the following line,

   vspider -cmdfile c:\ColdFusion8\verity\mysiteCmds.txt



5. Then you need to navigate to the CF Administrator (http://localhost/CFIDE/Administrator/) and add the collection under Verity Collections. Name it the same as you set it in the CmdFile, mysite. Be sure to select English Advanced.


6. Next you need a search form.
   

   
   <_cfform action="search.cfm" method="post" name="submitSearch" ><_INPUT class="btn" type="submit" value="SEARCH" /><_/cfform>



7. Last, you need to be able to display the results.
   

   
   <_cfsearch collection="mysite" name="results" criteria="#searchterm#" type="simple">
   <_cfloop query="results">
   <_cfset score2 = (score * 100)>
   <_span style="font-weight:bold ">#currentRow#. #score2#% : <_a href="#replace(url,"localhost/mysite/","www.mysite.com/")#">
   <_cfif title eq "">
   #replace(url,"localhost/mysite/","www.mysite.com/")#
   <_cfelse>
   #title#
   <_/cfif>
   <_/a><_/span>
   #left(summary, 150)#
   <_/cfloop>
   

   
   Remove the underscores, I had to add them so that the code would display.
   

Resources:

http://www.monkeyflash.com/coldfusion/using-vspider/
http://www.adobe.com/devnet/coldfusion/articles/vspider.html


Chris