Table of Contents

  • Introduction
  • Administrator
    · Introduction
    · Directories
    · Installing
    · Configuring
    · New profile wizard
    · Unleash the crawler
    · Logs
    · Advanced profile configuration
  • User
  • Appendix
  • Logs
    IntraSeek will create a few log files as it works. These can all be seen from the IntraSeek graphical configuration interface under the Logs tab.

    Main logs
    First of all, it is possible to view the main crawler log. It contains some general information on when crawlers were started and how their searches ended. Delete it when it grows too big.

    Below the main crawler log you will find the main engine log. In this log you can see which profiles are mounted, and when they were mounted. Every time the search module is restarted, something will appear here. Delete it when it grows too big.

    Query logs
    The query logs are found below the main engine log, in a new table with an entry for each profile. You can view the complete query log for any profile, as well as get a top 100 list of the most popular search terms. If you want to, you can delete these logs as well when they grow too big, unless collecting statistics for the most searched terms interests you.


    Note that error messages generated by the crawler if something goes really wrong will appear in the Roxen error log (and furthermore that they will not appear in any of these!)

    Site structure logs
    As IntraSeek sends its crawlers wandering through your site, it records all erroneous or strange links it finds and creates a log from this. One log is created for each profile.

    For example, you can find information like this in the log:


    Error: These URLs couldn't be found:


    1. http://www.compa.com/products/products/webhotel/jenspage.html:
      - Linked from: http://www.compa.com/products/products/webhotel/
      - Linked from: http://www.compa.com/

    In this case, IntraSeek has found a link to http://www.compa.com/products/products/webhotel/jenspage.html that it couldn't follow. In this example, if we look at the two web pages mentioned as 'Linked from' we find a link jenspage.html on each of them.

    Faulty links are not all that is recorded in this log, IntraSeek also reports web connections that failed, pages that were reported as 'forbidden' and illegal documents (i.e, containing non-indexable data such as binary files).