Search Section

The search uses a two-step approach to balance discovery with precision:

Fuzzy Discovery - Fuse.js finds potential matches with typo tolerance and weighted field matching across titles, content, tags, and metadata.

Exact Verification - Only shows results where the search term actually appears as a substring in the content, preventing false positives like "dors" matching "doors."

This eliminates the common trade-off between search recall and precision. Users get results that handle typos while ensuring all results contain their search terms.

The search component requires the metalsmith-search plugin to be properly configured in metalsmith.js build file.

Basic Plugin Installation

import search from 'metalsmith-search';

const metalsmith = Metalsmith(__dirname)
  .source('src')
  .destination('build')
  
  // Add search plugin EARLY in pipeline for clean content
  .use(search({
    // Default configuration processes clean markdown files
    indexLevels: ['page', 'section']
  }))
  
  // Content processing happens AFTER search indexing
  .use(layouts())
  .use(collections())
  
  .build((err) => {
    if (err) throw err;
    console.log('Build complete with search index!');
  });

Plugin Position

Place the search plugin before templating is applied for optimal content extraction

Configuration Options

File Processing:

Index Configuration:

Component Processing:

Content Processing:

Performance Options:

Complete Example:

.use(search({
  // File processing
  pattern: '**/*.md',
  ignore: ['**/search.md'],

  // Index configuration
  indexPath: 'search-index.json',
  indexLevels: ['page', 'section'],
  sectionsField: 'sections',

  // Component-based processing
  autoDetectSectionTypes: true,
  sectionTypeField: 'sectionType',

  // Content processing
  stripHtml: true,
  generateAnchors: true,
  maxSectionLength: 2000,
  chunkSize: 1500,
  minSectionLength: 50,

  // Frontmatter markdown processing
  processMarkdownFields: true,
  frontmatterFields: ['summary', 'intro', 'leadIn'],

  // Enhanced Fuse.js search configuration
  fuseOptions: {
    keys: [
      { name: 'title', weight: 10 },
      { name: 'tags', weight: 8 },
      { name: 'leadIn', weight: 5 },
      { name: 'prose', weight: 3 },
      { name: 'content', weight: 1 }
    ],
    threshold: 0.3,                     // Search sensitivity (0.0 = exact, 1.0 = loose)
    includeScore: true,                 // Include relevance scores
    includeMatches: true,              // Include match highlighting data
    minMatchCharLength: 3              // Minimum characters for match
  },

  // Performance options
  batchSize: 10,
  async: false
}))

The metalsmith-search plugin intelligently handles both structured content architecture and traditional long-form content.

Structured Content Sites

For sites using structured frontmatter with component sections:

---
title: "My Page"
sections:
  - sectionType: hero
    text:
      title: "Welcome"
      leadIn: "Get started"
      prose: "This content is automatically indexed"
  - sectionType: text-only
    text:
      prose: "More searchable content here"
---

Auto-Detection Features:

• Automatically discovers component types from sectionType fields
• Generates field mappings for discovered components
• No manual configuration needed when adding new components
• Debug with: DEBUG=metalsmith-search* node metalsmith.js

Traditional Markdown Sites

For traditional long-form content:

---
title: "My Article"
summary: "Brief description with **markdown**"
tags: ["web", "development"]
---

# Article Title

Long-form content that gets automatically chunked for search...

Intelligent Processing:

• Automatic content chunking for long articles
• Heading-based section navigation
• Frontmatter markdown processing
• Tag and metadata extraction

Custom Component Field Names

If your site uses different field names for component arrays:

.use(search({
  sectionsField: 'myComponents',  // Instead of default 'sections'
  indexLevels: ['page', 'section']
}))

Common alternative field names: components, blocks, content, pageComponents

The search component's client-side Fuse.js configuration must match the fields generated by the metalsmith-search plugin.

Server-Side Index Fields

The plugin generates these searchable fields in /search-index.json:

• pageName - Page title or filename
• title - Section or page title
• leadIn - Lead-in text from frontmatter
• prose - Main content text
• content - Combined full content
• tags - Content tags array
• url - Page or section URL
• type - Content type (page/section)
• sectionType - Component type (hero, text-only, etc.)

Client-Side Fuse.js Configuration

The search component uses this optimized configuration:

const fuseOptions = {
  keys: [
    { name: 'pageName', weight: 10 },   // Highest priority
    { name: 'title', weight: 8 },       // Section titles
    { name: 'leadIn', weight: 5 },      // Lead-in text
    { name: 'prose', weight: 3 },       // Main content
    { name: 'content', weight: 1 },     // Full content
    { name: 'tags', weight: 6 }         // Content tags
  ],
  threshold: 0.3,                       // Balanced sensitivity
  includeScore: true,
  includeMatches: true,
  minMatchCharLength: 3
};

Performance Optimization

Search Relevance Filtering:

• Results below 50% relevance are automatically filtered out
• Prevents low-quality matches from cluttering results
• Maintains focus on meaningful content matches

Dynamic Library Loading:

• Fuse.js (145KB) loaded from CDN only when search component is used
• Prevents impact on initial page load for pages without search
• Follows established component pattern for script loading

Enable debug output to troubleshoot plugin behavior:

DEBUG=metalsmith-search* node metalsmith.js

This shows detailed information about:

• Files being processed and content extraction
• Auto-detected component types and field mappings
• Index generation progress and performance metrics
• Batch processing status and error handling

Common Issues & Solutions

Empty search index:

• Verify plugin runs on correct file pattern (default: **/*.md)
• Check that markdown files contain valid frontmatter structure
• Ensure component sections use expected field names

Poor search results:

• Adjust threshold in fuseOptions (lower = more strict matching)
• Modify field weights to prioritize important content types
• Verify client-side Fuse.js keys match server-side index fields

Large bundle size:

• Use stripHtml: true to remove HTML markup from content
• Increase minSectionLength to filter out short content chunks (characters)
• Enable async: true and adjust batchSize for better performance (file count)

Missing anchor links:

• Ensure generateAnchors: true in plugin configuration
• Verify section components render IDs in templates
• Check that plugin runs before permalink processing

Performance Tips

For Large Sites:

.use(search({
  async: true,                    // Enable async processing
  batchSize: 50,                  // Larger batches (file count)
  maxSectionLength: 1500,         // Optimize content chunking (characters)
  chunkSize: 1000,                // Target chunk size (characters)
  stripHtml: true,                // Reduce index size (default)
  minSectionLength: 100           // Skip short sections (characters)
}))

Production Optimization:

// Search plugin positioned early for clean content access
const metalsmith = Metalsmith(__dirname)
  .source('src')
  .destination('build')
  
  // Search plugin EARLY - always runs to maintain pipeline position
  .use(search({
    indexPath: 'search-index.json',
    // Optimize for production builds
    minSectionLength: 100,        // Skip short sections (characters)
    batchSize: 20                 // Process files in batches (file count)
  }))
  
  .use(layouts())
  .use(collections())
  
  .build((err) => {
    if (err) throw err;
    console.log('Build complete with optimized search index!');
  });