Crawler
Patterns for web pages to ignore.
string[]
micromatch
{ exclusionPatterns: [ // Exclude all `/about` pages are except the `/about/ny` one. "http://www.google.com/about/**", "!http://www.google.com/about/ny", // Exclude all pages ending with `.html` "**.html", ]; }
Was this page helpful?