2011-12-14 20:05:27 +01:00
|
|
|
# HOW TO EDIT THIS FILE:
|
2013-11-22 08:29:22 +01:00
|
|
|
# The "handy ruler" below makes it easier to edit a package description.
|
|
|
|
# Line up the first '|' above the ':' following the base package name, and
|
|
|
|
# the '|' on the right side marks the last column you can put a character in.
|
|
|
|
# You must make exactly 11 lines for the formatting to be correct. It's also
|
|
|
|
# customary to leave one space after the ':' except on otherwise blank lines.
|
2012-05-19 22:59:44 +02:00
|
|
|
|
|
|
|
|-----handy-ruler------------------------------------------------------|
|
2011-12-14 20:05:27 +01:00
|
|
|
perl-www-robotrules: perl-www-robotrules (DB of robots.txt-derived perms)
|
|
|
|
perl-www-robotrules:
|
2012-05-19 22:59:44 +02:00
|
|
|
perl-www-robotrules: This perl module parses /robots.txt files as specified in "A
|
|
|
|
perl-www-robotrules: Standard for Robot Exclusion". Webmasters can use the /robots.txt
|
|
|
|
perl-www-robotrules: file to forbid conforming robots from accessing parts of their
|
|
|
|
perl-www-robotrules: web site
|
2011-12-14 20:05:27 +01:00
|
|
|
perl-www-robotrules:
|
2018-05-27 04:38:09 +02:00
|
|
|
perl-www-robotrules: https://metacpan.org/release/WWW-RobotRules
|
2011-12-14 20:05:27 +01:00
|
|
|
perl-www-robotrules:
|
|
|
|
perl-www-robotrules:
|
|
|
|
perl-www-robotrules:
|