Feature #27876
Add project id to robots.txt
| Status: | Closed | Start date: | ||
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% | |
| Category: | SEO | |||
| Target version: | 4.0.0 | |||
| Resolution: | Fixed |
Description
Bots found activities by project number and traverse to the whole history:
redmine:443 66.249.76.119 - - [02/Jan/2018:05:42:49 +0000] "GET /redmine/projects/3/activity?from=1979-03-13&show_files=1&user_id=3 HTTP/1.1" 200 8090 "-" "Mozilla/5.0 (compatible; Googlebot/2.1;)"
robots.txt looks like:
User-agent: * Disallow: /redmine/projects/support/repository Disallow: /redmine/projects/support/issues Disallow: /redmine/projects/support/activity Disallow: /redmine/issues/gantt Disallow: /redmine/issues/calendar Disallow: /redmine/activity Disallow: /redmine/search
Expected:
User-agent: * Disallow: /redmine/projects/support/repository Disallow: /redmine/projects/support/issues Disallow: /redmine/projects/support/activity Disallow: /redmine/projects/3/repository Disallow: /redmine/projects/3/issues Disallow: /redmine/projects/3/activity Disallow: /redmine/issues/gantt Disallow: /redmine/issues/calendar Disallow: /redmine/activity Disallow: /redmine/search
robot.diff 
(633 Bytes)
Associated revisions
add project id to robots.txt (#27876)
History
#1
Updated by Grischa Zengel over 3 years ago
I thinks it's better to block the whole numeric project:
User-agent: * Disallow: /redmine/projects/support/repository Disallow: /redmine/projects/support/issues Disallow: /redmine/projects/support/activity Disallow: /redmine/projects/3 Disallow: /redmine/issues/gantt Disallow: /redmine/issues/calendar Disallow: /redmine/activity Disallow: /redmine/search
#2
Updated by Toshi MARUYAMA over 3 years ago
- Tracker changed from Defect to Feature
#4
Updated by Grischa Zengel over 3 years ago
It works. Thanks.
User-agent: * Disallow: /redmine/projects/support/repository Disallow: /redmine/projects/support/issues Disallow: /redmine/projects/support/activity Disallow: /redmine/projects/3/repository Disallow: /redmine/projects/3/issues Disallow: /redmine/projects/3/activity Disallow: /redmine/issues/gantt Disallow: /redmine/issues/calendar Disallow: /redmine/activity Disallow: /redmine/search
#5
Updated by Toshi MARUYAMA over 3 years ago
- Subject changed from add numerical project to robots.txt to add project id to robots.txt
#6
Updated by Toshi MARUYAMA over 3 years ago
- Status changed from New to Closed
- Target version set to 4.0.0
- Resolution set to Fixed
I think it is no problem because project id is disclosed in project.xml and robots.txt hides non public projects.
source:trunk/app/controllers/welcome_controller.rb@17189#L26
#7
Updated by Go MAEDA about 3 years ago
- Category set to SEO
#8
Updated by Marius BALTEANU almost 3 years ago
- Subject changed from add project id to robots.txt to Add project id to robots.txt