HowTo to handle SVN repositories creation and access control with Redmine » History » Version 17

Nicolas Chuche, 2008-09-09 00:28

1 1 Jean-Philippe Lang
h1. HowTo to handle SVN repositories creation and access control with Redmine
2 1 Jean-Philippe Lang
3 2 Jean-Philippe Lang
{{>TOC}}
4 2 Jean-Philippe Lang
5 17 Nicolas Chuche
*This setup is deprecated, you should try [[Repositories_access_control_with_apache_mod_dav_svn_and_mod_perl]]*
6 17 Nicolas Chuche
7 1 Jean-Philippe Lang
h2. Overview
8 1 Jean-Philippe Lang
9 1 Jean-Philippe Lang
*This setup is not required if you just need to browse your repositories and changesets from Redmine.*
10 1 Jean-Philippe Lang
11 1 Jean-Philippe Lang
As of version 0.5.0, Redmine is able to handle Subversion repositories creation and access control.
12 1 Jean-Philippe Lang
13 17 Nicolas Chuche
Once you've done this extra setup, Redmine will create the repository for each of your projects. Users will be allowed to access the repositories using ssh+svn, according to their permissions defined in Redmine :
14 1 Jean-Philippe Lang
15 1 Jean-Philippe Lang
* for public projects : read access to the repository for any user, write access for project members only,
16 1 Jean-Philippe Lang
* for private projects : read/write access allowed to project members only.
17 1 Jean-Philippe Lang
18 8 Jean-Philippe Lang
User authentication is done using the same login/password as for Redmine access.
19 1 Jean-Philippe Lang
20 1 Jean-Philippe Lang
h2. Requirements
21 1 Jean-Philippe Lang
22 1 Jean-Philippe Lang
h3. Software
23 1 Jean-Philippe Lang
24 1 Jean-Philippe Lang
You need Redmine 0.5.0 or higher, running with MySQL[1].
25 1 Jean-Philippe Lang
26 5 Jean-Philippe Lang
Your SVN repositories must be hosted on a *nix system with the following packages:
27 5 Jean-Philippe Lang
* nss_mysql
28 5 Jean-Philippe Lang
* pam_mysql 0.7pre2 or higher, compiled with SHA1 support
29 1 Jean-Philippe Lang
30 1 Jean-Philippe Lang
Scripts used in this HowTo can be found in the /extra/svn directory of Redmine.
31 1 Jean-Philippe Lang
32 5 Jean-Philippe Lang
In this HowTo, we assume that:
33 5 Jean-Philippe Lang
* the redmine database is called @redmine@ and hosted on @localhost@
34 5 Jean-Philippe Lang
* the Subversion repositories are located in @/var/svn@
35 5 Jean-Philippe Lang
36 1 Jean-Philippe Lang
h3. Network considerations
37 1 Jean-Philippe Lang
38 5 Jean-Philippe Lang
The SVN host must be able to access both the Redmine database and HTTP server(s). In many cases, they will all be located on the same host.
39 1 Jean-Philippe Lang
40 1 Jean-Philippe Lang
h2. Setup
41 1 Jean-Philippe Lang
42 5 Jean-Philippe Lang
h3. Installing requires packages
43 5 Jean-Philippe Lang
44 5 Jean-Philippe Lang
Get nss_mysql and other necessary packages:
45 5 Jean-Philippe Lang
46 14 Jean-Philippe Lang
  apt-get install build-essential libnss-mysql libpam0g-dev libssl-dev libmysqlclient15-dev
47 5 Jean-Philippe Lang
48 5 Jean-Philippe Lang
Get and build @pam_mysql@:
49 5 Jean-Philippe Lang
50 5 Jean-Philippe Lang
<pre>
51 5 Jean-Philippe Lang
$ cd /usr/src
52 5 Jean-Philippe Lang
$ wget http://prdownloads.sourceforge.net/pam-mysql/pam_mysql-0.7RC1.tar.gz
53 5 Jean-Philippe Lang
$ tar xzf pam_mysql-0.7RC1.tar.gz
54 5 Jean-Philippe Lang
$ cd pam_mysql-0.7RC1
55 5 Jean-Philippe Lang
$ ./configure --with-openssl
56 5 Jean-Philippe Lang
$ make && make install
57 5 Jean-Philippe Lang
</pre>
58 5 Jean-Philippe Lang
59 1 Jean-Philippe Lang
h3. Preparing the Redmine database
60 1 Jean-Philippe Lang
61 1 Jean-Philippe Lang
Some views need to be added to the Redmine database. These views are used to authenticate users and retrieve their permissions.
62 1 Jean-Philippe Lang
63 3 Jean-Philippe Lang
1. Create the different views in your Redmine database :
64 1 Jean-Philippe Lang
65 7 Jean-Philippe Lang
  mysql --user=root redmine -p < create_views.sql
66 1 Jean-Philippe Lang
67 7 Jean-Philippe Lang
2. Create and grant privileges to 2 new mysql users (@redmine_nss@ and @redmine_pam@):
68 1 Jean-Philippe Lang
69 1 Jean-Philippe Lang
<pre>
70 7 Jean-Philippe Lang
mysql --user=root -p
71 1 Jean-Philippe Lang
mysql> create user redmine_nss@localhost identified by 'averylongpassword';
72 1 Jean-Philippe Lang
mysql> grant SELECT on redmine.nss_groups to redmine_nss@localhost;
73 1 Jean-Philippe Lang
mysql> grant SELECT on redmine.nss_users to redmine_nss@localhost;
74 1 Jean-Philippe Lang
mysql> grant SELECT on redmine.nss_grouplist to redmine_nss@localhost;
75 4 Jean-Philippe Lang
mysql> create user redmine_pam@localhost identified by 'averylongpassword';
76 4 Jean-Philippe Lang
mysql> grant SELECT on redmine.ssh_users to redmine_pam@localhost;
77 1 Jean-Philippe Lang
</pre>
78 1 Jean-Philippe Lang
79 8 Jean-Philippe Lang
h3. Configuring nss-mysql
80 1 Jean-Philippe Lang
81 1 Jean-Philippe Lang
3. Create the /etc/nss-mysql.conf as follows:
82 1 Jean-Philippe Lang
83 1 Jean-Philippe Lang
<pre>
84 1 Jean-Philippe Lang
conf.version = 2;
85 1 Jean-Philippe Lang
users.host = inet:localhost:3306;
86 1 Jean-Philippe Lang
users.database = redmine;
87 1 Jean-Philippe Lang
users.db_user = redmine_nss;
88 4 Jean-Philippe Lang
users.db_password = averylongpassword;
89 1 Jean-Philippe Lang
users.backup_database = nss_mysql_backup;
90 1 Jean-Philippe Lang
users.table = nss_users;
91 1 Jean-Philippe Lang
users.user_column = nss_users.username;
92 9 Jean-Philippe Lang
users.userid_column = nss_users.username;
93 1 Jean-Philippe Lang
users.uid_column = nss_users.uid;
94 1 Jean-Philippe Lang
users.gid_column = 100;
95 1 Jean-Philippe Lang
users.realname_column = nss_users.realname;
96 1 Jean-Philippe Lang
users.homedir_column = "/false/path";
97 1 Jean-Philippe Lang
users.shell_column = "/usr/local/bin/svnserve.wrapper";
98 1 Jean-Philippe Lang
groups.group_info_table = nss_groups;
99 1 Jean-Philippe Lang
groups.group_name_column = nss_groups.name;
100 1 Jean-Philippe Lang
groups.groupid_column = nss_groups.gid;
101 1 Jean-Philippe Lang
groups.gid_column = nss_groups.gid;
102 1 Jean-Philippe Lang
groups.password_column = "x";
103 1 Jean-Philippe Lang
groups.members_table = nss_grouplist;
104 1 Jean-Philippe Lang
groups.member_userid_column = nss_grouplist.username;
105 1 Jean-Philippe Lang
groups.member_groupid_column = nss_grouplist.gid;
106 1 Jean-Philippe Lang
</pre>
107 1 Jean-Philippe Lang
108 1 Jean-Philippe Lang
4. Install the svnserve wrapper
109 1 Jean-Philippe Lang
110 1 Jean-Philippe Lang
  sudo install svnserve.wrapper /usr/local/bin
111 1 Jean-Philippe Lang
112 4 Jean-Philippe Lang
5. Change /etc/nsswitch.conf
113 1 Jean-Philippe Lang
114 16 Jean-Philippe Lang
Add “mysql” at the end of the two lines passwd and group like that :
115 1 Jean-Philippe Lang
116 1 Jean-Philippe Lang
<pre>
117 1 Jean-Philippe Lang
passwd:         compat mysql
118 1 Jean-Philippe Lang
group:          compat mysql
119 1 Jean-Philippe Lang
</pre>
120 1 Jean-Philippe Lang
121 1 Jean-Philippe Lang
6. Test that all this stuff works :
122 1 Jean-Philippe Lang
123 1 Jean-Philippe Lang
You must have users in some project to verify.
124 1 Jean-Philippe Lang
125 1 Jean-Philippe Lang
<pre>
126 1 Jean-Philippe Lang
% getent passwd
127 1 Jean-Philippe Lang
[...]
128 1 Jean-Philippe Lang
user1:x:5002:100:user1 user1:/false/path:/usr/local/bin/svnserve.wrapper
129 1 Jean-Philippe Lang
user2:x:5003:100:user2 user2:/false/path:/usr/local/bin/svnserve.wrapper
130 1 Jean-Philippe Lang
131 1 Jean-Philippe Lang
% getent group
132 1 Jean-Philippe Lang
[...]
133 5 Jean-Philippe Lang
project1:x:5001:
134 5 Jean-Philippe Lang
project2:x:5002:
135 1 Jean-Philippe Lang
</pre>
136 1 Jean-Philippe Lang
137 1 Jean-Philippe Lang
h3. Authorize ssh pam to use mysql
138 1 Jean-Philippe Lang
139 4 Jean-Philippe Lang
7. Add these lines in @/etc/pam.d/ssh@ :
140 1 Jean-Philippe Lang
141 4 Jean-Philippe Lang
<pre>
142 4 Jean-Philippe Lang
auth sufficient pam_mysql.so \
143 4 Jean-Philippe Lang
verbose=1 \
144 4 Jean-Philippe Lang
user=redmine_pam \
145 4 Jean-Philippe Lang
passwd=averylongpassword \
146 4 Jean-Philippe Lang
host=localhost \
147 4 Jean-Philippe Lang
db=redmine \
148 4 Jean-Philippe Lang
table=ssh_users \
149 4 Jean-Philippe Lang
usercolumn=username \
150 4 Jean-Philippe Lang
passwdcolumn=password crypt=4
151 1 Jean-Philippe Lang
152 4 Jean-Philippe Lang
account sufficient pam_mysql.so \
153 4 Jean-Philippe Lang
verbose=1 \
154 4 Jean-Philippe Lang
user=redmine_pam \
155 4 Jean-Philippe Lang
passwd=averylongpassword \
156 4 Jean-Philippe Lang
host=localhost \
157 4 Jean-Philippe Lang
db=redmine \
158 4 Jean-Philippe Lang
table=ssh_users \
159 4 Jean-Philippe Lang
usercolumn=username \
160 4 Jean-Philippe Lang
passwdcolumn=password crypt=4
161 4 Jean-Philippe Lang
162 4 Jean-Philippe Lang
password sufficient pam_mysql.so \
163 4 Jean-Philippe Lang
verbose=1 \
164 4 Jean-Philippe Lang
user=redmine_pam \
165 4 Jean-Philippe Lang
passwd=averylongpassword \
166 4 Jean-Philippe Lang
host=localhost \
167 4 Jean-Philippe Lang
db=redmine \
168 4 Jean-Philippe Lang
table=ssh_users \
169 1 Jean-Philippe Lang
usercolumn=username \
170 1 Jean-Philippe Lang
passwdcolumn=password crypt=4
171 1 Jean-Philippe Lang
</pre>
172 1 Jean-Philippe Lang
173 1 Jean-Philippe Lang
Juste before
174 1 Jean-Philippe Lang
175 1 Jean-Philippe Lang
  @include common-auth
176 1 Jean-Philippe Lang
177 1 Jean-Philippe Lang
8. Test this against an existing Redmine user
178 1 Jean-Philippe Lang
179 6 Jean-Philippe Lang
Try to connect to the SVN host using a Redmine username (eg. jsmith):
180 1 Jean-Philippe Lang
181 6 Jean-Philippe Lang
<pre>
182 6 Jean-Philippe Lang
$ ssh jsmith@localhost
183 6 Jean-Philippe Lang
jsmith@localhost's password:
184 6 Jean-Philippe Lang
Could not chdir to home directory /false/path: No such file or directory
185 6 Jean-Philippe Lang
( success ( 1 2 ( ANONYMOUS EXTERNAL ) ( edit-pipeline ) ) )
186 6 Jean-Philippe Lang
</pre>
187 6 Jean-Philippe Lang
188 6 Jean-Philippe Lang
The chdir error is the expected result.
189 1 Jean-Philippe Lang
190 1 Jean-Philippe Lang
h3. Automating repository creation
191 1 Jean-Philippe Lang
192 8 Jean-Philippe Lang
Repository creation can be automated by running periodically the reposman script.
193 1 Jean-Philippe Lang
194 5 Jean-Philippe Lang
It takes 2 arguments:
195 1 Jean-Philippe Lang
196 5 Jean-Philippe Lang
    * @svn-dir@: path to the directory where your svn repositories are located
197 1 Jean-Philippe Lang
    * @redmine-host@: host name of your Redmine install
198 1 Jean-Philippe Lang
199 15 Nicolas Chuche
Perl and Ruby versions of this script are provided but the perl version is now deprecated.
200 1 Jean-Philippe Lang
201 8 Jean-Philippe Lang
Example using the Ruby version:
202 8 Jean-Philippe Lang
203 1 Jean-Philippe Lang
<pre>
204 15 Nicolas Chuche
./reposman.rb --svn-dir=/var/svn --redmine-host=localhost
205 1 Jean-Philippe Lang
repository /var/svn/project2 created
206 1 Jean-Philippe Lang
repository /var/svn/project1 created
207 1 Jean-Philippe Lang
mode change on /var/svn/project3
208 1 Jean-Philippe Lang
</pre>
209 1 Jean-Philippe Lang
210 1 Jean-Philippe Lang
Projects are retrieved from Redmine using a SOAP web service. This web service is disabled by default in Redmine.
211 16 Jean-Philippe Lang
To enable it, go to “Administration -> Settings” and check *Enable WS for repository management*.
212 1 Jean-Philippe Lang
213 13 Nicolas Chuche
Make sure this option is checked if you get this error when running reposman:
214 13 Nicolas Chuche
@Service description 'http://localhost/sys/service.wsdl' can't be loaded: 404 Not Found@
215 13 Nicolas Chuche
216 12 Nicolas Chuche
With a recent version of redMine/reposman.rb (re. 860 and later), reposman.rb can register the new repository
217 1 Jean-Philippe Lang
in redMine so that you have nothing to do and set the owner of repository to who you want to allow browsing private
218 15 Nicolas Chuche
repository in redMine. You can do that by using the @--url@ argument :
219 1 Jean-Philippe Lang
220 15 Nicolas Chuche
<pre>
221 15 Nicolas Chuche
ruby ./reposman.rb --redmine-host localhost:3000 --svn-dir /var/svn \
222 15 Nicolas Chuche
                   --url file:///var/svn/
223 15 Nicolas Chuche
</pre>
224 15 Nicolas Chuche
225 15 Nicolas Chuche
reposman will send back to Redmine the url of your repository. *be careful* when testing, one registered, you can't change the url in redmine).
226 15 Nicolas Chuche
227 15 Nicolas Chuche
Next time you create a project, reposman will informe Redmine that the repository was created and Redmine will save the repository url.
228 15 Nicolas Chuche
This way, the administrator won't have to enter the repositories urls manually in Redmine.
229 15 Nicolas Chuche
230 5 Jean-Philippe Lang
h3. Accessing the repositories
231 5 Jean-Philippe Lang
232 10 Jean-Philippe Lang
You can now access project1 repository using this url:
233 1 Jean-Philippe Lang
234 1 Jean-Philippe Lang
  svn+ssh://svnhost/project1
235 1 Jean-Philippe Lang
236 15 Nicolas Chuche
h2. What if you want to allow Redmine to browse private repository ?
237 15 Nicolas Chuche
238 15 Nicolas Chuche
The previous recipes allow you to create repository on the fly and anonymous browsing. But, if your project is private or if the project isn't on the same server, you won't be able to browse it in Redmine.
239 15 Nicolas Chuche
240 15 Nicolas Chuche
h3. Redmine and svn are on the same server
241 15 Nicolas Chuche
242 15 Nicolas Chuche
In this case, you just need to use the @--url@ option like in the previous item to register the repository and the @--owner@ argument to set the repository owner to the mongrel/apache user so that it can access the repositories.
243 15 Nicolas Chuche
244 15 Nicolas Chuche
<pre>
245 15 Nicolas Chuche
ruby ./reposman.rb --redmine-host localhost:3000 --svn-dir /var/svn \
246 15 Nicolas Chuche
                   --url file:///var/svn/ --owner MONGREL_USER
247 15 Nicolas Chuche
</pre>
248 15 Nicolas Chuche
249 15 Nicolas Chuche
BUT, you won't be able to separate svn and Redmine hosts in the future (in fact you will be able to but you would have to manually update the repositories urls in the database and that's bad). A better way to do this, if you think you will need to separate those two servers one day, is to do like you already have two servers. To do this, read the next recipe.
250 15 Nicolas Chuche
251 15 Nicolas Chuche
h3. Redmine and svn aren't on the same server
252 15 Nicolas Chuche
253 15 Nicolas Chuche
There's more than one way to do this, one could be to use a specific user to browse the repository with svnserve or svn+ssh but I don't like this way (don't ask why). Another way is to add a third access way (we already have svn+ssh for registered users and svnserve for anonymous users).
254 15 Nicolas Chuche
255 15 Nicolas Chuche
In the following, the Redmine server is known as redmine.my.domain and the svn as svn.my.domain. You need to have apache/apache2 and mod_dav_svn on the svn server.
256 15 Nicolas Chuche
257 15 Nicolas Chuche
1. configure your apache to serve the svn repository just for the Redmine server
258 15 Nicolas Chuche
259 15 Nicolas Chuche
Just add something like that in your @apache.conf@ or in a file in the directory @/etc/apache/conf.d@:
260 15 Nicolas Chuche
261 15 Nicolas Chuche
<pre>
262 15 Nicolas Chuche
   LoadModule dav_svn_module /usr/lib/apache2/modules/mod_dav_svn.so
263 15 Nicolas Chuche
   <Location /svn>
264 15 Nicolas Chuche
   DAV svn
265 15 Nicolas Chuche
   # this must be the path you give to reposman with -s,--svn-dir argument
266 15 Nicolas Chuche
   SVNParentPath "/var/svn"
267 15 Nicolas Chuche
   Order allow,deny
268 15 Nicolas Chuche
   Allow from ip.of.my.redmine.server
269 15 Nicolas Chuche
   </Location>
270 15 Nicolas Chuche
</pre>
271 15 Nicolas Chuche
272 15 Nicolas Chuche
Verify you can access it from your Redmine server.
273 15 Nicolas Chuche
274 15 Nicolas Chuche
2. change your reposman cron by adding the @--owner@ argument with the apache user :
275 15 Nicolas Chuche
276 15 Nicolas Chuche
<pre>
277 15 Nicolas Chuche
ruby ./reposman.rb --redmine-host http://redmine.my.domain/ --svn-dir /var/svn
278 15 Nicolas Chuche
                   --url http://svn.my.domain/svn/ --owner APACHE_USER
279 15 Nicolas Chuche
</pre>
280 15 Nicolas Chuche
281 15 Nicolas Chuche
h2. Web Service and Security
282 15 Nicolas Chuche
283 15 Nicolas Chuche
For the moment, the WS is open to everybody once actived and you surely don't want that someone register project for you. You can block access to the WS with apache/mongrel (if you don't use apache, I let you do your homework...) with the Location apache directive like this :
284 15 Nicolas Chuche
285 15 Nicolas Chuche
<pre>
286 15 Nicolas Chuche
<VirtualHost *:80>
287 15 Nicolas Chuche
   ServerName redmine.my.domain
288 15 Nicolas Chuche
   ServerAdmin webmaster@localhost
289 15 Nicolas Chuche
290 15 Nicolas Chuche
   <Location /sys>
291 15 Nicolas Chuche
      Order allow,deny
292 15 Nicolas Chuche
      Allow from ip.of.my.svn.server
293 15 Nicolas Chuche
   </Location>
294 15 Nicolas Chuche
295 15 Nicolas Chuche
   ProxyPass / http://localhost:3000/
296 15 Nicolas Chuche
   ProxyPassReverse / http://localhost:3000/
297 15 Nicolas Chuche
</VirtualHost>
298 15 Nicolas Chuche
</pre>
299 5 Jean-Philippe Lang
300 16 Jean-Philippe Lang
fn1. Other databases can not be used because of various problems: no pam module, no sha1 handling,...