enabling x_sendfile to efficiently send large files
Added by Barrett Lawson almost 13 years ago
Hi folks,
I had some problems with downloading large (>500MB) files using the "files" module. I've worked it out and thought I should share my solution.
My server is a win32 box running the BitNami stack: apache proxied to two mongrel instances.
Problem¶
- I would try to download the file and eventually get a 500 server error.
- redmine log showed that it handled the page fine
- apache log showed 200 OK for the GET of the files page.
- but one of the mongrel logs has lines like:
Thu Sep 22 16:24:35 -0400 2011: Error calling Dispatcher.dispatch #<NoMemoryError: failed to allocate memory>
Right, duh, it's a big file... so lets go see what the issue is.
Blamed on the ruby garbage collector:
http://www.redmine.org/boards/2/topics/13815?r=18407
Good information here, too:
http://www.ruby-forum.com/topic/134897
Or perhaps that rails is caching the file when passing it along to mongrel:
http://www.therailsway.com/2009/2/22/file-downloads-done-right#comments
the latter suggests using an X-sendfile header. Apache and lighttpd (can) support this header, which tells the server to ignore all output and just send the file. This eliminates unnecessary caching of the file to be downloaded.
worth noting: older search results on this issue suggest a ruby plugin that is deprecated by the built in support. Don't waste your time.
Solution¶
Configure Apache to support X_Sendfile¶
*install mod_xsendfile: I'm lazy and used a compiled binary from http://www.apachelounge.com/download/
*add it to httpd.conf
*configure it to run. I had to add:
<IfModule xsendfile_module>
XSendFile on
XSendFilePath "C:/Program Files/BitNami Redmine Stack/apps/redmine/files"
</IfModule>
Edit the attachments controller to use x_sendfile:¶
- add ":x_sendfile=>true" to the end of "def download" in attachments_controller.rb
Patch?¶
It seems like it would be easy enough to patch RM to create an option to enable x_sendfile for attachments. I'm new here, so should I do this or create an issue, or what?
Replies (8)
RE: enabling x_sendfile to efficiently send large files
-
Added by Aidin Abedi about 12 years ago
How did you upload large files from the beginning? Did you import them into redmine from the local filesystem?
RE: enabling x_sendfile to efficiently send large files
-
Added by Hank Hill almost 12 years ago
Good stuff! However, there is a change in Rails 3.
Now, instead of ":x_sendfile=>true", do the following:
In your production.rb, add the following lines:
# Specifies the header that your server uses for sending files config.action_dispatch.x_sendfile_header = "X-Sendfile"
And add to the apache directives (along with XSendFile on):
RequestHeader Set X-Sendfile-Type X-Sendfile
In CentOS/Fedora/RH, you can get mod_xsendfile from the epel repo.
RE: enabling x_sendfile to efficiently send large files
-
Added by @ go2null over 11 years ago
I did the following:
- I added the following section to
sites-available/mysite
RequestHeader Set X-Sendfile-Type X-Sendfile <IfModule xsendfile_module> XSendFile on XSendFilePath " /home/redmine/www/current/files" </IfModule>
- and the following to
/config/environments/production.rb
# Specifies the header that your server uses for sending files config.action_dispatch.x_sendfile_header = "X-Sendfile"
- I also installed
libapache2-mod-xsendfile
from Debian Sid (squeeze has 0.92, sid has 0.12 - which supportsXSendFilePath
) - and enabled
mod_headers
(forRequestHeader
).
The error I'm getting is:
File Not Found The file https://mysite/attachments/download/54/myfile.zip cannot be found. Please check the location and try again.
production.log
has this entry:
Started GET "/attachments/download/54/myfile.zip" for 10.10.9.99 at 2012-12-12 10:20:16 -0500 Processing by AttachmentsController#download as HTML Parameters: {"id"=>"54", "filename"=>"myfile.zip"} Current user: me (id=3) Sent file /home/redmine/www/releases/20121209221527/files/121212102012_myfile.zip (0.1ms) Completed 200 OK in 7ms (ActiveRecord: 1.9ms)
Any help appreciated.
I'm running Redmine on Debian 6 Squeeze
Environment: Redmine version 2.1.0.stable Ruby version 1.9.2 (x86_64-linux) Rails version 3.2.8 Environment production Database adapter Mysql2
RE: enabling x_sendfile to efficiently send large files
-
Added by @ go2null over 11 years ago
- Typo (spaces before path)
XSendFilePath " /home/redmine/www/current/files"
current
is a symlink, and web links point to actual file location.
RE: enabling x_sendfile to efficiently send large files
-
Added by @ go2null about 11 years ago
If I had read this, it would have prevented error 2.
2. current is a symlink, and web links point to actual file location.
RE: enabling x_sendfile to efficiently send large files
-
Added by Karel Pičman over 7 years ago
I've been using this feature for many years. In fact downloading of large files from Redmine is impossible without that option.
A few days ago I found that it's working no more. Although the setting remains without any change.
I've set in my additional_environment.rb:
# Specifies the header that your server uses for sending files config.action_dispatch.x_sendfile_header = "X-Sendfile"
But, this option is simply ignored and requests for files download doesn't contain the header X-Sendfile and are consequently processed by Redmine instead of the webserver.
Tested in Redmine 3.3.1. I've also tested it with a clear Redmine installation (without any plugins) with the same result.
Any ideas?
RE: enabling x_sendfile to efficiently send large files
-
Added by Toshi MARUYAMA over 7 years ago
FTR: #24646
RE: enabling x_sendfile to efficiently send large files
-
Added by Karel Pičman over 7 years ago
Yes, fixed and working again :-)