Mass File Upload by mask from server
Added by Valeriy Zhuk over 1 year ago
Hello everyone. In version 5.0.2, my script has stopped working, which used to add information about files to the database, allowing them to be downloaded later. It was a collection of daily builds of applications, where files were uploaded to the appropriate section based on specific "masks." Currently, only file information is being uploaded, but when attempting to download, a 404 error occurs. If files are attached manually, everything works fine. I am attaching the scripts. Please help. I'm sure that if it worked in older versions, it should work here as well.
Environment:
Redmine version 5.0.2.stable
Ruby version 2.7.4-p191 (2021-07-07) [x86_64-linux-gnu]
Rails version 6.1.6
Environment production
Database adapter Mysql2
Mailer queue ActiveJob::QueueAdapters::AsyncAdapter
Mailer delivery sendmail
Redmine settings:
Redmine theme Purplemine2-master (includes JavaScript)
SCM:
Subversion 1.14.1
Git 2.30.2
Filesystem
Redmine plugins:
no plugin installed
The main script is executed via a cron job, which in turn calls other scripts. I will simply refer to them as 1, 2, and 3 here.
1.sh
chmod -R g+rwx /home/devel/distr/*
echo Generating sql for distr...
cd /home/devel/distr/
echo ' deleting sql...'
/opt/redmine/1-update-distr-del-sql.sh > /opt/redmine/update-distr-del.sql
echo ' inserting sql...'
/opt/redmine/2-update-distr-ins-sql.sh > /opt/redmine/update-distr-ins.sql
echo Generating sql for nighlty distr...
cd /home/devel/distr/nightly
echo ' deleting sql...'
/opt/redmine/1-update-distr-del-sql.sh >> /opt/redmine/update-distr-del.sql
echo ' inserting sql...'
/opt/redmine/2-update-distr-ins-sql.sh >> /opt/redmine/update-distr-ins.sql
echo Exectuing deleting sql for distr...
/usr/bin/mysql --user=root --password=strondpass redmine < /opt/redmine/update-distr-del.sql
echo Exectuing inserting sql for distr...
/usr/bin/mysql --user=root --password=strongpass redmine < /opt/redmine/update-distr-ins.sql
echo Deleting older than 4 days nightly distr...
find /home/devel/distr/nightly/* -mtime +4 -exec rm {} \;
echo Done.
2.sh
for f in *
do
stat -c '%y %n %s' $f | cut --complement -b '19-34' | awk '{print "\
delete from attachments\
where\
container_type = \"Version\" and\
container_id in (\
select\
v.id as v_id\
from\
versions v, projects p\
where\
v.project_id = p.id and\
\"" $3 "\" regexp concat(\"^\", p.identifier, \"-\", v.name, \"[[:alnum:]*.period[:alnum:]*]*[[.hyphen.][:alnum:]*]*[.period](exe|deb|rpm)\")\
)\
;\
"}'
done
3.sh
for f in *
do
if [ -f $f ]; then
#
# Replace this command with bellow to enable md5 checksum.
#
# stat -t '%Y-%m-%d %H:%M:%S' $f | /sbin/md5 -qp $f | awk 'BEGIN {RS="";FS=" |\n"} {print "\
#
# Also add '\"" $22 "\",\' after value "application/octet-stream".
#
stat -c '%y %n %s' $f | cut --complement -b '19-34' | awk 'BEGIN {RS="";FS=" |\n"} {print "\
insert into attachments (\
container_id,\
container_type,\
filename,\
disk_filename,\
filesize,\
content_type,\
digest,\
downloads,\
author_id,\
created_on,\
description\
) values (\
(\
select\
v.id as v_id\
from\
versions v, projects p\
where\
v.project_id = p.id and\
\"" $3 "\" regexp concat(\"^\", p.identifier, \"-\", v.name, \"[[:alnum:]*.period[:alnum:]*]*[[.hyphen.][:alnum:]*]*[.period](exe|deb|rpm)\")\
),\
\"Version\",\
\"" $3 "\",\
\"" ENVIRON["PWD"] "/" $3 "\",\
" $4 ",\
\"application/octet-stream\",\
\"""\",\
0,\
3,\
\"" $1 " " $2 "\",\
\"" $3 "\"\
);\
"}'
fi
done
Old distributions are being deleted first, and then new ones are uploaded. This used to work in older versions, but not anymore. You thought that maybe an additional field is required – you checked the database table, and there haven't been any significant changes here.
It's possible that additional "parameters" need to be specified for the files, but I've already hit a dead end.