Home | HanG321 Blog

upload file on nginx – part 1: compile dynamic modules

June 20, 2017|CentOS, Linux, nginx|科技資訊, 電腦編程

Recently I was assigned to a task about adding a file upload feature to a existing nginx server on CentOS. Nginx was chosen years ago because it simply handles some static files only. If the file upload was a requirement, using apache is simpler as computation power isn’t a constraint in our case. Background CentOS 7 nginx 1.10.2 nginx-upload-module ngx_http_auth_pam_module Compile Dynamic modules Download upload and auth pam module source code. Then find out the current nginx version, download the source code and compile modules

## download module source code
$ wget https://github.com/sto/ngx_http_auth_pam_module/archive/master.zip -O ngx_http_auth_pam_module.zip
$ unzip ngx_http_auth_pam_module.zip
$ mv ngx_http_auth_pam_module-master ngx_http_auth_pam_module

$ wget https://github.com/vkholodkov/nginx-upload-module/archive/master.zip -O nginx-upload-module.zip
$ upzip nginx-upload-module.zip
$ mv nginx-upload-module-master nginx-upload-module
## modify the "config" file to support dynamic module,
## see https://github.com/vkholodkov/nginx-upload-module/issues/78

## install compile dependencies
$ sudo yum -y install pcre-devel zlib-devel pam-devel  libxslt-devel perl-ExtUtils-Embed GeoIP-devel \
 redhat-rpm-config openssl-devel gd-devel gperftools-devel gcc-c++

## find out the current version of nginx
$ nginx -v

## download source code
$ wget&nbsp;http://nginx.org/download/nginx-1.10.2.tar.gz

## list all compile parameters
$ nginx -V
## e.g.
configure arguments:
--prefix=/usr/share/nginx
--sbin-path=/usr/sbin/nginx
--modules-path=/usr/lib64/nginx/modules
--conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--http-client-body-temp-path=/var/lib/nginx/tmp/client_body
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi
--pid-path=/run/nginx.pid
--lock-path=/run/lock/subsys/nginx
--user=nginx
--group=nginx
--with-file-aio
--with-ipv6
--with-http_ssl_module
--with-http_v2_module
--with-http_realip_module
--with-http_addition_module
--with-http_xslt_module=dynamic
--with-http_image_filter_module=dynamic
--with-http_geoip_module=dynamic
--with-http_sub_module
--with-http_dav_module
--with-http_flv_module
--with-http_mp4_module
--with-http_gunzip_module
--with-http_gzip_static_module
--with-http_random_index_module
--with-http_secure_link_module
--with-http_degradation_module
--with-http_slice_module
--with-http_stub_status_module
--with-http_perl_module=dynamic
--with-mail=dynamic
--with-mail_ssl_module
--with-pcre
--with-pcre-jit
--with-stream=dynamic
--with-stream_ssl_module
--with-google_perftools_module
--with-debug
--with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
  --param=ssp-buffer-size=4 -grecord-gcc-switches
  -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'
--with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

## reuse the configs output from "nginx -V", now compile http auth pam module
$ tar zxvf nginx-1.10.2.tar.gz
$ cd nginx-1.10.2
$ ./configure --prefix=/usr/share/nginx ........... -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' \
 --add-dynamic-module=../ngx_http_auth_pam_module
$ make modules
## upload to to server
$ scp objs/ngx_http_auth_pam_module.so user@host:/usr/lib64/nginx/modules/

## repeat for nginx-upload-module
$ ./configure --prefix=/usr/share/nginx ........... -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' \
 --add-dynamic-module=../nginx-upload-module
$ make modules
$ scp objs/ngx_http_upload_module.so user@host:/usr/lib64/nginx/modules/

## create configs
echo 'load_module "/usr/lib64/nginx/modules/ngx_http_auth_pam_module.so";' >  /usr/share/nginx/modules/mod-http-auth-pam.conf
echo 'load_module "/usr/lib64/nginx/modules/ngx_http_upload_module.so";' >  /usr/share/nginx/modules/mod-http-upload.conf

## to validate config
$ nginx -tc /etc/nginx/nginx.conf

## download module source code

$ wget https://github.com/sto/ngx_http_auth_pam_module/archive/master.zip -O ngx_http_auth_pam_module.zip

$ unzip ngx_http_auth_pam_module.zip

$ mv ngx_http_auth_pam_module-master ngx_http_auth_pam_module

$ wget https://github.com/vkholodkov/nginx-upload-module/archive/master.zip -O nginx-upload-module.zip

$ upzip nginx-upload-module.zip

$ mv nginx-upload-module-master nginx-upload-module

## modify the "config" file to support dynamic module,

## see https://github.com/vkholodkov/nginx-upload-module/issues/78

## install compile dependencies

$ sudo yum -y install pcre-devel zlib-devel pam-devel libxslt-devel perl-ExtUtils-Embed GeoIP-devel \

redhat-rpm-config openssl-devel gd-devel gperftools-devel gcc-c++

## find out the current version of nginx

$ nginx -v

## download source code

$ wget http://nginx.org/download/nginx-1.10.2.tar.gz

## list all compile parameters

$ nginx -V

## e.g.

configure arguments:

--prefix=/usr/share/nginx

--sbin-path=/usr/sbin/nginx

--modules-path=/usr/lib64/nginx/modules

--conf-path=/etc/nginx/nginx.conf

--error-log-path=/var/log/nginx/error.log

--http-log-path=/var/log/nginx/access.log

--http-client-body-temp-path=/var/lib/nginx/tmp/client_body

--http-proxy-temp-path=/var/lib/nginx/tmp/proxy

--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi

--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi

--http-scgi-temp-path=/var/lib/nginx/tmp/scgi

--pid-path=/run/nginx.pid

--lock-path=/run/lock/subsys/nginx

--user=nginx

--group=nginx

--with-file-aio

--with-ipv6

--with-http_ssl_module

--with-http_v2_module

--with-http_realip_module

--with-http_addition_module

--with-http_xslt_module=dynamic

--with-http_image_filter_module=dynamic

--with-http_geoip_module=dynamic

--with-http_sub_module

--with-http_dav_module

--with-http_flv_module

--with-http_mp4_module

--with-http_gunzip_module

--with-http_gzip_static_module

--with-http_random_index_module

--with-http_secure_link_module

--with-http_degradation_module

--with-http_slice_module

--with-http_stub_status_module

--with-http_perl_module=dynamic

--with-mail=dynamic

--with-mail_ssl_module

--with-pcre

--with-pcre-jit

--with-stream=dynamic

--with-stream_ssl_module

--with-google_perftools_module

--with-debug

--with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong

--param=ssp-buffer-size=4 -grecord-gcc-switches

-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'

--with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

## reuse the configs output from "nginx -V", now compile http auth pam module

$ tar zxvf nginx-1.10.2.tar.gz

$ cd nginx-1.10.2

$ ./configure --prefix=/usr/share/nginx ........... -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' \

--add-dynamic-module=../ngx_http_auth_pam_module

$ make modules

## upload to to server

$ scp objs/ngx_http_auth_pam_module.so user@host:/usr/lib64/nginx/modules/

## repeat for nginx-upload-module

$ ./configure --prefix=/usr/share/nginx ........... -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' \

--add-dynamic-module=../nginx-upload-module

$ make modules

$ scp objs/ngx_http_upload_module.so user@host:/usr/lib64/nginx/modules/

## create configs

echo 'load_module "/usr/lib64/nginx/modules/ngx_http_auth_pam_module.so";' > /usr/share/nginx/modules/mod-http-auth-pam.conf

echo 'load_module "/usr/lib64/nginx/modules/ngx_http_upload_module.so";' > /usr/share/nginx/modules/mod-http-upload.conf

## to validate config

$ nginx -tc /etc/nginx/nginx.conf

HT-E5500W 跳Input

December 26, 2016|生活記事

今個Xmas 假期找到原因了, 原來係touch input 既問題. 宜家Streaming DLNA 唔使睇睇下冇聲 (input source switching automatically) http://www.avsforum.com/forum/109-home-theater-box/1481206-help-samsung-ht-e5500-keeps-changing-source-2.html#post23923247

convert .mkv to .mp4 without transcoding

October 3, 2016|codec, DSM, Linux, mint, XPEnology|科技資訊, 音樂電影

I have a Samsung smart TV, it can play AAC video format, but only with mp4 container, mkv file format is NOT supported. Here is a simple way to convert them. materials: synology linux 1. create NFS share on Synology Shared Folder > Edit > NFS Permission hostname/IP: 192.168.x.0/24 (allow whole subnet) then press “OK” 2. mount NFS from mint install nfs client and converter on Mint

sudo apt-get update
sudo apt-get install nfs-common rpcbind libav-tools

1 2	sudo apt-get update sudo apt-get install nfs-common rpcbind libav-tools

mount [Synology NAS IP address] : [mount path of shared folder] / [mount point on NFS client]

sudo mount diskstation:/volume2/data /mnt/dsdata

1	sudo mount diskstation:/volume2/data /mnt/dsdata

3. convert file

avconv -i input.mkv -c:v copy -c:a copy output.mp4

# batch convert
find /path -name '*.mkv' -not -path "*/@eaDir/*" -exec bash -c 'avconv -i "$1" -c:v copy -c:a copy "${1%.mkv}.mp4"' _ {} \;

avconv -i input.mkv -c:v copy -c:a copy output.mp4

# batch convert

find /path -name '*.mkv' -not -path "*/@eaDir/*" -exec bash -c 'avconv -i "$1" -c:v copy -c:a copy "${1%.mkv}.mp4"' _ {} \;

if your linux distro support ffmpeg other than avconv (libav-tools), use

ffmpeg -i input.mkv -codec copy output.mp4

1	ffmpeg -i input.mkv -codec copy output.mp4

reference: https://www.synology.com/en-global/knowledgebase/DSM/tutorial/File_Sharing/How_to_access_files_on_Synology_NAS_within_the_local_network_NFS http://askubuntu.com/questions/50433/how-to-convert-mkv-file-into-mp4-file-losslessly

...678 9...30...