Github Upload File and Expose as Url
Want to conversation with us ? Telegram channel : https://t.me/plik_rootgg
Plik
Plik is a scalable & friendly temporary file upload system (Wetransfer like) in golang.
Chief features
- Powerful control line customer
- Easy to use web UI
- Multiple data backend : File, OpenStack Swift, S3, Google Deject Storage
- Multiple metadata backend : Sqlite3, PostgreSQL, MySQL
- OneShot : Files are destructed after the first download
- Stream : Files are streamed from the uploader to the downloader (nothing stored server side)
- Removable : Give the ability to the uploader to remove files at any time
- TTL : Custom expiration date
- Password : Protect upload with login/pasgisword (Auth Basic)
- Comments : Add together custom message (in Markdown format)
- User authentication : Local / Google / OVH
- Upload restriction : Source IP / Token
- Administrator CLI and web UI
- Server side encryption (with S3 data backend)
- ShareX Uploader : Directly integrated into ShareX
- plikSharp : A .Net API client for Plik
- Filelink for Plik : Thunderbird Addon to upload attachments to Plik
Content Table
- Installation
- Configuration
- Data Backends
- Metadata Backends
- Customer CLI
- Go Customer
- HTTP API
- Admin CLI
- Hallmark
- Security
- Cross Compilation
- FAQ
- How To Contribute
Installation
From release
To run plik, it'southward very elementary :
$ wget https://github.com/root-gg/plik/releases/download/1.3.5/plik-ane.3.5-linux-amd64.tar.gz $ tar xzvf plik-1.three.5-linux-amd64.tar.gz $ cd plik-1.3.5-linux-amd64/server $ ./plikd Et voilà ! You now have a fully functional instance of Plik running on http://127.0.0.1:8080.
You can edit server/plikd.cfg to arrange the configuration to your needs (ports, ssl, ttl, backend params,...)
From sources
To compile plik from sources, you'll need golang and npm installed on your organization
Git clone or go become the project and simply run make :
$ make $ cd server && ./plikd Docker
Plik comes with multiarch docker images congenital for linux amd64/i386/arm/arm64:
- rootgg/plik:latest (latest release)
- rootgg/plik:{version} (release)
- rootgg/plik:dev (latest commit of principal)
Run into the Plik Docker reference
Plik also comes with some useful scripts to exam backend in standalone docker instances :
See the Plik Docker backend testing
Configuration
The configuration is managed using a TOML file plikd.cfg
Defining configuration parameters using environment variables
One can specify configuration parameters using env variable with the configuration parameter in screaming snake case
PLIKD_DEBUG_REQUESTS=truthful ./plikd For Arrays and config maps they must be provided in json format. Arrays are overridden but maps are merged
PLIKD_DATA_BACKEND_CONFIG='{"Directory":"/var/files"}' ./plikd Data backends
Plik is shipped with multiple data backend for uploaded files and metadata backend for the upload metadata.
- File databackend :
Shop uploaded files in a local or mounted file organization directory.
- Openstack Swift databackend : http://docs.openstack.org/developer/swift/
Openstack Swift is a highly available, distributed, eventually consistent object/blob shop which supports Server Side Encryption
-
Amazon S3
-
Google Cloud Storage
Metadata backends
- Sqlite3
Suitable for standalone deployment.
- PostgreSQL / Mysql
Suitable for distributed / High Availability deployment.
Cli client
Plik is shipped with a powerful golang multiplatform cli client (downloadable in web interface) :
Usage: plik [options] [FILE] ... Options: -h --help Testify this help -d --debug Enable debug mode -q --quiet Enable serenity mode -o, --oneshot Enable OneShot ( Each file volition exist deleted on first download ) -r, --removable Enable Removable upload ( Each file can be deleted by anyone at anymoment ) -S, --stream Enable Streaming ( It will block until remote user starts downloading ) -t, --ttl TTL Time before expiration (Upload will be removed in m|h|d) -n, --name Proper noun Set file proper name when piping from STDIN --server SERVER Overrides plik url --token TOKEN Specify an upload token --comments Annotate Set up comments of the upload ( MarkDown compatible ) -p Protect the upload with login and password --password PASSWD Protect the upload with login:countersign ( if omitted default login is "plik" ) -a Archive upload using default archive params ( encounter ~/.plikrc ) --archive MODE Annal upload using specified archive backend : tar|cipher --compress Mode [tar] Compression codec : gzip|bzip2|xz|lzip|lzma|lzop|compress|no --archive-options OPTIONS [tar|cipher] Boosted command line options -south Encrypt upload usnig default encrypt params ( see ~/.plikrc ) --not-secure Do non encrypt upload regardless of ~/.plikrc configurations --secure Mode Annal upload using specified archive backend : openssl|pgp --cipher Null [openssl] Openssl cipher to employ ( run across openssl assistance ) --passphrase PASSPHRASE [openssl] Passphrase or '-' to exist prompted for a passphrase --recipient RECIPIENT [pgp] Set recipient for pgp backend ( case : --recipient Bob ) --secure-options OPTIONS [openssl|pgp] Additional command line options --update Update client -five --version Prove client version For example to create directory tar.gz archive and encrypt it with openssl :
$ plik -a -s mydirectory/ Passphrase : 30ICoKdFeoKaKNdnFf36n0kMH Upload successfully created : https://127.0.0.ane:8080/ #/?id=0KfNj6eMb93ilCrl mydirectory.tar.gz : 15.lxx MB 5.92 MB/south Commands : curl -s 'https://127.0.0.1:8080/file/0KfNj6eMb93ilCrl/q73tEBEqM04b22GP/mydirectory.tar.gz' | openssl aes-256-cbc -d -pass laissez passer:30ICoKdFeoKaKNdnFf36n0kMH | tar xvf - --gzip Client configuration and preferences are stored at ~/.plikrc or /etc/plik/plikrc ( overridable with PLIKRC environement variable )
Quick upload using whorl only
curl --grade 'file=@/path/to/file' http://127.0.0.1:8080 When Hallmark is used and NoAnonymousUploads are enabled you tin quick upload using user tokens:
gyre --grade 'file=@/path/to/file' --header 'X-PlikToken: xxxx-xxx-xxxx-xxxxx-xxxxxxxx' http://127.0.0.one:8080 DownloadDomain configuration option must exist set for this to properly work.
Get customer
Plik now comes with a golang library to a higher place which the cli customer is built
Come across the Plik library reference
API
Plik server betrayal a HTTP API to manage uploads and become files :
See the Plik API reference
Admin CLI
Using the ./plikd server binary it'southward possible to :
- create/list/delete local accounts
- create/listing/delete user CLI tokens
- create/listing/delete files and uploads
- import / export metadata
Run across help for more details
Authentication
Plik tin cosign users using Local accounts or using Google or OVH APIs.
If source IP address restriction is enabled, user accounts can only be created from trusted IPs and so authenticated users can upload files without source IP brake.
Information technology possible to deny unauthenticated uploads totally ( NoAnonymousUploads ).
Admin users tin can access the admin dashboard and dispense every uploads.
-
Local :
- You tin manipulate local users with the server control line
$ ./plikd --config ./plikd.cfg user create --login root --proper name Admin --admin Generated password for user root is 08ybEyh2KkiMho8dzpdQaJZm78HmvWGC -
Google :
- You'll need to create a new awarding in the Google Developper Console
- You'll be handed a Google API ClientID and a Google API ClientSecret that you'll need to put in the plikd.cfg file.
- Do not forget to whitelist valid origin and redirect url ( https://yourdomain/auth/google/callback ) for your domain.
- Information technology is possible to whitelist only one or more email domains.
-
OVH :
- You'll demand to create a new application in the OVH API : https://eu.api.ovh.com/createApp/
- You'll exist handed an OVH application fundamental and an OVH application hugger-mugger fundamental that you'll need to put in the plikd.cfg file.
Once authenticated a user can generate upload tokens that can be specified in the ~/.plikrc file to authenticate the command line client.
Token = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" Security
Plik allow users to upload and serve any content as-is, but hosting untrusted HTML raises some well known security concerns.
Plik will endeavour to avert HTML rendering by overriding Content-Type to "text-manifestly" instead of "text/html".
Past default Plik sets a couple of security HTTP headers like Ten-Content-Blazon-Options, X-XSS-Protection, X-Frame-Options, Content-Security-Policy to disable sensible features of most recent browsers similar resources loading, xhr requests, iframes,... This volition however interruption features like audio/video playback, pdf rendering so it's possible to disable this behavior by setting the EnhancedWebSecurity configuration parameter to false
Along with that it is also strongly advised to serve uploaded files on a separate (sub-)domain to fight confronting phishing links and to protect Plik'southward session cookie with the DownloadDomain configuration parameter.
Cantankerous compilation
All binary are now statically linked. Clients can be safely cross-compiled for all os/architectures as they practice not rely on GCO (sqlite) Servers rely on CGO/sqlite need a cross-compilation fix surround.
make release will build release archives for amd64,i386,arm,arm64
To build a release with only specific architectures of the client
CLIENT_TARGETS="linux/amd64" releaser/release.sh To build only specific for only specific architectures
TARGETS="linux/amd64" releaser/release.sh To build with a specific cross compiler toolchain
TARGETS="linux/arm/v6" CC=arm-linux-gnueabihf-gcc releaser/release.sh FAQ
- Why is stream style broken in multiple instance deployement ?
Beacause stream mode isn't stateless. Equally the uploader request will block on ane plik instance the downloader request MUST go to the same case to succeed. The load balancing strategy MUST exist aware of this and route stream requests to the same instance by hashing the file id.
Hither is an example of how to achieve this using nginx and a little slice of LUA. Make sure your nginx server is built with LUA scripting back up. Yous might want to install the "nginx-extras" Debian bundle (>1.7.2) with congenital-in LUA support.
upstream plik { server 127.0.0.ane:8080; server 127.0.0.one:8081; } upstream stream { server 127.0.0.ane:8080; server 127.0.0.i:8081; hash $hash_key; } server { listen 9000; location / { fix $upstream ""; ready $hash_key ""; access_by_lua ' _,_,file_id = string.find(ngx.var.request_uri, "^/stream/[a-zA-Z0-9]+/([a-zA-Z0-nine]+)/.*$") if file_id == nothing so ngx.var.upstream = "plik" else ngx.var.upstream = "stream" ngx.var.hash_key = file_id end '; proxy_pass http://$upstream; } } - Redirection loops with DownloadDomain enforcement and opposite proxy
Invalid download domain 127.0.0.1:8080, expected plik.root.gg DownloadDomain check the Host header of the incoming HTTP asking, by default opposite proxies like Nginx or Apache mod_proxy does not forward this Header. Check the post-obit configuration directive :
Apache mod_proxy : ProxyPreserveHost On Nginx : proxy_set_header Host $host; - I have an error when uploading from client : "Unable to upload file : HTTP mistake 411 Length Required"
Under nginx < 1.three.9, you must enable HttpChunkin module to let transfer-encoding "chunked".
You might desire to install the "nginx-extras" Debian package with built-in HttpChunkin module.
And add together in your server configuration :
chunkin on; error_page 411 = @my_411_error; location @my_411_error { chunkin_resume; } - How to disable nginx buffering ?
By default nginx buffers large HTTP requests and reponses to a temporary file. This behaviour leads to unnecessary disk load and slower transfers. This should be turned off (>1.7.12) for /file and /stream paths. You might besides want to increase buffers size.
Detailed documentation : http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering
proxy_buffering off; proxy_request_buffering off; proxy_http_version i.1; proxy_buffer_size 1M; proxy_buffers viii 1M; client_body_buffer_size 1M; - Why hallmark does not work with HTTP connections when EnhancedWebSecurity is ready ?
Plik session cookies have the "secure" flag set when EnhancedWebSecurity is fix so they can only be transmitted over secure HTTPS connections.
- Build failure "/usr/bin/env: 'node': No such file or directory"
Debian users might need to install the nodejs-legacy package.
This bundle contains a symlink for legacy Node.js lawmaking requiring binary to exist /usr/bin/node (non /usr/bin/nodejs as provided in Debian). - How to accept and upload screenshots like a boss ?
alias pshot="scrot -s -e 'plik -q \$f | xclip ; xclip -o ; rm \$f'" Requires you to have plik, scrot and xclip installed in your $PATH.
scrot -s let you to "Interactively select a window or rectangle with the mouse" and so Plik will upload the screenshot and the url volition be direct copied to your clipboard and displayed by xclip. The screenshot is then removed of your home directory to avoid garbage.
How to contribute to the project ?
Contributions are welcome, feel free to open up issues and/or submit pull requests. Please be certain to also run/update the test suite :
make fmt make lint make test make test-backends Source: https://github.com/root-gg/plik
0 Response to "Github Upload File and Expose as Url"
Post a Comment