Gearing towards production
Contributors
Nate Coraor
Daniel Blankenberg
Abdulrahman Azab
Martin Čech last_modification Published: Jan 28, 2019
last_modification Last Updated: Apr 16, 2023
Production server
- Used by multiple people
- Designed to be resilient, scalable
- Designed to be easily managed
Speaker Notes
- A production server is a server ready for any number of users.
- It is designed to be scalable, easy to manage, and resilient in the face of many users.
- Running a production server is not difficult, thanks to ansible.
Configuration Options
Speaker Notes
- We will now cover many of the configuration options which are necessary for production.
class: reduce90
Configuring uWSGI
Important options in the uwsgi section of galaxy.yml
| option | description |
|---|---|
http |
port and IP on which to listen for HTTP requests, :8080 for port 8080 on all network interfaces. For a production server, you should use socket: and a proxy server. |
socket |
port and IP on which to listen for uWSGI protocol requests, 127.0.0.1:8000 for port 8000 on localhost. |
processes |
number of uWSGI web workers. |
threads |
number of threads for each web worker. |
offload-threads |
number of threads for internal routing requests, 1-2 is sufficient. |
master |
use uWSGI master process manager, defaults to false for dev compatibility but should be true in production. |
mule-reload-mercy |
time to gracefully wait for mule shutdown, set > monitor_thread_join_timeout galaxy option. |
Speaker Notes
- Production servers should use socket instead of http in the uwsgi listen section.
- This is because socket means using the uwsgi protocol which is more efficient.
- Processes and threads can be set, but the defaults are usually good.
Securing your Object IDs
- User facing object IDs
- Galaxy uses the Blowfish cipher to obscure integer IDs.
- Prevents guessing the hashes for various Galaxy objects.
id_secretis used as the Blowfish key.- Changing the default
id_secretis a must. - Changing
id_secretwill change & invalidate existing URLs (e.g. datasets, histories, workflows, etc).
- Setting
new_user_dataset_access_role_default_privateensures that even guessable IDs are still private
Speaker Notes
- An extremely important setting to change is the id secret.
- Galaxy generates a reversible hash for every numeric ID.
- You must change this, or an attacker can guess IDs and see potentially private data.
- There is an additional variable, new user dataset access role default private, which can be set.
- Then, even if IDs are guessed, the data remains private.
Customizing your “Brand”
| option | description |
|---|---|
brand |
appends “/{brand}” to the “Galaxy” text in the masthead. |
logo_url |
URL linked by the “Galaxy/brand” text. |
helpsite_url |
URL linked by the “Galaxy Help” link in the “Help” menu. |
wiki_url |
URL linked by the “Wiki” link in the “Help” menu. |
support_url |
URL linked by the “Support” link in the “Help” menu. |
citation_url |
URL linked by the “How to Cite Galaxy” link in the “Help” menu. |
search_url |
URL linked by the “Search” link in the “Help” menu. |
mailing_lists_url |
URL linked by the “Mailing Lists” link in the “Help” menu. |
screencasts_url |
URL linked by the “Videos” link in the “Help” menu. |
terms_url |
URL linked by the “Terms and Conditions” link on the registration form. |
qa_url |
URL linked by the “Galaxy Q&A” link in the “Help” menu. |
Speaker Notes
- Brand can be set to denote the identity of your server.
- Many URLs which appear in the top menu can be modified.
Adding a Notice Banner
| option | sample value |
|---|---|
message_box_visible |
true |
message_box_content |
Downtime scheduled Sunday at Noon to one. |
message_box_class |
warning (also: error, info) |
Speaker Notes
- Sometimes you wish to broadcast a message to your users, for example when you plan downtime.
- The message box allows doing this.
Update the welcome page
Welcome page is $GALAXY_ROOT/static/welcome.html and is the first thing that
users see. It is a good idea to extend it with things like:
- Server or Group News
- Downtimes/Maintenance periods
- New tools
- Publications relating to your Galaxy
- Current server usage / jobs running (e.g. https://usegalaxy.eu/)
.left[No restarting is necessary.]
Speaker Notes
- Updating the welcome page is important as all users see this page when accessing the server.
- This can be a great way to communicate lower-priority information to users.
- The European Galaxy server uses it to show usage graphs which users like.
Client Browser Security
| option | description |
|---|---|
sanitize_all_html |
by default, Galaxy sanitizes all “text/html” tool outputs. Setting to false potentially exposes users to XSS attacks. |
sanitize_whitelist_file |
manually override html sanitization for listed tools. Can set in admin interface. |
serve_xss_vulnerable_mimetypes |
certain filetypes (e.g. SVG) can contain JS that is vulnerable to XSS (Cross-site scripting) and are served as “plain/text” by default. |
allowed_origin_hostnames |
Returns Access-Control-Allow-Origin response header that matches the Origin header of the request. |
Speaker Notes
- Some tools produce HTML or SVG outputs.
- By default, galaxy sanitizes these files, as malicious ones could be used to attack users.
- If there are some tools you trust like fastqc, you might want to whitelist them.
Debugging
These should never be enabled on a public site.
| option | description |
|---|---|
debug |
enable debug options, preserves cluster job data on disk (but use cleanup_job for this). |
use_profile |
Python profiler on requests. |
use_printdebug |
Anything “print”ed within a Galaxy Web thread is exposed to user. |
use_interactive |
Enable live debugging in your browser. |
Speaker Notes
- There are debugging options if you need, but none of these should be enabled in production.
Configuring FTP
| option | description |
|---|---|
ftp_upload_dir |
Directory containing subdirectories matching users’ identifier (defaults to e-mail). |
ftp_upload_site |
Hostname of your FTP server provided to users in the help text. Must set to enable FTP import. |
ftp_upload_dir_identifier |
User attribute for subdirectory naming. ‘email’ (${user.email}) is default, but ‘id’ or ‘username’ are also common. |
ftp_upload_dir_template |
Python string template used to determine a users FTP upload directory (${ftp_upload_dir}/${ftp_upload_dir_identifier}). |
ftp_upload_purge |
Delete files after FTP import (true). |
Speaker Notes
- A common request is how should users upload large datasets.
- If you setup an FTP server, this can be integrated with Galaxy.
- Users can see their own FTP upload directory in Galaxy, if you configure it.
class: reduce90
Configuring Data Library Path Uploads
Admin options
| option | description |
|---|---|
library_import_dir |
Browse and upload files from configured directory. |
allow_path_paste |
Allow admins to paste any path to upload (e.g. file://), applies also to the general ‘upload tool’. |
User options
| option | description |
|---|---|
user_library_import_dir |
Root directory containing sub-directories named by user emails. A nifty setup is that the value is the same as for ftp_upload_dir allowing users to upload files via FTP and then import them either in history or data library. |
user_library_import_dir_auto_creation |
Create user’s folder upon login. |
user_library_import_symlink_whitelist |
Directories that users are allowed to symlink to. |
user_library_import_check_permissions |
If Galaxy usernames match system usernames enable this to use UNIX permissions. |
Speaker Notes
- Library management is a common problem for servers.
- How to manage the data and who can manage the data.
- There are several options in galaxy yml to allow users to manage libraries.
class: reduce90
Configuring SMTP
SMTP server
| option | description |
|---|---|
smtp_server |
host:port of SMTP server to use. Uses STARTTLS, but will fallback. |
smtp_username |
Username for SMTP server. |
smtp_password |
Password for SMTP server. STARTTLS recommended on SMTP server. |
smtp_ssl |
If SMTP server requires SSL from connection start, set to true. |
Addresses
| option | description |
|---|---|
error_email_to |
Address to send user error reports to. |
email_from |
Return address used in automatic user notifications. (<galaxy-no-reply@HOSTNAME>) |
mailing_join_addr |
Mailing list to subscribe users to during registration. |
Speaker Notes
- Setting up an SMTP server is useful.
- Emails can be sent to users if they forget their passwords.
- You can require email validation to ensure users provide valid emails.
galaxy.yml full options
a.k.a. Everything You Always Wanted to Know About galaxy.yml
Speaker Notes
- Everything covered in this slideset is also in the galaxy yaml file.
- You can find even more options to configure there.
Thank you!
This material is the result of a collaborative work. Thanks to the Galaxy Training Network and all the contributors!
Tutorial Content is licensed under
Creative Commons Attribution 4.0 International License.