Gearing towards production

name: inverse
layout: true
class: center, middle, inverse

</span></div>

</span></div>

---

# Gearing towards production

<div class="contributors-line">
		
	
<ul class="text-list">
			
			<li>
				<a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
    Nate Coraor</a>
			<li>
				<a href="/training-material/hall-of-fame/blankenberg/" class="contributor-badge contributor-blankenberg"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/blankenberg?s=36" alt="Daniel Blankenberg avatar" width="36" class="avatar" />
    Daniel Blankenberg</a>
			<li>
				<a href="/training-material/hall-of-fame/abdulrahmanazab/" class="contributor-badge contributor-abdulrahmanazab"><img src="https://avatars.githubusercontent.com/abdulrahmanazab?s=36" alt="Abdulrahman Azab avatar" width="36" class="avatar" />
    Abdulrahman Azab</a>
			<li>
				<a href="/training-material/hall-of-fame/martenson/" class="contributor-badge contributor-martenson"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/martenson?s=36" alt="Martin Čech avatar" width="36" class="avatar" />
    Martin Čech</a></li>
</ul>

</div>

<div class="footnote" style="bottom: 8em;">
  <i class="far fa-calendar" aria-hidden="true"></i><span class="visually-hidden">last_modification</span> Updated:   
  <i class="fas fa-fingerprint" aria-hidden="true"></i><span class="visually-hidden">purl</span><abbr title="Persistent URL">PURL</abbr>: <a href="https://gxy.io/GTN:S00019">gxy.io/GTN:S00019</a>
</div>

<div class="footnote" style="bottom: 5em;">

<i class="far fa-play-circle" aria-hidden="true"></i><span class="visually-hidden">video-slides</span> <a href="/training-material/videos/watch.html?v=/admin/tutorials/production/slides">Video slides</a> |

<i class="fas fa-file-alt" aria-hidden="true"></i><span class="visually-hidden">text-document</span><a href="slides-plain.html"> Plain-text slides</a> |

</div>

<div class="footnote" style="bottom: 2em;">
    <strong>Tip: </strong>press <kbd>P</kbd> to view the presenter notes
    | <i class="fa fa-arrows" aria-hidden="true"></i><span class="visually-hidden">arrow-keys</span> Use arrow keys to move between slides

</div>

???
Presenter notes contain extra information which might be useful if you intend to use these slides for teaching.

Press `P` again to switch presenter notes off

Press `C` to create a new window where the same presentation will be displayed.
This window is linked to the main window. Changing slides on one will cause the
slide to change on the other.

Useful when presenting.

---

# Production server

* Used by multiple people
* Designed to be resilient, scalable
* Designed to be easily managed

???

- A production server is a server ready for any number of users.
- It is designed to be scalable, easy to manage, and resilient in the face of many users.
- Running a production server is not difficult, thanks to ansible.

---

# Configuration Options

???

- We will now cover many of the configuration options which are necessary for production.

---
class: reduce90

# Configuring uWSGI

Important options in the `uwsgi` section of `galaxy.yml`

option | description
---- | ----
`http` | port and IP on which to listen for HTTP requests, `:8080` for port `8080` on all network interfaces. **For a production server, you should use `socket:` and a proxy server**.
`socket` | port and IP on which to listen for uWSGI protocol requests, `127.0.0.1:8000` for port `8000` on localhost.
`processes` | number of uWSGI *web workers*.
`threads` | number of threads for each web worker.
`offload-threads` | number of threads for internal routing requests, 1-2 is sufficient.
`master` | use uWSGI master process manager, defaults to `false` for dev compatibility but should be `true` in production.
`mule-reload-mercy` | time to gracefully wait for mule shutdown, set > `monitor_thread_join_timeout` galaxy option.

???

- Production servers should use socket instead of http in the uwsgi listen section.
- This is because socket means using the uwsgi protocol which is more efficient.
- Processes and threads can be set, but the defaults are usually good.

---
# Securing your Object IDs

* User facing object IDs
  * Galaxy uses the Blowfish cipher to obscure integer IDs.
  * Prevents guessing the hashes for various Galaxy objects.
  * `id_secret` is used as the Blowfish key.
  * Changing the default `id_secret` is a must.
  * Changing `id_secret` will change & invalidate existing URLs (e.g. datasets, histories, workflows, etc).
* Setting `new_user_dataset_access_role_default_private` ensures that even guessable IDs are still private

???

- An extremely important setting to change is the id secret.
- Galaxy generates a reversible hash for every numeric ID.
- You must change this, or an attacker can guess IDs and see potentially private data.
- There is an additional variable, new user dataset access role default private, which can be set.
- Then, even if IDs are guessed, the data remains private.

---
# Customizing your "Brand"

option | description
---- | ----
`brand` | appends "/{brand}" to the "Galaxy" text in the masthead.
`logo_url` | URL linked by the "Galaxy/brand" text.
`helpsite_url` | URL linked by the "Galaxy Help" link in the "Help" menu.
`wiki_url` | URL linked by the "Wiki" link in the "Help" menu.
`support_url` | URL linked by the "Support" link in the "Help" menu.
`citation_url` | URL linked by the "How to Cite Galaxy" link in the "Help" menu.
`search_url` | URL linked by the "Search" link in the "Help" menu.
`mailing_lists_url` | URL linked by the "Mailing Lists" link in the "Help" menu.
`screencasts_url` | URL linked by the "Videos" link in the "Help" menu.
`terms_url` | URL linked by the "Terms and Conditions" link on the registration form.
`qa_url` | URL linked by the "Galaxy Q&A" link in the "Help" menu.

???

- Brand can be set to denote the identity of your server.
- Many URLs which appear in the top menu can be modified.

---
# Adding a Notice Banner

option | sample value
---- | ----
`message_box_visible` | `true`
`message_box_content` | `Downtime scheduled Sunday at Noon to one.`
`message_box_class` | `warning` (also: `error`, `info`)

???

- Sometimes you wish to broadcast a message to your users, for example when you plan downtime.
- The message box allows doing this.

---
# Update the welcome page

Welcome page is `$GALAXY_ROOT/static/welcome.html` and is the first thing that
users see. It is a good idea to extend it with things like:
- Server or Group News
- Downtimes/Maintenance periods
- New tools
- Publications relating to your Galaxy
- Current server usage / jobs running (e.g. https://usegalaxy.eu/)

.left[No restarting is necessary.]

???

- Updating the welcome page is important as all users see this page when accessing the server.
- This can be a great way to communicate lower-priority information to users.
- The European Galaxy server uses it to show usage graphs which users like.

---
# Client Browser Security

option | description
---- | ----
`sanitize_all_html` | by default, Galaxy sanitizes all "text/html" tool outputs. Setting to false potentially exposes users to XSS attacks.
`sanitize_whitelist_file` | manually override html sanitization for listed tools. Can set in admin interface.
`serve_xss_vulnerable_mimetypes` | certain filetypes (e.g. SVG) can contain JS that is vulnerable to XSS (Cross-site scripting) and are served as "plain/text" by default.
`allowed_origin_hostnames` | Returns Access-Control-Allow-Origin response header that matches the Origin header of the request.

???

- Some tools produce HTML or SVG outputs.
- By default, galaxy sanitizes these files, as malicious ones could be used to attack users.
- If there are some tools you trust like fastqc, you might want to whitelist them.

---
# Debugging

These should never be enabled on a public site.

option | description
---- | ----
`debug` | enable debug options, preserves cluster job data on disk (but use `cleanup_job` for this).
`use_profile` | Python profiler on requests.
`use_printdebug` | Anything "print"ed within a Galaxy Web thread is exposed to user.
`use_interactive` | Enable live debugging in your browser.

???

- There are debugging options if you need, but none of these should be enabled in production.

---
# Configuring FTP

option | description
---- | ----
`ftp_upload_dir` | Directory containing subdirectories matching users' identifier (defaults to e-mail).
`ftp_upload_site` | Hostname of your FTP server provided to users in the help text. Must set to enable FTP import.
`ftp_upload_dir_identifier` | User attribute for subdirectory naming. 'email' (${user.email}) is default, but 'id' or 'username' are also common.
`ftp_upload_dir_template` | Python string template used to determine a users FTP upload directory (${ftp_upload_dir}/${ftp_upload_dir_identifier}).
`ftp_upload_purge` | Delete files after FTP import (true).

???

- A common request is how should users upload large datasets.
- If you setup an FTP server, this can be integrated with Galaxy.
- Users can see their own FTP upload directory in Galaxy, if you configure it.

---
class: reduce90

# Configuring Data Library Path Uploads

Admin options

option | description
---- | ----
`library_import_dir` | Browse and upload files from configured directory.
`allow_path_paste` | Allow admins to paste any path to upload (e.g. `file://`), applies also to the general 'upload tool'.

User options

option | description
---- | ----
`user_library_import_dir` | Root directory containing sub-directories named by user emails. A nifty setup is that the value is the same as for `ftp_upload_dir` allowing users to upload files via FTP and then import them either in history or data library.
`user_library_import_dir_auto_creation` | Create user's folder upon login.
`user_library_import_symlink_whitelist` | Directories that users are allowed to symlink to.
`user_library_import_check_permissions` | If Galaxy usernames match system usernames enable this to use UNIX permissions.

???

- Library management is a common problem for servers.
- How to manage the data and who can manage the data.
- There are several options in galaxy yml to allow users to manage libraries.

---
class: reduce90

# Configuring SMTP

SMTP server

option | description
---- | ----
`smtp_server` | host:port of SMTP server to use. Uses STARTTLS, but will fallback.
`smtp_username` | Username for SMTP server.
`smtp_password` | Password for SMTP server. STARTTLS recommended on SMTP server.
`smtp_ssl` | If SMTP server requires SSL from connection start, set to true.

Addresses

option | description
---- | ----
`error_email_to` | Address to send user error reports to.
`email_from` | Return address used in automatic user notifications. (`<galaxy-no-reply@HOSTNAME>`)
`mailing_join_addr` | Mailing list to subscribe users to during registration.

???

- Setting up an SMTP server is useful.
- Emails can be sent to users if they forget their passwords.
- You can require email validation to ensure users provide valid emails.

---
# galaxy.yml full options

a.k.a. Everything You Always Wanted to Know About `galaxy.yml`
* [config schema](https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/config_schema.yml)
* [config sample](https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/galaxy.yml.sample)
* [config docs](https://docs.galaxyproject.org/en/master/admin/config.html)

???

- Everything covered in this slideset is also in the galaxy yaml file.
- You can find even more options to configure there.

---

## Thank You!

This material is the result of a collaborative work. Thanks to the [Galaxy Training Network](https://training.galaxyproject.org) and all the contributors!

<div class="contributors-line">
		
<table class="contributions">
	
	<tr>
		<td><abbr title="These people wrote the bulk of the tutorial, they may have done the analysis, built the workflow, and wrote the text themselves.">Author(s)</abbr></td>
		<td>
			<a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
    Nate Coraor</a><a href="/training-material/hall-of-fame/blankenberg/" class="contributor-badge contributor-blankenberg"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/blankenberg?s=36" alt="Daniel Blankenberg avatar" width="36" class="avatar" />
    Daniel Blankenberg</a><a href="/training-material/hall-of-fame/abdulrahmanazab/" class="contributor-badge contributor-abdulrahmanazab"><img src="https://avatars.githubusercontent.com/abdulrahmanazab?s=36" alt="Abdulrahman Azab avatar" width="36" class="avatar" />
    Abdulrahman Azab</a><a href="/training-material/hall-of-fame/martenson/" class="contributor-badge contributor-martenson"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/martenson?s=36" alt="Martin Čech avatar" width="36" class="avatar" />
    Martin Čech</a>
		</td>
	</tr>

<tr class="reviewers">
		<td><abbr title="These people reviewed this material for accuracy and correctness">Reviewers</abbr></td>
		<td>
			<a href="/training-material/hall-of-fame/martenson/" class="contributor-badge contributor-badge-small contributor-martenson"><img src="https://avatars.githubusercontent.com/martenson?s=36" alt="Martin Čech avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-badge-small contributor-natefoo"><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/hexylena/" class="contributor-badge contributor-badge-small contributor-hexylena"><img src="https://avatars.githubusercontent.com/hexylena?s=36" alt="Helena Rasche avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/nsoranzo/" class="contributor-badge contributor-badge-small contributor-nsoranzo"><img src="https://avatars.githubusercontent.com/nsoranzo?s=36" alt="Nicola Soranzo avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/shiltemann/" class="contributor-badge contributor-badge-small contributor-shiltemann"><img src="https://avatars.githubusercontent.com/shiltemann?s=36" alt="Saskia Hiltemann avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/bgruening/" class="contributor-badge contributor-badge-small contributor-bgruening"><img src="https://avatars.githubusercontent.com/bgruening?s=36" alt="Björn Grüning avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/gmauro/" class="contributor-badge contributor-badge-small contributor-gmauro"><img src="https://avatars.githubusercontent.com/gmauro?s=36" alt="Gianmauro Cuccuru avatar" width="36" class="avatar" /></a></td>
	</tr>

</table>

</div>

</div>

Tutorial Content is licensed under <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.<br/>