Data source integration

Authors:

Overview
Questions:

How can I write a tool that can import data into Galaxy from an external database?

What are “data sources” and how do they function?

Is there any ready-to-use example?

Objectives:

Time estimation: 10 minutes

Supporting Materials:

FAQs

Published: Sep 30, 2016

Last modification: Oct 18, 2022

License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License. The GTN Framework is licensed under MIT

purl PURL: https://gxy.io/GTN:T00114

version Revision: 15

Data Source Integration

An important goal of Galaxy is scalability. A major bottleneck when it comes to analysis of big data sets is the time and space it takes of copying these data sets.

Galaxy provides an interface such that it can communicate with other servers to get data directly into the Galaxy environment of a user without the need of “downloading” the data. In this hands on, we will use the resource from DoRiNA Server (Blin et al. 2014). but the main point about this short section is: if you have a data source which you think is very important for your research with Galaxy let us know!

Hands-on: Hands on!

Create a new history called “doRiNA”

Go to Get Data::doRiNA search

Choose hg19 from the drop-down list -> Search Database

Leave everything as is and choose from the Regulators (set A) drop-down list “hsa-let-7astar-CLASH” -> Search doRiNA

Use the “Send to Galaxy” button

Notice the new History Item

You can see the tutorial section of the DoRiNA website for more detailed examples. That was very easy for all of you! If you want your database of choice to be accessible as easy as this let us know!

You've Finished the Tutorial

Key points

It is possible to couple an external data resource with a Galaxy server

The external data resource is accessed through his native interface

Data flows from the external data resource to the Galaxy server without the need of “downloading” the data

Frequently Asked Questions

Have questions about this tutorial? Check out the tutorial FAQ page or the FAQ page for the Development in Galaxy topic to see if your question is listed there. If not, please ask your question on the GTN Gitter Channel or the Galaxy Help Forum

References

Blin, K., C. Dieterich, R. Wurmus, N. Rajewsky, M. Landthaler et al., 2014 DoRiNA 2.0—upgrading the doRiNA database of RNA interactions in post-transcriptional regulation. Nucleic Acids Research 43: D160–D167. 10.1093/nar/gku1180

Feedback

Did you use this material as an instructor? Feel free to give us feedback on how it went.
Did you use this material as a learner or student? Click the form below to leave feedback.

Citing this Tutorial

Bérénice Batut, Saskia Hiltemann, Gianmauro Cuccuru, Helena Rasche, Data source integration (Galaxy Training Materials). https://training.galaxyproject.org/training-material/topics/dev/tutorials/data-source-integration/tutorial.html Online; accessed TODAY
Hiltemann, Saskia, Rasche, Helena et al., 2023 Galaxy Training: A Powerful Framework for Teaching! PLOS Computational Biology 10.1371/journal.pcbi.1010752
Batut et al., 2018 Community-Driven Data Analysis Training for Biology Cell Systems 10.1016/j.cels.2018.05.012

@misc{dev-data-source-integration,
author = "Bérénice Batut and Saskia Hiltemann and Gianmauro Cuccuru and Helena Rasche",
	title = "Data source integration (Galaxy Training Materials)",
	year = "",
	month = "",
	day = ""
	url = "\url{https://training.galaxyproject.org/training-material/topics/dev/tutorials/data-source-integration/tutorial.html}",
	note = "[Online; accessed TODAY]"
}
@article{Hiltemann_2023,
	doi = {10.1371/journal.pcbi.1010752},
	url = {https://doi.org/10.1371%2Fjournal.pcbi.1010752},
	year = 2023,
	month = {jan},
	publisher = {Public Library of Science ({PLoS})},
	volume = {19},
	number = {1},
	pages = {e1010752},
	author = {Saskia Hiltemann and Helena Rasche and Simon Gladman and Hans-Rudolf Hotz and Delphine Larivi{\`{e}}re and Daniel Blankenberg and Pratik D. Jagtap and Thomas Wollmann and Anthony Bretaudeau and Nadia Gou{\'{e}} and Timothy J. Griffin and Coline Royaux and Yvan Le Bras and Subina Mehta and Anna Syme and Frederik Coppens and Bert Droesbeke and Nicola Soranzo and Wendi Bacon and Fotis Psomopoulos and Crist{\'{o}}bal Gallardo-Alba and John Davis and Melanie Christine Föll and Matthias Fahrner and Maria A. Doyle and Beatriz Serrano-Solano and Anne Claire Fouilloux and Peter van Heusden and Wolfgang Maier and Dave Clements and Florian Heyl and Björn Grüning and B{\'{e}}r{\'{e}}nice Batut and},
	editor = {Francis Ouellette},
	title = {Galaxy Training: A powerful framework for teaching!},
	journal = {PLoS Comput Biol} Computational Biology}
}

                   

Congratulations on successfully completing this tutorial!

You can use Ephemeris's shed-tools install command to install the tools used in this tutorial.
shed-tools install [-g GALAXY] [-a API_KEY] -t <(curl https://training.galaxyproject.org/training-material/api/topics/dev/tutorials/data-source-integration/tutorial.json | jq .admin_install_yaml -r)
Alternatively you can copy and paste the following YAML
---
install_tool_dependencies: true
install_repository_dependencies: true
install_resolver_dependencies: true
tools: []

No feedback has been recieved yet for this training. Be the first one by filling in the feedback form from above.