| 28 |
Where Has Your Data Come From? Data Ancestry and Other Tales |
Dr. Tania Allard |
4:00 PM |
May 8 2019 |
Fuller Hall |
https://csvconf.com/speakers/#dr-tania-allard |
2019-05-08T16:00:00 |
Over the last few years, great improvements have been made around the areas of reproducible scientific computing research and FAIR (findable, accessible, interoperable and reusable) data. As a consequence, data scientists and researchers alike have started to incorporate modern software development practices in their workflows (i.e. version control, testing). More and more emphasis has been made on the need to look after the quality and validity of the software developed. But what about the data? Data validation and integrity is just as important as the adequacy of the code ingesting and processing the datasets.
In this talk, I will take a high-level look at concepts such as data lineage, provenance, continuous data validation and present real-world examples in which these concepts have been applied to different real-world data pipelines increasing not only the confidence of the results obtained but also the efficiency and integrity of the workflows themselves. |
https://csvconf.com/img/speakers-2019/tallard.jpg |
