The status of ASTERICS WP4, Data Access, Discovery and Interoperability (DADI), and the activities foreseen until the completion of the project in April 2019, will be presented.

KM3NeT is a large research infrastructure that will consist of a network of deep-sea neutrino detectors in the Mediterranean Sea with user ports for Earth and Sea sciences. As a partner experiment in the European Horizon 2020 initiative projects ASTERICS and KM3NeT-INFRADEV, the KM3NeT collaboration strives to foster the dissemination of scientific results to the scientific community and to citizen scientists. We will report on the first steps that have already been taken towards this goal. Also, an overview of the next milestones will be given, as detailed in the KM3NeT data management plan. The plan is based on FAIR data management guidelines and contains, amongst others, information on the open data access standards to be employed.

The LIGO/Virgo collaboration features an open science programme, the goal of which is to release data to the public, while providing the information and the tools necessary to understand and use the data. I will review the past and current activities related to this open science programme.

(To be announced)

I will report on recent efforts to create an open data specification for imaging atmospheric Cherenkov telescopes and the interplay of such a work with the design of high-energy modular open source analysis frameworks. Its usage should serve several purposes: enable the joint analysis of data from several observatories, be a vehicle for open data releases and legacy projects, and ease the checks on the reproducibility of results.
I will discuss, in this context, prototyping activities to export data into this format and the inclusion of the first real TeV data release from H.E.S.S. This data release can be seen as a test bench for the gateway prototype, based on IVOA standards, that is developed at the Paris Data Analysis Centre (PADC) in the context of the Cherenkov Telescope Array (CTA). This prototype is a web client for query and visualisation that integrates the IVOA Provenance data model, as well as Authentication & Authorization management.
I will finally present the specific needs and challenges of the CTA project concerning provenance information, one of the most complex use case for the definition of the IVOA Provenance Data Model.

(To be announced)

Why are DOI still an issue in Astronomy? Although in a broader community context (e. g., RDA) the usage of DOI for data sets is fairly uncontested, the pickup in astronomy is limited, and even within the IVOA the notion of DOI is indistinct. I would like to raise again the issue of using DOI for astronomical data sets.

We present the most recent outcomes of the Virtual Atomic and Molecular Data Center in these main fields:

1. Current Challenges in data publication and publication tools: We give an overview of the VAMDC Query Store (which is an implementation of the Data Citation RDA recommendation) by highlighting, in particular, the new DOI functionality: this makes it possible to automatically obtain DOIs for any data extracted from TAP services when a given user wishes to cite these data in publications.
2. Developments to facilitate date exploitation by astronomers: We present the renewed version of the SpectCol analysis tool (allowing to cross-match in an interoperable way, with spectroscopic and collisional data coming from heterogeneous databases), and a data conversion web service to convert any molecular data extracted from VAMDC into the widely used Hitran format.
3. Use and adoption of Virtual Observatory standards: After recalling the work we are doing for convergence between the IVOA-SLAP standard and the VAMDC-XSAMS one, we present a first prototype service for extracting SLAP-formatted data from the VAMDC infrastructure.

• About VAMDC more...
  The Virtual Atomic and Molecular Data Centre Consortium (VAMDC) is a consortium of institutes and research institutions that share a common technical and political framework for the distribution and curation of atomic and molecular data.
  From the technical point of view, VAMDC federate 30 heterogeneous databases (including VALD, Hitran, JPL, CDSM, Basecol, etc…) into an interoperable e-infrastructure. VAMDC is built adopting IVOA components (registry, TAP interfaces for discovering data, SAMP connectors for data communication between post-processors).

SPECTCOL is a graphical tool implemented in Java. It allows to manipulate and combine spectroscopic and collisional data coming from the databases (BASECOL, CDMS, HITRAN, JPL,…) using VAMDC technology.

SVOCat is an application intended to facilitate the publication of an astronomical catalogue, both as a web page and as a Virtual Observatory ConeSearch service, while also implementing the SSA and DataLink protocols where appropriate.

Set in motion by the IVOA, the publication of astronomical research data has gained more and more importance over the years. Big collaborations and traditional research projects alike have recognised the additional value of publishing their data. Many of these projects offer a dedicated website, which they also use to publish their data releases. Sadly, most of these websites are tailor-made for their particular use case and are therefore not transferable to future projects.
At the Leibniz Institute for Astrophysics Potsdam (AIP) we gained profound experience with both the maintenance and the development of such applications, and we have gained insight on both the dark and the bright sides of this approach. Based on these experiences, we use the Daiquiri framework to host a number of sites, e. g., the RAVE survey, the CosmoSim database, APPLAUSE (Archives of Photographic PLates for Astronomical USE), and the Gaia@AIP services.
Daiquiri is developed and maintained by the E-Science group at AIP. The Daiquiri software stack has now been released in its second incarnation based on the Python Django framework. The data sets are stored in SQL databases and published either to the public, or for a restricted groups of researchers. Access is possible via an interactive SQL query interface, the TAP protocol, or an IVOA Cone Search. The query parser package is part of the Daiquiri framework and allows to query the data using ADQL as well as the native backend SQL language. Currently, MySQL and PostgreSQL are supported.
Each user has her / his own private tablebase where the results of all queries are stored and can be retrieved and downloaded at any time. Along with data access, Daiquiri supports user management, contact messages, and meeting management, as well as an integration of Wordpress for documentation and project presentation. Daiquiri consists of a common code base, which is centrally maintained, and site-specific applications, which are highly customizable to project requirements. Daiquiri is open source software and is licensed under the Apache License 2.0. The source code is available on GitHub: https://github.com/aipescience/django-daiquiri

Aladin Desktop, Aladin Lite, Sitools / MIZAR, Stellarium, KStars, Firefly: more and more astronomical tools are adopting the IVOA HiPS system (Hierarchical Progressive Survey). This talk will present the HiPS history and its mechanism. And, concretely, what is required to transform an image collection into a HiPS and how to publish it for these viewers.

Fedora Astronomy is an effort to improve the astronomical software ecosystem. Many fundamental software packages in astronomy are free software and are required for the astronomers' daily work. An example is the Astropy package with its many affiliated packages for more specialised applications, but also the AstrOmatic software, such as SExtractor, and applications to access the Virtual Observatory.
Fedora Astronomy is organized as a Special Interest Group (SIG) that takes care of the integration of astronomical software into the Fedora Linux distribution. The advantage of this way of software distribution is a good integration into the operating system, and also between the different packages. Another task of the Fedora Astronomy SIG is the deployment of a ready-to-use Linux environment for the purposes of both amateur and professional astronomers, the Fedora Astronomy Lab. Finally, the Fedora Astronomy SIG discusses issues with upstream developers, for example changes in depending applications or licensing, and takes care of the cooperation with other distributions' projects such as Debian Astronomy.
The talk covers the organization of the Fedora Astronomy SIG as well as the current state of Fedora Astronomy and its position in the astronomical software ecosystem. Furthermore, possible improvements in scientific software distribution and schemes for cooperation between different Linux distributions are discussed.

The Space Science Data Center is an Italian facility of the Italian Space Agency, a multi-mission science operations, data processing, and data archiving center that provides support to several scientific space missions. The current status of the implementation activities related to VO techinques will be presented, then the potentiality of VO development will also be discussed.

(To be announced)

The LOFAR Radio Telescope supports many types of observations and data formats. I will present a short description of the telescope, what data formats we support, which ones of those are currently available in our archive and the problems we have encountered in trying to model and expose our data to the VO.

Radio astronomical data collections are becoming available through the VO. However, data from a new generation of interferometric facilities pose challenges to standard VO tools that need to be faced in order to properly expose scientific data products to the astronomical community. In this talk, I will overview the ongoing ASTRON / LOFAR efforts to adopt VO standards for its data collections and also the progress of relevant projects.

We present the Online Analysis System (OAS) platform, facilitating reduction and analytics of astronomical data. The version planned for public release in the coming months allows to effortlessly explore and exploit observations of ESA's INternational Gamma-Ray Laboratory (INTEGRAL). The platform can be accessed through a simple on-line interface, suitable for researchers not familiar with INTEGRAL data reduction but interested in astrophysical phenomena. A more evolved analysis configuration can be achieved through detailed formulation of the service requests through an HTTP API. Our goal is to combine both quick-look in-browser analysis (providing robust but simple results) and fine-grained control through the API for domain experts (allowing interoperability and integration in federated data analysis workflows).

The analysis results at different degrees of reduction are archived as static products, annotated and indexed with the respective provenance graphs. This structure allows to follow the lineage of the scientific products from the raw spacecraft data to the final scientific publication. These products can be referenced with unique identifiers, derived from their provenance, suitable for assigning citable DOI (e. g., through Zenodo). These references address statically available archive products, while at the same time referring to the online analysis platform, allowing to trace every analysis step, and to re-execute the pipeline, reproducing the result from scratch, optionally introducing changes that are testing the robustness of the result. Wherever possible, we will provide an interface conforming to the VO standards for astronomical data products.
A simple, immediate, automated, and reliable interface to the diverse astronomical data has proven to be most important in multi-messenger time domain astronomy. We discuss the services that have been developed for promptly sharing INTEGRAL data for transients, which were exploited in the recent observations of different multi-messenger transients. Finally, we discuss the integration of these services with the OAS platform.

With the current and upcoming astronomical catalogue sizes, the notion of computational cost management is becoming a very relevant requirement — from the very beginning of modeling data mining problems. This talk addresses various problems related to the computational costs that were encountered when creating a processing pipeline of the COSMOS catalogue, as well as the methodology used, i. e., parallel processing with vectorization in MATLAB. Finally, some results are presented on the performance of the automatic processing pipeline that was developed.

The Euclid satellite is an ESA mission scheduled for launch in 2020. It will observe an area of 15,000 deg² with two instruments, the Visible Imaging Channel (VIS) and the Near IR Spectrometer and imaging Photometer (NISP). Ground-based imaging data in griz from surveys such as the Dark Energy Survey or KIDS complement the Euclid data to enable photo-z determination. The mission investigates the distance-redshift relationship and the evolution of cosmic structures by measuring shapes and redshifts of galaxies and clusters of galaxies out to redshifts of about 2.
Processing the vast amount of Euclid data from raw images to object catalogs, to photometric redshifts and weak lensing, and eventually to scientific results poses large challenges for data processing and data distribution. To cope with those challenges, the Euclid Science Ground Segment is organized in nine Organizational Units (OUs) that implement the software for the various processing steps and seven Science Data Centers (SDCs) that optimize and run those processing steps on their computer clusters.
To test the data processing, a sequence of Science Data Challenges are undertaken that run Euclid Processing functions on simulated input data, which is based on a mock universe with realistic fluxes and morphologies from empirical descriptions.
We present the status of the Euclid project, discuss the challenges that were encountered so far, and show the latest result of the Science Data Challenge 4/5/6, which tests our processing down to the weak lensing and the spectral analysis.

The ALMA observatory has already delivered huge amounts of data. Those data are accessible to download via the ALMA science archive portal from their parent project id. Here we present ARTEMIX, a development from the Paris Observatory that aims to provide a quick-look access to the ALMA datacubes as well as a synthetic view of the observation and imaging parameters (positions / frequencies).

An update of the work that has been done on STOA — a workflow management system and data service being developed in house at Cambridge.

As a network of deep-sea neutrino telescopes in the Mediterranean, the KM3NeT research infrastructure will provide significant real-time capabilities for the detection of high energy astrophysical neutrinos and low energy supernova neutrino bursts. Within this context, automated multi-messenger analyses will allow a prompt release of information to the astroparticle community in case of detection. At the same time, external alerts can be exploited to improve the KM3NeT sensitivity to transient sources. These features will be fundamental for the integration of the experiment in the existing and future global alert networks, as well as for the development of a rich multi-messenger program within the ASTERICS and KM3NeT-INFRADEV H2020 initiative projects.

The IVOA has proposed a framework to describe provenance metadata for the dissemination of astronomical data collections. This effort re-uses concepts proposed by the W3C Provenance Data Model and interprets those in the IVOA ecosystem. The main description elements are presented, together with their implementation strategies.

This talk will present the construction of provenance information for the third data release of the APPLAUSE database.
The APPLAUSE (Archives of Photographic PLates for Astronomical USE) project processes digitised photographic plates from various plate archives of the Hamburg, Bamberg, Tartu, and Potsdam observatories. The plates are digitized with high-resolution flatbed scanners. In addition, the corresponding plate envelopes and observation logbooks are digitized, and further metadata are available in the database. With the pyplate software package, positions and magnitudes of objects are extracted from the scans and calibrated against modern catalogs. The 3rd Data Release catalogue, scheduled for July 2018, contains over 3 billion sources.
APPLAUSE is available at https://plate-archive.org.

The IVOA Provenance Data Model, currently in working draft stage, closely follows the quite general W3C (World Wide Web Consortium) recommendations for recording provenance of data. This means that the ensemble of provenance information actually makes up a web of arbitrarily structured relations, i. e., it features the properties of a graph in the sense of graph theory.
Thus, many meaningful queries about provenance such as "all persons that were directly or indirectly involved in creating a given image", "all images that depend on reductions from a certain calibration dataset", or "all intermediate processing steps between two datasets", may only be calculated via graph traversal, a feature not available in the IVOA ADQL query language. While in the long run ADQL should be probably extended with features from established graph query languages such as openCypher, we here explore the suitability for graph queries of presently available means for ADQL extension, namely, so-called "user"-defined functions.
Using (recursive) "common table expressions" (CTEs) that are available in virtually any SQL database commonly used in astronomical data centres, graph queries prove to be a feasible addition to existing ADQL implementations. Next to showing concrete examples, we discuss the benefits of using genuine graph databases, such as Bitnine Agens Graph.

This talk reviews the current status of VO protocols for discovery, description, and access of data 'cubes' or 'hyper cubes', with a focus on implementation tricks. It will also give the status of the ongoing work for time series.