Simulation Data Modeling
Table of Contents
This page is a brief discussion of efforts to describe simulations, both in the context of standards in the IVOA, and independently. To help set the stage, it provides a short introduction to the International Virtual Observatory Alliance (IVOA), and some of the standards that are relevant to publishing simulations.
International Virtual Observatory Alliance
About the IVOA
The IVOA is an organization composed of representatives from several national virtual observatory (VO) projects. Its goal is to produce standards that promote and facilitate the sharing of astronomical data.
Working & Interest Groups
Work in the IVOA is divided up into Working Groups, which can recommend standards, and Interest Groups, that focus on topics involving multiple Working Groups, outside of the standards track. Working Groups tend to focus on the nitty-gritty technical details, while Interest Groups have a broader scope.
Here are lists pulled from the IVOA's Who is Who? page.
Working Groups:
- Applications
- Data Access Layer
- Data Model
- Grid and Web Services
- Registry
- Semantics
- VO Event
- Query Language
- VOTable
Interest Groups:
- Astronomical Grid
- Data Curation & Preservation
- Theory
- Technical Coordination Group
- Standing Committee on Standards & Processes
InterOp Meetings
The IVOA has semi-annual Interoperability (InterOp) meetings, to hash out details of various standards. The IVOA Events page has a list of the previous and upcoming InterOps, along with other related events, such as ADASS. To get a feel for what gets covered during an InterOp, check out the May 2009 InterOp Agenda.
Current IVOA Standards
All of the IVOA standards are on the IVOA website, on the Documents & Standards page. This page also has various Technical Notes, and standards that are in process. Here we'll cover some of the more significant standards for publishing theory data.
Registry (Existing Standard)
"Registries" are servers that publish various "Resources" that they are aware of. From the IVOA perspective, publishing data (a Resource) to the VO equates to making it accessible online, and registering it in a Registry. For a summary of how to do this, read Ray Plante's How to Publish to the NVO.
VOTable (Existing Standard)
VOTable is an XML format for sharing astronomical data. As the name strongly implies, it is geared towards tabular data, and is intended to be compatible with the FITS format. In the IVOA, it is used for tables such as sources catalogs, spectra, or links to images. Over time, more and more software libraries have begun to support it, which makes it a convenient intermediate format.
Table Access Protocol (TAP) (Upcoming Standard)
The soon-to-be finished Table Access Protocol will provide simple access to relational databases. It supersedes the SkyNode Interface, which never became a standard. For theory data, it is the most important standard, because it doesn't distinguish between the type of data being presented. In particular, it does not focus on images or spectra whose data models contain an RA and DEC.
Using TAP, data providers will be free to publish arbitrary catalogs. The drawback is that interoperability between resources will be lost, since the contents of each service will not be standardized. E.g., two groups might publish simulated halo catalogs, but the tables and columns in each service will be different, which makes automated comparisons difficult.
Simulation Efforts (Progressing)
Information about the status of standards for simulations is spread out in many places. On the IVOA site, there is a wiki, which contains a lot of pages covering various topics. There is also a mailing list, with an archive.
The InterOp agendas and presentations are also useful for tracking the IVOA glacier.
There is a Google Code project, Volute, with an associated Subversion repository, where a few IVOA groups track their documents. The Theory Interest Groups uses this, as well.
Simulation Data Model
The Simulation Data Model describes the concepts (classes, relationships) in the theory realm, from a computational perspective. It is more of a conceptual model, rather than a physical model. I.e., it describes the abstract concepts represented by the, and not the structure of the data.
Simulation Database (SimDB)
Builds on the TAP service, by defining a fixed relational database schema using the Simulation Data Model.
Simulation Data Access Protocol (SimDAP)
Provides access to the data, using a standard operations such as download, cutout and extraction.
Simple Self-Described Service (S3)
The S3 protocol has been proposed as a mechanism to access micro-simulations, e.g., stellar evolution models, where the distinction between input data, parameters, and characterization is not clear. Hopefully, this can be unified with !SimDAP as a custom query, to provide a single interface.
Other Efforts
Some thoughts:
- Use standards, but don't wait for them.
- Sharing data now gets the science done, and builds experience and capabilities;
- this knowledge feeds back into the standards process.
This ideas were shown during the EuroVO Data Centre Alliance Workshop Theory in the Virtual Observatory (Proceedings). Several presenters showed how they were enabling collaborations, and publishing their data to a larger audience, without any current IVOA standards.
Computational Astrophysics Data Analysis Center (CADAC)
The CADAC is intended a community-driven resource for analyzing, sharing and publishing astrophysics simulations. It is based on our desire to publish several datasets from the Laboratory for Computational Astrophysics and a KITP Workshop on star formation.
CADAC's shortcomings lie in funding for storage and development. I have a request in for intramural funding over the next year for development, with some storage for University of California researchers. This is part of the bootstrapping process, and will hopefully lead to external funding.
CADAC Dev Site
SimCat
SimCat is the Django application that supports the simulation metadata in the CADAC. In the end, its goal is to support simple web service APIs, to simplify access from shared resources (such as NSF supercomputers), as well as IVOA standards such as !SimDB.
Attachments
- simplemodel.png (58.8 KB) - added by rpwagner 3 months ago.
