At the College Park Interop

Uneasy Logistics

This time, the event takes place in College Park, Maryland, in the metro area of Washington, DC. And that has been a bit of an issue with respect to “congregating”, because many of the regular Interop attendees were worried by news about extra troubles with US border checks. In consequence, we will only have about 40 on-site participants (rather than about 100, as is more usual for Interops); the missing people have promised to come in via Zoom, though.

Right now, in the closed session of the Technical Coordination Group, (TCG) where the chairs of the various Working and Interest Groups of the IVOA meet, this feels fairly ok. But then more than half of the participants are on-site here. Also, the room we are in (within the Edward St. John Learning and Teaching Center pictured above) is perfectly equipped for this kind of thing, what with microphones in each desk, and screens everywhere.

I am sure the majority-virtual situation will not work at all for what makes conferences great: the chats between the sessions. Let's see how the usual sessions – that mix talks and discussion in various proportions – will work in deeply hybrid.

TCG: Come again?

The TCG, among other things, has to worry about rather high-level, cross-working-group, and hence often boring topics. For instance, we were just talking about how to improve the RFC process, the way we discuss whether and how a draft standard (“Proposed Recommendation”) should become a standard (“Recommendation”). This, so far, happens on the Twiki, which is nice because it's stable over long times (20 years and counting). But it also sucks because the discussions are hard to follow and the link between comments and resulting changes is loose at best. For an example that should illustrate the problem, see the last RFC I ran.

Since we're sold to github/Microsoft for our current standards process anyway, I think I'd rather have the discussions on github, and in some way we will probably say as much in the next version of the Document Standards. But of course there are many free parameters in the details, which led to quite a bit more discussion than I had expected. I'm not sure whether we sometimes crossed the border to bikeshedding; but we probably didn't.

Here's another example of the sort of infrastruture talk we were having: There is now a strong move to express part of our standards' content machine-readably in OpenAPI (example for TAP). Again, there are interesting details: If you read a standard, how will you find the associated OpenAPI files? Since these specs will rather certainly include parts of other standards: how will that work technically (by network requests or in a single repository in which all IVOA OpenAPI specs reside)? And more importantly, can a spec say “I want to include a specific minor version“? Must it be minor version-sharp, and how would that fit with semantic versioning? Can it say “latest”?

This may appear very far removed from astronomy. But without having good answers as early as possible, we will quite likely repeat the mess we have had with our XML schemas (you would not guess how much curation went into this) and in particular their versioning. So, good thing there are the TCG sessions even if they sometimes are a bit boring.

Opening session (2025-06-02, 14:30)

The public part of the conference has started with Simon O'Toole's overview over what was going on in the VO in the past semester. Around page 36 of these slide set, updates from the Rubin Observatory say what I've been saying for a long time:

If you don't understand what they are talking about, don't worry too much: It's a fairly technical detail of writing VOTables, where we did a fix of something rather severly broken in 2012.

The entertaining part about it, though, is that later in the conference, when I will talk about the challenges of transitioning between incompatible versions of protocols, BINARY2 will be one of my examples for how such transitions tend to be a lot less successful than they could be. Seeing takeup by players of the size of Rubin almost proves me wrong, I think.

Charge to the Working Groups (2025-06-02, 15:30)

This is the session in which the chairs of the Working and Interest Groups discuss what they expect to happen in the next few days. Here is the first oddity of what I've just called deeply virtual: The room we are in has lots of screens along the wall that show the slides; but there is no slide display behind the local speaker:

If you design lecture halls: Don't do that. It really feels weird when you stand in front of a crowd and everyone is looking somewhere else.

Content-wise, let me stress that this detail from Grégory's DAL talk was good news to me:

This means that the scheme for distributing spectral line data that Margarida and I have been working on for quite a while now, LineTAP (last mentioned in the Malta post), is probably dead; the people who would mostly have to take it up, VAMDC, are (somewhat rightly) scared of having to do a server-side TAP implementation. Instead, they will now design a parameter-based interface.

Even though I have been promoting and implementing LineTAP for quite a while, that outcome is fine with me, because it seems that my central concern – don't have another data model for spectral lines – is satisfied in that that parameter-based interface (“SLAP2”) will build directly upon VAMDC's XSAMS model, actually adopting LineTAP's proposed table schema (or something very close) as the response table. So, SLAP2, evolved in this way, seems like an eminently sensible compromise to me.

Tess gave the Registry intro, and it promises a “Spring Cleaning Hackathon” for the VO registry. That'll be a first for Interop, but one that I have wished for quite a while, as evinced by my (somewhat notorious) Janitor post from 2023. I am fairly sure it will be fun.

Data Management Challenges (2025-06-03, 10:30)

Interops typically have plenary sessions with science topics, something like “the VO and radio astronomy“. This time, it's less sciency, it's about “Data Management” (where I refuse to define that term). If you look at the session programme, it's major projects telling you about their plans for how to deal with (mostly large) new data collections. For instance Euclid has to deal with several dozen petabytes, and they report 2.5 million async TAP queries in the three months from March, which seems incredibly much. I'd be really curious what people actually did. As usual: if you report metrics, make sure you give the information necessary to understand them (of course, that will usually mean that you don't need the metrics any more; but that's a feature, not a bug). In this case, it seems most of these queries are the result of web pages firing off such queries on rendering.

More relevant to our standards landscape, however, is that ESA wants to make the data available within their, cough, Science Data Platform, i.e., computers they control and that are close to the data. To exploit that property, in data discovery you need to somehow make it such that code running on the platform can find out file system paths rather than HTTP URIs – or in addition to them? We have already discussed possible ways to address such requirements in Malta, without a clear path forward yet that I remember. Pierre, the ESA speaker, did not detail their plan; let's hope there is time to review their plans in the discussion time.

In the talk from the Roman people, I liked the specification of their data reduction pipeline (p. 8 ff); I think I will use this as a reference for what sort of thing you would need to describe in a full provenance model for the output of a modern space telescope. On the other hand, this slide made me unhappy:

Admittedly, I don't really what use case these pre-baked table files serve, but I am rather sure that efficiency-wise having Parquet files (which they intend to use anyway) with VOTable metadata as per Parquet in the VO wouldn't make much of a difference; but you'd be much closer to proper VO metadata, which to me would sound like a big win.

The remaining two talks in the session covered fairly exotic instruments: SphereX, which scans the sky with spectral resolution, and COSI, a survey instrument for MeV gamma rays (like, for instance: ⁶⁰Fe) with the usual challenges for making something like an image out of what falls out of your detector, including the fact that the machines' point spread function is a cone:

How exciting.

Registry (2025-06-03, 12:30)

I'm on my home turf: The Registry Session, in which I will talk about how to deal with continuously updated resources. But before that, Renaud, the current chair of the Registry WG, pointed out something I did (and reported on here): Since yesterday, pyVO 1.7 is out and hence you can use the UAT constraint with semantics built-in:

Ha! The experience of agency! And I'm only having half a smiley here.

Later in the session, Gilles reported on the troubles that VizieR still has with the VOResource data model since many of their resources contain multiple tables with coordinates and hence multiple cone search services, and it is impossible in VODataService to say which service is related to which table. This is, indeed, a problem that will need some sort of solution. And I still think that the right solution would be to fix cone search rather than try and fiddle things in the registry.

He also submitted something where I'm not sure whether I ought to consider it a bug report; here are match counts for three different interfaces on top of roughly equivalent metadata:

I think we'll have to have second and third looks at this.

Tuesday (2025-05-03) Afternoon

I was too busy to blog during yesterday's afternoon sessions, Semantics (which I chaired in my function as WG chair emeritus because current chair and vice chair were only present remotely) and then, gasp, the session on major version transisitons. This last event was mainly a discussion session, where everyone agreed that (a) as a community, we should be able to change our standards in ways that break existing practices lest we become sclerotic and that (b) it's a difficult thing and needs management.

In my opening pitch, I mentioned a few cases where we didn't get the breaking changes right. Let's try to make it better next time, and at the session, some people signalled they'd be in on updating Simple Cone Search from the heap of outdated legacy that it now is into an elegant protocol nicely in line with modern VO standards. Now, if only I could bring myself to imagine the whole SCS2 business as something I would actually want to do.

If you are reading this and feel you would like to pull SCS2 along with me: Do write in.