THE METADATA MAZE
Every national open data portal has one thing in common: too much metadata and too little search.
CKAN is excellent at cataloguing datasets, but when you have thousands of them, “Search” suddenly becomes the weakest link. Users type a keyword, hit enter… and pray. No context, no filtering by structure, no preview of schema, no semantic hints, no dataset-to-dataset relationships.
That’s fine if your portal has 50 datasets. But what if it has 50,000?
WHY STANDARD CKAN SEARCH IS LIMITED
CKAN’s default search relies on Solr full-text indexing, which means:
- You can search titles, descriptions, and tags.
- You can’t easily filter by custom metadata fields (like organization type, update frequency, model, or spatial coverage).
- You can’t browse metadata structures, like you would browse tables or JSON schemas.
- You can’t compare datasets by attributes (like what format they use, or which Smart Data Model they follow).
When we’re building a Data Catalog-as-a-Service (DCaaS) and Data Space-as-a-Service (DSaaS), that’s a problem.
Especially when clients expect to explore, compare, make modifications on metadata, not just data.
HOW WE GOT THERE
We noticed this while testing national open data portals across Europe - Slovenia, Austria, Spain, Finland, Switzerland - all running CKAN with huge metadata collections.
So we decided to extend it.
Enter: Extended Metadata Browser (EMB).
WHAT IS EMB
Extended Metadata Browser (EMB) is a search / filter / query / sort / group / custom views / select / clone / manipulate / import / export / layer for CKAN, DCaaS, DSaaS that lets users see metadata in context - hierarchically, relationally, and visually.
Think of it as “Google for your metadata”, but smarter:
- It doesn’t just search; it browses.
- It lets users search, filter, use custom filter, query, sort, group datasets by any metadata field (standard or custom).
- It lets users to prepare custom views based on search, filter, query, sort, group, column selection and store them for later.
- It lets users to export (csv, Excel, pdf), or duplicate selection to the other dataset or resource.
- It lets users to control harvesters.
- It builds a tree of relationships (organization → dataset → resource → schema → field).
- It supports DCAT-AP and it’s fully compatible with FIWARE and EU Data Spaces standards.
In simple words: EMB lets you explore metadata, not just
look for it.
HOW IT WORKS
First, we added a control button into CKAN - the Magic Wand
Then, we connected the Magic Wand with our new component - Extended Metadata Browser (EMB).
And then Magic happens:
The EMB directly connects to CKAN database. On top of that, we added a smart UI component - a DataGrid-based browser with before mentioned features and integrated it into our DCaaS and DSaaS.
Each user, tenant (or subdomain) can have its own configuration:
- Which metadata fields are visible.
- Which filters are available.
- How results are displayed.
The result? Users can jump from “datasets tagged with FIWARE” to “datasets using Smart Data Model EnergyConsumption” in one click.
EMB, DCaaS, AND DSAAS - BETTER TOGETHER
Our Data Catalog-as-a-Service (DCaaS) and Data Space-as-a-Service (DSaaS) already provide CKAN-based dataset hosting, and DCAT-AP exposure.
With EMB integrated, they become a real metadata experience - not just a static portal.
- CKAN manages datasets.
- EMB lets you explore them.
- FIWARE provides interoperability.
- Together they form the foundation of a data catalog and data space.
Great stories have a personality. Consider telling a great story that provides personality. Writing a story with personality for potential clients will assist with making a relationship connection. This shows up in small quirks like word choices or phrases. Write from your point of view, not from someone else's experience.
Great stories are for everyone even when only written for just one person. If you try to write with a wide, general audience in mind, your story will sound fake and lack emotion. No one will be interested. Write for one person. If it’s genuine for the one, it’s genuine for the rest.
FUTURE STEPS
The next step is to make EMB context-aware - understanding that “water quality” and “drinking water” might belong together.
We’re preparing to work on semantic expansion using NGSI-LD and ontology mapping, which would make browsing metadata even more intuitive.
(Yes, that means EMB could one day guess what you mean before you finish typing. Scary? Maybe. Useful? Definitely.)
THE PARETO PRINCIPLE APPLIED AGAIN
Like everything we build under FIWAREBox, EMB follows the same rule:
INVEST 20%, GAIN 80%.
Instead of rebuilding CKAN’s core, we added a layer that immediately improves usability for all portals - national or local, public or private.
Sometimes the simplest improvement - “show me what’s inside” - delivers the biggest value.
DISAGREE, COMMENT, OR WISH TO KNOW MORE?
