Backport superset + cross-catalog lineage fixes onto 1.12.11 + Looker UI base env var by joaopamaral · Pull Request #16 · Automattic/OpenMetadata

joaopamaral · 2026-06-24T13:04:57Z

What

Branch cut from upstream tag 1.12.11-release, bundling two open a8c upstream fixes plus a Looker UI-link change. All changes are ingestion-only Python.

Backported fixes

Superset DataModel↔Dashboard lineage via Chart — cherry-picked from PR fix(superset): bridge DataModel <-> Dashboard lineage through the Chart node open-metadata/OpenMetadata#28240 (968ab13c, 31b54d43). Adds _get_dashboard_entity / _get_chart_entity helpers and bridges the lineage edge through the Chart node.
Cross-catalog view lineage — cherry-picked from PR fix(lineage): resolve cross-catalog view lineage via database-wildcard fallback open-metadata/OpenMetadata#28983 (676015df). Resolves cross-catalog view lineage via a database-wildcard fallback in sql_lineage.py.

New: Looker `LOOKER_UI_BASE` env var

hostPort is bound to the Looker API host (the SDK sets LOOKERSDK_BASE_URL = hostPort), so the sourceUrls built from it for the dashboard and chart-merge "View in Looker" links point at the API host instead of the browser UI.

Added a _ui_base helper that prefers the LOOKER_UI_BASE env var and falls back to hostPort when unset (behavior unchanged). Swapped at the two hand-built link sites (/dashboards/{id}, /merge?mid=). No schema change — configured via the ingestion job env, e.g. LOOKER_UI_BASE=https://looker-wyeast.a8c.com.

Why

Superset charts/datamodels now appear connected in the lineage graph.
Cross-catalog Trino/Iceberg view lineage resolves instead of dropping.
"View in Looker" links open the UI host, not the API host.

Notes

Mixin cherry-pick had one conflict (_get_datamodel signature) — resolved keeping both new helpers and 1.12.11's multi-line signature.
1.12.11 has no view sourceUrl (the buggy Fix #23539: Add looker explore source url open-metadata/OpenMetadata#26436 local-path code is not in this base), so the version-churn issue does not apply here and nothing was re-added.

Testing

py_compile clean on edited files.
test_superset_chart_lineage.py (from fix(superset): bridge DataModel <-> Dashboard lineage through the Chart node open-metadata/OpenMetadata#28240) included; run with the ingestion test env (metadata.generated must be built).

…rt node OpenMetadata's lineage graph traverses through explicit AddLineageRequest edges. For Superset the current behaviour was: - `Dashboard.dataModels` was never set, so the structural link from the dashboard to its datamodels was missing in the dashboard's Data Models panel. - `DashboardServiceSource.yield_datamodel_dashboard_lineage()` emitted a direct `DataModel -> Dashboard` lineage edge, so the lineage graph showed datamodels connected to the dashboard but skipped the chart entirely. Net result in production: the dashboard's lineage view showed `Table -> DataModel -> Dashboard` and the chart was invisible in the graph, even though it was a member of the dashboard. This change rewires the relationship so the chart is the bridge node: 1. `SupersetSourceMixin.yield_dashboard_lineage_details` now also emits `DataModel -> Chart` and `Chart -> Dashboard` edges for each chart on the dashboard. Combined with the existing `Table -> DataModel` edge the full chain in the lineage view becomes `Table -> DataModel -> Chart -> Dashboard`. 2. `SupersetSourceMixin.yield_datamodel_dashboard_lineage` is overridden to yield nothing, suppressing the base class' direct `DataModel -> Dashboard` edge so the datamodel doesn't render alongside the chart as two parallel paths into the dashboard. 3. `db_source.yield_dashboard` and `api_source.yield_dashboard` now send `Dashboard.dataModels=[]` (an empty list, not absent) on every PUT so any datamodel entries persisted from prior runs are deleted by the server. The relationship is fully represented by the lineage chain. Two helpers — `_get_chart_entity` and `_get_dashboard_entity` — resolve the Chart and Dashboard entities by FQN via `metadata.get_by_name`. If either isn't yet visible on the server (first run, before the entity POST has been committed), the bridge edges are skipped without crashing and the next run picks them up. Tests ----- Adds `tests/unit/topology/dashboard/test_superset_chart_lineage.py` with five Mockito-style unit tests: - `test_db_source_yields_empty_data_models` / `test_api_source_yields_empty_data_models`: Both sources produce `CreateDashboardRequest` with `dataModels=[]`. - `test_override_yields_no_edges`: the `yield_datamodel_dashboard_lineage` override produces zero edges. - `test_chart_bridge_edges_emitted`: when the chart and dashboard entities resolve, `yield_dashboard_lineage_details` emits both `DataModel -> Chart` and `Chart -> Dashboard` edges. - `test_no_chart_entity_skips_bridge_edges`: when the chart isn't yet resolvable on the server, both bridge edges are skipped gracefully. Three of the five tests fail on the unpatched code, exercising each of the three behaviour changes above. Manually verified in production (Superset metadata-DB connection mode) against a real Meltano dashboard: after this fix the dashboard's lineage graph renders `Table -> DataModel -> Chart -> Dashboard` with the chart visible as the bridge node, and the API confirms `Dashboard.dataModels` is cleared while all `DataModel -> Chart` and `Chart -> Dashboard` edges are present. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> # Conflicts: # ingestion/src/metadata/ingestion/source/dashboard/superset/mixin.py

…exing in test Two review comments from gitar-bot on open-metadata#28240: - mixin.yield_dashboard_lineage_details was calling self._get_dashboard_entity(dashboard_details) inside the per-chart loop, doing N identical metadata.get_by_name lookups for a dashboard with N charts. Hoist the call out of the loop and reuse the result. - test_superset_chart_lineage.py was wiring SupersetSourceMixin.__mro__[1]._get_add_lineage_request onto a mock, which silently breaks if a new intermediate base class is inserted. Reference DashboardServiceSource._get_add_lineage_request directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…d fallback When the same physical data is exposed through different catalogs across database services, a view in one catalog can reference a source table that another service ingested under a different database name. get_table_entities_ from_query searched every service (including crossDatabaseServiceNames) with the view's database name, so the source table under a different database never matched and no lineage edge was created -- even with processCrossDatabaseLineage and crossDatabaseServiceNames configured. Add a final database=None fallback (after the exact and schema fallbacks) so the FQN search wildcards the database (<service>.*.<schema>.<table>) and resolves the table across the configured cross-database services. It is scoped to service_names, which only holds the current service plus the explicitly allowed crossDatabaseServiceNames, so it cannot reach unrelated services. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Joao Amaral <7281460+joaopamaral@users.noreply.github.com>

hostPort is bound to the Looker API host (SDK LOOKERSDK_BASE_URL), so the dashboard and chart-merge sourceUrls built from it point at the API host instead of the browser UI. Add a _ui_base helper that prefers the LOOKER_UI_BASE env var and falls back to hostPort (unchanged behavior when unset). No schema change - configured via ingestion job env.

github-actions · 2026-06-24T13:05:28Z