Skip to content

chore: test correctness fixes#4094

Draft
d-v-b wants to merge 13 commits into
zarr-developers:mainfrom
d-v-b:test-correctness-fixes
Draft

chore: test correctness fixes#4094
d-v-b wants to merge 13 commits into
zarr-developers:mainfrom
d-v-b:test-correctness-fixes

Conversation

@d-v-b

@d-v-b d-v-b commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

claude-generated fixes of claude-discovered issues in our test suite

Downstream gpu marker was never registered

src/zarr/testing/conftest.py defined pytest_configure to register the gpu marker, but the pytest11 entry point is zarr.testing (src/zarr/testing/__init__.py), which has no pytest_configure — so downstream projects consuming StoreTests under --strict-markers never got the marker.

  • Add pytest_configure to src/zarr/testing/__init__.py (registers gpu, comment kept).
  • Delete the now-dead src/zarr/testing/conftest.py and its [tool.coverage.run] omit entry.
  • Import assert_bytes_equal only when pytest is available (it imports pytest), matching StoreTests.

zarr-python's own suite still registers gpu via the main markers list, so local behavior is unchanged. Verified the marker now appears via the plugin in an isolated empty directory.

Pin real exception messages on broad pytest.raises

Added verified match= regexes to ~16 high-value broad-builtin pytest.raises in tests/test_api.py, tests/test_group.py, and tests/test_store/test_core.py. Every message was confirmed against the actual code, not guessed.

Remove silently no-op assertions

In tests/test_experimental/test_cache_store.py, dropped the if hasattr(cached_store, "_is_key_fresh") guards (the method exists on CacheStore) so the TTL/freshness checks always run.

xfail/skip hygiene

Added reason= to two reason-less xfail params in tests/test_indexing.py. The env-specific unconditional skip on test_setitem_repeated_index was left as-is (repeated-index assignment has undefined ordering and fails broadly, so no correct skipif exists).

Verification

  • pytest tests/test_api.py tests/test_group.py tests/test_store/test_core.py tests/test_experimental/test_cache_store.py tests/test_indexing.py: 1413 passed, 25 skipped, 5 xfailed.
  • Full-suite collection: 6479 tests, no errors. ruff/mypy pre-commit pass.

Part of #195

dependabot Bot and others added 11 commits May 31, 2026 19:28
…#176)

Bumps the actions group with 8 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` |
| [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` |
| [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` |
| [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` |
| [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` |
| [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` |
| [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` |



Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6
- [Release notes](https://github.com/prefix-dev/setup-pixi/releases)
- [Commits](prefix-dev/setup-pixi@1b2de7f...5185adf)

Updates `codecov/codecov-action` from 6.0.0 to 6.0.1
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@57e3a13...e79a696)

Updates `github/issue-metrics` from 4.2.2 to 4.2.7
- [Release notes](https://github.com/github/issue-metrics/releases)
- [Commits](github-community-projects/issue-metrics@c9e9838...1e38d5e)

Updates `j178/prek-action` from 2.0.3 to 2.0.4
- [Release notes](https://github.com/j178/prek-action/releases)
- [Commits](j178/prek-action@6ad8027...bdca6f1)

Updates `actions/upload-artifact` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v7...043fb46)

Updates `actions/download-artifact` from 7.0.0 to 8.0.1
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...3e5f45b)

Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.13.0...cef2210)

Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](zizmorcore/zizmor-action@b1d7e1f...5f14fd0)

---
updated-dependencies:
- dependency-name: prefix-dev/setup-pixi
  dependency-version: 0.9.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: codecov/codecov-action
  dependency-version: 6.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: github/issue-metrics
  dependency-version: 4.2.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: j178/prek-action
  dependency-version: 2.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/download-artifact
  dependency-version: 8.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: pypa/gh-action-pypi-publish
  dependency-version: 1.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.5.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Move downstream `gpu` pytest marker registration from
  `src/zarr/testing/conftest.py` into the pytest11 plugin entry point
  `src/zarr/testing/__init__.py` (the conftest was never loaded by
  downstream consumers of StoreTests under --strict-markers). Delete the
  now-dead conftest and drop its `[tool.coverage.run] omit` entry. As a
  side benefit, `assert_bytes_equal` is now imported only when pytest is
  available (it imports pytest), matching StoreTests.

- Add verified `match=` regexes to broad/bare `pytest.raises` in
  tests/test_api.py (test_create float shape/chunks, test_save_errors,
  test_save no-arrays), tests/test_group.py (KeyError key names,
  require_group TypeError), and tests/test_store/test_core.py
  (invalid zarr_format, make_store_path type, normalize_path). Each
  message was confirmed against the actual code.

- Remove `hasattr(cached_store, "_is_key_fresh")`-guarded assertions in
  tests/test_experimental/test_cache_store.py that could silently no-op;
  `_is_key_fresh` exists on CacheStore, so assert it directly.

- Add `reason=` to the two reason-less xfail marks in
  tests/test_indexing.py (structured dtypes unsupported in v3). The
  unconditional env-specific skip is left as-is (the operation is broadly
  broken, including locally, so no correct skipif condition exists).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jun 22, 2026
@d-v-b d-v-b changed the title Test correctness fixes chore: test correctness fixes Jun 22, 2026
@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.50%. Comparing base (a63ebd5) to head (68ec724).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4094   +/-   ##
=======================================
  Coverage   93.50%   93.50%           
=======================================
  Files          90       90           
  Lines       11979    11982    +3     
=======================================
+ Hits        11201    11204    +3     
  Misses        778      778           
Files with missing lines Coverage Δ
src/zarr/testing/__init__.py 90.90% <100.00%> (+3.40%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

conftest.py was removed from zarr.testing in this branch, but the
auto-API page, its mkdocs nav entry, and the testing index link still
referenced it, breaking `mkdocs build --strict`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rRWaxXfZ3ZmbiZYWZoDBP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs release notes Automatically applied to PRs which haven't added release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant