chore(deps): update dependency kafka-python to v3

[renovate-bot]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
kafka-python (changelog)	==2.0.6 → ==3.0.0

---

Release Notes

dpkp/kafka-python (kafka-python)

v3.0.0

Compare Source

This is a major release with significant changes to kafka-python internals to simplify networking and feature development. It introduces a new networking layer (kafka.net) and a dynamic protocol system that uses JSON schema files imported from Apache Kafka. It substantially refactors and expands the Admin client, including breaking changes to some API signatures, and it lands a long list of KIP features/changes across the producer, consumer, admin, and networking/metadata clients. Protocol support across kafka-python is now at or beyond the apache kafka 3.0 baseline.

Breaking Changes

Python Compatibility

• Drop python2 support (#​2699, #​2774, #​2837, #​2846)

Default Configuration Changes

• KIP-679: Update Producer defaults -- enable_idempotence=True, acks='all' (#​3013)
• KIP-735: Increase default consumer session_timeout_ms from 10s to 45s (#​3030)
• Rename api_version_auto_timeout_ms -> bootstrap_timeout_ms; default 30s (#​3028)
• Producer: Remove deprecation warning for buffer_memory config (now raises) (#​3047)

Abstract Interface Changes

• Serializer/Deserializer: Pass headers to serialize/deserialize (#​3046)
• Partitioner: pass both key/value, serialized and unserialized, to partition() (#​3045)

Admin API Changes

• Admin: change response shapes to simple dicts (#​2883)
• Admin: groups apis renames; add offsets, members apis (#​2892)
• Admin: Deprecate NewTopics/NewPartitions in favor of simple dicts (#​2869)

Consumer API Changes

• Consumer: use assignor instances, not classes (#​2775)

Error Hierarchy

• KafkaError subclass Exception not RuntimeError (#​2932)
• Make IncompatibleBrokerVersion a subclass of UnsupportedVersionError (#​2924)
• Eliminate NoBrokersAvailableError (#​2942)
• KafkaProtocolError is not retriable (#​2941)
• Prefer raised Exceptions to assert / AssertionError (#​3042)

Old Networking Stack Removal

• Remove kafka.client_async / kafka.conn legacy modules (#​2918)
• Drop version probes for pre-0.10/ApiVersionsRequest brokers

SASL Module Rename

• Rename kafka.sasl -> kafka.net.sasl

Networking (kafka.net)

Complete refactor of the networking layer using a bespoke event-loop supporting async/await (but no asyncio yet). All three clients (Admin, Consumer, Producer) use a dedicated IO thread that drives a selector-based event loop. AdminClient/KafkaConsumer leverage a built-in io thread supplied by kafka.net, KafkaProducer continues to use its existing background Sender thread for now.

Async IO Substrate

• kafka.net: Refactored async networking modules (generator-based coroutines) (#​2812)
• kafka.net: Network IO thread (#​2965, #​2980, #​2961, #​2968, #​2963)
• KafkaNetClient: drop-in replacement for KafkaClient using kafka.net (#​2816)
• KafkaNetSocket: interface class for socket and proxy connections (#​2992)
• kafka.net.manager: Add call_soon(coro) and run(coro) for sync/async bridge (#​2862)
• kafka.net.manager: Bootstrap is sync/blocking (#​2919)
• connection: short-circuit send/recv when closed (#​2967)
• Clamp broker_version to user-supplied api_version (#​3052)

Transports and Proxies

• Default SSLContext -> PROTOCOL_TLS_CLIENT; minimum version TLS 1.2 (#​2807)
• Enable TCP KeepAlive as default socket option (#​2904)
• kafka.net: HTTP CONNECT proxy support (RFC 7231 s4.3.6) (#​2990)

SASL Authentication

• kafka.net: Validate SASL/SCRAM iterations (#​3026)
• SASL: Prefer node hostname to IP address when building mechanisms (#​3003)

Timeouts and Connection Management

Per-request and per-stage timeouts replace the old single client-wide timeout.

• KIP-601: support socket_connection_setup_timeout_ms w/ exponential backoff (#​3027)
• kafka.net: Bootstrap if needed for metadata refresh (#​2825)
• kafka.net: Raise UnknownBrokerIdError when connection fails because node_id is not in metadata (#​2876)

Futures and Wakeups

The Future primitive gains __await__ and a faster slotted implementation; cross-thread wakeups are factored out into a reusable helper.

• Future.await support (#​2811)
• Future: slots and callback/errback inlining optimization (#​2848)
• Future: clear callbacks/errbacks when done to avoid reference cycles (#​2891)
• WakeupNotifier primitive for cross-thread/task wakeups (#​2925, #​2933)

Concurrency and Error Handling

Defensive checks throughout the kafka.net event loop and transport stack: improved socket I/O error handling, RuntimeErrors on misuse of the IO thread, and lock-based detection of concurrent access.

• kafka.net.selector: Break scheduled heapq ties when tasks share scheduled_at (#​3007)
• kafka.net.selector: Monitor slow tasks with slow_task_threshold_secs (#​2946)
• kafka.net.selector: Use threading.Lock() to detect concurrent access to poll() (#​2945)
• kafka.net.selector: Track pending tasks to prevent gc before completion (#​2950)
• kafka.net.selector: Support reschedule(when, task); idempotent unschedule (#​2939)
• kafka.net.selector: Raise RuntimeError on concurrent access to net.poll or wakeup() (#​2938)
• kafka.net.selector: Raise RuntimeError on run/call_at/call_soon_threadsafe after closed (#​2971)
• kafka.net.selector: Catch unhandled exceptions in IO thread (#​2970)
• kafka.net.selector: Improve error handling on sock read/write (#​2995)
• kafka.net.selector: Check locks in _poll_once; add net.drain() (#​2949)
• kafka.net.transport: Close connection on socket write error (#​2973)

Protocol

A new JSON-schema-based dynamic protocol generator now replaces the legacy hand-written protocol classes (moved to kafka.protocol.old).

Dynamic Protocol Classes

Protocol classes are now generated from the upstream Apache Kafka JSON schemas.

• Dynamic protocol classes using upstream json schemas (#​2727, #​2745, #​2779, #​2782, #​2787, #​2810, #​3037)
• Add .pyi type annotation stubs for generated protocol classes (#​2784)
• Migrate all internal usage to new protocol classes (#​2764, #​2765, #​2766, #​2767, #​2768, #​2772)
• Refactor treatment of versioned ApiMessage classes (#​2739)
• Lookup request header -> response class via ResponseClassRegistry (#​2730)
• Manual json schema edits to restore fields dropped in Apache 4.0 (#​2738)

New Types and Messages

• Add UUID protocol type (#​2703)
• Add UnsignedInt16 type
• KIP-893: support nullable structs (#​2889)
• Replace kafka.structs BrokerMetadata/PartitionMetadata with MetadataResponse structs (#​2794)
• Add ConsumerProtocol data schemas (#​2754)
• Add StickyAssignorUserData json schema (#​2755)

Performance

• Protocol encode/decode optimization with inline compile/exec (#​2785)
• Protocol benchmarks and profiling (make bench-protocol) (#​2783)

Protocol Fixes

• Fix TaggedFields encoding/decoding (#​2725, #​2745, #​2779)
• Fix VarInt/VarLong encoding (#​2706)
• Fix CompactBytes encoding of struct data (#​2782)
• Fix compiled encode_into for None arrays (#​2790)
• Validate network frame size (#​3019)
• Validate DataContainer version (#​2759)

Helpers and Debugging

• DataContainer.to_dict() helper (#​2758, #​2872, #​2879)
• Store in-flight request headers only for protocol parser (#​2723)
• Debug log send/recv bytes from protocol parser (#​2707)
• Adjust protocol debug logging; add KAFKA_PYTHON_PROTOCOL_DEBUG_LOG (#​2719)
• CoordinatorType enum (GROUP/TRANSACTION/SHARE) (#​3049)

Broker Version Check

Broker version inference is consolidated into a single BrokerVersionData helper that tracks the broker's reported API versions and infers a broker version string. ApiVersionsRequest is always sent on connect.

BrokerVersionData

• BrokerVersionData: consolidated version checks (#​2795)
• BrokerVersionData: infer up to 4.3 (#​2835, #​2836, #​3032)
• BrokerVersionData: support request min/max version (#​2868)
• BrokerVersionData: Fix IncompatibleBrokerVersion errors; add str (#​2804)

ApiVersionsRequest

• Always send ApiVersionsRequest on connect (#​2802)
• Improve ApiVersionsRequest fallbacks (#​2803, #​2817, #​2821)

Client Bootstrap

• Fix bootstrap connection error handling (#​2831)
• ClusterMetadata: bootstrap_brokers(), set_topics/add_topics/metadata_request (#​2805, #​2792, #​2796, #​2797)
• Move bootstrap_brokers fallback from brokers() to least_loaded_node() (#​2809)
• KafkaClient.least_loaded_node: pass bootstrap_fallback=True if needed (#​2830)
• Consumer: add explicit bootstrap() method (#​2975)
• Admin: explicit bootstrap on init (#​2864)
• Manage metadata refresh logic in ClusterMetadata via attached manager (#​2920)
• kafka.cluster: End refresh loop on close(); refresh loop catches KafkaError (#​2935, #​2936)
• Respect metadata backoff in KafkaNetClient (#​2854)

Consumer

KafkaConsumer drops the dedicated HeartbeatThread in favor of scheduled async tasks on the kafka.net IO thread. Internals have been substantially refactored to migrate from future callbacks to async/await syntax. Feature support added for incremental cooperative rebalancing (KIP-429), rack-aware fetch (KIP-392), and log truncation detection (KIP-320/KIP-595).

Threading and IO

All consumer network I/O now flows through the shared kafka.net IO thread; consumer.poll() no longer drives the event loop directly.

• Consumer: use background thread for all network io; drop HeartbeatThread (#​2965)
• Consumer: send all requests from net io thread (#​2980)
• Consumer: simplify poll() with fetcher.fetch_records (#​2960)
• Consumer: sleep in poll() if timeout, no records, and no fetchable partitions (#​3039)
• Consumer: drop poll loop optimizations for pending offset resets and rejoins (#​2959)
• Consumer: _update_fetch_positions -> _refresh_committed_offsets; dont poll in position() (#​2958)
• Consumer: convert fetcher reset_offsets/send_list_offsets_requests to async def
• Consumer: use new proto attrs in Fetcher (ListOffsets/Fetch) (#​2923)

Group Membership and Rebalance

• KIP-429: Incremental Cooperative Rebalance (#​2999)
• KIP-429: Add on_partitions_lost rebalance listener hook (#​3016)
• KIP-559: Bump JoinGroup v7 / SyncGroup v5; verify protocol_type/name (#​2998)
• Rebalance Listener: raise errors, block fetches on revoked, pre-revoke unsubscribed (#​3018)
• Consumer: call RebalanceListener on close (#​3020)
• Preserve member id on IllegalGeneration error (#​3017)

Fetch

• KIP-320: Detect and handle Log Truncation (#​2978)
• KIP-392: Rack-aware fetch from closest replica (#​2986, #​2996)
• KIP-595: Fetch v12 support; check for log truncation and leader updates in response (#​3021)
• KAFKA-7548: Retain fetched data for paused partitions (#​2981)
• KAFKA-9212: Leader Epoch unreliable before Metadata v9 (#​2997)

Offsets and Commits

• KIP-447: OffsetFetch v7 set require_stable flag if read_committed (#​3023)
• Consumer: handle retriable errors in offsets_for_times (#​3022)
• Consumer: Improve retriable offset fetch error handling (#​2977)
• Consumer: Retain subscription TopicPartitionState when possible (#​3001)
• Consumer: drop max_version clamp on ListOffsetsRequest (#​3012)

Configuration

• KIP-602: Support client_dns_lookup in Consumer/Producer/Admin config (#​3004)
• Consumer: default request_timeout_ms 30s; use request-specific timeout for JoinGroup (#​3011)

Consumer Fixes

• Retry metadata request if tracked topics have retriable errors (#​2832)
• Fix KeyError in KafkaConsumer.committed() (#​2710)
• Fix Fetcher._fetch_offsets_by_times retry handling (#​2833)

Producer

KafkaProducer gains a sticky partitioner (KIP-480), enabled-by-default idempotence (KIP-679), tightened transaction handling, and a faster send/encode path.

Transactions and Idempotence

• KIP-360 (pt1): Add transaction manager state and helper methods (#​2852)
• KIP-360 (pt2): Bump producer epoch with InitProducerIdRequest v3 on 2.5+ brokers (#​2853)
• KIP-447: Include group metadata in TxnOffsetCommitRequest (#​2984, #​2988)
• KIP-654: Txn Producer aborts with non-fatal TransactionAbortedError (#​3010)
• KAFKA-5793: Tighten up semantics of OutOfOrderSequenceNumber (pt1) (#​2843)
• Enable producer idempotence with max_in_flight_requests_per_connection > 1 (#​2841)
• Producer: Enforce guaranteed message order when idempotence_enabled (#​2937)
• Producer: treat InvalidProducerEpochError as ProducerFencedError (#​2885)
• Producer: Improve transaction manager retriable error handling (#​2884)
• Producer: Use new-style request construction in Txn Manager (#​2983)

Partitioning

• KIP-480: StickyPartitioner for KafkaProducer (#​2982, #​2989)

Batching and Send Path

Split-and-resend oversized batches instead of failing; avoid redundant validation and buffer copies on hot send-path.

• KIP-126: Allow KafkaProducer to split and resend oversized batches (#​2839)
• Refactor producer.send / _wait_on_metadata for fast path (#​2850)
• Only ensure_valid_topic_name new topics on send (#​2849)
• Avoid unnecessary bytes/bytearray copies on send (#​2845)
• MemoryRecordsBuilder.build() returns bytearray buffer not bytes copy (#​2842)
• Drop PartitionResponse namedtuple; consolidate response and error handling (#​2851)

Producer Fixes

• Fixup Sender error class logging (#​2828)

Admin Client

Split KafkaAdminClient into focused mixin classes (cluster, topics, configs, groups, ACLs, log dirs, etc), and convert request-sending path to async def methods that run on the kafka.net IO thread. Support for new KIPs using new protocol stack.

Refactor and Async Migration

The admin client interface remains sync but wraps a fully-async internal api (does not support asyncio yet). Adds cached coordinator lookups and a mixin structure to separate logical resource groups.

• Admin: split into functional mixin classes (#​2873, #​2877, #​2882)
• Admin: convert request paths to async; cache coordinator_ids (#​2851, #​2862, #​2863, #​2866, #​2867, #​2870, #​2871, #​3050)
• Admin: refactor _send_request_to_controller error handling (#​2751)

Batch Protocol Support

• KIP-699: FindCoordinatorRequest v4 -- multi-group support (#​3025)
• KIP-709: OffsetFetch v8 -- use batch interface when available (#​3024)

New Cluster and Quorum APIs

• Admin: describe_metadata_quorum (#​2914)
• Admin: cluster features describe/update (#​2908, #​3053)
• Admin: cluster get_broker_version_data / api_versions (#​2903)

Configs

• Admin: list_config_resources (requires 4.1+) (#​2900)
• Admin: support incremental alter configs (#​2901)
• Admin: Improve alter_configs w/ filters and missing keys (#​2899)

Topics and Partitions

• KIP-516: Support topic id (uuid) for describe topics (#​3031)
• Admin: re-use config processing for CreateTopicsResponse (#​3036)
• Admin: wait_for_topics() and create_topics() wait_for_metadata option (#​2856)
• Admin: list_partition_offsets (#​2894)
• Admin: Expand partitions api support (#​2890)
• Admin: Cleanup alter_partition_reassignments (#​3002)

Groups

• Admin: include latest offsets and lag in list_group_offsets; add reset_group_offsets (#​2897)
• Admin: support group state/type filters for list_groups (#​2910)
• Admin: add extended group reset options (#​2911)
• Admin: Dont return MemberToRemove as key in remove_group_members dict result (#​2893)

Transactions

• Admin: implement KIP-664 hanging-transaction tooling (#​3051)

Log Dirs

• Admin: alter_log_dirs (#​2912)
• Admin: Fix describe_log_dirs, accept optional topics/brokers (#​2881)

Reliability

• Admin: retry delete_records / list_partition_offsets on NotLeaderForPartitionError (#​2976)

CLI

The CLI adds shared parser config, SASL/SSL connection support across all subcommands, and several new admin subcommands (acls, configs alter, users).

Common Infrastructure

• Add kafka-python cli script to wrap admin/consumer/producer (#​3034)
• kafka.cli: common parser args; support sasl/ssl connections (#​2887)
• kafka.cli: common configuration for logging and connect kwargs (#​2906)
• Add --enable-logger/--disable-logger to cli options (#​2798)
• Set default cli log level => CRITICAL (#​2760)

Admin CLI

• admin cli: acls, users, configs alter (#​2888)
• admin cli: refactor admin group/command parsers; consumer/producer option groups (#​2909)
• admin cli: --id support for describe-topics
• admin cli: close() after running command
• admin cli: dont print stacktrace for BrokerResponseError or ValueError (#​2895)
• admin cli: catch AttributeError and print_help() (#​2880)
• admin cli: fix describe_configs (#​2875)

Consumer CLI

• consumer cli does not require group_id (#​3044)

Compatibility / Misc

Small quality-of-life additions to the public API surface.

• Support context manager interface for consumer/producer/admin (#​2969)
• Make IncompatibleBrokerVersion a subclass of UnsupportedVersionError (#​2924)
• Add OffsetSpec / IsolationLevel to kafka imports (#​2898)
• Errors: subclasses for RetriableError and InvalidMetadataError (#​3041)
• Helper classes: DefaultSerializer and JsonSerializer (#​3046)

Fixes

Codec and Python-3-compatibility fixes that aren't specific to a single client.

• Fix zstd multi-frame decompression failure (#​2717)
• Use time.monotonic() instead of time.time() for elapsed time calculations (#​2714)
• Fix deprecated log.warn() -> log.warning()
• REF: Switch to deque.copy() for Python 3 compatibility (#​2712)

Tests

A new in-memory MockBroker / MockTransport enables deterministic protocol-level tests, and the integration test fixtures have been substantially consolidated.

MockBroker and Fixtures

• MockBroker / MockTransport for deterministic protocol tests (#​2861, #​2902)
• Organize test files into consumer/ producer/ admin/ directories (#​2844)
• Consolidate consumer integration tests (#​2857)
• Consolidate more pytest fixtures (admin/client) (#​2921)
• Integration fixtures use contextmanager (#​2966)
• Refactor coordinator test fixtures; prefer MockBroker to patched_coord (#​2953)
• Support SSL transports in KafkaFixture (#​2806)
• Simplify interacting with KafkaFixture (client factories, create_topics) (#​2808)
• Use producer/consumer/admin factories in producer_integration tests (#​2829)
• Set api_version for integration tests; fix test_group missing consumers (#​2824)
• Close local fixtures in tests (#​2962)
• Catch exceptions in fixture.open() -> close() (#​2907)
• Create topic on 0.8.2 broker to fix bootstrap (#​2896)

Reliability

• Reduce timing flakiness in test_group (#​3006)
• Test timeouts: use pytest-timeout method=thread and add faulthandler

Project Infra

• Enable pylint workflow; disable/fix all outstanding errors (#​2701)
• Add coverage step to CI (#​2786)
• CI test matrix updated to python 3.14 / kafka 4.3 (#​2836, #​3032)
• Add lint-unicode make target; include in make lint test (#​2993)
• pyproject updates: no more universal wheels; requres py3.8+; fix license files; drop setup.py (#​3033)
• Bump actions/cache from 4 to 5; actions/upload-artifact from 4 to 7 (#​2702, #​2789)

v2.3.2

Compare Source

2.3.2 (Jun 3, 2026)

Fixes

• kafka.conn: Improve error handling for sasl authenticate mechanisms
• kafka.net: Validate SASL/SCRAM iterations (#​3026)
• KafkaProtocol: validate network frame size (backport of #​3019)
• Clear _callbacks/_errbacks list when future is_done to avoid reference cycles (#​2891)

v2.3.1

Compare Source

Fixes

• Fix TaggedFields value encoding; add test coverage (#​2725)
• Fix zstd multi-frame decompression failure (#​2717)
• Fix KeyError in KafkaConsumer.committed() (#​2710)
• Fix VarInt/VarLong encoding; move tests to test/protocol/ (#​2706)
• Fix Fetcher._fetch_offsets_by_times retry handling (#​2833)
• Fixes to support integration testing with external KAFKA_URI (#​2838)
• Minor py2 fixes: consumer integration test; dont pip install python-snappy

v2.3.0

Compare Source

CLI

• python -m cli interfaces for kafka.admin, kafka.consumer, kafka.producer (#​2650)

Producer

• KIP-654: Abort transaction with pending data with TransactionAbortedError (#​2662)
• KafkaProducer: Handle UnknownProducerIdError (#​2663)
• KIP-467: Augment ProduceResponse error messaging for specific culprit records (#​2661)

Consumer

• KIP-345: Consumer group static membership (#​2625)
• KIP-207: Add ListOffsetsRequest v5 / handle OffsetNotAvailableError (#​2657)
• Fetcher: Add missing argument in debug log (#​2665)

AdminClient

• KIP-430: Return Authorized Operations in Describe Responses (#​2656)
• Add send_request() and send_requests() to KafkaAdminClient (#​2649)

Maintenance

• Remove old/unused kafka.protocol.pickle (#​2653)
• Switch protocol code to getattr/setattr from dict (#​2654)
• Drop unused imports (#​2651)

Project Infra

• Bump github/codeql-action from 3 to 4 (#​2678)
• Bump actions/setup-python from 5 to 6 (#​2674)
• Bump actions/setup-java from 4 to 5 (#​2673)
• Bump actions/checkout from 4 to 5 (#​2669)
• Bump actions/checkout from 5 to 6 (#​2694)
• NixOS helpers

v2.2.20

Compare Source

2.2.20 (Jun 3, 2026)

Fixes

• kafka.conn: Improve error handling for sasl authenticate mechanisms
• kafka.net: Validate SASL/SCRAM iterations (#​3026)
• KafkaProtocol: validate network frame size (backport of #​3019)
• Clear _callbacks/_errbacks list when future is_done to avoid reference cycles (#​2891)

v2.2.19

Compare Source

Fixes

• Fix TaggedFields value encoding; add test coverage (#​2725)
• Fix zstd multi-frame decompression failure (#​2717)
• Fix KeyError in KafkaConsumer.committed() (#​2710)
• Fix VarInt/VarLong encoding; move tests to test/protocol/ (#​2706)
• Fix Fetcher._fetch_offsets_by_times retry handling (#​2833)
• Fixes to support integration testing with external KAFKA_URI (#​2838)
• Minor py2 fixes: consumer integration test; done pip install python-snappy

v2.2.18

Compare Source

Fixes

• Add ProducerBatch.lt for heapq (#​2698)

v2.2.17

Compare Source

Fixes

• Add internal poll to consumer.position() (#​2696)
• Initiate Coordinator Reconnect w/ Backoff from Heartbeat Thread (#​2695)

Networking

• SOCKS5: support looking up names remotely (jschwartzenberg / #​2666)

Documentation

• Add transactional_id to KafkaProducer Keyword Arguments docstring

v2.2.16

Compare Source

Fixes

• Fix thread not waking up when there is still data to be sent (gqmelo / #​2670)
• Ensure timeout is checked after each fetch position update in Consumer.position() (k61n / #​2668)

v2.2.15

Compare Source

Fixes

• Fix KafkaProducer broken method names (llk89 / #​2660)
• Fix spelling mistake in KafkaConsumer docs (Xeus-CC / #​2659)

v2.2.14

Compare Source

Fixes

• python2 fixups (#​2655)
• Fix coordinator lock contention during close() (#​2652)

v2.2.13

Compare Source

Fixes

• Use client.await_ready() to simplify blocking wait and add timeout to admin client (#​2648)
• Fixup import style in example.py

Documentation

• update README kafka version badge to include 4.0

v2.2.12

Compare Source

Fixes

• Fix construction of final GSSAPI authentication message (#​2647)
• Avoid RuntimeError on mutated _completed_fetches deque in consumer fetcher (#​2646)
• Throw exception on invalid bucket type (#​2642)

v2.2.11

Compare Source

Fixes

• Do not ignore metadata response for single topic with error (#​2640)
• Fix decoding bug in AWS_MSK_IAM mechanism (#​2639)
• Add synchronized decorator; add lock to subscription state (#​2636)
• Update build links in documentation (#​2634)

v2.2.10

Compare Source

Fixes

• Set the current host in the SASL configs (#​2633)
• Fix sasl gssapi plugin: do not rely on client_ctx.complete in auth_bytes() (#​2631)

v2.2.9

Compare Source

Fixes

• Do not reset fetch positions if offset commit fetch times out (#​2629)

Logging / Error Messages

• More / updated debug logging for coordinator / consumer (#​2630)

v2.2.8

Compare Source

Fixes

• Wait for next heartbeat in thread loop; check for connected coordinator (#​2622)
• Acquire client lock in heartbeat thread before sending requests (#​2620)

Logging / Error Messages

• Log all SyncGroupResponse errors as info+
• More coordinator / heartbeat logging (#​2621)
• Fix timeout seconds error message in KafkaProducer (#​2627)
• Update offset commit error handling; use RebalanceInProgressError if applicable (#​2623)

v2.2.7

Compare Source

Fixes

• Minor Heartbeat updates: catch more exceptions / log configuration / raise KafkaConfigurationError (#​2618)

v2.2.6

Compare Source

Fixes

• Only disable heartbeat thread once at beginning of join-group (#​2617)

v2.2.5

Compare Source

Fixes

• Fix producer busy loop with no pending batches (#​2616)
• Fixup py27 fetcher test failure

v2.2.4

Compare Source

Fixes

• Do not reset_generation after RebalanceInProgressError; improve CommitFailed error messages (#​2614)
• Fix KafkaConsumer.poll() with zero timeout (#​2613)
• Fix Fetch._reset_offsets_async() KeyError when fetching from multiple nodes (#​2612)

v2.2.3

Compare Source

Fixes

• Ignore leading SECURITY_PROTOCOL:// in bootstrap_servers (#​2608)
• Only create fetch requests for ready nodes (#​2607)

v2.2.2

Compare Source

Fixes

• Fix lint errors

v2.2.1

Compare Source

Fixes

• Fix KafkaProducer broken method names (llk89 / #​2660)
• Fix spelling mistake in KafkaConsumer docs (Xeus-CC / #​2659)

v2.2.0

Compare Source

KafkaProducer

• KIP-98: Add idempotent producer support (#​2569)
• KIP-98: Transactional Producer (#​2587)
• KIP-98: Add offsets support to transactional KafkaProducer (#​2590)
• Prefix producer logs w/ client id and transactional id (#​2591)
• KAFKA-5429: Ignore produce response if batch was previously aborted
• KIP-91: KafkaProducer delivery_timeout_ms
• Default retries -> infinite
• Expand KafkaProducer docstring w/ idempotent and transactional notes
• RecordAccumulator: Use helper method to get/set _tp_locks; get dq with lock in reenqueue()

KafkaConsumer

• KIP-98: Add Consumer support for READ_COMMITTED (#​2582)
• KIP-394: handle MEMBER_ID_REQUIRED error w/ second join group request (#​2598)
• KAFKA-5078: Defer fetch record exception if iterator has already moved across a valid record
• KAFKA-5075: Defer consumer fetcher exception if fetch position has already increased
• KAFKA-4937: Batch offset fetches in the Consumer
• KAFKA-4547: Avoid resetting paused partitions to committed offsets
• KAFKA-6397: Consumer should not block setting positions of unavailable partitions (#​2593)

Potentially Breaking Changes (internal)

• Rename CorruptRecordException -> CorruptRecordError
• Rename Coordinator errors to generic not group (#​2585)
• Rename ClusterMetadata.add_group_coordinator -> add_coordinator + support txn type
• Use SaslAuthenticationFailedError in kafka.conn connection failure; Drop unused AuthenticationFailedError
• Remove old/unused errors; reorder; KafkaTimeout -> retriable
• Drop log_start_offset from producer RecordMetadata

Internal

• MemoryRecords iterator; MemoryRecordsBuilder records() helper
• Convert DefaultRecordsBuilder.size_in_bytes to classmethod

Fixes

• Resolve datetime deprecation warnings (#​2589)
• Avoid self refcount in log messages; test thread close on all pythons
• Fix client.wakeup() race from producer/sender close
• Fix ElectionNotNeededError handling in admin client

Tests

• Move integration tests and fixtures to test/integration/; simplify unit fixtures (#​2588)
• Expand Sender test coverage (#​2586)
• py2 test fixups
• Drop unused KafkaClient import from test_fetcher

v2.1.6

Compare Source

Fixes

• Only create fetch requests for ready nodes (#​2607)

v2.1.5

Compare Source

Fixes

• Fix python2.7 errors (#​2578)

Improvements

• Move benchmark scripts to kafka.benchmarks module (#​2584)
• Use slots for metrics (#​2583)
• Pass metrics_enabled=False to disable metrics (#​2581)
• Drop unused kafka.producer.buffer / SimpleBufferPool (#​2580)
• Raise UnsupportedVersionError from coordinator (#​2579)

v2.1.4

Compare Source

Fixes

• Dont block pending FetchRequests when Metadata update requested (#​2576)
• Fix MetadataRequest for no topics (#​2573)
• Send final error byte x01 on Sasl OAuth failure (#​2572)
• Reset SASL state on disconnect (#​2571)
• Try import new Sequence before old to avoid DeprecationWarning

Improvements

• Update Makefile default to 4.0 broker; add make fixture
• Improve connection state logging (#​2574)

v2.1.3

Compare Source

Fixes

• Fix crash when switching to closest compatible api_version in KafkaClient (#​2567)
• Fix maximum version to send an OffsetFetchRequest in KafkaAdminClient (#​2563)
• Return empty set from consumer.partitions_for_topic when topic not found ([#

✂ Note

PR body was truncated to here.

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Never, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[gemini-code-assist]

Code Review

This pull request updates the kafka-python dependency version from 2.0.6 to 3.0.0 in dataflow/snippets/requirements.txt. There are no review comments, and I have no feedback to provide.