#1541)

* bug: fixes discrepancy btwn python-api-core & bigquery re object default timeout

* Fix: loosen ipywidget dependency (#1504)

* fix: updates ipywidget dependency

* fix: updates ipywidget version number

* chore(main): release 3.6.0 (#1490)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* docs: Remove < 3.11 reference from README (#1502)

* chore(python): upgrade gcp-releasetool in .kokoro [autoapprove] (#1508)

Source-Link: googleapis/synthtool@5f2a608
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:8555f0e37e6261408f792bfd6635102d2da5ad73f8f09bcb24f25e6afb5fac97

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: add `connection_properties` and `create_session` to `LoadJobConfig` (#1509)

* feat: added `connection_properties` and `create_session` in load job

* chore(deps): update all dependencies (#1501)

* chore(deps): update all dependencies

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* chore(deps): update all dependencies (#1513)

* feat: add default_query_job_config property and property setter to BQ client (#1511)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [feature request](https://togithub.com/googleapis/python-bigquery/issues/1512) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes 
- [feature request](https://togithub.com/googleapis/python-bigquery/issues/1512)🦕
- [internal bug](https://b.corp.google.com/issues/271044948)

* chore(deps): update all dependencies (#1514)

* chore(deps): update dependency charset-normalizer to v3.1.0 (#1518)

* chore(main): release 3.7.0 (#1507)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob (#1521)

* feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob

Note for google-cloud-bigquery developers: This also refactors these classes so
that `_set_properties` does not modify the `_properties` dictionary in-place.
Doing so was also mutating the request object, making it difficult to debug
what request was _actually_ sent. Before this change, many tests hallucinated
that the request was always equal to the response.

* E           google.api_core.exceptions.BadRequest: 400 Clone operation with write disposition WRITE_TRUNCATE is not supported. Please try again with WRITE_EMPTY.

* chore(deps): Update nox in .kokoro/requirements.in [autoapprove] (#1527)

Source-Link: googleapis/synthtool@92006bb
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:2e247c7bf5154df7f98cce087a20ca7605e236340c7d6d1a14447e5c06791bd6

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: add default LoadJobConfig to Client (#1526)

* feat: add bool, int, float, string dtype to to_dataframe (#1529)

* fix: loosen ipywidgets restrictions further to address ipython compatibility issues (#1531)

* fix: loosen ipywidgets restrictions further to address ipython compatibility issues

* include ipywidgets in prerelease deps

* show all package versions

* add ipykernel dependency

* ipykernel in noxfile

* oops

* chore(main): release 3.8.0 (#1525)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* fix: keyerror when the load_table_from_dataframe accesses a unmapped dtype dataframe index (#1535)

* feat: expose query job on dbapi cursor (#1520)

Co-authored-by: Tim Swast <swast@google.com>

* chore(main): release 3.9.0 (#1537)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* chore: update tests to be compatible with pandas 2.0 (#1538)

* chore: update tests to be compatible with pandas 2.0

* use StringDtype without storage argument

* avoid Float64Dtype on older pandas

* chore(deps): update all dependencies (#1522)

* chore: updates minimum version of bqstorage (#1542)

* chore: updates minimum version of bqstorage

* removes unneeded test

* updates linting, removes unneeded comment

* updates conditional checks, comments, adds test

* Removes test, adds pragma no cover

* Removes test

* fix linting error

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Atsushi Yamamoto <yamaatsushi927@gmail.com>
Co-authored-by: gcf-owl-bot[bot] <78513119+gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Shobhit Singh <just.shobhit@gmail.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Co-authored-by: chelsea-lin <124939984+chelsea-lin@users.noreply.github.com>
Co-authored-by: Tim Swast <swast@google.com>
Co-authored-by: r1b <robert.cole.jensen@gmail.com>

Fixes #15871

## Summary

`Connection.transaction_checkout()` currently calls
`Transaction.begin()` explicitly before the first query, which sends a
standalone `BeginTransaction` gRPC RPC. This is unnecessary because the
`Transaction` class already supports **inline begin** — piggybacking
`BeginTransaction` onto the first `ExecuteSql`/`ExecuteBatchDml` request
via `TransactionSelector(begin=...)`.

This PR removes the explicit `begin()` call, letting the existing inline
begin logic in `execute_sql()`, `execute_update()`, and `batch_update()`
handle transaction creation. This saves **one gRPC round-trip per
transaction** (~16ms measured on the emulator).

### What changed

**`google/cloud/spanner_dbapi/connection.py`** —
`transaction_checkout()`:
- Removed `self._transaction.begin()` on
[L413](https://github.com/googleapis/python-spanner/blob/v3.63.0/google/cloud/spanner_dbapi/connection.py#L413)
- The transaction is now returned with `_transaction_id=None`
- Updated docstring to explain the inline begin behavior

**`tests/unit/spanner_dbapi/test_connection.py`**:
- Added `test_transaction_checkout_does_not_call_begin` to assert
`begin()` is not called

**`tests/mockserver_tests/test_dbapi_inline_begin.py`** (new):
- 9 mockserver tests verifying inline begin behavior for read-write
transactions
- Covers: no `BeginTransactionRequest` sent, first `ExecuteSqlRequest`
uses `TransactionSelector(begin=...)`, transaction ID reuse on second
statement, rollback, read-only unaffected, retry after abort

**`tests/mockserver_tests/test_tags.py`**:
- Updated 4 read-write tag tests: removed `BeginTransactionRequest` from
expected RPC sequences, adjusted tag index offsets

**`tests/mockserver_tests/test_dbapi_isolation_level.py`**:
- Updated 4 isolation level tests: verify isolation level on
`ExecuteSqlRequest.transaction.begin.isolation_level` instead of
`BeginTransactionRequest.options.isolation_level`

### Why this is safe

The inline begin code path already exists and is battle-tested —
`Session.run_in_transaction()` creates a `Transaction` without calling
`begin()` and relies on the same inline begin logic ([session.py
L566](https://github.com/googleapis/python-spanner/blob/v3.63.0/google/cloud/spanner_v1/session.py#L566)).

Specific safety analysis:

1. **`transaction_checkout()` callers always execute SQL immediately**:
It's only called from `run_statement()` → `execute_sql()` and
`batch_dml_executor` → `batch_update()`. Both set `_transaction_id` via
inline begin before any commit/rollback path.

2. **`execute_sql`/`execute_update`/`batch_update` handle
`_transaction_id is None`**: They acquire a lock, use
`_make_txn_selector()` which returns `TransactionSelector(begin=...)`,
and store the returned `_transaction_id` ([transaction.py
L612-L623](https://github.com/googleapis/python-spanner/blob/v3.63.0/google/cloud/spanner_v1/transaction.py#L612-L623)).

3. **`rollback()` handles `_transaction_id is None`**: Skips the RPC —
correct when no server-side transaction exists ([transaction.py
L163](https://github.com/googleapis/python-spanner/blob/v3.63.0/google/cloud/spanner_v1/transaction.py#L163)).

4. **`commit()` handles `_transaction_id is None`**: Falls back to
`_begin_mutations_only_transaction()` for mutation-only transactions
([transaction.py
L263-L267](https://github.com/googleapis/python-spanner/blob/v3.63.0/google/cloud/spanner_v1/transaction.py#L263-L267)).

5. **Retry mechanism is compatible**: `_set_connection_for_retry()`
resets `_spanner_transaction_started=False`, so replayed statements go
through `transaction_checkout()` again, create a fresh `Transaction`,
and use inline begin.

### PEP 249 conformance

This change is fully conformant with [PEP 249 (DB-API
2.0)](https://peps.python.org/pep-0249/). The spec does not define a
`begin()` method — transactions are implicit. The PEP author [clarified
on the DB-SIG mailing
list](https://mail.python.org/pipermail/db-sig/2010-September/005645.html)
that *"transactions start implicitly after you connect and after you
call `.commit()` or `.rollback()`"*, and the mechanism by which the
driver starts the server-side transaction is an implementation detail.
Deferring the server-side begin to the first SQL execution (as psycopg2
and other mature DB-API drivers do) is the standard approach. The
observable transactional semantics — atomicity between
`commit()`/`rollback()` calls — are unchanged.

### Performance impact

Before (4 RPCs per read-write transaction):
```
BeginTransaction → ExecuteSql (read) → ExecuteSql (write) → Commit
```

After (3 RPCs per read-write transaction):
```
ExecuteSql (read, with inline begin) → ExecuteSql (write) → Commit
```

Measured ~16ms savings per transaction on the Spanner emulator.

### Context

This optimization was identified while profiling SQLAlchemy/DBAPI
performance against Spanner. The DBAPI was created ~2 years before
inline begin support was added to the Python client library (inline
begin landed in [PR
googleapis/python-spanner#740](googleapis/python-spanner#740),
Dec 2022). The `run_in_transaction` path was updated to use inline
begin, but `transaction_checkout` was not.

## Test plan

- [x] All 198 DBAPI unit tests pass (`tests/unit/spanner_dbapi/`)
- [x] New unit test verifies `begin()` is not called by
`transaction_checkout()`
- [x] 9 new mockserver tests verify inline begin RPC behavior
(`tests/mockserver_tests/test_dbapi_inline_begin.py`)
- [x] 8 existing mockserver tests updated for inline begin expectations
(`test_tags.py`, `test_dbapi_isolation_level.py`)
- [x] All 53 DBAPI system tests pass against Spanner emulator
(`tests/system/test_dbapi.py`)
- [ ] CI system tests (will run automatically)

---------

Co-authored-by: rahul2393 <irahul@google.com>
Co-authored-by: Knut Olav Løite <koloite@gmail.com>