feat(bq_driver): add AllowedProjects DSN option to scope SQLTables metadata enumeration - #1617
Open
Rigbyfab4 wants to merge 3 commits into
Open
feat(bq_driver): add AllowedProjects DSN option to scope SQLTables metadata enumeration#1617Rigbyfab4 wants to merge 3 commits into
Rigbyfab4 wants to merge 3 commits into
Conversation
added 3 commits
July 27, 2026 19:42
Power Query's Navigator opens a connection and immediately calls SQLTables with wildcard catalog/schema arguments. GetFilteredProjectIds answered that by enumerating every project the authenticated principal can reach via the projects.list REST API, then listing datasets for each one. In a large enterprise organization the principal can typically *see* far more projects than it holds BigQuery permissions on. Listing datasets for such a project fails with a permission error, and because GetResultSetForDatasets surfaces the first error from its parallel fan-out, one inaccessible project fails the whole SQLTables call. The result is that the Navigator tree cannot be built at all, even though the user only wanted the handful of projects they do have access to. Enumeration is also slow and quota-heavy at that scale, but the permission failure is the blocking issue. Add an AllowedProjects DSN option naming the projects to expose. When set, GetFilteredProjectIds filters that list and skips projects.list entirely, so no inaccessible project is ever contacted. The existing filter semantics are preserved: entries are matched against the catalog argument with the same LIKE pattern / SQL_ATTR_METADATA_ID rules used elsewhere. AllowedProjects and AdditionalProjects stay orthogonal and composable: if both are set, enumeration starts from the allowlist and still appends the AdditionalProjects ids on top. Also add a pick list to the Advanced Options dialog so non-technical users need not hand-type project ids. "Load Projects" reuses the existing DriverForm::GetCatalogAndDataset path to populate a checkbox list view, preserving ticks across reloads and keeping saved ids the account no longer reports. The DSN key remains usable headlessly for automation. Tests: unit coverage for FilterAllowedProjects and ALLOWEDPROJECTS parsing, a dialog round-trip test, and an integration test asserting that a project the principal can reach but did not allow-list is absent from SQLTables.
The Advanced Options dialog asked for a fixed 720x525 window and centred it on the raw screen metrics. On a display whose usable height is smaller than that -- a 1280x720 panel at 150% scaling leaves 672px of work area -- the lower part of the dialog, including the OK and Cancel buttons, fell below the screen edge with no way to reach it. Adding the allowed-projects pick list made this worse by growing the layout, but the dialog already did not fit such a display before that. Clamp the window height to the desktop work area and give the window a vertical scrollbar, so a layout taller than the available space stays fully reachable. Scrolling moves the child controls via ScrollWindow, which keeps the existing absolute control positions untouched. The scroll range is derived from the laid-out content height and refreshed on WM_SIZE; the scrollbar, mouse wheel, page keys and thumb dragging are all handled. Widen the frame by the scrollbar width. The scrollbar is carved out of the client area, so without this every right-aligned control loses those pixels and the edit boxes are clipped. Accumulate mouse wheel movement rather than dividing each message by WHEEL_DELTA. High-resolution wheels and precision trackpads report fractions of a notch, which integer division rounded away to nothing, so those devices could not scroll the dialog at all. Also honour the system SPI_GETWHEELSCROLLLINES setting instead of assuming three lines. Also centre the window inside the work area rather than the full screen, so it is not positioned partly behind the taskbar.
Clicking a blank part of the DSN setup dialog or the Advanced Options dialog could open a browser at the BigQuery ODBC/JDBC drivers page. Both dialogs hit-test WM_LBUTTONDOWN against the documentation hyperlink's rectangle, but the CreateHyperlinkLabel calls that would create that control are currently commented out. GetDlgItem therefore returns NULL, GetClientRect(NULL, &rect) fails and leaves 'rect' holding whatever was on the stack, and PtInRect matches that uninitialised rectangle -- so clicks on unrelated parts of the dialog launch the browser. Which clicks match varies between runs, since it depends on stack contents. Bail out when the control does not exist or its rectangle cannot be read, and zero-initialise the rectangle so it can never hold garbage.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Rigbyfab4
force-pushed
the
feature/allowed-projects
branch
from
July 28, 2026 18:46
b8de255 to
90e0976
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
AllowedProjectsDSN option to scopeSQLTablesmetadata enumerationProblem
Power Query for Excel opens a connection and immediately calls
SQLTableswithwildcard catalog/schema arguments to populate its Navigator tree. Today
GetFilteredProjectIdsanswers that by enumerating every project theauthenticated principal can reach via the
projects.listREST API, then callingdatasets.listfor each one.In a large organization a principal can typically see far more projects than it
holds BigQuery permissions on.
datasets.liston such a project fails with apermission error, and because
GetResultSetForDatasetssurfaces the first errorfrom its parallel fan-out, one inaccessible project fails the entire
SQLTablescall. The Navigator tree cannot be built at all, even though theuser only wanted the handful of projects they do have access to.
The enumeration is also slow and quota-heavy at that scale, but the permission
failure is what blocks the user outright.
Neither existing mechanism helps:
AdditionalProjectsonly adds projects on top of the full enumeration.ListProjectsParentrestricts via the Cloud Resource Manager hierarchy, whichrequires
cloudresourcemanager.googleapis.comaccess and has no UI.Neither is an explicit allowlist, and neither prevents the broad call.
Change
Add an
AllowedProjectsDSN option naming the projects to expose. When set,GetFilteredProjectIdsfilters that list and skipsprojects.listentirely,so no inaccessible project is ever contacted.
Existing filter semantics are preserved: entries are matched against the catalog
argument using the same ODBC LIKE-pattern /
SQL_ATTR_METADATA_IDrules usedelsewhere in the file, with surrounding whitespace trimmed and blank entries
ignored.
AllowedProjectsandAdditionalProjectsremain orthogonal and composable: ifboth are set, enumeration starts from the allowlist and still appends the
AdditionalProjectsids.The DSN key is fully usable headlessly for automation. For non-technical users
there is also a pick list in the Advanced Options dialog: a Load Projects
button populates a checkbox list via the existing
DriverForm::GetCatalogAndDatasetpath, preserving ticks across reloads andretaining saved ids the account no longer reports.
SQLColumnsis unaffected — its catalog argument is always a single resolvedproject, so it never triggers account-wide enumeration.
Commits
feat(bq_driver): add AllowedProjects DSN option to scope SQLTablesfix(bq_driver): keep the Advanced Options dialog within the screenfix(bq_driver): stop empty dialog areas acting as a documentation linkCommits 2 and 3 fix pre-existing defects, not regressions from this feature.
They are separated so they can be reviewed or dropped independently. See
"Pre-existing issues" below for why they surfaced now.
Testing
Unit tests (all passing):
FilterAllowedProjects— 8 cases: wildcard, LIKE patterns,_single-character wildcard, non-matching pattern, blank/whitespace entries,
empty allowlist,
SQL_ATTR_METADATA_IDexact match, and%treated as aliteral under
SQL_ATTR_METADATA_ID.ConnectionHandle—ALLOWEDPROJECTSparsing, set and unset, including thatit stays independent of
AdditionalProjects.AdvanceOptions—SetValues/ResetToDefaultsround-trip, and a pick-listtest that ticks two saved ids, unticks one, clicks OK and asserts only the
remaining id is saved.
Integration test
CatalogTest.SQLTables_AllowedProjects: enumerates everyreachable project, then re-connects with
AllowedProjectsset to a subset andasserts the returned catalog set is exactly that subset — so a project the
principal can reach but did not allow-list must be absent. This is what proves
the enumeration was skipped rather than merely filtered.
Manual verification on Windows against a real BigQuery account:
account; ticking a subset and pressing OK persisted
AllowedProjectsto theDSN and re-opening the dialog restored the ticks.
project. A second project the principal could reach, but had not allow-listed,
was correctly absent, and a table loaded successfully through Power Query.
Note: the Win32 unit tests are not built by CI (see below), so the dialog tests
above were run locally with a temporary build workaround.
Pre-existing issues encountered
All of the following sit in Win32 dialog code that no CI path compiles:
ci/gha/builds/windows-cmake-integration.shsets-DODBC_UNIT_TESTING=OFF(required, because
fuzztestfails to configure under MSVC withCompiler MSVC is not supported), while the Linux builds that do compile unittests exclude these files via the
if (WIN32)guard inbq_driver.cmake. Thatsingle gap is likely why the following accumulated unnoticed, and may be worth
addressing on its own.
Fixed here (commits 2 and 3):
720px height centred on raw screen metrics. On a 1280x720 display at 150%
scaling only 672px of work area is available, leaving the OK and Cancel
buttons unreachable below the screen edge. Now clamped to the work area with
a vertical scrollbar.
WM_LBUTTONDOWNagainst the documentation hyperlink's rectangle, but theCreateHyperlinkLabelcalls are commented out.GetDlgItemreturns NULL,GetClientRect(NULL, &rect)fails leavingrectuninitialised, andPtInRectthen matches arbitrary clicks — launching a browser from emptyparts of the dialog.
Not fixed, reported for triage:
driver_form_test.ccdoes not compile. It callsDriverForm::TestODBCConnection(section)with one argument at lines 172, 181and 191; the signature changed in commit
5a10f6b9. Error C2660.ConvertLogLevel.ValidateLogLevelConversionfails. The test expects""for an invalid log level; the implementation returns
"0"(kLogOff).Two dialog tests hang indefinitely.
AdvanceOptionsTest.ShowWindowand acase in
ProxyOptionsTestcall a form'sShow(), whose modalGetMessageloop never receives
WM_QUIT. (The new pick-list test avoids this by postingWM_QUITbefore callingShow().)User DSNs cannot be used to connect.
GetSectionWin(
internal/utils.cc) hardcodesRegOpenKeyEx(HKEY_LOCAL_MACHINE, ...), so aDSN under
HKEY_CURRENT_USERis invisible when connecting — a DSN created viathe User DSN tab of the ODBC administrator cannot be used at all.
This looks like an oversight rather than a design decision, since the driver
is otherwise scope-aware: the write path uses
SQLWriteDSNToIni/SQLWritePrivateProfileString, and the config-dialog read path usesSQLGetPrivateProfileString. Only the connect path hardcodes the hive, so thedriver can create and re-edit a User DSN that it then cannot connect with.
The failure is also misleading rather than merely inconvenient: with no values
found,
stoi("")throws inCreateAuthand its catch block silently defaultsto
kServiceAccount, so the user is told"The path to the file can't be empty" — an error about credentials — when the
real problem is that the DSN was never read.
Suggested fix: have the DSN read path consult
HKEY_CURRENT_USERbeforeHKEY_LOCAL_MACHINE, matching the precedence ODBC itself uses, or switch itto
SQLGetPrivateProfileStringas the config dialog already does. Worthnoting for whoever picks this up:
GetSectionWinis also used to read thetrace/logging configuration, so changing it in place affects more than DSN
lookups — adding a DSN-specific reader may be the safer shape. Deliberately
left out of this PR as it is unrelated to this feature and deserves its own
review.
Checklist
clang-formatand the top-level.clang-formatci/cloudbuild/build.sh -t checkers-pr --docker— not run locally (noDocker available on the development machine); formatting was applied with
the pinned
clang-formatdirectlySQLTables_AllowedProjectstest compiles but was not run in CI conditions🤖 Generated with Claude Code