Skip to content

Fix #9113 Exception when reading Paimon table via Hive proxy user in … - #9116

Merged
JingsongLi merged 1 commit into
apache:masterfrom
tyoushinya:issue/9113-reading-paimon-data-exception
Aug 10, 2026
Merged

Fix #9113 Exception when reading Paimon table via Hive proxy user in …#9116
JingsongLi merged 1 commit into
apache:masterfrom
tyoushinya:issue/9113-reading-paimon-data-exception

Conversation

@tyoushinya

Copy link
Copy Markdown
Contributor

…Kerberos-enabled environment

Purpose

When querying Paimon external tables via Hive in a Kerberos-secured cluster with proxy user (e.g., Hue, JDBC through HiveServer2), the query fails with:
org.apache.hadoop.hive.ql.parse.SemanticException: Proxy user is not supported
at org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer.transform(SimpleFetchOptimizer.java:125)
The root cause is that PaimonStorageHandler#getDataFieldsJsonStr passes null as the Configuration to HiveSchema.extract:
// Before

static String getDataFieldsJsonStr(Properties properties) {
    HiveSchema hiveSchema = HiveSchema.extract(null, properties);
    ...
}

With a null Configuration:
Kerberos credentials are invisible: HiveUtils.extractCatalogConfig(null) returns an empty Options, so Paimon security settings (e.g., paimon.security.kerberos.login.keytab, paimon.security.kerberos.login.principal) configured in Hadoop Configuration are not loaded.
Filesystem initialization fails silently: FileIO.get() cannot obtain a properly authenticated filesystem under the proxy user's doAs context. The schema file fetch silently fails, causing SimpleFetchOptimizer to fall back to a direct-access code path that rejects proxy users.
The fix passes the actual Configuration instance (which carries Kerberos credentials and proxy user context) through the call chain:
// After

static String getDataFieldsJsonStr(Configuration conf, Properties properties) {
    HiveSchema hiveSchema = HiveSchema.extract(conf, properties);
    ...
}

This ensures HiveSchema.extract can properly initialize the filesystem with Kerberos authentication under the proxy user's UGI, allowing the schema to be read correctly and the query to proceed via the normal MR/Tez execution path instead of falling back to SimpleFetchOptimizer.

Tests

Added 4 unit tests in HiveTableSchemaTest:

  1. testGetDataFieldsJsonStrWithNullConf: Verifies JSON output correctness when conf is null (backward-compatible, local FS).
  2. testGetDataFieldsJsonStrWithConf: Verifies JSON output correctness when conf is a real Configuration instance (the fixed code path).
  3. testGetDataFieldsJsonStrWithEmptyDDLAndPaimonTable: Verifies JSON roundtrip with empty DDL and an existing Paimon table schema.
  4. testGetDataFieldsJsonStrRoundtrip: Verifies that serializing and deserializing the JSON output preserves field names, types, and comments.

@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 6208af5 into apache:master Aug 10, 2026
12 checks passed
@tyoushinya
tyoushinya deleted the issue/9113-reading-paimon-data-exception branch August 10, 2026 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants