Skip to content

HIVE-29241: Distinguish the default location of the database by catalog #6267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zhangbutao wants to merge 2 commits into
base: master
Choose a base branch
from
Open

HIVE-29241: Distinguish the default location of the database by catalog #6267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e299310
HIVE-29241: Distinguish the default location of the database by catalog
zhangbutao Apr 11, 2026
0a57566
HIVE-29241: Distinguish the default location of the database by catalog
zhangbutao Apr 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestMiniClusters.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
import org.apache.hadoop.hive.llap.LlapItUtils;
import org.apache.hadoop.hive.llap.daemon.MiniLlapCluster;
import org.apache.hadoop.hive.llap.io.api.LlapProxy;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
import org.apache.hadoop.hive.ql.exec.Utilities;
import org.apache.hadoop.hive.ql.exec.tez.TezSession;
import org.apache.hadoop.hive.ql.lockmgr.zookeeper.CuratorFrameworkSingleton;
Expand Down Expand Up @@ -544,6 +545,7 @@ private void setFsRelatedProperties(HiveConf conf, boolean isLocalFs, FileSystem
// Different paths if running locally vs a remote fileSystem. Ideally this difference should not
// exist.
Path warehousePath;
Path warehouseCatPath;
Path jarPath;
Path userInstallPath;
if (isLocalFs) {
Expand All @@ -554,23 +556,27 @@ private void setFsRelatedProperties(HiveConf conf, boolean isLocalFs, FileSystem
// Create a fake fs root for local fs
Path localFsRoot = new Path(path, "localfs");
warehousePath = new Path(localFsRoot, "warehouse");
warehouseCatPath = new Path(localFsRoot, "catalog");
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't the catalog be under the warehousePath?

Warehouse Root
  └── Catalog (default or named)
        └── Database (.db)
              └── Table

/warehouse/<catalog_name>/<database_name>.db/<table_name>/   # non default catalog
/warehouse/hive/<database_name>.db/<table_name>/             # default catalog
/warehouse/hive/default.db/<table_name>/                     # default catalog & database

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't the catalog be under the warehousePath?

No. The warehousePath is the path of the default catalog, under which the paths of Hive databases reside. If the catalog path is placed under the warehousePath, it will cause confusion with the path information of Hive databases under the default catalog.

The path information for the default catalog's databases is controlled by the property metastore.warehouse.dir, which defaults to /user/hive/warehouse. The path for non-default catalogs is controlled by the new added propertymetastore.warehouse.catalog.dir, which defaults to /user/hive/catalog/warehouse.

/user/hive/catalog/warehouse/<catalog_name>/<database_name>.db/<table_name>/     # non default catalog
/user/hive/warehouse/<database_name>.db/<table_name>/            				 # default catalog
/user/hive/warehouse/default.db/<table_name>/                     				 # default catalog & database

jarPath = new Path(localFsRoot, "jar");
userInstallPath = new Path(localFsRoot, "user_install");
} else {
// TODO Why is this changed from the default in hive-conf?
warehousePath = new Path(fsUriString, "/build/ql/test/data/warehouse/");
warehouseCatPath = new Path(fsUriString, "/build/ql/test/data/catalog/");
jarPath = new Path(new Path(fsUriString, "/user"), "hive");
userInstallPath = new Path(fsUriString, "/user");
}

warehousePath = fs.makeQualified(warehousePath);
warehouseCatPath = fs.makeQualified(warehouseCatPath);
jarPath = fs.makeQualified(jarPath);
userInstallPath = fs.makeQualified(userInstallPath);

conf.set(CommonConfigurationKeysPublic.FS_DEFAULT_NAME_KEY, fsUriString);

// Remote dirs
conf.setVar(ConfVars.METASTORE_WAREHOUSE, warehousePath.toString());
MetastoreConf.setVar(conf, MetastoreConf.ConfVars.WAREHOUSE_CATALOG, warehouseCatPath.toString());
conf.setVar(ConfVars.HIVE_JAR_DIRECTORY, jarPath.toString());
conf.setVar(ConfVars.HIVE_USER_INSTALL_DIR, userInstallPath.toString());
// ConfVars.SCRATCH_DIR - {test.tmp.dir}/scratchdir
Expand Down
4 changes: 2 additions & 2 deletions parser/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1132,10 +1132,10 @@ createCatalogStatement
: KW_CREATE KW_CATALOG
ifNotExists?
name=identifier
catLocation
catLocation?
catalogComment?
(KW_PROPERTIES catprops=properties)?
-> ^(TOK_CREATECATALOG $name catLocation ifNotExists? catalogComment? $catprops?)
-> ^(TOK_CREATECATALOG $name catLocation? ifNotExists? catalogComment? $catprops?)
;

catLocation
Expand Down
27 changes: 23 additions & 4 deletions ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@

package org.apache.hadoop.hive.ql.ddl.catalog.create;

import com.google.common.base.Strings;

Check warning on line 21 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'com.google.common.base.Strings'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HLqs0Fbzs2joUkd&open=AZ2B7HLqs0Fbzs2joUkd&pullRequest=6267
import org.apache.hadoop.hive.metastore.CatalogUtil;
import org.apache.hadoop.hive.metastore.api.Catalog;
import org.apache.hadoop.hive.ql.QueryState;
import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
Expand All @@ -29,6 +31,7 @@
import org.apache.hadoop.hive.ql.parse.HiveParser;
import org.apache.hadoop.hive.ql.parse.SemanticException;

import java.util.HashMap;
import java.util.Map;

/**
Expand All @@ -43,16 +46,19 @@
@Override
public void analyzeInternal(ASTNode root) throws SemanticException {
String catalogName = unescapeIdentifier(root.getChild(0).getText());
String locationUrl = unescapeSQLString(root.getChild(1).getChild(0).getText());
outputs.add(toWriteEntity(locationUrl));

String locationUrl = null;
boolean ifNotExists = false;
String comment = null;
Map<String, String> props = null;
Map<String, String> props = new HashMap<>();

for (int i = 2; i < root.getChildCount(); i++) {
for (int i = 1; i < root.getChildCount(); i++) {
ASTNode childNode = (ASTNode) root.getChild(i);
switch (childNode.getToken().getType()) {
case HiveParser.TOK_CATALOGLOCATION:

Check warning on line 58 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

'case' child has incorrect indentation level 8, expected level should be 6. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02F3oiSNCgpa0-uA&open=AZzh02F3oiSNCgpa0-uA&pullRequest=6267
locationUrl = unescapeSQLString(childNode.getChild(0).getText());

Check warning on line 59 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

'block' child has incorrect indentation level 10, expected level should be 8. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02F3oiSNCgpa0-uB&open=AZzh02F3oiSNCgpa0-uB&pullRequest=6267
outputs.add(toWriteEntity(locationUrl));

Check warning on line 60 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

'block' child has incorrect indentation level 10, expected level should be 8. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02F3oiSNCgpa0-uC&open=AZzh02F3oiSNCgpa0-uC&pullRequest=6267
break;
case HiveParser.TOK_IFNOTEXISTS:
ifNotExists = true;
break;
Expand All @@ -61,16 +67,29 @@
break;
case HiveParser.TOK_PROPERTIES:
props = getProps((ASTNode) childNode.getChild(0));
break;

Check warning on line 70 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

'block' child has incorrect indentation level 10, expected level should be 8. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02F3oiSNCgpa0-uD&open=AZzh02F3oiSNCgpa0-uD&pullRequest=6267
default:
throw new SemanticException("Unrecognized token in CREATE CATALOG statement");
}
}

// Set default type to hive if not specified
checkCatalogType(props);

CreateCatalogDesc desc = new CreateCatalogDesc(catalogName, comment, locationUrl, ifNotExists, props);
Catalog catalog = new Catalog(catalogName, locationUrl);

rootTasks.add(TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc)));
outputs.add(new WriteEntity(catalog, WriteEntity.WriteType.DDL_NO_LOCK));
}

private static void checkCatalogType(Map<String, String> props) throws SemanticException {
String catalogType = props.get(CatalogUtil.TYPE);
// Normalize and validate catalog type (fail fast on invalid input)
try {
props.put(CatalogUtil.TYPE, CatalogUtil.normalizeCatalogType(catalogType));
} catch (IllegalArgumentException e) {
throw new SemanticException(String.format("type '%s' is not valid", catalogType));
}
}
}
7 changes: 6 additions & 1 deletion ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogOperation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

import org.apache.hadoop.hive.metastore.api.AlreadyExistsException;
import org.apache.hadoop.hive.metastore.api.Catalog;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
import org.apache.hadoop.hive.ql.ErrorMsg;
import org.apache.hadoop.hive.ql.ddl.DDLOperation;
import org.apache.hadoop.hive.ql.ddl.DDLOperationContext;
Expand All @@ -37,7 +38,11 @@ public CreateCatalogOperation(DDLOperationContext context, CreateCatalogDesc des

@Override
public int execute() throws Exception {
Catalog catalog = new Catalog(desc.getName(), desc.getLocationUri());
String catLocationUri = Optional.ofNullable(desc.getLocationUri())
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor. i think cat prefix in local var is redundant here since we are inside CreateCatalog class

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, i didn't get you here. Could you please describe it in detail? Thank you.

.orElse(MetastoreConf.getVar(context.getConf(),
MetastoreConf.ConfVars.WAREHOUSE_CATALOG) + "/" + desc.getName());

Catalog catalog = new Catalog(desc.getName(), catLocationUri);
catalog.setDescription(desc.getComment());
Optional.ofNullable(desc.getCatlogProperties())
.ifPresent(catalog::setParameters);
Expand Down
10 changes: 5 additions & 5 deletions ql/src/java/org/apache/hadoop/hive/ql/ddl/database/create/CreateDatabaseAnalyzer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,21 @@
package org.apache.hadoop.hive.ql.ddl.database.create;

import java.util.Map;
import java.util.Optional;

import org.apache.commons.lang3.tuple.Pair;
import org.apache.hadoop.hive.metastore.api.DataConnector;
import org.apache.hadoop.hive.metastore.api.Database;
import org.apache.hadoop.hive.metastore.api.DatabaseType;
import org.apache.hadoop.hive.metastore.api.PrincipalType;
import org.apache.hadoop.hive.ql.ErrorMsg;
import org.apache.hadoop.hive.ql.QueryState;
import org.apache.hadoop.hive.ql.ddl.DDLUtils;
import org.apache.hadoop.hive.ql.exec.TaskFactory;
import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory.DDLType;
import org.apache.hadoop.hive.ql.ddl.DDLWork;
import org.apache.hadoop.hive.ql.hooks.ReadEntity;
import org.apache.hadoop.hive.ql.hooks.WriteEntity;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;
import org.apache.hadoop.hive.ql.parse.ASTNode;
import org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer;
import org.apache.hadoop.hive.ql.parse.HiveParser;
Expand All @@ -51,10 +52,9 @@ public CreateDatabaseAnalyzer(QueryState queryState) throws SemanticException {
@Override
public void analyzeInternal(ASTNode root) throws SemanticException {
Pair<String, String> catDbNamePair = DDLUtils.getCatDbNamePair((ASTNode) root.getChild(0));
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about record CatalogDb(String catalog, String database) {}
please see #6379 (comment)

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using a Pair here is already simple and clear enough, there's no need to use the record CatalogDb.

String catalogName = catDbNamePair.getLeft();
if (catalogName != null && getCatalog(catalogName) == null) {
throw new SemanticException(ErrorMsg.CATALOG_NOT_EXISTS, catalogName);
}
String catalogName = Optional.ofNullable(catDbNamePair.getLeft())
.orElse(HiveUtils.getCurrentCatalogOrDefault(conf));

String databaseName = catDbNamePair.getRight();

boolean ifNotExists = false;
Expand Down
63 changes: 46 additions & 17 deletions ql/src/java/org/apache/hadoop/hive/ql/ddl/database/create/CreateDatabaseOperation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import org.apache.hadoop.hive.ql.ddl.DDLOperationContext;
import org.apache.hadoop.hive.ql.exec.Utilities;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;
import org.apache.hadoop.hive.ql.session.SessionState;

/**
Expand All @@ -56,12 +57,12 @@ public int execute() throws HiveException {
if (desc.getManagedLocationUri() != null) {
database.setManagedLocationUri(desc.getManagedLocationUri());
}
makeLocationQualified(database); // TODO catalog. Add catalog prefix for db location. Depend on HIVE-29241.
makeLocationQualified(database);
if (database.getLocationUri().equalsIgnoreCase(database.getManagedLocationUri())) {
throw new HiveException("Managed and external locations for database cannot be the same");
}
} else if (desc.getDatabaseType() == DatabaseType.REMOTE) {
makeLocationQualified(database); // TODO catalog. Add catalog prefix for db location. Depend on HIVE-29241.
makeLocationQualified(database);
database.setConnector_name(desc.getConnectorName());
database.setRemote_dbname(desc.getRemoteDbName());
} else { // should never be here
Expand All @@ -81,34 +82,62 @@ public int execute() throws HiveException {
}

private void makeLocationQualified(Database database) throws HiveException {
String catalogName = database.getCatalogName().toLowerCase();
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the names be normalized here: special chars/encoding lowercase?

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this stage, the database is being created, and the prerequisite is that a valid catalog already exists — there is no need to double validate the catalog here.

String dbName = database.getName().toLowerCase();
boolean isDefaultCatalog = HiveUtils.isDefaultCatalog(catalogName, context.getConf());

// -------- External location --------
if (database.isSetLocationUri()) {
database.setLocationUri(Utilities.getQualifiedPath(context.getConf(), new Path(database.getLocationUri())));
} else {
// Location is not set we utilize WAREHOUSE_EXTERNAL together with database name
String rootDir = MetastoreConf.getVar(context.getConf(), MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL);
if (rootDir == null || rootDir.trim().isEmpty()) {
// Fallback plan
LOG.warn(String.format(
"%s is not set, falling back to %s. This could cause external tables to use to managed tablespace.",
MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL.getVarname(), MetastoreConf.ConfVars.WAREHOUSE.getVarname()));
rootDir = MetastoreConf.getVar(context.getConf(), MetastoreConf.ConfVars.WAREHOUSE);
}
Path path = new Path(rootDir, database.getName().toLowerCase() + DATABASE_PATH_SUFFIX);
String qualifiedPath = Utilities.getQualifiedPath(context.getConf(), path);
database.setLocationUri(qualifiedPath);
String rootDir = getExternalRootDir(isDefaultCatalog);
Path path = buildDbPath(rootDir, catalogName, dbName, isDefaultCatalog);
database.setLocationUri(Utilities.getQualifiedPath(context.getConf(), path));
}

// -------- Managed location --------
if (database.isSetManagedLocationUri()) {
database.setManagedLocationUri(Utilities.getQualifiedPath(context.getConf(),
new Path(database.getManagedLocationUri())));
} else {
// ManagedLocation is not set we utilize WAREHOUSE together with database name
String rootDir = MetastoreConf.getVar(context.getConf(), MetastoreConf.ConfVars.WAREHOUSE);
Path path = new Path(rootDir, database.getName().toLowerCase() + DATABASE_PATH_SUFFIX);
String rootDir = MetastoreConf.getVar(
context.getConf(),
isDefaultCatalog
? MetastoreConf.ConfVars.WAREHOUSE
: MetastoreConf.ConfVars.WAREHOUSE_CATALOG
);

Path path = buildDbPath(rootDir, catalogName, dbName, isDefaultCatalog);
String qualifiedPath = Utilities.getQualifiedPath(context.getConf(), path);
if (!qualifiedPath.equals(database.getLocationUri())) {
database.setManagedLocationUri(qualifiedPath);
}
}
}

private Path buildDbPath(String rootDir, String catalogName, String dbName, boolean isDefaultCatalog) {
return isDefaultCatalog
? new Path(rootDir, dbName + DATABASE_PATH_SUFFIX)
: new Path(rootDir + "/" + catalogName, dbName + DATABASE_PATH_SUFFIX);
}

private String getExternalRootDir(boolean isDefaultCatalog) {
MetastoreConf.ConfVars externalVar = isDefaultCatalog
? MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL
: MetastoreConf.ConfVars.WAREHOUSE_CATALOG_EXTERNAL;

String rootDir = MetastoreConf.getVar(context.getConf(), externalVar);
if (rootDir != null && !rootDir.trim().isEmpty()) {
return rootDir;
}

MetastoreConf.ConfVars fallbackVar = isDefaultCatalog
? MetastoreConf.ConfVars.WAREHOUSE
: MetastoreConf.ConfVars.WAREHOUSE_CATALOG;

LOG.warn("{} is not set, falling back to {}. This could cause external tables to use managed tablespace.",
externalVar.getVarname(), fallbackVar.getVarname());

return MetastoreConf.getVar(context.getConf(), fallbackVar);
}
}
4 changes: 4 additions & 0 deletions ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -553,4 +553,8 @@ public static String getCurrentCatalogOrDefault(Configuration conf) {
.map(SessionState::getCurrentCatalog)
.orElseGet(() -> getDefaultCatalog(conf));
}

public static boolean isDefaultCatalog(String catName, Configuration conf) {
return catName == null || catName.isEmpty() || getDefaultCatalog(conf).equals(catName);
}
}
2 changes: 1 addition & 1 deletion ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@
import static org.apache.hadoop.hive.metastore.client.utils.HiveMetaStoreClientUtils.deepCopyFieldSchemas;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.compareFieldColumns;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getColumnNamesForTable;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog;

Check warning on line 145 in ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HN4s0Fbzs2joUkh&open=AZ2B7HN4s0Fbzs2joUkh&pullRequest=6267

Check warning on line 145 in ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Unused import - org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HN4s0Fbzs2joUki&open=AZ2B7HN4s0Fbzs2joUki&pullRequest=6267
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getPvals;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.isExternalTable;

Expand Down Expand Up @@ -181,7 +181,7 @@
}

private boolean isDefaultCatalog(String catName) {
return catName == null || catName.isEmpty() || getDefaultCatalog(conf).equals(catName);
return HiveUtils.isDefaultCatalog(catName, conf);
}

/**
Expand Down
32 changes: 27 additions & 5 deletions ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import org.apache.hadoop.fs.PathFilter;
import org.apache.hadoop.hive.common.repl.ReplConst;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.Warehouse;

Check warning on line 30 in ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'org.apache.hadoop.hive.metastore.Warehouse'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HMts0Fbzs2joUkf&open=AZ2B7HMts0Fbzs2joUkf&pullRequest=6267
import org.apache.hadoop.hive.metastore.api.Database;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
Expand All @@ -39,6 +40,7 @@
import org.apache.hadoop.hive.ql.hooks.WriteEntity;
import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;
import org.apache.hadoop.hive.ql.metadata.Partition;
import org.apache.hadoop.hive.ql.metadata.Table;
import org.apache.hadoop.hive.ql.parse.repl.DumpType;
Expand Down Expand Up @@ -388,14 +390,34 @@

private static void updateIfCustomDbLocations(Database database, Configuration conf) throws SemanticException {
try {
String whLocatoion = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL,
MetastoreConf.getVar(conf, MetastoreConf.ConfVars.WAREHOUSE));
Path dbDerivedLoc = new Path(whLocatoion, database.getName().toLowerCase() + DATABASE_PATH_SUFFIX);
String catName = database.getCatalogName();
String dbName = database.getName().toLowerCase();
boolean isDefaultCatalog = HiveUtils.isDefaultCatalog(catName, conf);

// external warehouse root
String whLocation = MetastoreConf.getVar(conf,
isDefaultCatalog ? MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL : MetastoreConf.ConfVars.WAREHOUSE_CATALOG_EXTERNAL,
MetastoreConf.getVar(conf,
isDefaultCatalog ? MetastoreConf.ConfVars.WAREHOUSE : MetastoreConf.ConfVars.WAREHOUSE_CATALOG));

if (!isDefaultCatalog) {
whLocation = new Path(whLocation, catName).toString();
}

Path dbDerivedLoc = new Path(whLocation, dbName + DATABASE_PATH_SUFFIX);
String defaultDbLoc = Utilities.getQualifiedPath((HiveConf) conf, dbDerivedLoc);
database.putToParameters(ReplConst.REPL_IS_CUSTOM_DB_LOC,
Boolean.toString(!defaultDbLoc.equals(database.getLocationUri())));
String whManagedLocatoion = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.WAREHOUSE);
Path dbDerivedManagedLoc = new Path(whManagedLocatoion, database.getName().toLowerCase()

// managed warehouse root
String whManagedLocatoion = MetastoreConf.getVar(conf,
isDefaultCatalog ? MetastoreConf.ConfVars.WAREHOUSE
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see #6267 (comment)

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replied in #6267 (comment)

: MetastoreConf.ConfVars.WAREHOUSE_CATALOG);

if (!isDefaultCatalog) {
whManagedLocatoion = new Path(whManagedLocatoion, catName).toString();
}
Path dbDerivedManagedLoc = new Path(whManagedLocatoion, dbName
+ DATABASE_PATH_SUFFIX);
String defaultDbManagedLoc = Utilities.getQualifiedPath((HiveConf) conf, dbDerivedManagedLoc);
database.getParameters().put(ReplConst.REPL_IS_CUSTOM_DB_MANAGEDLOC, Boolean.toString(
Expand Down
14 changes: 10 additions & 4 deletions ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import org.apache.hadoop.hive.ql.QueryState;
import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;

Check warning on line 31 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'org.apache.hadoop.hive.ql.metadata.HiveUtils'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02K6oiSNCgpa0-uJ&open=AZzh02K6oiSNCgpa0-uJ&pullRequest=6267

Check warning on line 31 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Unused import - org.apache.hadoop.hive.ql.metadata.HiveUtils. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02K6oiSNCgpa0-uK&open=AZzh02K6oiSNCgpa0-uK&pullRequest=6267
import org.apache.hadoop.hive.ql.metadata.Table;
import org.apache.hadoop.hive.ql.parse.PartitionTransform;
import org.apache.hadoop.hive.ql.parse.TransformSpec;
Expand All @@ -40,6 +41,8 @@
import java.util.ArrayList;
import java.util.List;

import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog;

public abstract class AbstractRepository implements QueryHistoryRepository {
protected Logger LOG = LoggerFactory.getLogger(getClass());
@VisibleForTesting
Expand Down Expand Up @@ -82,9 +85,12 @@
db = hive.getDatabase(QUERY_HISTORY_DB_NAME);
if (db == null) {
LOG.warn("Database ({}) for query history table hasn't been found, auto-creating one", QUERY_HISTORY_DB_NAME);
String location = getDatabaseLocation(QUERY_HISTORY_DB_NAME);
// TODO catalog. The Hive Query History functionality is currently limited to the default catalog. Depend on HIVE-29561

Check warning on line 88 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Complete the task associated to this TODO comment. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HOZs0Fbzs2joUkj&open=AZ2B7HOZs0Fbzs2joUkj&pullRequest=6267

Check warning on line 88 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Line is longer than 120 characters (found 127). 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HOZs0Fbzs2joUkk&open=AZ2B7HOZs0Fbzs2joUkk&pullRequest=6267
db = new Database(QUERY_HISTORY_DB_NAME, QUERY_HISTORY_DB_COMMENT,
location, null);
null, null);
db.setCatalogName(getDefaultCatalog(conf));
String location = getDatabaseLocation(db);
db.setLocationUri(location);
hive.createDatabase(db, false);
}
return db;
Expand All @@ -93,8 +99,8 @@
}
}

private String getDatabaseLocation(String databaseName) throws Exception {
return warehouse.getDefaultExternalDatabasePath(databaseName).toUri().toString();
private String getDatabaseLocation(Database db) throws Exception {

Check warning on line 102 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Replace generic exceptions with specific library exceptions or a custom exception. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02K6oiSNCgpa0-uH&open=AZzh02K6oiSNCgpa0-uH&pullRequest=6267
return warehouse.getDefaultExternalDatabasePath(db).toUri().toString();
}

protected Table initTable(Hive hive, Database db) {
Expand Down
Loading
Loading