Skip to content

classification xgboost train bug#16

Open
shenzhengntu wants to merge 1 commit into
postgresml:masterfrom
shenzhengntu:patch-1
Open

classification xgboost train bug#16
shenzhengntu wants to merge 1 commit into
postgresml:masterfrom
shenzhengntu:patch-1

Conversation

@shenzhengntu

Copy link
Copy Markdown

\set ON_ERROR_STOP true
\timing on

-- 清理旧数据
DROP EXTENSION IF EXISTS pgml CASCADE;
DROP SCHEMA IF EXISTS pgml CASCADE;
CREATE EXTENSION pgml;

-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');
-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');

-- =============================================
-- 分类算法 Python vs Rust 对比测试
-- 指标: f1, 差异阈值: < 0.05 (random_forest < 0.1)
-- =============================================

-- === linear 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', 'classification', 'pgml.breast_cancer', 'malignant', algorithm => 'linear', runtime => 'python'); SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'linear', runtime => 'rust'); -- === xgboost 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'python', hyperparams => '{"n_estimators": 10}'); SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'rust', hyperparams => '{"n_estimators": 10}');

-- xgboost rust "roc_auc" always 0.5-----

INFO: Metrics: {"roc_auc": 0.5, "log_loss": 10.55341, "f1": 0.50526315, "precision": 0.33802816, "recall": 1.0, "accuracy": 0.33802816, "mcc": NaN, "fit_time": 0.006182069, "score_time": 0.000807744} INFO: Comparing to deployed model f1: Some(0.9292929172515868) WARNING: New model's f1 is not better than current model. New: 0.5052631497383118, Current 0.9292929172515868 WARNING: Not deploying newly trained model.
project | task | algorithm | deployed
-----------------------------+----------------+-----------+----------
Rust Classification Compare | classification | xgboost | f

the reason is not update bst train ,lead to "roc_auc": 0.5,the normal value should be around 0.9.

\set ON_ERROR_STOP true
\timing on

-- 清理旧数据
DROP EXTENSION IF EXISTS pgml CASCADE;
DROP SCHEMA IF EXISTS pgml CASCADE;
CREATE EXTENSION pgml;

-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');
-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');

-- =============================================
-- 分类算法 Python vs Rust 对比测试
-- 指标: f1, 差异阈值: < 0.05 (random_forest < 0.1)
-- =============================================

-- === linear 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', 'classification', 'pgml.breast_cancer', 'malignant', algorithm => 'linear', runtime => 'python');
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'linear', runtime => 'rust');
-- === xgboost 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'python', hyperparams => '{"n_estimators": 10}');
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'rust', hyperparams => '{"n_estimators": 10}');

-- xgboost rust "roc_auc" always 0.5-----

INFO:  Metrics: {"roc_auc": 0.5, "log_loss": 10.55341, "f1": 0.50526315, "precision": 0.33802816, "recall": 1.0, "accuracy": 0.33802816, "mcc": NaN, "fit_time": 0.006182069, "score_time": 0.000807744}
INFO:  Comparing to deployed model f1: Some(0.9292929172515868)
WARNING:  New model's f1 is not better than current model. New: 0.5052631497383118, Current 0.9292929172515868
WARNING:  Not deploying newly trained model.
           project           |      task      | algorithm | deployed
-----------------------------+----------------+-----------+----------
 Rust Classification Compare | classification | xgboost   | f


the reason is not update bst train
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant