classification xgboost train bug#16
Open
shenzhengntu wants to merge 1 commit into
Open
Conversation
\set ON_ERROR_STOP true
\timing on
-- 清理旧数据
DROP EXTENSION IF EXISTS pgml CASCADE;
DROP SCHEMA IF EXISTS pgml CASCADE;
CREATE EXTENSION pgml;
-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');
-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');
-- =============================================
-- 分类算法 Python vs Rust 对比测试
-- 指标: f1, 差异阈值: < 0.05 (random_forest < 0.1)
-- =============================================
-- === linear 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', 'classification', 'pgml.breast_cancer', 'malignant', algorithm => 'linear', runtime => 'python');
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'linear', runtime => 'rust');
-- === xgboost 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'python', hyperparams => '{"n_estimators": 10}');
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'rust', hyperparams => '{"n_estimators": 10}');
-- xgboost rust "roc_auc" always 0.5-----
INFO: Metrics: {"roc_auc": 0.5, "log_loss": 10.55341, "f1": 0.50526315, "precision": 0.33802816, "recall": 1.0, "accuracy": 0.33802816, "mcc": NaN, "fit_time": 0.006182069, "score_time": 0.000807744}
INFO: Comparing to deployed model f1: Some(0.9292929172515868)
WARNING: New model's f1 is not better than current model. New: 0.5052631497383118, Current 0.9292929172515868
WARNING: Not deploying newly trained model.
project | task | algorithm | deployed
-----------------------------+----------------+-----------+----------
Rust Classification Compare | classification | xgboost | f
the reason is not update bst train
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
\set ON_ERROR_STOP true
\timing on
-- 清理旧数据
DROP EXTENSION IF EXISTS pgml CASCADE;
DROP SCHEMA IF EXISTS pgml CASCADE;
CREATE EXTENSION pgml;
-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');
-- 加载数据集
SELECT pgml.load_dataset('breast_cancer');
-- =============================================
-- 分类算法 Python vs Rust 对比测试
-- 指标: f1, 差异阈值: < 0.05 (random_forest < 0.1)
-- =============================================
-- === linear 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', 'classification', 'pgml.breast_cancer', 'malignant', algorithm => 'linear', runtime => 'python'); SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'linear', runtime => 'rust'); -- === xgboost 分类 ===
SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'python', hyperparams => '{"n_estimators": 10}'); SELECT * FROM pgml.train('Rust Classification Compare', algorithm => 'xgboost', runtime => 'rust', hyperparams => '{"n_estimators": 10}');
-- xgboost rust "roc_auc" always 0.5-----
INFO: Metrics: {"roc_auc": 0.5, "log_loss": 10.55341, "f1": 0.50526315, "precision": 0.33802816, "recall": 1.0, "accuracy": 0.33802816, "mcc": NaN, "fit_time": 0.006182069, "score_time": 0.000807744} INFO: Comparing to deployed model f1: Some(0.9292929172515868) WARNING: New model's f1 is not better than current model. New: 0.5052631497383118, Current 0.9292929172515868 WARNING: Not deploying newly trained model.
project | task | algorithm | deployed
-----------------------------+----------------+-----------+----------
Rust Classification Compare | classification | xgboost | f
the reason is not update bst train ,lead to "roc_auc": 0.5,the normal value should be around 0.9.