[SYSTEMDS-3948] Row-wise Sparsity Estimator#2466
Conversation
…ise sparsity estimator works for the matrix multiplication and bind test cases for now
…to the unity tests for sparsity estimation
…ontainer for row wise sparsity vectors to simplify access and allow storing it with chain nodes
…mator with element-wise and single operations
…wise and single operations NOTE: using average case estimation per row
Baunsgaard
left a comment
Baunsgaard
left a comment
There was a problem hiding this comment.
Some comments. hope it helps.
| } | ||
|
|
||
|
|
||
| // Row Wise Sparsity Estimator |
There was a problem hiding this comment.
You can optionally have all of these classes extend another class where you add these tests. Instead of copying the code on all of them.
| private final static int k = 300; | ||
| private final static double sparsity = 0.2; | ||
| // private final static OpCode eqzero = OpCode.EQZERO; | ||
| // private final static OpCode eqzero = OpCode.EQZERO; |
There was a problem hiding this comment.
Suggestion remove this edit, or do a followup PR that cleans it up.
| // Row Wise Sparsity Estimator | ||
| // @Test | ||
| // public void testRowWiseEqzero() { | ||
| // runSparsityEstimateTest(new EstimatorRowWise(), m, k, sparsity, eqzero); | ||
| // } | ||
|
|
||
| // @Test | ||
| // public void testRowWiseDiag() { | ||
| // runSparsityEstimateTest(new EstimatorRowWise(), m, m, sparsity, diag); | ||
| // } |
| case PLUS: | ||
| case NEQZERO: | ||
| case EQZERO: | ||
| return new MatrixCharacteristics(nodeLeft.getRows(), nodeLeft.getCols(), |
There was a problem hiding this comment.
consider calling the getters above the switch case, to avoid all these repeated calls.
| } | ||
| } | ||
|
|
||
| public static class RSVector { |
There was a problem hiding this comment.
Wrapping the double[] primitive is not really good practice. This adds an indication that the JIT compiler have to resolve, and slows you down. I would suggest not adding this to the code. and instead on the individual functions code up the solution without this functional indirection.
2 participants
Hi,
this PR adds the row-wise sparsity estimator from:
Lin, Chunxu, Wensheng Luo, Yixiang Fang, Chenhao Ma, Xilin Liu and Yuchi Ma;
On Efficient Large Sparse Matrix Chain Multiplication;
Proceedings of the ACM on Management of Data 2 (2024): 1 - 27.
Note that the row sparsity propagation, as described in the publication, applies to MM chains only. Other operations use fallback methods for sparsity estimation w/ row sparsity vectors, which create a cut in the sparsity estimation DAG.